Grant Application: Improving Devel::Cover
Tue, 03-Apr-2012 by
Karen Pauley
edit post
<![CDATA[
We have received the following grant application from Paul Johnson.
Before we vote on this proposal we would like to get feedback and endorsements from the Perl community. Please leave feedback in the comments or send email with your comments to karen at perlfoundation.org.
**Name**: Paul Johnson
**Project Title**: Improving Devel::Cover
**Synopsis**:
In the past few months Booking.com has donated €100,000 to The Perl Foundation to aid the further development of the Perl 5 programming language and the craigslist Charitable Fund has donated $100,000 towards Perl 5 maintenance and for general use by the Perl Foundation.
I'd like to apply for a grant to improve Devel::Cover modelled on the successful grants currently in progress wherein David Mitchell and Nicholas Clark are working on improving the Perl 5 core.
My work situation is currently evolving from one in which I have a single full-time job to one in which I may have a number of smaller concurrent projects to work on, and so I would be able to make space to work on Devel::Cover for a day or two per week, or perhaps even more intensely for short periods. This grant would facilitate that.
**Benefits to Perl 5**:
Devel::Cover is the Perl code coverage tool. Perl is noted for its QA and testing focus, and I like to think that Devel:Cover is an important part of that. It is one of the more fully featured and unobtrusive coverage tools of any software language. In a similar fashion to the Perl core, for the most part it just works.
However, Devel::Cover is now a little over ten years old and for the majority of that time it would be safe to say that it has been primarily in maintenance mode. This is mainly because as the primary developer I have been unable to put much time into the module, and in particular I have not been able to give the kind of sustained effort required to solve some of the more tricky bugs or to add new features. I'd like to rectify that and ensure that Devel::Cover remains useful, relevant and one of the best coverage tools available for any language.
**Project Details**:
There are three main areas that are in need of attention: bugs, features and ease of development.
**Bugs**: There are currently 70 bugs on RT and various others that I have received by mail, that have been reported in other ways, or that I have found myself. Some of these are many years old because they are the sort of tricky problem which can't be solved in one evening after work.
**Features**: New features fall into two areas. The first of these overlaps with bugs somewhat. As Perl progresses, ingenious developers have created modules which bend and mold Perl's syntax and semantics, sometimes in ways which mean that Devel::Cover can no longer provide coverage.
Dancer falls into this category, as does MooseX::Declare and probably everything which uses Devel::Declare. Quite likely there are other modules in this category and as some of these modules become more popular the lack of coverage data becomes more apparent and may become an impediment to their continued uptake.
The challenge here is to recognise how these modules extend Perl's syntax and semantics, to collect coverage, and to map the results back to the enhanced syntax or altered semantics in a way which allows the developer to understand what the results mean. It's possible, and even likely, that the best way to accomplish this, in some cases, will be to improve or extend perl's core API.
The second area relates to brand new features. For example, as testing becomes a standard practice some code now has extensive test suites, and these can take a long time to execute. This leads to a natural reluctance to run the test suite as often as might be desirable. Devel::Cover could assist here by providing an optimum test ordering, or by reporting which tests exercise given changes.
Building on the same foundation as this work comes mutation coverage. This is the idea that making any functional change to your program should cause some test to fail. Coverage comes into the picture here because this useful technique becomes far too expensive unless you only execute tests which exercise the mutation.
And then there are new coverage criteria, such as path coverage or regular expression coverage. As far as I am aware, no coverage tool for any language provides regular expression coverage.
The Devel::Cover TODO list contains many more ideas for improvements in different areas.
**Ease of Development**: Many people have contributed to Devel::Cover, for which I am very grateful. But even during times when I have been relatively inactive there hasn't been sustained, substantial input from others. There are probably as many reasons for this as there are potential contributors, and I'm certainly not complaining about it, but I do want to ensure that anyone who is interested in developing Devel::Cover encounters as few difficulties as possible in doing so. For the overall health of the project it would be very nice if there were a few people who were in a position to take over maintenance if necessary. I want to remove roadblocks to this goal.
**Deliverable Elements**:
I propose to adopt a similar model to the successful grants currently in progress wherein David Mitchell and Nicholas Clark are working on improving the Perl 5 core. In those grants there are intentionally no pre-defined deliverables for the projects because the nature of the work does not lend itself such an arrangement.
I intend to devote 400 hours to work on improving Devel::Cover, paid by the hour at the same below-commercial rate as Dave and Nick. In a similar manner to them, I would post a weekly summary on the perl-qa mailing list detailing activity for the week, allowing the grant managers and anyone else who is interested to verify that the claimed hours tally with actual activity, and thus allow early flagging of any concerns. Missing two weekly reports in a row without prior notice would be grounds for one of my grant managers to terminate this grant.
Exactly as Dave and Nick do, once per calendar month I would claim an amount equal to $50 x hours worked. I would issue a report similar to the weekly ones, but summarising the whole month. The report would need to be signed off by one of the grant managers before I get paid. Note that this means I am paid entirely in arrears.
At the time of my final claim, I would also produce a report summarising the activity across the whole project period.
Also, (the "nuclear option"), the grant managers would be allowed, at any time, to inform the board that in their opinion the project is failing, and the TPF board may then, after allowing me to present my side of things, decide whether to terminate the project at that point (i.e. to not pay me for any hours worked after I was first informed that a manager had "raised the alarm").
The specific tasks mentioned above are examples of some of the things I'd like to get done. By their nature it's difficult to estimate the amount of effort required, hence the set-up of this grant.
This model has worked well for core development. This grant can be viewed as something of a proof of concept too, to see whether the model can be extended to modules which do not ship with the core. In this case Devel::Cover has a very close relationship with the core, and could be considered to have similar characteristics and problems. It is now fairly mature, it basically just works, it has a fair number of old bugs which haven't been fixed because they are "hard", relatively few people are actively working on it, and new features, which are necessary to keep it useful and relevant, need to be carefully worked into old, stable code. As such, this grant may be well suited to test this model.
**Project Schedule**:
I expect that I can deliver 400 hours of work in approximately four or five months. If I do not manage to do so, I will continue work on the grant unless the grant managers decide that I shouldn't. If circumstances permit, I may be able to finish earlier.
As described above in "deliverable elements", specific tasks in "project details" are examples of the types of things that I intend to work on. I certainly won't be able to complete everything on the TODO list during this grant, and it may turn out that other tasks are a better use of TPF's money in my opinion, and that of my grant managers and the Perl QA group.
I am available to start work on this project immediately.
**Bio.**:
I wrote Devel::Cover and have maintained it for over ten years. I have also written commercial code coverage tools. I've been using Perl for almost as long as Perl has been around. Git tells me that my first core patch was applied 14 years ago. I've spoken at eight YAPC::EU conferences. I helped to lead The Perl Foundation's presence in the Google Code-in programme 2011/2012.
**Amount Requested**: $20,000
**Endorsed by**: Nicholas Clark, Ricardo Signes, Florian Ragwitz
**Suggestions for Grant Manager**: Florian Ragwitz & Ricardo Signes
]]>
Comments (15)
For what it's worth, I heartily endorse this grant proposal.
He's maintained it for over 10 years and it just works. I say pay the man to improve it.
Can't think of any reason for not supporting this application.
Devel::Cover is an important part of the Perl ecosystem. Modifying it to match enhancements to Perl itself and to improve coverage areas can only benefit us all.
+1 from me.
Yes, Yes, Yes.
Devel::Cover is now almost a mandatory, inseparable tool in a Perl developer's toolkit.
Paul Johnson has written Devel::Cover and so has the most intimate understanding of it. He has put a lot of work recently to modernize it and bring it up to par with distribution and code standards that the community has evolved. He's an integral part of the Perl community and therefore understands all of these new standards.
I fully support this grant!
+1
I agree that Devel::Cover is an important tool that gets more important the bigger a project gets. It is therefore necessary for code coverage to be able to keep up as Perl evolves.
Having met Paul at several conferences and hackathons, I'm confident that this is money well spent.
I therefore fully endorse this grant.
I agree - it's important to have this module maintained and developed further. Paul Johnson has put in a lot of work over the years and if he need a grant to take it to the next level I think that's really worth it.
Very good use for that money!
+1
Devel::Cover is one of those tools that makes perl great and is worth every penny of support
I am all for it. Devel::Cover is an invaluable tool and investing to improve it is a great usage of grant money. Paul Johnson has done an outstanding job on the module and he is quite obviously the best person to work on it.
This is a very important project that has now become a standard in many developers tool chains. Though not Perl Core it is synonymous with Perl QA and Perl 5. I think it is important that the tpf shows a vested interest in pushing this important work to the next level.
It would be a real privilege to secure some time from Paul who is very well respected in the community as a project leader, organiser (GCi and GSoC) and programmer.
Needless to say the chance to shout how great it is having three respected programmers improving Perl 5 hasn't passed me by either.
I have no overview in the available budget which is left for Dave and Nick and I'd hate to see doing Nick and Dave doing less work for the 20.000 needed here.
But if this is no problem, I of course loved to see Devel::Cover to support newer dynamic syntax extensions (old-style and new-style) and 20.000 seems to be worth it.
*raises*both*arms*
I definitely support this.
Even moderate improvements would be helpful; a strong development effort and significant upgrades would make a huge difference to our ability to take advantage of Devel::Cover to improve our code and reduce the number of bugs that our users end up seeing.
I support this grant. Devel::Cover is an excellent tool and it isn't keeping up with the more dynamic nature of perl syntax. I'd like to see bugs closed and new syntax extensions supported before other features are considered.
Paul has a good track record supporting Devel::Cover on a very minimal, part-time basis and a focused effort on his part should lead to greater benefits.
