We have received the following grant application from Paul Johnson.
Before we vote on this proposal we would like to get feedback and endorsements from the Perl community. Please leave feedback in the comments or send email with your comments to karen at perlfoundation.org.
Name: Paul Johnson
Project Title: Improving Devel::Cover
In the past few months Booking.com has donated €100,000 to The Perl Foundation to aid the further development of the Perl 5 programming language and the craigslist Charitable Fund has donated $100,000 towards Perl 5 maintenance and for general use by the Perl Foundation.
I'd like to apply for a grant to improve Devel::Cover modelled on the successful grants currently in progress wherein David Mitchell and Nicholas Clark are working on improving the Perl 5 core.
My work situation is currently evolving from one in which I have a single full-time job to one in which I may have a number of smaller concurrent projects to work on, and so I would be able to make space to work on Devel::Cover for a day or two per week, or perhaps even more intensely for short periods. This grant would facilitate that.
Benefits to Perl 5:
Devel::Cover is the Perl code coverage tool. Perl is noted for its QA and testing focus, and I like to think that Devel:Cover is an important part of that. It is one of the more fully featured and unobtrusive coverage tools of any software language. In a similar fashion to the Perl core, for the most part it just works.
However, Devel::Cover is now a little over ten years old and for the majority of that time it would be safe to say that it has been primarily in maintenance mode. This is mainly because as the primary developer I have been unable to put much time into the module, and in particular I have not been able to give the kind of sustained effort required to solve some of the more tricky bugs or to add new features. I'd like to rectify that and ensure that Devel::Cover remains useful, relevant and one of the best coverage tools available for any language.
There are three main areas that are in need of attention: bugs, features and ease of development.
Bugs: There are currently 70 bugs on RT and various others that I have received by mail, that have been reported in other ways, or that I have found myself. Some of these are many years old because they are the sort of tricky problem which can't be solved in one evening after work.
Features: New features fall into two areas. The first of these overlaps with bugs somewhat. As Perl progresses, ingenious developers have created modules which bend and mold Perl's syntax and semantics, sometimes in ways which mean that Devel::Cover can no longer provide coverage.
Dancer falls into this category, as does MooseX::Declare and probably everything which uses Devel::Declare. Quite likely there are other modules in this category and as some of these modules become more popular the lack of coverage data becomes more apparent and may become an impediment to their continued uptake.
The challenge here is to recognise how these modules extend Perl's syntax and semantics, to collect coverage, and to map the results back to the enhanced syntax or altered semantics in a way which allows the developer to understand what the results mean. It's possible, and even likely, that the best way to accomplish this, in some cases, will be to improve or extend perl's core API.
The second area relates to brand new features. For example, as testing becomes a standard practice some code now has extensive test suites, and these can take a long time to execute. This leads to a natural reluctance to run the test suite as often as might be desirable. Devel::Cover could assist here by providing an optimum test ordering, or by reporting which tests exercise given changes.
Building on the same foundation as this work comes mutation coverage. This is the idea that making any functional change to your program should cause some test to fail. Coverage comes into the picture here because this useful technique becomes far too expensive unless you only execute tests which exercise the mutation.
And then there are new coverage criteria, such as path coverage or regular expression coverage. As far as I am aware, no coverage tool for any language provides regular expression coverage.
The Devel::Cover TODO list contains many more ideas for improvements in different areas.
Ease of Development: Many people have contributed to Devel::Cover, for which I am very grateful. But even during times when I have been relatively inactive there hasn't been sustained, substantial input from others. There are probably as many reasons for this as there are potential contributors, and I'm certainly not complaining about it, but I do want to ensure that anyone who is interested in developing Devel::Cover encounters as few difficulties as possible in doing so. For the overall health of the project it would be very nice if there were a few people who were in a position to take over maintenance if necessary. I want to remove roadblocks to this goal.
I propose to adopt a similar model to the successful grants currently in progress wherein David Mitchell and Nicholas Clark are working on improving the Perl 5 core. In those grants there are intentionally no pre-defined deliverables for the projects because the nature of the work does not lend itself such an arrangement.
I intend to devote 400 hours to work on improving Devel::Cover, paid by the hour at the same below-commercial rate as Dave and Nick. In a similar manner to them, I would post a weekly summary on the perl-qa mailing list detailing activity for the week, allowing the grant managers and anyone else who is interested to verify that the claimed hours tally with actual activity, and thus allow early flagging of any concerns. Missing two weekly reports in a row without prior notice would be grounds for one of my grant managers to terminate this grant.
Exactly as Dave and Nick do, once per calendar month I would claim an amount equal to $50 x hours worked. I would issue a report similar to the weekly ones, but summarising the whole month. The report would need to be signed off by one of the grant managers before I get paid. Note that this means I am paid entirely in arrears.
At the time of my final claim, I would also produce a report summarising the activity across the whole project period.
Also, (the "nuclear option"), the grant managers would be allowed, at any time, to inform the board that in their opinion the project is failing, and the TPF board may then, after allowing me to present my side of things, decide whether to terminate the project at that point (i.e. to not pay me for any hours worked after I was first informed that a manager had "raised the alarm").
The specific tasks mentioned above are examples of some of the things I'd like to get done. By their nature it's difficult to estimate the amount of effort required, hence the set-up of this grant.
This model has worked well for core development. This grant can be viewed as something of a proof of concept too, to see whether the model can be extended to modules which do not ship with the core. In this case Devel::Cover has a very close relationship with the core, and could be considered to have similar characteristics and problems. It is now fairly mature, it basically just works, it has a fair number of old bugs which haven't been fixed because they are "hard", relatively few people are actively working on it, and new features, which are necessary to keep it useful and relevant, need to be carefully worked into old, stable code. As such, this grant may be well suited to test this model.
I expect that I can deliver 400 hours of work in approximately four or five months. If I do not manage to do so, I will continue work on the grant unless the grant managers decide that I shouldn't. If circumstances permit, I may be able to finish earlier.
As described above in "deliverable elements", specific tasks in "project details" are examples of the types of things that I intend to work on. I certainly won't be able to complete everything on the TODO list during this grant, and it may turn out that other tasks are a better use of TPF's money in my opinion, and that of my grant managers and the Perl QA group.
I am available to start work on this project immediately.
I wrote Devel::Cover and have maintained it for over ten years. I have also written commercial code coverage tools. I've been using Perl for almost as long as Perl has been around. Git tells me that my first core patch was applied 14 years ago. I've spoken at eight YAPC::EU conferences. I helped to lead The Perl Foundation's presence in the Google Code-in programme 2011/2012.
Amount Requested: $20,000
Endorsed by: Nicholas Clark, Ricardo Signes, Florian Ragwitz
Suggestions for Grant Manager: Florian Ragwitz & Ricardo Signes