David Mitchell has submitted a grant proposal, which if accepted would make use of a portion of the funding generously provided to TPF by "Booking.com":http://www.booking.com/.
Before the Board votes on this proposal we would like to get feedback and endorsements from the Perl community. Please leave feedback in the comments or send email with your comments to karen at perlfoundation.org.
Grant Title: Fixing perl5 core bugs
Name: David Mitchell
Amount Requested: $25,000
Synopsis
Recently, booking.com donated $50K for the "further development and maintenance of the Perl programming language". I would like part of that money to to be used to fund me for approximately six months to devote 50% of my time fixing "hard" core perl5 bugs.
Benefits to the Perl Community
There are currently approximately 1200 open and 300 new bug reports in the perl5 bug queue. Although some of these are of the "5.003_08 does not build on platform X" variety, many are current: for example, almost 500 of them were created after the release of 5.10.0. As the perl core has become more and more gnarly, and the pool of experienced but active core hackers has declined, these bugs are just piling up and not getting fixed, especially the hard ones. With this funding, I would would be able to devote serious time and effort to making a dent in this queue.
Note that unlike many large open source projects, perl has no paid developers devoted to bug fixing.
Deliverables
Unusually for a TPF grant, there are not clear-cut deliverables for this project. I intend to devote 500 hours of my time over the next six months fixing perl core bugs. The net result will be a list of bug numbers that have been diagnosed, and (hopefully) fixed. Because it's impossible to predict in advance how difficult a bug is going to be to diagnose and fix (or indeed whether it is even fixable), I can't commit in advance to a fixed list of bugs that I will fix over the course of the grant. Nor is it realistic to have a bounty per fixed bug; I would end up not getting rewarded for time spent on difficult bugs, and conversely I would have a strong incentive to cherry-pick easy bugs, defeating the purpose of the grant.
Therefore, monitoring of my progress will become important (see below).
Project Details
I think this has been fully covered above.
Inch-stones
Note that due to the length and scale of this project, it is suggested that there be two project managers, who can spread the monitoring load between them as they see fit.
Since this project is heavily based on hours worked and the monitoring thereof, I would post a weekly summary on the p5p mailing list which details, for each bug worked on that week, how many hours were spent on diagnosis and fixes, plus any bug status changes. This frequent feedback would allow the grant managers and active core developers (who will be aware of any recent commits and other activity of mine) to observe whether my claimed hours bear any relation to actual activity and results, and thus allow early flagging of any concerns.
Missing two weekly reports in a row without prior notice would be grounds for terminating the project.
Once per calendar month I would claim an amount equal to $50 x hours worked. I would issue a report similar to the weekly ones, but summarizing the whole month. The report would need to be signed off by one of the project managers before I get paid. Note that this means I am paid entirely in arrears.
At the time of my final claim, I would also produce a report summarising the activity across the whole project period.
Also, (the "nuclear option"), I suggest that either of the project managers be allowed, at any time, to inform the board that in their opinion the project is failing badly, and that the TPF board may then, after allowing me to present my side of things, to vote whether to terminate the project at that point (i.e. to not pay me for any hours worked after I was first informed that a manager had "raised the alarm").
To ensure that at there are at least some visible results for the hours spent, I would be required have closed at least one bug per 20 hours before being able to claim money for those hours. (I would hope to close more bugs than that, but by setting a low baseline, I'm not tying my hands, while still allowing TPF to have something visible for publicity purposes during the interim.)
Project Schedule
I am available to start work on this project immediately.
The project is expected to take six months. I am self-employed, which allows me a good deal of flexibility. By promising approximately 50% of my time, this gives me the ability to continue with my existing commitments to other clients, while deferring seeking new clients. As such, the weekly hours I devote to perl are likely to be highly variable, but hopefully averaging out to about 20 hours per week. If for some reason I find that I have spent less than 500 hours at the end of the six months, then I will continue the project until until the 500 hours been spent, with the proviso that that the TPF board are free to terminate the project at any time after the six months. Conversely, if I manage to devote more than 20 hours per week, then my monthly payments will be accordingly larger, and the project will terminate early (once the 500 hours are spent).
Note that it is currently my intention that the after six months I will apply for a further $25K extension, although there is no obligation for me to do so, nor for TPF to approve it.
Bio
I'm a freelance UNIX sysadmin and programmer living in the UK. I have been using perl since 1993, and have been fixing core perl 5 bugs since 2001. I have had commit rights since 2003 and I was pumpking for the 5.10.1 perl release.
In short, I am one of only a handful of active people who understand large parts of the perl internals and who can thus fix "hard" bugs.
I like the way this grant proposal is organized, particularly how it builds in accountability.
If people like Nicholas Clark and other core contributors were to endorse this, I think this would be a no-brainer for funding.
I'm not really familiar with a lot that p5p does, nor am I qualified to comment on whether or not these 1500 bugs are serious or not. But this seems like a great idea to me. Having a number that large of bugs isn't good for Perl's reputation of stability and progress. And it's hard to find volunteers to do bug triage (which I assume a lot of this work will be).
This grant is for a lot of money, but at the same time there are a lot of precautions built in to make sure that money won't be payed out without accompanying work. And it would be nice to have someone being payed to work on Perl's core even if it is part time.
I'm acquainted with the quality of Dave's patches, and being a long-standing perl5 committee, I fully support this grant.
Generally, I like the idea and I like the strict accountability framework that Dave M. has designed. I think the gnarly bugs pile up because fixing them takes concentrated time and this would provide for an expert to spend that concentrated time consistently for several months.
I would like to see a few things added to this grant:
On the first point -- we've built up huge technical debt allowing so many bug reports to pile up that it's hard to know where Dave's time is best spent. I would support him spending the first month doing nothing but triage on the roughly 500 "high" or "medium" severity tickets to determine (a) if they correctly classified, (b) whether they can be closed as fixed/end-of-life and (c) a rough sense of difficulty. I'd then like to see the remaining time spent on the higher severity tickets across a mix of difficulties (banging out some easier ones and diving into some trickier ones).
On the second point -- I believe closing irrelevant or non-replicable tickets is also valuable work. I wouldn't want to see that become the bulk of Dave's effort, but I wouldn't want him to avoid doing so for lack of "credit". I would expect the managers to consider such closings as counting towards "quota" while providing some degree of oversight (above and beyond p5p members) that such closings are appropriate.
On the third point -- there were a number of meta-tickets created to track blocking, non-blocking, "needs expert assessment" and "end-of-life-wontfix" issues. I'd like to see Dave M. work within (or improve) that framework as he goes and to focus particularly on the "needs expert assessment" meta-ticket. This would provide means for community members to flag particular tickets for Dave to review, giving them a voice in guiding the effort.
I have confidence in David's ability and commitment. I fully support this proposal.
I fully support this proposal.
I fully support this proposal as well. Dave is one of the few people with the ability and experience to address many of the hard core perl bugs. This grant will give him some time to do so in a useful, concentrated manner. His work on perl-5.10.1 is more than ample evidence that he is capable of understanding difficult core issues, and has a good sense of the appropriate balance between maintaining backwards compatibility and the occasional necessity to break things in order to move forward.
1. On triage: It is certainly likely that a considerable amount of time will effectively be devoted to triaging efforts, but I think it would be ill-advised for TPF to require too much specifically in that area. Bug reports vary greatly in how easy they are to triage. Especially for the hard problems that Dave proposes to tackle, the time and effort required to do steps (a), (b), and (c) is such that it often wouldn't make sense to stop there and simply report it as a "hard" problem. Instead, that's the precise best time to try to solve it. Saying "triage it in month 1, but don't fix it until month 2" might involve re-learning everything that was learned in month 1 about the bug.
More broadly, however, I completely agree -- as Dave goes through the bug database, he should leave plenty of notes and comments on tickets if he has anything useful to add.
2. & 3.: I agree with both of these points. Indeed, I'd hope the meta-ticket process would evolve in useful ways as a result of this process.
I approve of this proposal. I note that proposal is specifically about "hard"
core bugs, not directly about managing the bug queue. I agree with this.
Historically we have at times had various unsung (volunteer) heroes quietly
dig through it. (I'm aware of Bram, Steve Peters and Schwern, and doubtless
unaware of others.) Usually it's possible to knock it back to about 1300 open
bugs, but rarely further, because those that remain are too hard to be fixed
in an a volunteer's available time.
Hence why we need to pay someone to break out of this loop.
I like the proposed mechanisms for ensuring accountability. To my mind they
strike a good compromise between verifying that work is being done, whilst not
attempting to micromanage or dictate which bugs should be investigated.
Specifically I like the plan for payment by hours worked, rather than bugs
closed, because it will make for better quality fixes. The opposite would mean
a subconscious bias towards the quickest fix for the symptoms, in order to
close the bug, rather than investigating and fixing the root cause. The
opposite would also lead to more bugs being closed, but simpler bugs,
rather than the harder bugs that would likely never otherwise be addressed.
I understand where David Golden is coming from in his suggestion, but I do not
agree with it formally becoming part of the proposal.
Specifically, I object to the idea of spending the first month solely triaging
tickets as
The planets don't align like this very often. TPF has a cash drawer bulging a bit larger than usual due to the generous booking.com donation, requiring an ambitious grant of significant and lasting benefit to Perl. Many good things are happening in Perl 5 core development, but the proposal is unfortunately quite accurate about the size and nature of the bug queue, and the tendency of very part-time volunteers to punt on the really hard ones. Jesse is demonstrating all sorts of creative leadership but quite explicitly said when he took up the 5.12 patch pumpkin, "I am not a perlguts rockstar."[1]
Dave is. Obviously. The exact thing that is hardest to find, that is most urgently needed, and that you can't put an ad in the paper for even if you have a lot more money than TPF does is exactly what Dave is offering at exactly the price TPF can afford. The proposal shows an excellent balance of simplicity, flexibility, autonomy, accountability, and a "start yesterday" mentality.
Please accept the proposal as is and allow discussions of other things Dave might do to go on in parallel with getting some bugs fixed; they can always become part of what I hope will be a renewed grant 500 hours later.
[1] http://www.nntp.perl.org/group/perl.perl5.porters/2009/10/msg152913.html
Responding to Nicholas' points:
Given the size of the grant, I think it's appropriate to have an early read on where Dave is planning to spend his time. That's why I suggested triaging the high/medium severity bugs (~500). If Dave thinks that's too much time just on triage, that's fine. I'd still encourage triage of just the high severity bugs. My point is that I want to see a plan of attack to ensure that his efforts are directed towards the most important areas. I support "up to" (not mandating) a month of triage/planning as valuable thinking work. I'm not saying he should only triage during that time. I'm saying that we should be willing to pay for a thoughtful approach, which I expect will take a few weeks to figure out.
On the "expert assessment" meta tickets -- again, I don't suggest this be mandated for the reasons you suggest. But then again, I think the list of people who can attach tickets to a meta ticket is limited so you already have a first filter. (I could be wrong.) My suggestion is that he triage in a similar way, using those tickets and that he try to assess the higher severity tickets in the "expert assessment" queue. I think that it's just as frustrating for the community to do bug triage and then have no experts assess the ones that need it. The meta-ticket system broadly seems to be working well for Perl 5.11.X efforts, so reinforcing what's working seems like a good idea. It shouldn't consume his efforts, but I think it's a valuable area for at least some of his time and attention.
I support this proposal, and I don't think there should be more formalities than proposed - Dave should be able to concentrate on hacking as much as possible.
I support this proposal as-is. I don't think Dave will have any trouble finding relevant bugs to work on, especially once he starts sending reports to perl5-porters which can give him good ones to work on in the very unlikely case that he can't find any.
I fully support this proposal as is. I've known Dave through his work on the Perl core for almost six years now. I fully trust that Dave will continue working much as he has during that time by tackling the hard and higher priority bugs.
I believe in the objective of this proposal. I hope that it gets approved for everyone's benefit.