January 2014 Archives

We have received the following Perl 5 grant application from Ricardo Signes.

Before we vote on this proposal we would like to have a period of community consultation that will last seven days. Please leave feedback in the comments or if you prefer send email with your comments to karen at perlfoundation.org.


Ricardo Signes

Project Title

Perl QA Hackathon 2014

Amount Requested



This grant will be used to pay for travel for Ricardo Signes to and from the Perl QA Hackathon held in France in Q2 2014.

Benefits to Perl 5

I have attended five of the six Perl QA Hackathons (Oslo, Birmingham, Amsterdam, Paris, and Lancaster) and have, at each of them, been able to contribute several solid work days of very productive work to the infrastructure behind the CPAN and related tools. Specifically, I was one of the chief implementors of the new CPAN Testers platform (Metabase) and built the Fake CPAN system for testing CPAN tools, and several reusable software libraries that are used to power both Metabase and Fake CPAN. In 2012 and 2013, I worked on refactoring PAUSE, adding tests, improving maintainability, fixing bugs, and adding features. PAUSE the system which processes contributor uploads to the CPAN, manages CPAN contributor identity, and builds the CPAN indexes used by CPAN clients to locate libraries for installation.

In previous years, I also spent a significant amount of time working with other attendees on their contributions, and plan to do the same this year. This is one of the several reasons that attendance in person is incomparably superior to "virtual attendance."

Deliverable Elements

The QA Hackathon does not have a set agenda, so promising specific work product from it up front seems unwise. I have detailed, above, the sort of work that I am almost certain to do, however. Further, I will provide a public, written report of my activities at the Hackathon.

I hope, in particular, to: work on PAUSE, specifically to enable the almost-complete increase in distribution name permissions lockdown; to discuss issues of the toolchain in the core perl distribution; and to process enhancement requests for CPAN::Meta and Software::License. If time permits, I'd like to work on the incorporation of the new Pod::Simple-based Pod::Checker into core, which will require substantial recoding of our podchecker tests. I expect a number of other topics will come and also get some of my time and labor.

The hackathon takes place over the course of four days, with eight to ten hour workdays. I'll probably also be working on the travel and in the evenings.

Any software that I produce will be released under the Perl 5 standard license terms.

Applicant Biography

I have been building software in Perl professionally for about thirteen years. I am a frequent contributor of original software to the CPAN and a frequent contributor to, or maintainer of, other popular CPAN libraries. I am also a contributor to the core Perl 5 project, and its current project lead.

I have been the recipient of TPF grants four times before, all of which were successful.

Tony Cook writes:

Approximately 30 tickets were worked on, and 10 patches were applied.

- perl #120384 This month I re-worked Data::Dumper's pure-perl and XS output of hash keys to be consistent with both each other, and with the Quotekeys configuration.

- perl #82948 Also on Data::Dumper, I modified the code that dumps qr// objects to output them as qr/original re/flags rather than the qr/(?^flags:original re)/ form that tends to expand each time it is serialized.

5.34#114350 simple xs
#114350 fix the xs, tests, comment with patch
#114350 expose sdbm_prep
#114350 minor re-work, documentation, win32 testing, comment
#114350 documentation updates and comment
#114350 rebase, retest, apply, perldelta
1.45#117475 produce a draft and comment
2.00#118651 review, research POSIX readdir, write patch, test, comment
#118651 review smoke-me results, apply to blead and close
3.84#118843 reproduce and try to debug
#118843 experimentation and research, comments
#118843 report Socket.pm bug (irc)
0.50#119623 try to setup odd block size RAID (ended up with 4k blocks)
8.66#120384 write tests, ensure numbers quoted in Useperl
#120384 more tests, work on bugs in key quoting
#120384 more key quoting, new tests, fix XS
#120384 fix unicode digit handling, comment updates,
re-work XS number matching
#120384 add 1\x{660} test to quotekeys.t, produce a patch and comment
#120384 rebase and test, apply to blead and perldelta
1.35#120505 test and comment
#120505 test and apply
1.33#120531 review lib/perl5db.pl
0.33#120572 review and comment
0.43#120619 review and comment
#120619 follow-up
0.70#120634 research and comment
8.12#120635 work on a test (on Tux's HP-UX box)
#120635 get it working, and failing correctly, update
AUTHORS, push a smoke-me
#120635 review smoke reports, review test from OP and adapt
#120635 adaption continued, testing, apply
#120635 investigate solaris sem.t failures and fix
1.47#120645 research, test and comment
#120645 research and respond
2.35#120670 review, experiment, research and comment
#120670 try to reproduce again, comment
#120670 produce patch for core headers and comment
1.63#120685 try to make patch fail
#120685 retest, apply and perldelta
0.12#120687 test with base.pm fix and close
0.90#120727 review and mostly reject
#120727 test build with new patch, apply, push
0.46#120731 review and comment
#120731 review new patch and apply
0.68#120748 review and comment
0.43#120752 same as 118237, comment and merge tickets
1.13#120758, #62746 review code, C standard, tickets
0.17#120764 comment
0.27#120802 (sec) reproduce and start bisect
0.08#120806 review, test_porting and apply
0.08#120808 comment
0.45#120694 start bisect
0.43#74650 review patch and comment
7.65#81074 re-work patch and test on Win32
#81074 diagnose smoke failure, research, fix, start win32
test run, comment
#81074 discuss with leont
#81074 comment
#81074 start on code
#81074 signal handling code, start tests
5.87#82948 actually post original patch, work on quoting
#82948 more quoting, realize it can't work due to code blocks
#82948 test on older perls, install various perlbrew perls in parallel
#82948 fix backward portability, apply
0.47fix format string warning

58.69 hours total

Dave Mitchell writes:

I spent the majority of my time last month working on a function called Perl_re_intuit_start() in the regex engine.

This function is one of the major optimisations in the regex engine. For each compiled pattern, it is noted what is the longest fixed and floating strings that must appear in the string for the match to succeed, along with what char class (if any) the pattern must start with.

Using these heuristics, re_intuit_start() tries to quickly scan the string (using fast Boyer-Moore) to locate these known substrings. This allows us to either quickly reject the string as not matching, or to find a minimum start position in the string, before handing off to the normal NFA automaton.

Unfortunately it's a bit flaky when it comes to handling utf8-encoded strings, especially long ones. There are lots of bits of code that do length calculations on bits of the string, for example along the lines of "if the maximum start position plus the length of the floating string is less than the end of the string less minlen chars, then..." which ends up doing a lot of calculations of utf8 lengths of the substring between two pointers. Which for long strings becomes grindingly slow.

Also, chunks of the code are marked with reassuring comments like:

/* XXXX May be hopelessly wrong for UTF... */

The problem with this function being mainly an optimisation, is that as long as it doesn't falsely reject valid strings or choose to high a starting point, it won't fail any tests: even if it makes the match unnecessarily slow. So errors are hard to spot.

The code's also hard to understand. (Even ignoring 'fail:' and variants, it has 17 separate labels that can be goto'ed).

So I've spent most of last month going through this function, trying to understand how bits of it work, fixing issues, and generally cleaning up, refactoring and better documenting the code. I'm nowhere near finished yet.


5:32 RT#120426 Strange and unwarranted underflow in string-to-number
0:17 RT#120675 Unexpected tainting via regex using locale
38:33 RT#120692 Slow global pattern match with input from utf8
7:01 fix smoke issues
2:07 look at Compress-Raw-Zlib and -g3 issue
14:14 process p5p mailbox

67:44 Total (HH::MM)

4.4 weeks
67.7 total hours
15.3 average hours per week

As of 2013/12/31: since the beginning of the grant:

11.3 weeks
183.3 total hours
16.2 average hours per week

There are 217 hours left on the grant.

Jeffrey Ryan Thalhammer reported:

This past May, The Perl Foundation awarded a grant to fund development of a couple features in Pinto. Pinto is a robust tool for curating a private repository of CPAN modules, so you can build your application with the right modules every time. This is my fourth progress report on that work.

I've done quite a lot of work on Pinto the last few weeks. And today, I pushed a development release to CPAN that has a lot of really cool features and improvements. However, none of this work is tied to the TPF grant.

But now the backlog is cleared up a bit, so I hope to finally finish off the grant work. Look for another update next month.

PS: I'll have a lot more to say about the new Pinto features when I make a production release in a week or so. For a sneak peek, look at the change log.

Dave Mitchell writes:

I spent the majority of my time last month fixing compiler warnings in smokes; along with fixing a few bugs, "working" my p5p mailbox etc.

Over the last month I have averaged 17.1 hours per week.

As of 2013/11/30: since the beginning of the grant:

6.9 weeks
115.6 total hours
16.8 average hours per week

There are 284 hours left on the grant.

7:55 RT#119847 formats loop chewing RAM for non SvPVs
0:43 RT#120292 Can't find any loadable formatter class...
4:29 RT#120446 Slow global pattern match if match fails
39:43 fix smoke issues
3:29 help move Cwd and List-Utils distros
0:56 look at maint-5.18
15:55 process p5p mailbox

73:10 Total (HH::MM)

About TPF

The Perl Foundation - supporting the Perl community since 2000. Find out more at www.perlfoundation.org.

About this Archive

This page is an archive of entries from January 2014 listed from newest to oldest.

December 2013 is the previous archive.

February 2014 is the next archive.

Find recent content on the main index or look in the archives to find all content.


OpenID accepted here Learn more about OpenID
Powered by Movable Type 6.2.2