Improving Perl 5: Grant Report for Month 14
Tue, 04-Dec-2012 by
Karen Pauley
edit post
<![CDATA[
_Nicholas Clark writes:_
As per my grant conditions, here is a report for the November period.
There was no report for October, as I wasn't able to do any (paid) work, due to an unforseen bureaucratic trip hazard.
An "interesting" surprise emerged as a side effect of moving. It turned out that I need some paperwork to comply with something about three levels of prerequisites back from what I actually need to do. The process to get the paperwork was kicked off as soon as I became aware that it was needed, but took about 5 weeks, with no way to accelerate the process. I now have a certified translation of the obscure and slow-to-obtain document, completed the rest of the paperwork within an hour, and was back in business after lunch.
Yves has been doing some magnificent work on looking for faster/better hashing algorithms for Perl 5's hashes. Not only has he provided various alternative algorithms in hv.h, selectable at compile time, he also did all the unfun admin work of identifying and fixing bugs that this exposed in modules shipped in the core, and getting those fixes reported and applied in their upstream CPAN distributions.
With all the ducks apparently nicely in a row, blead changed from the old default of Bob Jenkins' one-at-a-time hash to MurmurHash3, which is faster, ready for the 5.17.6 release. Unfortunately, it was only at this point that we discovered that the code as-was wouldn't build on HP-UX. I was able to figure out a more portable fix just in time to make the release, which avoided Ricardo being stuck with a messy problem.
I continued looking at hashing functions. In particular, the vulnerability that we solved back in 2003 which most everyone else ignored until last year, "http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2011-4815":http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2011-4815 doesn't seem to be the end of the story.
It looks like all the brouhaha that created prompted research by Jean-Philippe Aumasson and Daniel J. Bernstein, which led among other things to the creation of SipHash, "https://www.131002.net/siphash/siphash.pdf":https://www.131002.net/siphash/siphash.pdf
The paper introducing SipHash details an approach that will recover the key used in Python's hash randomisation, but it's not clear how easy it is to pull this off in practice. There is also a bug open in Python's bug tracker stating that the implementation of randomisation is actually far far less effective than it purports to be, with only 8 bits of the key actually used, not 32: "http://bugs.python.org/issue14621":http://bugs.python.org/issue14621
Ruby has had a second CVE related to hashing:
"http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2012-5371":http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2012-5371
PHP had to issue a second CVE due to a bug in their fix for the first CVE:
"http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2012-0830":http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2012-0830
With this they raised the stakes from denial of service to potential remote code exploit ("There's been a little complication with my complication")
Having carefully checked over the causes and implications of each of the above, I don't think that any of the issues described there affect any supported releases of Perl 5. (Under the annual release policy, currently perl5-porters considers is 5.16.x, 5.14.x and 5.12.x supported. But actually everything back to 5.8.1 is fine.)
I've got a sneaking suspicion that this story still has legs, and that someone will pop up with some new surprise or twist. Hence I'm keeping an eye open to spot any more developments in this decade old saga, in case there is action Perl 5 needs to take.
I also fixed a small portability issue in a test added to t/op/fork.t
The new test used ulimit -u to lower the number of user processes to provoke fork failure. Whilst the test was careful to ensure that it only attempted this if the shell is bash or zsh, it turns out that one some platforms (at least HP-UX and AIX), bash offers ulimit -u, but will always error if its use is attempted. So probe that ulimit -u can be used, and skip the test if it can't.
A more detailed breakdown summarised from the weekly reports.
|Hours | Activity |
| 1.50 | BeOS |
| 48.50 | PERL_HASH |
| | PERL_HASH (and HP-UX)|
| 0.50 | RT #115928 |
| 0.25 | dist/threads-shared/t/stress.t |
| 0.75 | reading/responding to list mail |
| 0.25 | t/op/fork.t |
**51.75 hours total**
]]>
Category:
(none)
Comments (1)
Oops. I failed to note that I removed support for building on BeOS. BeOS was an operating system for personal computers developed by Be Inc, initially for their BeBox hardware. The OS Haiku was written as an open source replacement/continuation for BeOS, and its perl port is current and actively maintained.
The BeOS port had not been updated since 2004.
Support for the EPOC port was removed the previous month, and MPE/iX the month before. The next "victim" is Rhapsody, which will be removed before the v5.17.8 release.
The list of suspected "special biologist word for stable" platforms is here:
https://metacpan.org/module/RJBS/perl-5.16.0/pod/perldelta.pod#Platforms-with-no-supporting-programmers:
(yes, with a colon at the end. Dear blog formatting software, please take note)
