September 2013 Archives

Tony Cook writes:

The most interesting changes this month were probably:

- 115928 - a common rand() implementation

- 117265 - failing system calls on embedded NULs

- 3330 - warning on ++$foo when $foo isn't a number, or magically
incrementable

- 85104 - preserving $^E across signal handlers

perl #115928 This month I implemented perl's new rand() implementation, avoiding the portability issues of random number generators on various platforms. This does not provide a cryptographically usable rand(), but means that some platforms now generate a larger selection of random numbers, rather than the say 32768 different possible numbers on Win32. This wasn't applied until September.

perl #117265 I did some clean up of the safesyscalls patch and finally applied it this month.

perl #3330 This adds a warning when the user attempts to use perl's increment operator on a string that isn't a number and doesn't match /^[A-Za-z]+[0-9]+/.

perl #85104 I produced an alternative fix for this in July, but hadn't written any tests until now. I first attempted to generate SIGINT and SIGBREAK using kill, hopefully trying to test a "real" operating system delivered signal, but the signal wasn't being delivered, even when I went direct to the API using GenerateConsoleCtrlEvent(). Unfortunately Windows would never deliver the event without signalling the whole console, possibly killing the TEST or harness process too.

I ended up using a thread and perl's Win32 signal emulation to test that $^E is preserved.

Hague Grant Application Accepted

2 Comments

I'm pleased to announce that diakopter's grant proposal, Wrap up Perl 5 Interop for Rakudo Perl 6 on MoarVM, has been accepted. I would like to thank everyone who took the time to comment on the proposal.

If you would like to help fund grants like this or any of our other projects please visit our donation system.

[Editor's note: Paul has posted 3 Grant reports together due to a very hectic workload and the need to do the work but hold on the reporting of the work. This is a large undertaking and I would like to thank him for his diligence, and tenacity, in filing the paperwork]

In accordance with the terms of my grant from TPF this is the monthly report for my work on improving Devel::Cover covering August 2013.

This month I released Devel::Cover versions 1.07 and 1.08.

Version 1.07 was primarily concerned with getting Devel::Cover to work with recent Perl releases: 5.18.1, 5.19.1, 5.19.2 and 5.19.3. The main problem was with 5.18.1 which is a stable, maintenance release of Perl and which was causing Devel::Cover the fail its tests rather dramatically. And the tests were showing up a real problem which stopped Devel::Cover working with Perl 5.8.1 at all.

The original bug report against Perl, filed by Jim Keenan, is RT #119351: Perl 5.18.1 breaks Devel::Cover. And the analysis of the problem is at https://rt.perl.org/rt3//Public/Bug/Display.html?id=119351#txn-1245355

The minimal test case is:

$ perl5.18.0 MB -E 'say B::main_cv>GV // "undef"'
B::SPECIAL=SCALAR
$ perl5.18.1 MB -E 'say B::main_cv>GV // "undef"'
undef

which shows the regression from 5.18.0 and, indeed, from every stable version since 5.6.1, which is the first version that Devel::Cover works with and, I think, the first version which supported this construct. The reason that Devel::Cover has, and always had, a minimum requirement of Perl 5.6.1 was because that was the first version of Perl which supported sufficient introspection capabilities (B::) to make Devel::Cover possible.

The offending commit was e6c4c33, which was an important fix for a crash related to lexical subroutines, but which had this unfortunate and unintended consequence.

Further discussion in RT #119413: Reconsider e6c4c33 to ext/B/ in light of test failures, suggests that we will see a Perl 5.18.2 release fixing this regression sooner rather than later.

On a personal note, I'm annoyed that I didn't test any of the 5.18.1 release candidates against Devel::Cover. Doing so could have prevented 5.18.1 being released with this regression. I do usually test the release candidates against all the modules I use personally or in my work, but this release caught me during a rather busy time.

In any case, irrespective of what happens with upcoming Perl releases, Devel::Cover 1.07 and later work with every stable version from 5.6.1 to 5.18.1, and should work with 5.18.2 whether or not this regression is fixed.

The same regression, unsurprisingly, was in bleadperl, and was causing Devel::Cover to fail with recent development releases. The fix for 5.18.1 also worked for the development releases, but Perl 5.19.2 and 5.19.3 had another problem. The problem was that line numbers reported for certain constructs changed and whilst Devel::Cover still worked, the tests were failing because the line numbers against which coverage was reported didn't match.

This problem was reported in Perl RT #118931 and shows that the problem started with 2179133 and was fixed with bf1b738. But since there are some cpantesters running 5.19.2 and 5.19.3, I caused the problematic tests to be skipped on those versions. So as of the end of August, The current version of Devel::Cover should pass its tests against all stable versions of Perl since 5.6.1 and all 5.19.x development releases.

As usual, there were also some patches applied, pull requests merged, and other bugs fixed.

As I mentioned in previous reports, I'm still rather busy with other work, but I hope that I will be able to start devoting more time to Devel::Cover towards the end of September.

**Closed Github tickets**

  • 66 test failed on Perl 5.18.1
  • 67 typo fixes
  • 68 Devel::Cover 1.06: many test failures with Perl 5.18.1

**Merged pull requests**

  • 67 typo fixes

Closed RT tickets

* #88027 [PATCH] POD encoding

Fixed cpantesters reports

  • http://www.cpantesters.org/cpan/report/f55d93ca-ef02-11e2-86a0-c002f2ff63fb
  • http://www.cpantesters.org/cpan/report/30be31c6-f1b6-11e2-afc2-f2e1da45041b
  • http://www.cpantesters.org/cpan/report/064f36d0-f1eb-11e2-bf5c-b8f3f1ff63fb
  • http://www.cpantesters.org/cpan/report/fbc3b37c-f356-11e2-9ec7-5be4f1ff63fb
  • http://www.cpantesters.org/cpan/report/8cb777fe-f3d6-11e2-a1a6-d9e588f30751
  • http://www.cpantesters.org/cpan/report/01337d58-f576-11e2-bcc8-eaa4f1ff63fb
  • http://www.cpantesters.org/cpan/report/d9066a0a-f4be-11e2-ac5a-1eb5f1ff63fb
  • http://www.cpantesters.org/cpan/report/cd493678-fdf3-11e2-bc11-107f5388eb25
  • http://www.cpantesters.org/cpan/report/d1eb5ae0-038d-11e3-acb5-a176cd95e79a
  • http://www.cpantesters.org/cpan/report/48e7199c-03c3-11e3-8a37-e2882d12fa71
  • http://www.cpantesters.org/cpan/report/0a95b548-041f-11e3-8a37-e2882d12fa71
  • http://www.cpantesters.org/cpan/report/f17470be-0418-11e3-935d-85093788882e
  • http://www.cpantesters.org/cpan/report/33dc0f64-0560-11e3-8a37-e2882d12fa71
  • http://www.cpantesters.org/cpan/report/c535a3e6-0752-11e3-bc0a-b75d6d822b3f
  • http://www.cpantesters.org/cpan/report/56faf65a-0767-11e3-8b51-dc8ed8c219ae
  • http://www.cpantesters.org/cpan/report/0f4fba8e-eff2-11e2-85ee-142989f30751
  • http://www.cpantesters.org/cpan/report/c0f0efc6-efd7-11e2-8541-50faf1ff63fb
  • http://www.cpantesters.org/cpan/report/bea64e9c-f2a1-11e2-b883-7597f1ff63fb
  • http://www.cpantesters.org/cpan/report/7e018316-f61b-11e2-9843-3cfc88f30751

You can see the commits at https://github.com/pjcj/Devel--Cover/commits/master

Hours worked

  • 21/08 5:20
  • 24/08 4:15
  • Total 9:35

Total hours worked on grant: 294:30

[Editor's note: Paul has posted 3 Grant reports together due to a very hectic workload and the need to do the work but hold on the reporting of the work. This is a large undertaking and I would like to thank him for his diligence, and tenacity, in filing the paperwork]

In accordance with the terms of my grant from TPF this is the monthly report for my work on improving Devel::Cover covering August 2013.

This month I released Devel::Cover versions 1.07 and 1.08.

Version 1.07 was primarily concerned with getting Devel::Cover to work with recent Perl releases: 5.18.1, 5.19.1, 5.19.2 and 5.19.3. The main problem was with 5.18.1 which is a stable, maintenance release of Perl and which was causing Devel::Cover the fail its tests rather dramatically. And the tests were showing up a real problem which stopped Devel::Cover working with Perl 5.8.1 at all.

The original bug report against Perl, filed by Jim Keenan, is RT #119351: Perl 5.18.1 breaks Devel::Cover. And the analysis of the problem is at https://rt.perl.org/rt3//Public/Bug/Display.html?id=119351#txn-1245355

The minimal test case is:

$ perl5.18.0 MB -E 'say B::main_cv>GV // "undef"'
B::SPECIAL=SCALAR
$ perl5.18.1 MB -E 'say B::main_cv>GV // "undef"'
undef

which shows the regression from 5.18.0 and, indeed, from every stable version since 5.6.1, which is the first version that Devel::Cover works with and, I think, the first version which supported this construct. The reason that Devel::Cover has, and always had, a minimum requirement of Perl 5.6.1 was because that was the first version of Perl which supported sufficient introspection capabilities (B::) to make Devel::Cover possible.

The offending commit was e6c4c33, which was an important fix for a crash related to lexical subroutines, but which had this unfortunate and unintended consequence.

Further discussion in RT #119413: Reconsider e6c4c33 to ext/B/ in light of test failures, suggests that we will see a Perl 5.18.2 release fixing this regression sooner rather than later.

On a personal note, I'm annoyed that I didn't test any of the 5.18.1 release candidates against Devel::Cover. Doing so could have prevented 5.18.1 being released with this regression. I do usually test the release candidates against all the modules I use personally or in my work, but this release caught me during a rather busy time.

In any case, irrespective of what happens with upcoming Perl releases, Devel::Cover 1.07 and later work with every stable version from 5.6.1 to 5.18.1, and should work with 5.18.2 whether or not this regression is fixed.

The same regression, unsurprisingly, was in bleadperl, and was causing Devel::Cover to fail with recent development releases. The fix for 5.18.1 also worked for the development releases, but Perl 5.19.2 and 5.19.3 had another problem. The problem was that line numbers reported for certain constructs changed and whilst Devel::Cover still worked, the tests were failing because the line numbers against which coverage was reported didn't match.

This problem was reported in Perl RT #118931 and shows that the problem started with 2179133 and was fixed with bf1b738. But since there are some cpantesters running 5.19.2 and 5.19.3, I caused the problematic tests to be skipped on those versions. So as of the end of August, The current version of Devel::Cover should pass its tests against all stable versions of Perl since 5.6.1 and all 5.19.x development releases.

As usual, there were also some patches applied, pull requests merged, and other bugs fixed.

As I mentioned in previous reports, I'm still rather busy with other work, but I hope that I will be able to start devoting more time to Devel::Cover towards the end of September.

**Closed Github tickets**

  • 66 test failed on Perl 5.18.1
  • 67 typo fixes
  • 68 Devel::Cover 1.06: many test failures with Perl 5.18.1

**Merged pull requests**

  • 67 typo fixes

Closed RT tickets

* #88027 [PATCH] POD encoding

Fixed cpantesters reports

  • http://www.cpantesters.org/cpan/report/f55d93ca-ef02-11e2-86a0-c002f2ff63fb
  • http://www.cpantesters.org/cpan/report/30be31c6-f1b6-11e2-afc2-f2e1da45041b
  • http://www.cpantesters.org/cpan/report/064f36d0-f1eb-11e2-bf5c-b8f3f1ff63fb
  • http://www.cpantesters.org/cpan/report/fbc3b37c-f356-11e2-9ec7-5be4f1ff63fb
  • http://www.cpantesters.org/cpan/report/8cb777fe-f3d6-11e2-a1a6-d9e588f30751
  • http://www.cpantesters.org/cpan/report/01337d58-f576-11e2-bcc8-eaa4f1ff63fb
  • http://www.cpantesters.org/cpan/report/d9066a0a-f4be-11e2-ac5a-1eb5f1ff63fb
  • http://www.cpantesters.org/cpan/report/cd493678-fdf3-11e2-bc11-107f5388eb25
  • http://www.cpantesters.org/cpan/report/d1eb5ae0-038d-11e3-acb5-a176cd95e79a
  • http://www.cpantesters.org/cpan/report/48e7199c-03c3-11e3-8a37-e2882d12fa71
  • http://www.cpantesters.org/cpan/report/0a95b548-041f-11e3-8a37-e2882d12fa71
  • http://www.cpantesters.org/cpan/report/f17470be-0418-11e3-935d-85093788882e
  • http://www.cpantesters.org/cpan/report/33dc0f64-0560-11e3-8a37-e2882d12fa71
  • http://www.cpantesters.org/cpan/report/c535a3e6-0752-11e3-bc0a-b75d6d822b3f
  • http://www.cpantesters.org/cpan/report/56faf65a-0767-11e3-8b51-dc8ed8c219ae
  • http://www.cpantesters.org/cpan/report/0f4fba8e-eff2-11e2-85ee-142989f30751
  • http://www.cpantesters.org/cpan/report/c0f0efc6-efd7-11e2-8541-50faf1ff63fb
  • http://www.cpantesters.org/cpan/report/bea64e9c-f2a1-11e2-b883-7597f1ff63fb
  • http://www.cpantesters.org/cpan/report/7e018316-f61b-11e2-9843-3cfc88f30751

You can see the commits at https://github.com/pjcj/Devel--Cover/commits/master

Hours worked

  • 21/08 5:20
  • 24/08 4:15
  • Total 9:35

Total hours worked on grant: 294:30

Devel::Cover Grant Report (July)

No Comments

[Editor's note: Paul has posted 3 Grant reports together due to a very hectic workload and the need to do the work but hold on the reporting of the work. This is a large undertaking and I would like to thank him for his diligence, and tenacity, in filing the paperwork)

In accordance with the terms of my grant from TPF this is the monthly report for my work on improving Devel::Cover covering July 2013.

This month I released Devel::Cover versions 1.05 and 1.06. The releases contain a number of patches and pull requests along with various little cleanups and other small fixes.

The rest of the work this month was spent on p5cover. This is the project to get test coverage of the perl core and the core modules. The perl.gcov Makefile target was recently removed from the perl core so I needed to update the p5cover code slightly. I also improved the p5cover process, but there is still work needed here which I am hoping to get completed soon. The 1.06 release was purely the facilitate the p5cover work.

As I mentioned in June's report, I'm still heavily involved in other work, so
I wasn't able to invest much time in Devel::Cover for July.

Closed Github tickets
* 63 cpancover: Errors in write_csv()
* 54 Test fails when coverage enabled

Merged pull requests
* 65 Typo fix
* 64 Correct errors in write_csv().
* 53 Improve -coverage options, fix -ignore_re for .gcov files

Closed RT tickets
* #34888 Fix pod coverage for multiple packages in a file.

You can see the commits at https://github.com/pjcj/Devel--Cover/commits/master

Hours worked

  • 16/07 1:30
  • 17/07 6:30
  • 18/07 1:45
  • Total 9:45

Total hours worked on grant: 284:55

Devel::Cover Grant Report (June)

No Comments

[Editor's note: Paul has posted 3 Grant reports together due to a very hectic workload and the need to do the work but hold on the reporting of the work. This is a large undertaking and I would like to thank him for his diligence, and tenacity, in filing the paperwork)

In accordance with the terms of my grant from TPF this is the monthly
report for my work on improving Devel::Cover covering June 2013.

This month I released Devel::Cover 1.04. The release contains a number of fixes from last month, along with performance improvements.

The performance improvements are part of work package I started at the end of May and which continued into the beginning of June. They are primarily related to the checking of whether or not a file should be reported on for coverage. This is a relatively expensive operation that has to be performed frequently. As I mentioned in the previous report, some of this work is done by a Perl subroutine, and ideally this should be rewritten in XS. I didn't do that work this month, but instead I focussed on reducing the number of times that subroutine was called.

Devel::Cover adds quite a performance overhead to running a test suite, and this is something that discourages its use, so I feel it is important to try to keep the code as lean as possible.

The other Devel::Cover work from this month was just small fixes in various areas.

Unfortunately, I have been very busy with other work this month, and that is likely to continue throughout July and August. I'm keen that a large majority of my grant work should be spent getting to grips with some of the larger problems that I would otherwise be unable to tackle, and I will probably not be in a position to tackle such problems until after my other work subsides somewhat, which should be in September.

You can see the commits at https://github.com/pjcj/Devel--Cover/commits/master

**Hours worked**

  • 01/06 4:20
  • 22/06 1:15
  • Total 5:35

Total hours worked on grant: 275:10

Tony Cook writes:

The most interesting changes this month were probably:

  • 116190 - making -F imply -a and -a imply -n.
  • 116989 - S_croak_memory_wrap
  • 117265 - safesyscalls
  • 74798 - Data::Dumper XS

perl #116190 The original request in this ticket was that -F without -a should warn or fail with an error. The problem with that, as discussed, is that there's a perfectly reasonable thing to do instead: have -F switch on -a, and Aristotle Pagaltzis produced a patch to do just that.

The OP then suggested that -a should then imply -n, which is where it sat for 5 months. I added changes to improve the portability of the new test, to make -a imply -n and to avoid a warnings test from blocking on STDIN.

perl #116989 The story of this ticket starts in October last year when a space optimization was added for the Perl_croak_nocontext() call generated by the MEM_WRAP_CHECK() macro.

In February this year this change was found to be the cause of probe failures: compiler flag checks done by ccflags.SH in perl and library checks by Time-HiRes.

In March, S_croak_memory_wrap() was changed into an exported API function instead of a static function, with the hope that this would fix the problem (but it turned out it didn't.)

By the time we realized that, we were in the lead up to 5.18, and so while a real fix for the issue was added by Nicholas in April, the reinstatement of the original S_croak_memory_wrap() was left until 5.19.

This finally happened in d1decf2b.

perl #117265 This patch fixes a long standing bug in perl, where the string supplied to open() and other functions dealing with the filesystem would simply ignore any characters after an embedded NUL in a filename.

Reini Urban provided a patch to fix that, but the original behaviour was a bit extreme, croaking when warnings were enabled.

A second version of the patch didn't croak, but did treat the warning as a severe warning, which are always enabled. It also behaved incorrectly for functions taking multiple filenames, such as unlink(), ignoring any filename after the first name containing NUL but returning them as successful operations. eg. unlink "foo\0bar", "idontexist" would return 1 without attempting to remove "idontexist'.

I ended up largely rewriting the patch to correct those problems.

We have received a new grant application, under the Perl 5 Core Maintenance Fund, from David Mitchell.

Before we vote on this proposal we would like to get feedback and endorsements from the Perl community. Please leave feedback in the comments or send email with your comments to karen at perlfoundation.org.

Name: David Mitchell

Project Title: Maintaining the Perl 5 Core

Synopsis

Having recently finished a three-year TPF grant (plus extensions) devoted to fixing 'hard' core perl 5 bugs, I'd like to start a new grant that covers generally similar activity, but which increases the scope of the chargeable activities: from being solely "hard bugs", to also include more general maintenance activities, such as looking at performance issues, investigating smoke failures, helping getting perl into a releasable state etc. These are all activities I already do, but I have been unable to claim for them before.

Benefits to Perl 5 Core Maintenance

As a long-time core perl developer, this grant will enable me to continue to devote significant time and effort to maintaining the perl core, both in terms of fixing bugs, and in terms of more general maintenance.

Deliverable Elements

Like my previous Maintenance Fund grant, and like similar grants for Nicholas Clark and Tony Cook, there are intentionally no pre-defined deliverables for this project.

Instead, I intend to work for a stipend of 50 USD per hour.

As before, I will post a weekly summary on the perl5-porters mailing list which details how many hours were spent on each activity, allowing grant managers and active core developers (who will be aware of any recent commits and other activity of mine) to observe whether my claimed hours bear any relation to actual activity and results, and thus allow early flagging of any concerns.

Missing two weekly reports in a row without prior notice would be grounds for terminating the project.

Once per calendar month, I would claim an amount equal to $50 x hours worked. I would issue a report similar to the weekly ones, but summarizing the whole month. The report would need to be signed off by one of the project managers before I get paid. Note that this means I am paid entirely in arrears.

At the time of my final claim, I would also produce a report summarising the activity across the whole project period.

Also, (the "nuclear option"), the grant managers would be allowed, at any time, to inform the board that in their opinion the project is failing, and that the TPF board may then, after allowing me to present my side of things, to decide whether to terminate the project at that point (i.e. to not pay me for any hours worked after I was first informed that a manager had "raised the alarm").

I would anticipate that I will apply for for further extensions once the initial grant hours have been used up.

Project Details

I intend largely to continue the activities that I already do on core maintenance, which may include (this list is non-exhaustive):

  • Fixing bugs, both hard and easy.
  • Monitoring smokes for failures and compiler warnings, and where possible fixing issues - especially in the lead-up to a new release.
  • Improving performance where I see the opportunity.
  • Code refactoring, where the performance or maintenance benefits outweigh the cost of code churn.
  • Reading and replying to the perl5-porters mailing list.
  • Acting as release pumpking for a maint release where essential: for example I was responsible for the 5.14.4 security release. (But I'm not intending to become a regular maint pumpking.)

Project Schedule

I am available to start work on this project immediately.

I intend to devote 400 hours of my time over approximately the next five or six months: around 20 hours per week on average, although the individual weeks may vary wildly.

Bio

I'm a freelance UNIX sysadmin and programmer living in the UK. I have been using perl since 1993, and have been fixing core perl 5 bugs since 2001. I have had commit rights since 2003 and I was pumpking for the 5.10.1 and 5.14.4 perl releases. I am known for fixing "hard" things in the perl core.

Endorsed by: Jesse Vincent, Marcus Holland-Moritz

Amount Requested: 20,000 USD.

Suggestions for Grant Manager

Jesse Vincent and Marcus Holland-Moritz have kindly agreed to continue in this role.

About TPF

The Perl Foundation - supporting the Perl community since 2000. Find out more at www.perlfoundation.org.

Recent Comments

  • Nicholas Clark: Karl's choice of words are uncannily close to what I read more
  • Karl Williamson: I think Tony did an outstanding job, and I was read more
  • Ricardo Signes: Yes, please, with all possible speed. read more
  • Ron Savage: I'm with Craig. The depth of Tony's understanding is something read more
  • Craig Berry: Tony's patience and skill in executing the initial grant have read more
  • Ron Savage: Hi Tony Well done! That's a lot of valuable work read more
  • Karen Pauley: I don't think that the current Perl 5 Core Fund read more
  • diakopter: Karen, will there be a "Perl 6 core" fund? Or read more
  • Nicholas Clark: Dave's grant-funded work on the Perl core has been incredibly read more
  • Ricardo Signes: I am strongly in favor of this grant being granted! read more

About this Archive

This page is an archive of entries from September 2013 listed from newest to oldest.

August 2013 is the previous archive.

October 2013 is the next archive.

Find recent content on the main index or look in the archives to find all content.

OpenID accepted here Learn more about OpenID
Powered by Movable Type 4.38