May 2013 Archives

It took some time to get the grant results. In fact, grantees are aware of the status of their grants for about a week, but we were dealing with some internal details before posting the results.

In this round the Grant Committee did not vote for rejection of any grant. That is good, but the committee does not have funds to accept all grants at once. This lead to a ranking of the proposals, and the two most voted will be funded:

  • Next Release of Pinto With Key Features by Jeffrey Ryan Thalhammer
  • YACT - Yet Another Conference Tool by Torsten Raudssus (Getty)

The other two grants will be kept for discussion in our next grant round, as stated in the Committee rules of operation.

We have received the following grant application, under the Perl 5 Core Maintenance Fund, from Tony Cook.

Before we vote on this proposal we would like to get feedback and endorsements from the Perl community. Please leave feedback in the comments or send email with your comments to karen at perlfoundation.org.

Project Title: Maintaining Perl 5

Name: Tony Cook

Synopsis

Free up one of the Perl 5 core's contributors to work non-stop on making Perl 5 better.

Benefits to Perl 5 Core Maintenance

This grant provides the pumpking with a development resource to target as he or she wills, while still providing for more general bug fixes and other improvements to the perl core.

Deliverables

I propose to adopt the same model as Dave and Nicholas's successful ongoing grants.

Like their grants, there are intentionally no pre-defined deliverables for this project. I intend to devote around 260 hours (about 20 hours a week) over the next 3 months to work on improving the core, paid by the hour at the same below-commercial rate as Dave and Nicholas. Some weeks I may be able to more than 20 hours, if acceptable this will consume more hours and end the grant earlier.

Like them, I would post a weekly summary on the p5p mailing list detailing activity for the week, allowing the grant managers and active core developers to verify that the claimed hours tally with actual activity, and thus allow early flagging of any concerns. Likewise, missing two weekly reports in a row without prior notice would be grounds for one of my grant managers to terminate this grant.

Exactly as Nicholas and Dave do, once per calendar month I would claim an amount equal to $50 x hours worked. I would issue a report similar to the weekly ones, but summarising the whole month. The report would need to be signed off by one of the grant managers before I get paid. Note that this means I am paid entirely in arrears.

At the time of my final claim, I would also produce a report summarising the activity across the whole project period.

Also, (the "nuclear option"), the grant managers would be allowed, at any time, to inform the board that in their opinion the project is failing, and that the TPF board may then, after allowing me to present my side of things, to decide whether to terminate the project at that point (i.e. to not pay me for any hours worked after I was first informed that a manager had "raised the alarm").

I view this grant as a proof of concept - if it goes well for everyone involved, I expect to apply to extend it.

Project Details

I think that the work that I would do to improve Perl 5 would mostly fall into one of four main classes: code reviews, bug fixing, helping other contributors, and adding features - with bug fixes the most prominent and adding features the least.

In one major sense what I'm offering is different to Nicholas's or Dave's grant: the current pumpking would be able to assign tasks directly.

Ideally this would be done with some consultation with myself, so a large complex task involving parts of the core I'm unfamiliar with isn't assigned (or is assigned with reasonable expectations on time). Of course, if too many tasks are negotiated into non-existence, the grant can be terminated.

Some possible tasks, based on discussions over the last several months:

  • readpipe(@list)
  • core exception objects
  • a git hook to prevent changes in the left of a merge

Success metric: completion of tasks assigned.

Otherwise I'd work on:

  • Reviews of patches submitted to perlbug, possibly committing them

This will improve my core knowledge, and provide more timely feedback to non-committers using their time to help perl.

Metric: number and complexity of patches applied or commented on.

  • Fixing bugs I select from the perl5 Request Tracker queue.

While I wouldn't necessarily be working on the the harder bugs that Dave targets, this would help bring the total bug count down, and reduce the noise in the Request Tracker queue.

Metric: number and complexity of issues fixed.

  • Fixing systemic issues in perl, such as the mis-use of I32 and U32 in the perl core.

Metric: complexity of issue solved.

  • Contributing to discussion on the perl5-porters mailing list.

For the grant, I'm specifically not proposing to:

  1. Be a release manager. This doesn't prevent me volunteering to act as a release manager, but that wouldn't be counted towards this grant.
  2. Act as language designer - I don't feel that I'm good at this.

Project Schedule

I expect that I can deliver 260 hours of work in approximately 3 months.

I am available to start work on this project immediately.

Bio

I'm a freelance programmer living in Australia. I've been irregularly contributing to perl since 2008 and a committer since 2010. My contributions have varied from build system fixes, to UTF-8 handling, to portability fixes. I've been programming in C for 25 years and in perl for 20.

Endorsed By

Ricardo Signes, Nicholas Clark, H.Merijn Brand

Amount Requested: $13,000

Suggestions for Grant Manager

Ricardo Signes, Marcus Holland-Moritz

Dave Mitchell writes:

This month I worked on three 5.18 blocker tickets; all three being regressions related to my jumbo re_eval fix back in 5.17.1.

The first, which I continued working on from last month, was the "Regexp::Grammars" bug.

Basically, my reworking of the /(?{})/ implementation assumed that a constant string segment like "foo" in /foo..../ would indeed be constant; but in the presence of

    use overload::constant qr => sub { bless [], ... }

the "constant" can be anything but, including an overloaded or REGEXP object. So the concatenation of the pattern's string segments didn't handle all the extra stuff like doing overloading properly or extracting out pre-compiled code blocks from qr// objects.

This is now fixed.

The second issue concerned handling arrays embedded within literal regexes, e.g. /...@a.../. This was partially to fix a regression from 5.16.x where, if @a contained a qr/...( {...}).../, then suddenly you'd need a 'use re eval' where you didn't need one before: RT #115004. But it also enhances the behaviour of array interpolation relative to 5.16.x too, especially relating to closures and overloading.

Basically, the traditional behaviour of run-time patterns such as /a${b}c/ was to concatenate the pattern components together, then pass it to the regex engine. My 5.17.1 jumbo re_eval fix changed that so that the list of args was preserved and passed as-is to the regex engine. This meant that the engine could do things like extract out existing optrees from code blocks in something like $b = qr/...(?{...}).../, rather than having to recompile them. So closures work properly.

The thing I missed back then was applying the same new handling to arrays as well as scalars. Until my fix, /a@{b}c/ would be parsed as

    regcomp('a', join($", @b), 'c')

This meant that the array was flattened and its contents stringified before hitting the regex engine.

I've now changed it so that it is parsed as

    regcomp('a', @b, 'c')

(but where the array isn't flattened, but rather just the AV itself is pushed onto the stack, c.f. push @b, ....).

As well as handling closures properly, it also means that 'qr' overloading is now handled with interpolated arrays as well as with scalars:

    use overload 'qr' => sub { return  qr/a/ };
    my $o = bless [];
    my @a = ($o);
    "a" =~ /^$o$/; # always worked
    "a" =~ /^@a$/; # now works too

As well as the new handling of arrays, the pattern concatenation code within Perl_re_op_compile was heavily reworked, resulting in fixing a utf8 edge case, and generally simplifying the code, including enabling the removal of a clunky if (0) { label: ... } bit of code.

This issue is now fully fixed.

The third issue concerned how caller() and SUB work within regex code blocks. It turns out that since my re_eval jumbo fix, code blocks in literal matches were displaying an extraneous extra stack frame. This code:

    #!/usr/bin/perl
    use Carp;
    sub f3 { croak() }
    sub f2 { "a" =~ /a(?{f3(3)})/ }
    sub f1 { f2(2) }
    f1(1);

gives the following results:

5.16.3:
        main::f3(3) called at (re_eval 1) line 1
        main::f2(2) called at /home/davem/tmp/p line 6
        main::f1(1) called at /home/davem/tmp/p line 7
5.17.10:
        main::f3(3) called at /home/davem/tmp/p line 5
        main::f2 called at /home/davem/tmp/p line 5
        main::f2(2) called at /home/davem/tmp/p line 6
        main::f1(1) called at /home/davem/tmp/p line 7
blead:
        main::f3(3) called at /home/davem/tmp/p line 5
        main::f2(2) called at /home/davem/tmp/p line 6
        main::f1(1) called at /home/davem/tmp/p line 7

In addition, the SUB token, which returns a reference to the current subroutine, was returning a ref to the hidden anonymous sub which is now used to implement closure behaviour correctly for code blocks within qr//'s; that is,

    $r = qr/foo(?{...})bar/;

is supposed to behave like

    $r = sub { /foo/  && do {...} && /bar/ }

as far as closures are concerned. The trouble is, the anon sub was never designed to be called directly, and in fact perl SEGVs if you do attempt to call it. The workaround for this is to skip regex calls on the context stack when looking for the CV for SUB; this has the effect of SUB always returning the sub which executed the pattern match, regardless of what direct code blocks (/(?{})/), or indirect code blocks ( $r = qr/(?{})/; /a$r/ ) have been called. I have documented this as subject to change for now.

Over the last month I have averaged 8.8 hours per week

As of 2013/04/30: since the beginning of the grant:

164.5 weeks
1656.6 total hours
10.1 average hours per week

There are 43 hours left on the grant.

Report for period 2013/04/01 to 2013/04/30 inclusive

Summary

Effort (HH::MM):

3:08 diagnosing bugs
35:27 fixing bugs
0:00 reviewing other people's bug fixes
0:00 reviewing ticket histories
0:00 review the ticket queue (triage)
-----
38:35 Total

Numbers of tickets closed:

3 tickets closed that have been worked on
0 tickets closed related to bugs that have been fixed
0 tickets closed that were reviewed but not worked on (triage)
-----
3 Total

Short Detail

7:35 [perl #113928] caller behaving unexpectedly in re-evals
19:33 [perl #115004] perl 5.17.x can't use @var in regexp, but only $var
11:27 [perl #116823] Regexp::Grammars broken since 5.17.1

Joel Berger wrote:

Alien::Base Final Report

Summary

With this report I end my grant for Alien::Base. I consider it to be a reasonable success and have hope that the project will continue further. It became, as perhaps I should have expected, a larger project than anticipated; the problems were rarely the anticipated ones.

In the end Alien::Base faced two major problems: (program/script) compile-time linking of the library and the localization of the built binary library on Mach (read Mac) platforms. The the linking problem is solved, running use Alien::MyLibrary; in the dependent module dynamically loads the module as expected.

The Mach problem should be solved, though part of the problem is that the solution must of necessity be different for proper users and CPANtesters. Mach binaries must hard-code the path to themselves in the binary, this requires fore-knowledge of the final install location of the library. Steps have been taken to ensure that the install location is set correctly for common (perl Build.PL && ./Build && ./Build install) use. CPANtesters unfortunately do not run the install phase, but rather append the blib directory into @INC. I hope that I have fixed this problem, however I have waited months and yet have seen no Mac reports on either of my test modules Acme::Alien::DontPanic and Acme::Ford::Prefect; until these reports come in I cannot assess the ability of my recent work to compensate for this so-called "blib-scheme".

I am happy to report that I have received much interest in the project and even some patches to the code-base. I am confident that the project will continue to progress; for my part I will continue to contribute and shepherd patches/pull-requests. Certainly the system is still in a rather basic form, for example HTTP support is still rather fragile, but the major architecture exists and will continue to improve.

Deliverables

  • Alien::Base is released to CPAN and is even being used by several early-adopter projects.
  • The documentation and tests are included
  • Alien::Base::ModuleBuild can fetch, build and install libraries. By design they are localized to the current Perl installation
  • The system is perlbrew/local::lib compatible and as a result does not require root privileges
  • FTP and HTTP are supported, though both could use some polish, bundled binaries are also supported
  • Library metadata is extracted from pkgconfig data or may be generated if needed
  • Alien::GSL does NOT yet depend on Alien::Base, though a branch on its github development site is ready to be uploaded to PAUSE once I deem Alien::Base fully stable.

Thanks

I would especially like to that my ever-patient project coordinator Makoto Nozaki for his help coordinating the project, and for far longer than expected. As the project progressed beyond its expected life, my Ph.D. Thesis and Defense took more and more of my time and so I want to thank both the Perl Foundation and the community at-large for being patient with me as well.

I also want to thank all of those people who have in any way encouraged or contributed to Alien::Base:

  • Christian Walde (Mithaldu) for productive conversations about component interoperability
  • kmx for writing Alien::Tidyp from which I drew many of my initial ideas
  • David Mertens (run4flat) for productive conversations about implementation
  • Mark Nunberg (mordy, mnunberg) for graciously teaching me about rpath and dynamic loading
  • Torsten Raudssus (Getty) for productive conversations about usability
  • giatorta, alexrj, jtpalmer, rsimoes, preaction for patches/pull-requests
  • Any others I have forgotten

Closing

This process has been taxing at times, but still rewarding. I hope that Alien::Base will be a useful part of the CPAN toolchain in the future as the Perl renaissance progresses!

Original article at Joel Berger [blogs.perl.org].

The Grants Committee will evaluate this report and make decision on the grants payment. Feel free to give us your inputs at the comments field.

For this quarter, TPF Grants Committee have four different proposals. Who invite the Perl Community to comment on the proposals and their relevance to the community. Please comment on each grant on their specific page.

Name:

Torsten Raudssus (Getty)

Amount Requested:

$3000

Synopsis

The current Act software and their instances are an often discussed topic in the world of Perl. The migrating of those instances, and the move forward to more modern Perl solutions in the system are often discussed. Last year we were able to address and start a concept at the Quack and Hack Europe 2012, we called it YACT - Yet Another Conference Toolkit, which targets to be a "in place" replacement for the existing Act software. It accesses the same database, and (should) be usable with the same templates. It also implements the idea of using git (and so GitHub) to maintain the Act setup for a conference.

YACT will not replace the Act name, Act will stay as the name for the complete project/platform/service, YACT is just a specific implementation state.

Additionally YACT is supposed to be also a management for Perl Monger groups and so should animate people to startup a Perl Monger group "instantly" and get all the nice shiny fuzz to manage it. This also means that we need a more flexible theme based style for this. A better support for localization is also a very important point here.

The current state is a very good database setup already, but the web interface setup is not really started and no administrative tools are added yet. There is still lots of work todo. (Current state: "/github.com/perl-act/yact" in https:)

Name:

Jozef Kutej

Amount Requested:

2000 €

Synopsis

Create similar page to http://deb.perl.it/ for RPM world.

Name:

Neil Bowers

Amount Requested:

$1500

Synopsis

A review of the main modern web frameworks for Perl, somewhat in the style of the other reviews I've done: http://neilb.org/reviews/

Name:

Jeffrey Ryan Thalhammer

Amount Requested:

$3000.00

Synopsis

Pinto is a turnkey solution for constructing and managing local CPAN-like repositories of Perl modules. This purpose of this grant proposal is to obtain funding for development of the next release of Pinto, which will include specific features described below.

The Pinto project is less than 2 years old, but it has already gained a modest user base and is potentially relevant to a wide audience within the Perl community (i.e. anyone that uses CPAN module). Funding development on Pinto will help the Perl community by providing them with better tools for managing their CPAN modules.

About TPF

The Perl Foundation - supporting the Perl community since 2000. Find out more at www.perlfoundation.org.

Recent Comments

  • Nicholas Clark: Karl's choice of words are uncannily close to what I read more
  • Karl Williamson: I think Tony did an outstanding job, and I was read more
  • Ricardo Signes: Yes, please, with all possible speed. read more
  • Ron Savage: I'm with Craig. The depth of Tony's understanding is something read more
  • Craig Berry: Tony's patience and skill in executing the initial grant have read more
  • Ron Savage: Hi Tony Well done! That's a lot of valuable work read more
  • Karen Pauley: I don't think that the current Perl 5 Core Fund read more
  • diakopter: Karen, will there be a "Perl 6 core" fund? Or read more
  • Nicholas Clark: Dave's grant-funded work on the Perl core has been incredibly read more
  • Ricardo Signes: I am strongly in favor of this grant being granted! read more

About this Archive

This page is an archive of entries from May 2013 listed from newest to oldest.

April 2013 is the previous archive.

June 2013 is the next archive.

Find recent content on the main index or look in the archives to find all content.

OpenID accepted here Learn more about OpenID
Powered by Movable Type 4.38