Recently in Grants Category

Dave Mitchell writes:

Apart from a bit of time spent on a few miscellaneous bugs, I mainly continued working on refactoring re_intuit_start(). During the course of the month I merged two sets of commits back into blead. At this point I have now audited (and where appropriate fixed), the whole of the body of code for general correctness, and in particular for utf8 bugs and inefficiencies, and for correct treatment of anchors, especially \G.

In fact it was \G that originally set me off looking at this function. Originally the regex optimiser was skipped completely when a pattern contained \G. As part of my work last summer in refactoring how the "intelligence" of matching was distributed between pp_match(), regexec_flags() and re_intuit_start(), I changed it so that re_intuit_start() was called even in the presence of \G. This triggered a couple of bug reports, and in the course of fixing them, I realised I didn't really understand re_intuit_start(). The more I studied it, the less I understood it. It turned out to have lots of subtleties, most of which were undocumented. It used just two generic character pointers s and t, in scope throughout the 600 lines function, which were used to mean different things at different times. It had 32 gotos but no loop structures like while(). You get the general picture.

The code now has only(!) 20 gotos, and variables with more meaningful names, like 'rx_origin', and with a lot more code comments.

There's still a lot more I'd like to do with the code. In particular, there are a couple of significant optimisations that I'm aware of that are now possible, and still considerable scope for simplification; but I'm leaving any further work until after 5.20 is out.

Summary

32:45 RT#120692 Slow global pattern match with input from utf8
1:36 [perl #121230] process group kill on Win32 broken in 5.17.2
3:41 [perl #121362] Bleadperl recently introduced a new SEGV
0:32 [perl #121484] /m causing false negative
2:07 [perl #72028] execute permission missing from some utils
18:55 process p5p mailbox

59:36 Total (HH::MM)

4.4 weeks
59.6 total hours
13.5 average hours per week

As of 2014/03/31: since the beginning of the grant:

24.1 weeks
395.1 total hours
16.4 average hours per week

There are 5 hours left on the grant.

The Grants Committee is pleased to announce the voting results of the March round.

The following grant is approved and funded:

The following grant is approved but not funded as specified in 3.4 of the Rules of Operation. This will be reconsidered in the next round according to 1.2 of the rules.

The following grant applications were rejected:

Tom Hukins has been assigned as a Grant Manager for the funded grant.

We would like to express gratitude for those who took time to give feedback on these proposals.

As a reminder, we will again call for grant proposals on May 1st.

Jeffrey Ryan Thalhammer reported:

This past May, The Perl Foundation awarded a grant to fund development of a couple features in Pinto. Pinto is a robust tool for curating a private repository of CPAN modules, so you can build your application with the right modules every time. This is my fifth progress report on that work.

Pinto 0.0995 was just shipped to CPAN and it includes a merge command. At the moment, it is only capable of doing a fast-forward merge (that is, when the target stack hasn't changed since you last forked or merged it).

I still plan to support other types of merges, but I actually think fast-forward merging will cover the vast majority of scenarios. Most of the time, stacks are short-lived. You make a copy, upgrade some packages, and then build your app. If your tests fail, you probably throw the new stack away and stay with your current packages. If your tests pass, then you immediately merge the new stack back into the original.

This release also contains reset and revert commands that are similar (but not identical) to the same commands you find in git. All the new commands are experimental, so the interface and behavior are subject to change.

So I'm not calling the grant work "done" just yet. Once these new commands have been battle-tested a bit, then we'll see where things stand.

Dave Mitchell writes:

I spent about half my time last month continuing to work on fixing and refactoring the Perl_re_intuit_start() function, which is the main run-time optimisation facility in the regex engine.

My work so far was merged back into blead on 8th February; I've since done some more work which hasn't been pushed yet.

I also fixed a regression in maint-5.18 regarding whether a variable is seen in a regex that has a hash char class and an embedded newline, like

m{[#]
$foo}x

A side-effect of working on that led me to waste about 5 hours trying to get maint-5.18 to pass under clang/address-sanitiser. The failures appeared random, and bisecting showed that they were "fixed" in blead by a commit that should have made no difference; but cherry-picking that commit made the problem go away. It was also sensitive to re-compiles: touch, but don't change perl.h, and recompile, and the problem changed or went away. Eventually I decided it was probably a weird clang issue, and abandoned the effort.

The rest of my time was spent on a few miscellaneous issues, plus ploughing through my p5p mailbox.

Summary

5:10 RT #119125 /[#]/ and codeblocks in maint
35:14 RT#120692 Slow global pattern match with input from utf8
1:37 [perl #121230] process group kill on Win32 broken in 5.17.2
0:46 [perl #121336] valgrind errors in Perl_hv_iternext_flags
5:00 clang/asan weirdness on maint
1:00 fix locale.t spurious warning output
23:39 process p5p mailbox

72:26 Total (HH::MM)

4.0 weeks
72.4 total hours
18.1 average hours per week

As of 2014/02/28: since the beginning of the grant:

19.7 weeks
335.6 total hours
17.0 average hours per week

There are 64 hours left on the grant.

The Grants Committee got four grant proposals for this round. Before the Committee members vote, we would like to solicit feedback from the Perl community on these proposals.

Review the proposals below and please comment on each page. They are listed in the order in which they were received.

We have received the following grant application. Please leave feedback in the comments field by March 22nd, 2014.

Automated generation of DWIM Perl

  • Name:

    Gabor Szabo

  • Email:

    undisclosed

  • Amount Requested:

    $6,000

Synopsis

DWIM Perl is a "batteries included" Perl distribution for Microsoft Windows and for Linux. On Windows it is based on Strawberry Perl, on Linux it is compiled from the source released by the Perl 5 Porters. Both versions include a few hundred additional modules from CPAN.

The grant will enable me to further work on it, and specifically to finish automating the release of new versions.

Benefits to the Perl Community

DWIM Perl can make it much easier and faster for beginners to get started using Perl. It is useful both for people who would like to program in Perl, and for people who "just" want to run existing scripts or applications.

DWIM Perl makes it much easier and faster to set up a server running a Perl-based application. For example on a $5/month VPS. This reduces the advantage other languages might have in the area of deployment.

DWIM Perl can be the foundation of a "Perl Cloud", where people can develop on either a Windows or Linux based machine, and then easily deploy to a cheap VPS that has the same version of Perl with the same set of CPAN modules.

The grant will allow me to cerate a tool for automated releases and semi-automated upgrade of the parts. This will allow me (or anyone else) to easily create a new, up-to-date release of DWIM Perl.

It will also make it easy to create similar "batteries included" Perl distributions for the needs of companies, or open source projects using Perl.

Deliverables

  • A script that given a configuration file can build a DWIM Perl distribution for Linux and for Windows.
  • Configuration files ready to build DWIM Perl for Linux using 5.18.2, and for Windows using Strawberry 5.18.2.1.
  • A new release of DWIM Perl for Windows based on Strawberry Perl 5.18.2.1 using a scripted generation.
  • A new version of DWIM Perl for Linux based on 5.18.2
  • DWIM Perl for Linux based on 5.19.X.
  • Bug reports or even patches to modules that don't install cleanly.
  • All the source will be placed on GitHub.

The Windows version will be an installation process. The Linux version will be a tar.gz file to be unzipped.

Project Details

In order to get started using Perl with any reasonable modernity, people need to find and install hundreds of CPAN modules. This is an additional obstacle to getting started with Perl. Not only can it take hours to install all the required modules, there is always a chance that the current version of some of the modules on CPAN do not install cleanly. Several important CPAN modules require additional, non-perl packages. Locating and installing them can be additional issue.

Having a Perl distribution with "batteries included" means that newbies can download, install, and run. A lot less friction in getting started with Perl.

Each DWIM Perl distribution will include

The Windows version will also include wxPerl and Padre.

The Linux version will include Starman.

Inch-stones

On Linux, there is already a relatively good script, located at https://github.com/dwimperl/dwimperl-5-18-linux, but there are several modules that require external dependencies, that are not included.

In the Windows version, Strawberry already includes these "problematic" modules so the main thing that is missing is the automation.

For both versions the handling of the stable CPAN snapshot is missing.

  • Research how the CPAN snapshot should be handled: Options that should be checked are: Carton, Pinto
  • Do a "market research" asking people for lists of prerequisites they have in their Perl-based applications. Both open source and closed-source applications. This will provide valuable input as to what needs to be included in DWIM Perl.
  • Create a new release of both versions using a simple script, and with one module in the CPAN snapshot.

    The Linux script will be based on https://github.com/dwimperl/dwimperl-5-18-linux

  • Add tests that can be run post installation to verify the problematic modules work. (Mostly the ones that require external libraries.)

  • Write script to update the sh-bang lines of the perl scripts after relocation. (This might be submitted to p5p as well.)
  • Include DBD::mysql in Linux.
  • Include DBD::Pg in Linux.
  • Include POE in Linux.
  • Include PAR::Packer in Linux.
  • Include XML::Parser in Linux.

There will be several intermediate DWIM Perl releases, marking the finish of each inch-stone.

Project Schedule

I can start immediately after the grant is accepted on March 31, 2014. I expect the project to be done within 2-3 months.

Completeness Criteria

Assuming the evaluator has access to the right operating system (MS Windows or Linux) she will be able to run the build script and create a new version of DWIM Perl with minimal manual intervention.

Bio

I have created the DWIM Perl project in 2011, originally as a vehicle to distribute Padre, the Perl IDE on Windows. Later the Linux version was added in order make it easy to deploy Perl-based applications on Linux-based servers.

I have been using Perl since 1995 and teaching it since 2000. I am the original author of Padre, the Perl IDE. I am also the maintainer of a number of modules on CPAN https://metacpan.org/author/SZABGAB.

In addition, I run the Perl Weekly newsletter and the Perl Maven site.

We have received the following grant application. Please leave feedback in the comments field by March 22nd, 2014.

Perl::Lint - Yet Another Static Analyzer for Perl5

  • Name:

    Taiki Kawakami (@moznion)

  • Email:

    undisclosed

  • Amount Requested:

    $800

Synopsis

This project aims to create a fast and flexible static analyzer for Perl5 that has compatibility with Perl::Critic, with the goal to be light and fast enough to allow near real-time check of Perl source code while you code (e.g. hook it in your editor)

Why do I fork Perl::Critic?

Currently Perl::Critic uses PPI to analyze perl source code, which is relatively slow. The processing time can be greatly shortened by about 30 times the current value by replacing PPI with Compiler::Lexer and Compiler::Parser, which are new tools created by Masaaki Goshima who authored PerlMotion.

Using these modules are the main ingredients to the speed up, but in my previous attempts introduce these changes to Perl::Critic, it has proven difficult to do this without changing the current codebase significantly thereby causing significant trouble for the current user base. It would especially affect the already-existing plugins for Perl::Critic.

Rather than breaking existing functionality, I propose to build a new tool that is feature-wise compatible to Perl::Critic, but with much faster processing speed.

Benefits to the Perl Community

While Perl::Critic is an extremely useful tool, we have seen many colleagues avoiding its use because of its slowness. Improving the execution speed of Perl::Critic would lower the barrier to use in your day-to-day hacking activities, thus also leading to a better quality code, and improved development efficiency. With a fast enough Perl::Critic, one might even be able to embedded in an editor for seamless and continuous checking as you write.

Deliverables

The new module which is compatible with Perl::Critic. And this module is much faster than Perl::Critic because it uses Compiler::Lexer and Compiler::Parser instead of PPI.

Project Details

This project aims to create a MUCH faster version of Perl::Critic, using Compiler::Lexer and Compiler::Parser

Compiler::Lexer and Compiler::Parser are tokenizer and parser for Perl5 written in C++, which offers unparalleled speed compare to pure-perl competitors used in PPI. They are used in projects such as PerlMotion which transpiles Perl5 code to Objective-C.

A real life example replacing PPI with Compiler::Lexer can be found in Perl::MinimumVersion and Perl::MinimumVersion::Fast . The former is implemented using PPI, and the latter uses Compiler::Lexer. A simple benchmark shows that Perl::MinimumVersion::Fast is about 33 times faster than Perl::MinimumVersion (https://metacpan.org/pod/Perl::MinimumVersion::Fast#BENCHMARK). Applying this to Perl::Critic, one can expect at least an order of magnitude faster processing.

Therefore goal of this project is to create this much faster replacement while still keeping compatibility with Perl::Critic.

Inch-stones

  • Analyze functions of Perl::Critic (half to one day)

    Extract items that Perl::Critic currently checks

  • Implement tokenizer (two or three weeks)

    Parse source code written in perl and divide to tokens by Compiler::Lexer and Compiler::Parser.

  • Implement code analyzer (two or three weeks)

    Analyze and evaluate tokens by checking with inspection rules

  • Implement a system to define inspection rules (one or half week)

    Implement a system to define inspection rules on this phase, as well as a basic set of rules that are frequently used (mostly ported from original Perl::Critic rules)

Project Schedule

Period: within two months When can I start: May 2014

Completeness Criteria

Pass all of tests for Perl::Critic, benchmark against Perl::Critic, and release the module to CPAN.

Bio

I am a software engineer and graduate student (majoring in high-performance computing and software engineering) in Japan. I'm experienced in systems development for about five years, and I have worked with perl for about 2 years. I have contributed to a few open source projects (mostly involving perl) and have released a few CPAN modules. Please refer to the following web sites for details:

https://github.com/moznion https://metacpan.org/author/MOZNION

I am also a member of Compiler::Lexer development team, so I know the inner implementation of these modules which are crucial for this project. I believe I can apply those skills to this project.

ACKNOWLEDGEMENT

While the original proposal was written by Taiki Kawakami, Daisuke Maki was responsible for editing this proposal's English.

About TPF

The Perl Foundation - supporting the Perl community since 2000. Find out more at www.perlfoundation.org.

Recent Comments

  • Ron Savage: I support granting this extension. read more
  • Steffen Mueller: Dave's work has been consistently outstanding. He's working well below read more
  • mirod: +1 read more
  • Makoto Nozaki: There was a question why the fourth proposal got 9 read more
  • Makoto Nozaki: Will, As you may know, the Grants Committee was unable read more
  • Makoto Nozaki: Will, Grants Committee representative will get in touch with you read more
  • William N. Braswell, Jr.: Mr. Wright, Again I apologize for confusing your office, obviously read more
  • Dan Wright: Mr. Braswell, Your statements are blatantly false. On November 27th read more
  • William N. Braswell, Jr.: Mr. Wright, I thought you were the past TPF grants read more
  • William N. Braswell, Jr.: Mr. Timmermans, RPerl works fine on my free & open read more

About this Archive

This page is an archive of recent entries in the Grants category.

Conferences is the previous category.

GSoC is the next category.

Find recent content on the main index or look in the archives to find all content.

OpenID accepted here Learn more about OpenID
Powered by Movable Type 4.38