May 2012 Archives

The deadline for submitting proposals for hosting YAPC::NA 2013 is June 1st, which is only a few days away. The original post has more details.

Joel Berger wrote:

This month's work on Alien::Base started really exciting. I had tracked down several bugs and was honing in on full Linux compatibiliy. Turns out much of my testing problems had been in the test suite, wherein I mimiced make/Makefile with a perl script (in the name of Xplatform) but made some incorrect assumptions.

The problem came when I went to look at OS X. The problem I have found is that Mac bakes the full path to the library into the library itself during compiling/linking. It can be changed manually as long as the new path is shorter than the old one! Yes there are work-arounds but they would involve delving into the library's Makefile, which I have been ardently trying to avoid (opens a pandoras box). The other mechanism would be to target the library to install in the final place directly, but this breaks the usual workflow of installing a Perl module, in which perl Build.PL and ./Build do not do anything permenant, waiting until ./Build install to move the files to their final places.

This problem cropped up a few weeks ago and since then my work on Alien::Base has been residing in the grey-matter, hoping a solution would present itself. I don't want to say "forget Mac" because the whole point of Alien is to be as cross-platform as possible, still I keep coming back to thinking that perhaps an early release might have to forgo Mac initially.

My research has picked up, some of you may have seen my PerlGSL and related modules arrive this month; these modules are not purely a diversion from Alien::Base but rather they have been critial in some simulations I am developing. These and my thesis have taken some time from Alien::Base but I knew that (and disclosed it) going into the grant.

I am targeting a Linux release before next month's grant report as most of the API seems sufficient, though I am reluctant to release while the Mac issue is outstanding like this. Hopefully the Perl Foundation will be patient with me as I deal with these pressing matters both inside and outside of the Alien::Base project.

Original article at Joel Berger [blogs.perl.org].

The Grant Committee informs that there is a final result about the 2012Q2 Grant Proposals. The following two grants were voted for rejection:

  • Easy installation of non-CPAN dependencies by Alexandr Ciornii
  • Perl module for Linked Data by Tope Omitola

Also, the following grant was voted for funding:

  • Spanish Localization of the Perl Core Documentation by Enrique Nell and Joaquin Ferrero

David Golden reported:

Current progress

Three of the eight deliverables are complete and a fourth is substantially complete:

  1. Publish a Chef cookbook for Perl interpreter deployment
  2. Publish a Chef cookbook for CPAN module deployment
  3. Publish a Chef cookbook for Plack application deployment
  4. Improve the Pantry command line tool (80%)

Pantry

I released new versions of Pantry to CPAN, which add support for command-line configuration of Chef nodes and roles for use with Chef Solo.

For example, here is how one might use it to create node and role configuration data in a project directory. The following commands create properly structured JSON configuration data files for a "web" role (that has NTP and nginx deployed, with nginx as user 'nobody') and for the "www.example.com" server that fulfills that role:

$ mkdir my-project && cd my-project
$ pantry init
$ pantry create role web
$ pantry apply role -r ntp -r nginx
$ pantry apply role --default nginx.user=nobody
$ pantry create node www.example.com
$ pantry apply node www -R web
Assuming the 'ntp' and 'nginx' cookbooks are copied into the 'cookbooks' directory of the project folder (whether from the Opscode community site or from a git repository), the following commands would configure the server by copying all necessary files via ssh/rsync and invoking 'chef-solo' on the server:
$ ssh-add ~/path/to/ssh/key/id_dsa
$ pantry sync www

The server must already be bootstrapped with chef-solo, but this step will be described in the teaching deliverables. I may explore tools like Vagrant to help people get virtual images bootstrapped for the tutorials.

Pantry needs some additional work to support different deployment environments, but I've put that on the back burner until the teaching deliverables are underway.

Other CPAN modules

I've also been working on getting Metabase code in shape to be used for the real-world tutorial. This took a bit longer than I anticipated as some of it had not been touched in some time and evolution of dependencies (e.g. Moose, Catalyst, CHI) broke some things that needed fixing.

Between Pantry and the Metabase work, over the last month, I released 10 distributions directly or indirectly related to my work on this grant:

I also shaved various yaks in my build tools. For those who use similar tools, you may appreciate some of the improvements in these:

Current work and next steps

I've spent a larger-than-expected chunk of time updating the Metabase codebase, but I now have complete proofs-of-concept for the following:

  • Deploying MongoDB nodes via pantry/chef-solo
  • Metabase web server using the new MongoDB backend (tested locally only)

The next step will be deploying Metabase via pantry/chef and then testing the two-node system. After that is complete, I will start drafting the teaching deliverables for the grant.

This quarter, The Perl Foundation Grants Committee has three grant proposals. One of them was not funded in the last round, given to budget restrictions. Accordingly with GC rules it should be voted again in this round. Together with that, we have two new proposals. You can read their descriptions bellow, and you are invited to comment on their own posts. Your feedback is important for the Grants Committee.

The received grant proposals are:

Name:
    Enrique Nell - Language Lead
    Joaquin Ferrero - Tech Lead
Amount Requested:
    USD$2,000

Synopsis

Spanish is the third most commonly used language on the Internet after English and Mandarin. It is also the second most studied language and second language in international communication, after English, in the world. Currently, there are 400 million native speakers and Spanish is the official language in 21 countries. However, the number of contributions to CPAN from the Spanish-speaking community is much lower than expected, considering these figures.

Our goal is to translate the Perl core documentation into Spanish, in order to make it available to a wider public through the POD2::ES distribution. In this process, we are using and developing sustainable procedures that reuse previous translations and provide a quick update for each new Perl release.

We are requesting a grant to boost our work on POD2::ES.

Name:

Tope Omitola

Kjetil Kjernsmo will assist Tope with Perl and CPAN

Amount Requested:

How much is your project worth?

$500

Synopsis

The Semantic Web community has developed a vocabulary named VoID, which is a World Wide Web Consortium Interest Group Note, a de facto standard: http://www.w3.org/TR/void/. The goal of this project is to generate such descriptions, partly automatically, partly by hand-maintained descriptions, using RDF::Trine.

Name:

Alexandr Ciornii (CHORNY)

Amount Requested:

$2000

Synopsis

Many CPAN modules depend on non-CPAN libraries (or programs). Installation of such module requires that library be installed before installing CPAN module. For beginners it is more confusing as library may be installed, but its headers may not. I'd like to automate this step and install corresponding Linux package as part of module install.

Currently main module of this project has codename CPAN::non, it will be renamed later after discussion in module-authors mailing list.

I am pleased to announce that Paul Johnson's grant application, Improving Devel::Cover, has been successful. The grant will be split in two phases, the second phase starting after the completion of the first.

I would like to thank everyone who provided feedback on this grant, both on our blog and during the consultation phase.

We have received the following grant application under the Perl 5 Core Maintenance Fund from Jess Robinson.

Before we vote on this proposal we would like to get feedback and endorsements from the Perl community. Please leave feedback in the comments or send email with your comments to karen at perlfoundation.org.

Name: Jess Robinson

Project Title: Improving Cross compilation of Perl 5

Synopsis:

Perl 5 has several scripts and tools for cross-compiling it for other platforms. These are somewhat inconsistent and scattered across the codebase. My aim is to complete a cross-compilation target for Android, with smoker, and to prepare and document ways to add more targets without specialising for each of them.

The implementation goal is to allow direct cross compilation (no changes to actual Perl C-code, just using the hints system and configuration of cross-compiler paths and flags) to as many current and future platforms as possible, without needing knowledge of the build system internals. Ideally this will automate as much as possible the collection of data for the config files. Only platforms on which we can't run arbitrary compiled C code (eg Android) should get pre-defined config files in core, and if these become unmaintained (not updated when new features are added to Configure) they should be flagged as stale.

Benefits to Perl 5 Core Maintenance:

This work will remove duplicate build scripts and "specialised" code created for various cross compilation environments and either them replace with a generic version, or leave them out completely. Either way this will allow Core contributors to have a better overview of which specialised code needs to be maintained.

Deliverable Elements:

I propose to work this as a pay-as-you-go grant, similar to Nicholas and Dave. The initial request is for 200 hours at $50/hour, over 2 months. If the project proves to need less time than this, we may consider it done and end early, with the payment covering the actual number of hours worked. Alternately, if it requires more time I would be willing to work longer at the same rate to complete unfinished tasks.

The actual deliverables will be determined by the project, but will cover at least the delivery of a working Android cross-compilation build environment, together with a smoke environment to ensure that it continues to compile.

I will deliver weekly summary reports on progress via the p5p mailing list, and an end of month report summarising the whole month. I would then claim for the hours worked in that month.

The grant managers would be allowed at any time to inform the board that the project is failing, and the TPF board, after allowing me to present my side, decide to terminate the project.

Project Details:

I intend to spend time on the following items:

1. Bring my existing Android cross compiling environment and copy of Perl up to date and working. The current branch is based on perl 5.15.1, records work in progress, and uses an NDK two versions out of date. The branch needs rebasing and some restructuring to eliminate extraneous intermediate stages. The version of NDK used doesn't upgrade cleanly to newer ones. Fixing this will require trying to make the use of the NDK more generic and easier to configure in case of more path/binary renaming in future versions. approx 50 hours.

2. Set up a smoker for Android cross-compilation to ensure that the build continues to work. - 10 hours.

3. Refactor the working Android cross compiling approach to minimise divergence from the regular hosted build process, simplifying it as much as possible.

In particular:

  • fully exploit the fact that it now uses ./Configure -Dmksymlinks to separate the host and target build trees, which wasn't done previously.
  • use lib/buildcustomize.pl to simplify the cases where the host miniperl driving the build needs to report configuration values for the target system.
  • assume that a functional /usr/bin/perl is present when cross-compiling, and convert top level driver scripts from /bin/bash to Perl for easier maintenance and improved portability.


[Flexible guess. - 30 hours]

4. Identify a second target platform to cross compile to, representative of a small consumer device running Linux or *BSD on a non-x86 chip, and which can be run under emulation for testing. One plausible candidate is the Raspberry Pi, an ARM Linux device, because developer virtual machines exist, are documented and actively maintained. Get cross compilation to this device working, using the toolchain documented in Cross/README.new, and also using the approach documented in INSTALL. - Unknown.

Refactor the cross compilation code described in the previous steps to

  • take advantage of the simplifications possible with -Dmksymlinks and buildcustomize.pl approaches used by the Android port
  • minimise the amount of duplication and overlap with the regular hosted build process
  • share with the Android port as much code as is practical
  • maximise maintainability
  • aim toward an approach that where possible is generic, minimising the use of pre-requisite "canned" config files.

5. Document this with a "Howto cross-compile Perl" guide.

6. Set up a smoker for cross-compilation to the small-unix target to ensure that the build continues to work. - 10 hours.

7. Review the various Cross compilation scripts and tools existing today in the core code, to determine which are still required using the revised approaches above. Tidy up by removing / deprecating un-used or obsolete parts.

Stretch goals:

1. Devise and implement a way to perform an installation of the resulting build onto the target smoke platforms (using an emulator), run the perl 5 core test suite, and gather the results to report back to the current smoke system. - Unknown

2. Investigate ways to cross-compile CPAN modules, especially XS modules, and install onto the target device. Likely this would be several steps, starting with modules that don't probe the local system within their Makefile.PL, and build them as part of the main cross build. Then work to decouple building modules from building the perl core, possibly including making the cross-building setup installable on the host machine. - Unknown, estimate 25 hours.

I estimate that the main items 1 to 7 will take around 200 hours to complete. I will work on each item in turn (though not necessarily in the above order) and in stages to provide usable patches during the project.

These items were prepared with much help from Nicholas Clark, many thanks to him.

Project Schedule:

I expect to be working on this alongside other work, and realistically to be spending about half my time on it. Thus I am asking for 200 hours over 2 months.

I am available to start work on this immediately.

Bio.:

I have worked with Perl professionally for the last 7 years, and I spend much of my spare time supporting and encouraging the use of Perl.

I took the Perl 5.10 changes and additions from the android-scripting repo and ported them to enable cross-compiling of Perl 5.15 for Android devices last year, this work can be seen on github: https://github.com/castaway/perl/tree/x-compile-android.

Endorsed by: Nicholas Clark, Ricardo Signes

Amount Requested: $10,000

About TPF

The Perl Foundation - supporting the Perl community since 2000. Find out more at www.perlfoundation.org.

Recent Comments

  • idn: Ah crap, that should have been Jozef not Alberto.. read more
  • idn: Hi Alberto, Do you plan to import the existing RPMs read more
  • idn: This is an excellent plan getty! I'd vote for a read more
  • Neil Bowers: I think this is well worth funding (and more so read more
  • autarch.urth.org: The Perl community needs more well thought out and documented read more
  • autarch.urth.org: I'm all for improving ACT, and if the current maintainer read more
  • Jeffrey Ryan Thalhammer: This definitely gets my support. I'm thinking of putting on read more
  • Paul Seamons: While I like reading reviews, I'm not sure I'd pay read more
  • Jeffrey Ryan Thalhammer: tempire: It is probably impossible to do full justice to read more
  • Jeffrey Ryan Thalhammer: I'd love to see this proposal funded. Neil always writes read more

About this Archive

This page is an archive of entries from May 2012 listed from newest to oldest.

April 2012 is the previous archive.

June 2012 is the next archive.

Find recent content on the main index or look in the archives to find all content.

OpenID accepted here Learn more about OpenID
Powered by Movable Type 4.38