We have received the following grant application under the Perl 5 Core Maintenance Fund from Jess Robinson.
Before we vote on this proposal we would like to get feedback and endorsements from the Perl community. Please leave feedback in the comments or send email with your comments to karen at perlfoundation.org.
Name: Jess Robinson
Project Title: Improving Cross compilation of Perl 5
Perl 5 has several scripts and tools for cross-compiling it for other platforms. These are somewhat inconsistent and scattered across the codebase. My aim is to complete a cross-compilation target for Android, with smoker, and to prepare and document ways to add more targets without specialising for each of them.
The implementation goal is to allow direct cross compilation (no changes to actual Perl C-code, just using the hints system and configuration of cross-compiler paths and flags) to as many current and future platforms as possible, without needing knowledge of the build system internals. Ideally this will automate as much as possible the collection of data for the config files. Only platforms on which we can't run arbitrary compiled C code (eg Android) should get pre-defined config files in core, and if these become unmaintained (not updated when new features are added to Configure) they should be flagged as stale.
Benefits to Perl 5 Core Maintenance:
This work will remove duplicate build scripts and "specialised" code created for various cross compilation environments and either them replace with a generic version, or leave them out completely. Either way this will allow Core contributors to have a better overview of which specialised code needs to be maintained.
I propose to work this as a pay-as-you-go grant, similar to Nicholas and Dave. The initial request is for 200 hours at $50/hour, over 2 months. If the project proves to need less time than this, we may consider it done and end early, with the payment covering the actual number of hours worked. Alternately, if it requires more time I would be willing to work longer at the same rate to complete unfinished tasks.
The actual deliverables will be determined by the project, but will cover at least the delivery of a working Android cross-compilation build environment, together with a smoke environment to ensure that it continues to compile.
I will deliver weekly summary reports on progress via the p5p mailing list, and an end of month report summarising the whole month. I would then claim for the hours worked in that month.
The grant managers would be allowed at any time to inform the board that the project is failing, and the TPF board, after allowing me to present my side, decide to terminate the project.
I intend to spend time on the following items:
1. Bring my existing Android cross compiling environment and copy of Perl up to date and working. The current branch is based on perl 5.15.1, records work in progress, and uses an NDK two versions out of date. The branch needs rebasing and some restructuring to eliminate extraneous intermediate stages. The version of NDK used doesn't upgrade cleanly to newer ones. Fixing this will require trying to make the use of the NDK more generic and easier to configure in case of more path/binary renaming in future versions. approx 50 hours.
2. Set up a smoker for Android cross-compilation to ensure that the build continues to work. - 10 hours.
3. Refactor the working Android cross compiling approach to minimise divergence from the regular hosted build process, simplifying it as much as possible.
- fully exploit the fact that it now uses ./Configure -Dmksymlinks to separate the host and target build trees, which wasn't done previously.
- use lib/buildcustomize.pl to simplify the cases where the host miniperl driving the build needs to report configuration values for the target system.
- assume that a functional /usr/bin/perl is present when cross-compiling, and convert top level driver scripts from /bin/bash to Perl for easier maintenance and improved portability.
[Flexible guess. - 30 hours]
4. Identify a second target platform to cross compile to, representative of a small consumer device running Linux or *BSD on a non-x86 chip, and which can be run under emulation for testing. One plausible candidate is the Raspberry Pi, an ARM Linux device, because developer virtual machines exist, are documented and actively maintained. Get cross compilation to this device working, using the toolchain documented in Cross/README.new, and also using the approach documented in INSTALL. - Unknown.
Refactor the cross compilation code described in the previous steps to
- take advantage of the simplifications possible with -Dmksymlinks and buildcustomize.pl approaches used by the Android port
- minimise the amount of duplication and overlap with the regular hosted build process
- share with the Android port as much code as is practical
- maximise maintainability
- aim toward an approach that where possible is generic, minimising the use of pre-requisite "canned" config files.
5. Document this with a "Howto cross-compile Perl" guide.
6. Set up a smoker for cross-compilation to the small-unix target to ensure that the build continues to work. - 10 hours.
7. Review the various Cross compilation scripts and tools existing today in the core code, to determine which are still required using the revised approaches above. Tidy up by removing / deprecating un-used or obsolete parts.
1. Devise and implement a way to perform an installation of the resulting build onto the target smoke platforms (using an emulator), run the perl 5 core test suite, and gather the results to report back to the current smoke system. - Unknown
2. Investigate ways to cross-compile CPAN modules, especially XS modules, and install onto the target device. Likely this would be several steps, starting with modules that don't probe the local system within their Makefile.PL, and build them as part of the main cross build. Then work to decouple building modules from building the perl core, possibly including making the cross-building setup installable on the host machine. - Unknown, estimate 25 hours.
I estimate that the main items 1 to 7 will take around 200 hours to complete. I will work on each item in turn (though not necessarily in the above order) and in stages to provide usable patches during the project.
These items were prepared with much help from Nicholas Clark, many thanks to him.
I expect to be working on this alongside other work, and realistically to be spending about half my time on it. Thus I am asking for 200 hours over 2 months.
I am available to start work on this immediately.
I have worked with Perl professionally for the last 7 years, and I spend much of my spare time supporting and encouraging the use of Perl.
I took the Perl 5.10 changes and additions from the android-scripting repo and ported them to enable cross-compiling of Perl 5.15 for Android devices last year, this work can be seen on github: https://github.com/castaway/perl/tree/x-compile-android.
Endorsed by: Nicholas Clark, Ricardo Signes
Amount Requested: $10,000