Grant Proposal: Inline::C(PP) Module Support

Mon, 15-Sep-2014 by Makoto Nozaki edit post

_We have received the following grant application "Inline::C(PP) Module Support". Please leave feedback in the comments field by September 25th, 2014._ # Inline::C(PP) Module Support - Name - Ingy dÃ¶t Net - David Oswald - Amount Requested USD $2,000 ## Synopsis Make Inline::C and Inline::CPP the best choice for writing "XS" modules. ## Benefits to the Perl Community In 2000, Inline.pm and Inline::C brought XS from "hard things possible" to "hard things simple". People with basic knowledge of Perl and C could combine the two, without having to learn the entirety of the black art of XS. Soon after, support was added for C++, Java and ~25 other languages. Inline was consider rather successful. The missing piece of all this is that Inline (C, C++) was never really polished to write CPAN modules; CPAN dists that would handle the compilation parts at install/build time, and then become nearly indistinguishable from plain XS modules after installation. A common pattern has been for module programmers to start with Inline::C and then use it to learn XS, so that they can release the modules to CPAN. Often Inline::C has been used to generate the XS which is then pasted into a module distribution with a few adaptations to fit the distribution's framework. Though very successful, Inline should go further to simplify and remove the need for authors to manually work through tedious and error-prone steps. In the summer of 2014, a number of core Inline developers came together to get Inline up to modern standards. We've decided the next big move is to polish up Inline so that it is ModuleReadyâ„¢; so that it supports module authorship and distribution. This will allow many more people to use C and C++ to make Perl modules, without ever needing to learn XS boilerplate, which is a significant barrier to entry. Those who already know XS will still be pleased to rediscover that Inline is an easy way to create XS, and that these enhancements make it a good choice as a basis for XS module distributions. ## Deliverables - Allow compilation of inlined C code to happen during a module's build time, and then allow Inline to get out of the way to a greater degree than currently happens. - Make `use Inline` not trigger a C compile or a C source check when running installed. - Provide support integration for: - Dist::Zilla - Module::Install - ExtUtils::MakeMaker - Module::Build - Make sure that the right things happen at test and build time (vs runtime). - Test that all works properly with C++. - Provide support for C modules where the C code is not 'inlined' - Like YAML::XS - Release a few current XS modules using Inline. - YAML::XS - String::Slice ## Project Details Inline development has resurged in #inline on irc.perl.org. The primary maintainers of Inline, Inline::C and Inline::CPP (C++) are the ones who want to make this happen. These three modules above have already undergone considerable refactoring, modernization and bug fixing in the past 3 months, with many releases to show for it. The goals for this grant are not conceptually hard, but require some focus. This grant will give us the time to focus. There is no reason to expect this work will take more than 2 months. ## Inch-stones 1. Identify existing (or create new) modules that use different 'XS' strategies. These will be the driving test cases. They should include: - Inlined C - External C - Currently using XS - Trying to ship with Inline - C++ 2. Adjust Inline to know about development vs build vs installed runtimes. There are a few strategies. Likely there will be a dependency on a small module, say Inline::Module, that knows how to DTRTs, at the right times. 3. Facilitate making Inline and Inline::C/CPP authoring and build tools for distributions, not just runtime tools. 4. Make sure that Inline::C and Inline::CPP work right. In particular, upgrade the grammar tests and parsing to handle many C and C++ constructs that have been found to be lacking in current parsing solutions. 5. Release real modules to CPAN that exercise each of the new development strategies. Watch for results on cpantesters. 6. Write automation helpers: - Dist::Zilla plugin - Module::Install plugin - Documentation for ExtUtils::MakeMaker and Module::Build 7. Write documention for how to be an "XS" author without learning XS boilerplate. Likely this will include a tutorial. ## Project Schedule David and Ingy see no reason this can't be done in two months. One month for code and modules. One month for documentation and testing. We hope to work on other grants, each about 2 months, so there is good reason to get this one done, done right, and out of the way. ## Completeness Criteria Release of these modules to CPAN (with above support): - [Inline](https://metacpan.org/pod/Inline) - [Inline::C](https://metacpan.org/pod/Inline::C) - [Inline::CPP](https://metacpan.org/pod/Inline::CPP) - [YAML::XS](https://metacpan.org/pod/YAML::XS) - [Math::Fast::PrimeSieve](https://metacpan.org/pod/Math::Fast::PrimeSieve) ## Bio Ingy dÃ¶t Net is the original creator of Inline and Inline::C. His primary focus in Perl is to bring the "hard" things into the hands of beginners. David Oswald has been maintaining Inline::CPP for four years, and is the only person to have made releases on Inline::CPP since 2003. Ingy and David work well together and have decided to collaborate on a number of big projects that benefit Perl and Software Development. Inline was the obvious first choice.

Category: Grants

Comments (11)

[ 1408 ] | Mon, 15-Sep-2014 by curtisjewell

Thank you for proposing this one!

I'd like to see this one first, with "IO::All Redux" being my personal #2, although others will have different priorities.

[ 1409 ] | Mon, 15-Sep-2014 by perlpilot

I really have no other comment than: +1

[ 1410 ] | Mon, 15-Sep-2014 by anonymous

I think this one is quite important. We're planning to release a CPAN module which uses Inline::C and I don't want any nasty surprises.

[ 1411 ] | Mon, 15-Sep-2014 by hjansen

I think this is the most interesting of the proposals in the September round: +1

[ 1412 ] | Mon, 15-Sep-2014 by markfowler

I think this is a worthwhile project.

I would like to raise "inchstone 7" to a higher prominence and would encourage this grant to be adjusted so that inchstone 1-6 include documentation deliverables for each stage marking the "completion" of these steps.

[ 1413 ] | Tue, 16-Sep-2014 by reiniurban

+1
They already made big progress, and hopefully they are making even more. Now even a full Inline:CPP with all the new c++11 std and boost enhancements seems to be doable.

[ 1414 ] | Tue, 16-Sep-2014 by ingy

Thanks for the feedback!

We also think this proposal is an obvious win. It is bid the lowest because we believe it is the closest to getting done.

We are planning to do weekly blog posts with all of the proposals that are granted. The relevant texts of those posts will certainly be rolled into documentation.

â€” Ingy and David

[ 1415 ] | Sun, 21-Sep-2014 by makoto

[ 1416 ] | Sun, 21-Sep-2014 by ingy

Makoto,

Thanks for bringing this up. I think it's in everyone's best interest to talk about it.

In 2006 I was encouraged by a TPF member to submit a grant proposal. I was a bit reluctant to accept money for personal work, but I wanted to help out; to do good things for the community. So I went for it.

The proposal was to port PyYaml to YAML::Perl. PyYaml (by the author of libyaml) was the best YAML framework at the time.

IIRC, the proposal was $3000, with $1500 granted up front. I started down the path and a lot of code was ported. At some point during this I realized that it was critical to bind libyaml (which was brand new) to Perl and I made YAML::XS, which is by far the most solid YAML module in Perl.

After more time I talked to the TPF and we agreed to not go further on the grant. YAML::Perl was working for basics but not fully done. YAML::XS was working well. We decided that it was time to scrap the grant. I was not awarded the remaining $1500 (though it was a close vote) and I respected that decision. While the grant did not go as proposed, I felt that Perl came out well ahead for it.

This time around the idea to work on a grant is completely mine (and David's). I chose to work with someone else specifically to make sure things went on track. I like working with David a lot and he is very good at getting things done.

Also we have made several proposals with many more in the wings, and so it is in our best interest to make sure we get things done on time and done well. There is no need for the TPF to pay us money up front.

Finally in regards to this proposal, it is really the most straightforward of the 4. That's why it has the smallest pricetag. Yet the implications of lowering the bar to authoring XS modules seems like a big return on investment.

I hope that the TPF and the community is willing to give me another chance. I will work my hardestâ€¦ not only to not let you down, but to give you something truly great.

â€” Ingy dÃ¶t Net

[ 1417 ] | Thu, 02-Oct-2014 by leontimmermans

This sounds sensible. In fact I'd argue Inline isn't really usable without this.

[ 1418 ] | Mon, 06-Oct-2014 by wbraswell

Congrats ingy & davido!

I put quite a bit of effort into Inline::CPP to create a fix (with #inline's help) for the "Infamous Namespace Hack", part 4 of which finally culminated in Inline::CPP v0.63, which is currently the most recent code on Github and CPAN.

https://github.com/daoswald/Inline-CPP/commit/8ddcf88830ec83340b23531214ae69f678c9f349

I think it is important to point out, for the benefit of the community, that even in RPerl's as-yet-unfinished state, the RPerl test suite is (I believe) the most comprehensive test of Inline::C(PP) of which I am aware, other than Inline::C(PP) themselves.

https://github.com/wbraswell/rperl/tree/master/t

You will note I began by implementing the Inline::C(PP) tests from the already-existing POD, then continued on with over one thousand other tests, all of which focus on pushing the boundaries of Inline::C(PP).

The first 1 or 2 deliverables of this grant sound like they may make use of ingy's Module::Compile, which I also plan to use myself for RPerl both independently of, and related to Inline::C(PP), so I'm especially excited about that!

https://metacpan.org/pod/Module::Compile

Thanks TPF for granting the money, thanks #inline, and thanks again to ingy and davido and even our old friend nwatkiss for Inline::C(PP), without which RPerl might not exist at all. :-)

Grant Proposal: Inline::C(PP) Module Support

Comments (11)

Sign in to add comment

About TPF

Categories

Popular Tags

Recent Tags

Get Perl

Links