Grant Proposal: Automated generation of DWIM Perl

Category: Grants

Comments (7)

Please support this. Dwim Perl is a very handy ‘everything included’ distribution of Perl it's useful to be able to point beginners at, or ask Windows sys-admins to install for supporting Perl applications.

It would be great to have this updated from v5.14.

And Gabor has a proven record in the Perl community of getting things done. The proposal seems well-thought-through, so it seems likely to succeed as planned.

This would be a boon to the Perl community; please support it.

I think DWIM perl is a great idea and I have great respect for Gabor's work to evangelize Perl in many forms.

However, the overall size of the grant request seems excessively large for the effort involved.

I was the original creator of Strawberry Perl, including its automated build tools. It was implemented over the course of a YAPC (based, admittedly, on the Vanilla Perl contest run by Alias previously).

As I expect satisfying Linux dependencies will be easier than satisfying Windows ones, I don't see why that part of the effort would be much larger. The other inchstones seem relatively straightforward.

My general impression is the same as David's. This is not a bad idea, but I don't think the money matches the necessary work, nor do I think it's the best spending of the limited community funds we have available.

I look at this grant and my first thought is that $6000 is equivalent to 120 hours of Tony Cook or Dave Mitchell. Much that I'd like to see this grant delivered, if funding this grant prevents Tony's open application from being renewed, then I think that TPF's prioritisation is wrong.

I have more specific concerns about the proposal.

$6000 is a lot of money. At the below-commercial rate that P5CMF has been paying grants, that equates to 120 hours. Is there really 3 weeks' full-time work needed for the deliverables here?

The work described is just to set things up for the first release. How is ongoing maintenance going to be handled? Experience both from core development and installing dependency trees from CPAN is that you can't assume that integrating a bunch of modules will work perfectly every time. The proposal acknowledges this too - there is always a chance that the current version of some of the modules on CPAN do not install cleanly. Upgrading one module can cause another's test suite to fail. Module upgrades may be intentionally incompatible - documented changes to an API. An end-user has the choice of whether to make such an upgrade, based on their individual use case. A distribution has to be one size fits all. The plan lists a lot of modules - the problems are likely to scale as O(n²) with the number of modules, not O(n). So this could become a big pain. If DWIM Perl can't kickstart without a grant, how will it avoid becoming "special biologist word" without further grants?

I see that Task::Kensho is listed. My understanding is that Task::Kensho lists "current best practice", and will drop modules from its list of recommendations if a better module emerges. This makes sense for it. But for DWIM Perl, what's the deprecation policy for removing modules? If someone builds their business codebase on DWIM Perl, can they upgrade safely to get bug fixes and other improvements, or will they run the risk of modules they were using being pulled from under them because Task::Kensho dropped them? Or is the plan for DWIM Perl to be a one-off? You say where people can develop on either a Windows or Linux based machine, and then easily deploy to a cheap VPS that has the same version of Perl with the same set of CPAN modules which would imply that there will only be one release of DWIM Perl ever. Otherwise, how would such a deployment work, as differing DWIM Perls would have different module versions.

You state A large part of Task::Kensho. Which modules do you think you will need to omit, and for what reasons? It's really not clear from the proposal, and it's hard to evaluate without more clarity on this.

Why all of Catalyst, Dancer and Mojolicious? (and hence all their dependency chains). This strikes me as a lot of work for little gain. You can't please everyone, so don't kill yourself trying. Surely the aim of this distribution is to be a reasonable out-of-the-box set of defaults for people who aren't aware of the choice on CPAN, so why the need to offer (and confuse) with choice here?

While Do a "market research" asking people for lists of prerequisites they have in their Perl-based applications. Both open source and closed-source applications. This will provide valuable input as to what needs to be included in DWIM Perl sounds laudable, I fear that it won't generate an actionable list. A lot of people will have a lot of different ideas, likely contradictory, and likely suggesting multiple modules that perform the same task. Also, is this item that costly to set up? It would have made a much better grant proposal if this had been run before applying, so that the project scope would be clearer.

You mention that several important CPAN modules require additional, non-perl packages. Locating and installing them can be additional issue and you specifically list DBD::mysql, DBD::Pg and XML::Parser. Are you proposing to bundle source for expat and for the MySQL and PostreqSQL client libraries? Or are you going to assume the same behaviour as core perl (and Python and I believe PHP) - if the C libraries are not present, the relevant extension is not built? Each of yes and no will lead to pain, but in different places.

By Linux do you mean any architecture that Linux builds on where Perl, expat and the databases run? Debian runs on more than 10 architectures, Fedora on at least 6, and perl will build on all of them and more. Or did you just mean x86 and x86_64?

The whole proposal strikes me as a highly ambitious "big bang" approach to trying to solve what is a legitimate problem. It would feel a lot less risky, and I'd be a lot more comfortable, with a proposal that started smaller and iterated based on feedback and experience. For example, start by taking just Task::Kenso and producing a distribution that builds on Win32, current CentOS and current Ubuntu, and see how that goes.

DWIM Perl looks like an interesting idea.

Whereas most grant proposals request funds for a project of fixed scope, keeping DWIM Perl up to date seems more like an ongoing operational matter.

DWIM Perl's existing Github repositories show that so far only Gabor has contributed to this interesting idea.

As time passes, DWIM Perl will need to keep up to date with new releases of Perl, CPAN modules and the supported operating systems.

For an ongoing operational task like this, I would expect to see more people involved so that work can continue if the maintainer becomes busy. Perl 5 itself and large CPAN modules all take such an approach.

I support the ideas behind DWIM Perl, but I consider funding its one developer for such a large amount at this stage a risky proposition. Having more contributors developing and releasing this project will significantly reduce that risk.

I understand people are worried about the size of the grant. After all previously these grants
were limited to 3,000 and only recently was the limit raised to $10,000. As I undersand
this was done in order to allow bigger projects without cutting them up in small pieces.
The only bigger grants were the ones given to Nicholas Clark, Tony Cook, and Dave Mitchell,
but as far as I know those come from a different budget.

Anyway, let me try to respond to some of the concerns raised here.

@David Golden, if I understand you correctly you imply that this whole things should be a few hours work or a few days at most. I understand that you could do this during a YAPC but I doubt I can do what I planned for DWIM Perl in such a short time, in a way that will be maintainable long-term.

@Nicholas Clark, if you have checked out the already existing releases of DWIM Perl, you probably saw that there were releases of both the Linux and Windows versions. They were partially automated, but still a large part was maual work.

Module upgrade is indeed not straight forward. That's why I need to make sure that the back-end I select (Pinto, Carton, etc.) can handle fixing module versions and easy upgrade and downgrade. This will still require manual work from me, but far less than it is today, that will allow me to handle the ongoing maintenance on a volunteer base in my free time.
My experience with the previous releases is that there are not too many cases when I can't just upgrade to the latest in most distributions.

I am not sure what do you mean by "If DWIM Perl can't kickstart without a grant, ...". First of all DWIM Perl exists, so it has kickstarted. Second, do you actually say that in order to get a grant I need to prove that I don't need a grant?

Task::Kensho the general guideline is to include everything that can be installed on the platform.

Backward compatibility and deprecation policy: There is no commitment to backward compatibility. If a module stops working it will be removed from DWIM Perl. Otherwise they can be kept in the distribution as including them does not incur a big cost. Modules might change in a backward incompatible way, but DWIM Perl is not committed to maintain backward compatibility.

OTOH the whole point of the grant is to build to a tool that companies can use to easily build their Perl distribution consisting the standard perl + a bunch of CPAN modules at specific versions including external dependencies. They can then decide to either go with DWIM Perl or build their own distribution.

Why all of Catalyst, Dancer and Mojolicious? - while reducing the need to choose can certainly help beginner, the primary objective of DWIM Perl is to eliminate the need to install. OTOH having only those 3 is already a big reduction in choices.

Market research: I am not sure how can the idea of what to include be contradictory? There are no mutually exclusive modules on CPAN, are there? Also, market research is not the same as asking for an order. In the end I'll have to make a decision what to include in each release. Oh and before someone misunderstands, DWIM Perl already hase a nice user-base. So there is no doubt that it already serves a market.

External dependencies: I am planning to include the already compiled part of the MySQL and PostreqSQL clients along with expat and other dependencies. Please check on how it is being done for some other libraries.

By Linux I just mean x86 and x86_64. At least for the grant.

@Tom Hukins, besides p5p, which other perl project has multiple developers?

As I understand Strawberry Perl always had one maintainer. DBIx::Class, Catalyst have minimal contributions outside the main developer. This seems to be how most of the Perl-related projects operate.
BTW The grant is requested to make it easy for anyone to build DWIM Perl and thus either to contribute or to take over.

TPF is going to pay any money only after the project has finished successfully. So the only risk is that TPF allocates the money for a few month. If after a few months the project has not succeeded, the money is freed again for other grants.

As far as I know TPF grants are not for ongoing maintenance, but for short and fixed projects.

What are the flags required for an Silent install for dwimperl-


Sign in to add comment