2012Q1 Grant Proposal: Easy installation of non-CPAN dependencies
Sat, 04-Feb-2012 by
Alberto Simões
edit post
<![CDATA[
<dl>
<dt><strong>Name:</strong></dt>
<dd>
<p>Alexandr Ciornii (CHORNY)</p>
</dd>
<dt><strong>Email:</strong></dt>
<dd>
<p>[hidden email]</p>
</dd>
<dt><strong>Amount Requested:</strong></dt>
<dd>
<p>$2000</p>
</dd>
</dl>
<p>
</p>
<h2>Synopsis</h2>
<p>Many CPAN modules depend on non-CPAN libraries (or programs). Installation of such module requires
that library be installed before installing CPAN module. For beginners it is more confusing
as library may be installed, but its headers may not. I'd like to automate this step and install
corresponding Linux package as part of module install.</p>
<p>Currently main module of this project has codename CPAN::non, it will be renamed later after discussion in
module-authors mailing list.</p>
<h2>Benefits to the Perl Community</h2>
<ul>
<li>
<p>For any user installing CPAN modules, as it will become much simpler.
It is especially important when installing many modules or in case of inexpirienced user.</p>
</li>
<li>
<p>Module authors, because module installation will become easier and they
would not need to periodically explain that users need to install non-CPAN library.</p>
</li>
<li>
<p>CPAN/OS porters, because dependencies will be listed more clearly,
in future it may be also automated.</p>
</li>
<li>
<p>Package tools like Shipwright or carton can use CPAN::non to install OS packages.</p>
</li>
</ul>
<p>
</p>
<h2>Deliverables (to CPAN)</h2>
<p>New version of Linux::Distribution. XML::Parser will automatically install expat
(of course if user agrees). Module for yum interaction.
CPAN::non and Module::Install plugin will be released.</p>
<p>
</p>
<h2>Project Details</h2>
<p>CPAN modules that want to get rid of this problem usually bundle libraries either
directly or as Alien:: CPAN distributions. This creates several problems:</p>
<dl>
<dt><strong>More work for authors and OS package maintainers</strong></dt>
<dt><strong>Duplication</strong></dt>
<dt><strong>Necessity to update library if serious problem is found</strong></dt>
<dt><strong>Necessity to replicate additional patches for every OS</strong></dt>
</dl>
<p>But it is easy to install such dependency with OS tools. CPAN module will have configuration file
that will specify build and runtime dependencies. CPAN::non will be bundled with module, or
installed using configure_requires in META.yml/json. In future OS vendors may include it into their
versions of Perl.</p>
<p>CPAN::non will detect version of Linux, check if dependency and its headers are installed,
find out which name has dependency and headers on this OS from configuration file, install it.</p>
<p>For yum there is no module CPAN, I plan to write module for it. Also there is no general module
that would be dist-neutral, so I will write it too.</p>
<p>Module::Install plugin will be convenient as Module::Install allows automating bundling of run-time
dependencies.</p>
<p>XML::Parser will be used as a first test for CPAN::non as I have comaint rights for it.</p>
<p>
</p>
<h2>Inch-stones</h2>
<ol>
<li><strong>Improve tests and release new version of Linux::Distribution.</strong>
</li>
<li><strong>Test Linux::APT. Write/submit patches if necessary.</strong>
</li>
<li><strong>Install CentOS and Fedora. Write a module for yum interaction</strong>
</li>
<li><strong>Write and test an OS-independent module for software package installation (apt and yum)</strong>
</li>
<li><strong>CPAN::non - prototype</strong>
</li>
<li><strong>CPAN::non - write code</strong>
</li>
<li><strong>CPAN::non - test</strong>
</li>
<li><strong>CPAN::non - documentation</strong>
</li>
<li><strong>Use it in XML::Parser for expat</strong>
</li>
<li><strong>Module::Install plugin</strong>
</li>
<li><strong>Blog about it, help CPAN authors use it, improve main module if necessary</strong>
</li>
</ol>
<p>
</p>
<h2>Project Schedule</h2>
<p>The project will take 2-3 months. I will start right after grant approval.</p>
<p>
</p>
<h2>Completeness Criteria</h2>
<ul>
<li>
<p>New versions of Linux::Distribution and XML::Parser.</p>
</li>
<li>
<p>CPAN::non, Module::Install plugin and module for yum released.</p>
</li>
</ul>
<p>
</p>
<h2>Future</h2>
<p>This project is a working solution, but after it will be adopted, more interesting
things may be implemented, like support for FreeBSD, even Windows, support in tools for
OS package generation, etc.</p>
<p>
</p>
<h2>Bio</h2>
<p>I maintain many CPAN modules and fixed bugs in many more. Participated in
such projects as Strawberry Perl, Padre, ExtUtils::MakeMaker and others.
Contributed to Module::Install.
My CPAN account: <a href="http://search.cpan.org/~chorny/">http://search.cpan.org/~chorny/</a> and Ohloh:
<a href="https://www.ohloh.net/accounts/chorny/positions">https://www.ohloh.net/accounts/chorny/positions</a>.</p>
<p>For 9 years I was a lead programmer of credit card payment system InterCardPro.
Currently work for kariera.md.</p>
]]>
Comments (8)
just having a perl interface to yum would be excellent. but it only has an api via python, no c based libraries (making Yum::XS possible) or anything else sensible. you may have to wrap the thing with expect... :(
why redhat doesnt just give up on rpms is beyond me...
I've been using Perl for years but always dreaded installing certain modules because of the problems with external libraries, and (to me) the frustrating error messages - dylib this and header that. I often just gave up and it is only now I realize how much I was missing. So I think a project like this is very good, especially if the needs of inexperienced users are taken into account (e.g. error messages that point to places where you might find help - and I don't mean man pages).
Hey, how this grant would relate to the other Alien:: grant that is also waiting approval?
Relationship between our projects is similar to relation between installing software from source and from pre-built packages. Both ways have advantages and disadvantages. Of course, installation from source with Alien::Base::ModuleBuild means that you will get newest version. Also you will be able to compile with other parameters that your vendor did. But complexity of maintaining Alien:: distribution (for ex. author needs to apply patches for every version of OS/distribution that does not compile out of box, similarly to OS vendors) means that most CPAN authors will not create them or will not supply patches. User will also be required to monitor dependency for vulnerabilities. And of course, it will require compiler/linker installed (some web hosting servers do not have gcc, but are willing to install CPAN modules) and all libraries that this library depends on (building Alien packages for them would be too complex), including headers. Most sysadmins prefer to have non-CPAN libraries to be installed from vendor packages, and in future CPAN::non will be able to simplify building vendor packages from CPAN modules.
IMHO, both ways are useful. As Joel Berger writes, Alien::Base::ModuleBuild could even use this project. Or vice versa :).
Most sysadmins prefer to have non-CPAN libraries to be installed from vendor packages, and in future CPAN::non will be able to simplify building vendor packages from CPAN modules.
This is what lead me to my core concept that all installations should target File::ShareDir; this makes library installations essentially indistinguishable from CPAN installations.
You are correct that complex libraries with deep dependencies chains or OS specific patches would be harder. It is for this reason that I will have a handler for downloading specifically pre-built binary packages (built for the specific Alien:: module).
May I ask, what will be the fallback if CPAN::non doesn't support the local package manager?
I'm going to say the same thing for all the grant requests this round ...
As a general rule, I think grants should be used to fund development on projects that are already established.
Speculative projects are just that, speculative. Even if the grant is completed, it's really hard to know whether the resulting work will be useful to anyone.
When funding work on an existing project, we already have some idea of whether or not that project has traction in the wider Perl community.
Most OS’es would be supported, support of yum and apt means 90-95% of Linux installations. In future I plan to add FreeBSD, possibly Mandriva/Mageia, SUSE, Solaris and Cygwin. For Windows I planned to use same method as you plan: installing prebuilt archives. If OS/distribution is unsupported, error message would be printed with general instructions how to find out which package to install. Or use Alien::Base::ModuleBuild to see if it will be able to install :).
What I would really love is a good interface to apt. I have tried to write a script to run immediately after installing a fresh Ubuntu. I finally gave up because I couldn't get useful information about what was installed, installable, which repositories were available, etc etc. That would be a great thing. Once that was running, I would be happy to incorporate it into Alien::Base or support its existence in some other installer mechanism.
