2012Q1 Grant Proposal: Easy installation of non-CPAN dependencies

8 Comments
Name:

Alexandr Ciornii (CHORNY)

Email:

[hidden email]

Amount Requested:

$2000

Synopsis

Many CPAN modules depend on non-CPAN libraries (or programs). Installation of such module requires that library be installed before installing CPAN module. For beginners it is more confusing as library may be installed, but its headers may not. I'd like to automate this step and install corresponding Linux package as part of module install.

Currently main module of this project has codename CPAN::non, it will be renamed later after discussion in module-authors mailing list.

Benefits to the Perl Community

  • For any user installing CPAN modules, as it will become much simpler. It is especially important when installing many modules or in case of inexpirienced user.

  • Module authors, because module installation will become easier and they would not need to periodically explain that users need to install non-CPAN library.

  • CPAN/OS porters, because dependencies will be listed more clearly, in future it may be also automated.

  • Package tools like Shipwright or carton can use CPAN::non to install OS packages.

Deliverables (to CPAN)

New version of Linux::Distribution. XML::Parser will automatically install expat (of course if user agrees). Module for yum interaction. CPAN::non and Module::Install plugin will be released.

Project Details

CPAN modules that want to get rid of this problem usually bundle libraries either directly or as Alien:: CPAN distributions. This creates several problems:

More work for authors and OS package maintainers
Duplication
Necessity to update library if serious problem is found
Necessity to replicate additional patches for every OS

But it is easy to install such dependency with OS tools. CPAN module will have configuration file that will specify build and runtime dependencies. CPAN::non will be bundled with module, or installed using configure_requires in META.yml/json. In future OS vendors may include it into their versions of Perl.

CPAN::non will detect version of Linux, check if dependency and its headers are installed, find out which name has dependency and headers on this OS from configuration file, install it.

For yum there is no module CPAN, I plan to write module for it. Also there is no general module that would be dist-neutral, so I will write it too.

Module::Install plugin will be convenient as Module::Install allows automating bundling of run-time dependencies.

XML::Parser will be used as a first test for CPAN::non as I have comaint rights for it.

Inch-stones

  1. Improve tests and release new version of Linux::Distribution.
  2. Test Linux::APT. Write/submit patches if necessary.
  3. Install CentOS and Fedora. Write a module for yum interaction
  4. Write and test an OS-independent module for software package installation (apt and yum)
  5. CPAN::non - prototype
  6. CPAN::non - write code
  7. CPAN::non - test
  8. CPAN::non - documentation
  9. Use it in XML::Parser for expat
  10. Module::Install plugin
  11. Blog about it, help CPAN authors use it, improve main module if necessary

Project Schedule

The project will take 2-3 months. I will start right after grant approval.

Completeness Criteria

  • New versions of Linux::Distribution and XML::Parser.

  • CPAN::non, Module::Install plugin and module for yum released.

Future

This project is a working solution, but after it will be adopted, more interesting things may be implemented, like support for FreeBSD, even Windows, support in tools for OS package generation, etc.

Bio

I maintain many CPAN modules and fixed bugs in many more. Participated in such projects as Strawberry Perl, Padre, ExtUtils::MakeMaker and others. Contributed to Module::Install. My CPAN account: http://search.cpan.org/~chorny/ and Ohloh: https://www.ohloh.net/accounts/chorny/positions.

For 9 years I was a lead programmer of credit card payment system InterCardPro. Currently work for kariera.md.

8 Comments

just having a perl interface to yum would be excellent. but it only has an api via python, no c based libraries (making Yum::XS possible) or anything else sensible. you may have to wrap the thing with expect... :(

why redhat doesnt just give up on rpms is beyond me...

I've been using Perl for years but always dreaded installing certain modules because of the problems with external libraries, and (to me) the frustrating error messages - dylib this and header that. I often just gave up and it is only now I realize how much I was missing. So I think a project like this is very good, especially if the needs of inexperienced users are taken into account (e.g. error messages that point to places where you might find help - and I don't mean man pages).

Relationship between our projects is similar to relation between installing software from source and from pre-built packages. Both ways have advantages and disadvantages. Of course, installation from source with Alien::Base::ModuleBuild means that you will get newest version. Also you will be able to compile with other parameters that your vendor did. But complexity of maintaining Alien:: distribution (for ex. author needs to apply patches for every version of OS/distribution that does not compile out of box, similarly to OS vendors) means that most CPAN authors will not create them or will not supply patches. User will also be required to monitor dependency for vulnerabilities. And of course, it will require compiler/linker installed (some web hosting servers do not have gcc, but are willing to install CPAN modules) and all libraries that this library depends on (building Alien packages for them would be too complex), including headers. Most sysadmins prefer to have non-CPAN libraries to be installed from vendor packages, and in future CPAN::non will be able to simplify building vendor packages from CPAN modules.
IMHO, both ways are useful. As Joel Berger writes, Alien::Base::ModuleBuild could even use this project. Or vice versa :).

Most sysadmins prefer to have non-CPAN libraries to be installed from vendor packages, and in future CPAN::non will be able to simplify building vendor packages from CPAN modules.

This is what lead me to my core concept that all installations should target File::ShareDir; this makes library installations essentially indistinguishable from CPAN installations.

You are correct that complex libraries with deep dependencies chains or OS specific patches would be harder. It is for this reason that I will have a handler for downloading specifically pre-built binary packages (built for the specific Alien:: module).

May I ask, what will be the fallback if CPAN::non doesn't support the local package manager?

I'm going to say the same thing for all the grant requests this round ...

As a general rule, I think grants should be used to fund development on projects that are already established.

Speculative projects are just that, speculative. Even if the grant is completed, it's really hard to know whether the resulting work will be useful to anyone.

When funding work on an existing project, we already have some idea of whether or not that project has traction in the wider Perl community.

Most OS’es would be supported, support of yum and apt means 90-95% of Linux installations. In future I plan to add FreeBSD, possibly Mandriva/Mageia, SUSE, Solaris and Cygwin. For Windows I planned to use same method as you plan: installing prebuilt archives. If OS/distribution is unsupported, error message would be printed with general instructions how to find out which package to install. Or use Alien::Base::ModuleBuild to see if it will be able to install :).

What I would really love is a good interface to apt. I have tried to write a script to run immediately after installing a fresh Ubuntu. I finally gave up because I couldn't get useful information about what was installed, installable, which repositories were available, etc etc. That would be a great thing. Once that was running, I would be happy to incorporate it into Alien::Base or support its existence in some other installer mechanism.

Leave a comment

About TPF

The Perl Foundation - supporting the Perl community since 2000. Find out more at www.perlfoundation.org.

Recent Comments

  • Joel Berger: What I would really love is a good interface to read more
  • Alexandr Ciornii: Most OS’es would be supported, support of yum and apt read more
  • autarch.urth.org: I'm going to say the same thing for all the read more
  • Joel Berger: Most sysadmins prefer to have non-CPAN libraries to be installed read more
  • Alexandr Ciornii: Relationship between our projects is similar to relation between installing read more
  • Alberto Simões: Hey, how this grant would relate to the other Alien:: read more
  • tangent: I've been using Perl for years but always dreaded installing read more
  • djzort.myopenid.com: just having a perl interface to yum would be excellent. read more

About this Entry

This page contains a single entry by Alberto Simões published on February 4, 2012 1:31 PM.

Perl 5 Grant Application: Travel to QA Hackathon was the previous entry in this blog.

2012Q1 Grant Proposal: EPublisher Website is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

OpenID accepted here Learn more about OpenID
Powered by Movable Type 4.38