2012Q1 Grant Proposal: Cooking Perl with Chef

12 Comments

Cooking Perl with Chef

Name:

David Golden (DAGOLDEN/xdg)

Email:

[Hidden]

Amount Requested:

USD $2000

Synopsis

I propose to create ready-to-use tools and supporting tutorials that teach Perl developers (and the sysadmins that support them) how to deploy Perl applications repeatable and reliably using the Chef configuration management tool.

Benefits to the Perl Community

  • First-class support for Perl in Chef, just like other web development languages

  • Open cookbooks and tutorials to encourage sharing techniques and practices for app deployment

Deliverables

Code deliverables:

  • Publish a Chef cookbook for Perl interpreter deployment

  • Publish a Chef cookbook for CPAN module deployment

  • Publish a Chef a cookbook for Plack application deployment

  • Improve the Pantry command line tool for configuring servers with Chef Solo

    • Add command-line configuration of "nodes" (servers) and "roles" (configuration patterns)

    • Add support for multiple environments (e.g. "dev", "test", "production") with different configuration options per environment

Note: in Chef a "cookbook" encapsulates all code and metadata needed to automate configuration of particular resources. C.f. About Cookbooks (Opscode wiki)

Teaching deliverables:

  • Publish a presentation-style introduction to using Perl with Chef

  • Publish a document-style "hello world" tutorial with step by step instructions for deploying a simple Plack app

  • Publish a document-style "real world" tutorial with step by step instructions for deploying a complete application stack for a real application

  • Release a video demonstration of "hello world" deployment, as either a screencast or a recording of a conference/seminar presentation

Project Details

Motivation (aka "the problem")

Automated configuration management tools are increasingly part of the toolbox for Internet technology companies, but support for Perl and Perl application frameworks lags behind other dynamic languages. This makes Perl harder to integrate into an environment that already uses such tools and makes it harder for Perl-oriented companies to adopt such tools.

While these difficulties can be overcome, deploying Perl applications with configuration management tools remains an idiosyncratic black art, with the knowledge of how to do it locked up in the minds (or companies) that have done it. For someone new to configuration management, the learning curve is steep.

Approach (aka "the solution")

Over the last couple months, I have been experimenting with deploying Perl applications using the Chef configuration management tool. I demonstrated my "proof-of-concept" code at the Orlando Perl Workshop in January. I continue to tinker with this code and seek a grant to support and motivate turning this experiment into public resources for the Perl community at large.

Using configuration management tools for Perl applications requires addressing several issues that go beyond what typical Perl/CPAN toolchain programs deal with, such as:

  • Idempotent deployment automation

  • Versioned library dependency management

  • Application restart when configuration (including dependencies) changes

  • Multiple perls and multiple apps deployed simultaneously on a single server

My proof-of-concept combines existing tools like perlbrew, local::lib, carton, Plack, and so on to achieve these goals. While the proof-of-concept tools work, they need further refinement, documentation end testing before they are release-ready for general use.

Once these cookbooks and tools are created, I want to make them easy to learn. Since not everyone learns the RTFM way, I propose to create two step-by-step tutorials that walk someone though deploying an application from bare (virtual) metal to live server.

The first tutorial will be the configuration management equivalent of "hello world" -- it will be the smallest code that can show the essential components and steps of the solution. Since not everyone learns from reading tutorials and since there appears to be popular appeal for screencast tutorials, I will also record a video of this tutorial. (Or, if an appropriate conference/seminar opportunity arises to make such a demonstration during the grant period, I will provide that instead.)

The second tutorial will be a "real world" one that shows how a full application -- including supporting, non-Perl components like a web-server, database, caching layer, etc. -- can be deployed and configured. I have not settled on an application for this tutorial, but I consider the CPAN Testers Metabase or the Jitterbug continuous integration server to be leading candidates.

Inch-stones

See "Deliverables" for general work-blocks.

Each of the code deliverables will be broken into "coding", "documentation" and "release" inch-stones.

Written teaching documents will be broken into "rough draft", "peer review", "final draft" and "release" inch-stones.

The video deliverable will be broken into "storyboard", "recording", "post-production" and "release" inch-stones.

Project Schedule

The project is estimated to take three months. I will begin immediately upon being awarded the grant.

Completeness Criteria

All code and teaching deliverables will be published online under open licenses.

  • Cookbooks will be published to the Chef Community Cookbooks site (which is like CPAN for Chef).

  • Pantry will be published to CPAN.

  • Tutorials will be published either to my blog or to an appropriate Perl tutorials site.

  • Video will be published to YouTube and/or Perl-specific video sites.

Code, presentations and documents will also be made available via Github for community access and improvement.

Bio

I have a long history contributing to the Perl 5 community:

  • Active Perl 5 Porter responsible for two dev releases, several new features and the occasional bug fix.

  • Developer of the first several releases of Strawberry Perl.

  • Prolific CPAN author. Contributor to or co-maintainer of a large subset of the CPAN toolchain.

  • Leader of the CPAN Meta Spec v2 revision process

  • Lead architect and manager of the CPAN Testers Metabase that collects nearly a million CPAN Testers reports every month.

  • Moderately-frequent blogger and conference/seminar presenter on Perl-related topics.

12 Comments

I like that this builds on an already successful project (Chef). David has a good track record of getting things done, which is a big plus.

Does this need any buy-in from the Chef developers, or can it stand on its own?

Also, why Chef and not Puppet or some other tool? Is there any agreement in the Perl community as to what the best choice is? FWIW, we've started using Puppet at my $DAY_JOB.

Overall, I think this looks like a useful project.

Definitely seems useful. Too often we in Perland are in a bubble and don't participate fully with outside non-perl projects.

My only nibble is like autarch, I wonder why Chef and not Puppet. In my experience Puppet is more popular and usage is on the rise.

My experience has been the opposite. Chef is more popular among the developers I've run into.

I'm hoping this happens. I'm just learning Chef and trying to automate the configuration of our Perl servers. I was probably going to end up writing some of this myself.

Chef is a Ruby-based tool.
Ruby breaks backward compatibility too often even with minor language version upgrade.

So, It's harder to maintain working language version across servers than do real sysadmin tasks.

I can't understand who use such unstable thing for tasks requiring robustness.

I think it's better to raise fund for Rex( http://rexify.org/ ) instead of Chef.
It's Perl's counterpart to Chef.

I have absolutely zero doubt that David will deliver what he promises. His track record in following through on what he starts is excellent.

As for the choice of tool, I'd also prefer to see puppet used as well, simply because $work uses it heavily. That being said, I believe that David's work on Chef would be valuable, not only standing on its own, but also as research and example on how other volunteers can build similar setups with other tools. His inclusion of documentation and learning material in the grant may go a long way towards sharing the design work.

I left a comment before, but it seems lost in the moderation queue, so I'll try again.

Re: Chef developer support -- no direct support needed. Their community cookbook site seems open to just about any reasonable contribution.

Re: Chef vs puppet -- I think this is like emacs-vs-vi. I looked at both and Chef fit my brain better. Support for standalone mode with Chef Solo also seems better than the equivalent with Puppet, which has nice properties that I won't go into here, but might blog about someday.

As Steffen said, I think the techniques/patterns I'm developing for Chef could be translated to Puppet by a reasonably skilled Puppet hacker.

I'd like to see better integration for Perl projects in such projects. Companies often want use the same tools for all applications they develop (no matter what language the application is written in). So I support this grant application!

I think it is very important to have such integration for Perl with multiple tools and I have seen what David is capable of so I am fully supporting this request.

IMHO integrating with Chef is a good first step but I'd like to see plans, e.g. a follow-up grant, either from David or from someone else, to create the integration with Puppet and Rex as well.

I am all for this. ... Assuming that some parts are built in a platform agnostic way.

What I would really love is a good interface to apt. I have tried to write a script to run immediately after installing a fresh Ubuntu. I finally gave up because I couldn't get useful information about what was installed, installable, which repositories were available, etc etc. That would be a great thing. Once that was running, I would be happy to incorporate it into Alien::Base or support its existence in some other installer mechanism.

Shoot, the above was meant for the CPAN::non post. Too many tabs fail I think. Sorry about that.

A huge +1 to this grant. It will be extremely useful in many contexts.

Leave a comment

About TPF

The Perl Foundation - supporting the Perl community since 2000. Find out more at www.perlfoundation.org.

Recent Comments

  • leto.net: A huge +1 to this grant. It will be extremely read more
  • Joel Berger: Shoot, the above was meant for the CPAN::non post. Too read more
  • Joel Berger: What I would really love is a good interface to read more
  • Brad Gilbert: I am all for this. ... Assuming that some parts read more
  • szabgab.com: I think it is very important to have such integration read more
  • Renée Bäcker: I'd like to see better integration for Perl projects in read more
  • dagolden.com: I left a comment before, but it seems lost in read more
  • Steffen Mueller: I have absolutely zero doubt that David will deliver what read more
  • camel: Chef is a Ruby-based tool. Ruby breaks backward compatibility too read more
  • Brian Meeker: My experience has been the opposite. Chef is more popular read more

About this Entry

This page contains a single entry by Alberto Simões published on February 9, 2012 6:01 PM.

2012Q1 Grant Proposal: Alien::Base - Base classes for Alien:: modules was the previous entry in this blog.

2012Q1 Grant Proposals is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

OpenID accepted here Learn more about OpenID
Powered by Movable Type 4.38