2013Q2 GP: Next Release of Pinto With Key Features

3 Comments
Name:

Jeffrey Ryan Thalhammer

Amount Requested:

$3000.00

Synopsis

Pinto is a turnkey solution for constructing and managing local CPAN-like repositories of Perl modules. This purpose of this grant proposal is to obtain funding for development of the next release of Pinto, which will include specific features described below.

The Pinto project is less than 2 years old, but it has already gained a modest user base and is potentially relevant to a wide audience within the Perl community (i.e. anyone that uses CPAN module). Funding development on Pinto will help the Perl community by providing them with better tools for managing their CPAN modules.

Benefits to the Perl Community

The CPAN ecosystem is one of the most valuable assets of the Perl Community. However, it is not stable because authors can freely add or remove modules. As a result, end-users cannot rely on the public CPAN to provide a consistent set of Perl module dependencies from one day to the next. Sudden "breakage" of dependencies is a common frustration among Perl users.

Furthermore, the inner workings of CPAN itself (i.e. PAUSE) are not readily accessible. The code is arcane, has minimal testing, and can't be installed in the usual fashion. So despite its enormous value to the Perl Community, the CPAN infrastructure is largely unusable to those wishing to locally distribute their own private modules using the standard Perl toolchain.

Pinto solves both of these problems with a complete, robust, and extensible application. Pinto users can easily create local CPAN-like repositories that are stable and will yield the exact same modules every time. Pinto also duplicates the indexing functionality of PAUSE, so they can use these repositories to locally distribute their private modules as well. Pinto even provides novel tools for managing upgrades to dependent modules.

These capabilities are beneficial to the Perl Community because:

  • They make it safer to incorporate more CPAN modules into their work, thereby encouraging more use (and reuse) of CPAN modules.

  • They reduce the risk of breakage from upgrading CPAN modules, thereby enabling more frequent upgrades.

  • They facilitate sharing of private Perl modules within an organization, thereby promoting the growth of Perl.

For more details about Pinto, see the "More About Pinto" section below.

Deliverables

The deliverable will be a release of Pinto (and possibly other attendant modules) that support the features described below in the "Project Details" section. All software will be released under the same terms as Perl itself and delivered via the CPAN.

Deliverables

The deliverable will be a release of Pinto (and possibly other attendant modules) that support the features described below in the "Project Details" section. All software will be released under the same terms as Perl itself and delivered via the CPAN.

Project Details

This proposal covers the addition of two key features to Pinto:

Repository Migration Mechanism

Pinto will continue to evolve and future versions will undoubtedly introduce non-compatible changes. To grow and maintain its user base, Pinto needs a way to migrate existing repositories into a format that is compatible with future versions of Pinto. This migration may involve changes to the internal database schema as well as changes to file formats and directory structures.

To be clear, the migration mechanism will only work for Pinto repositories created with the *next* release of Pinto or later. It will not work for Pinto repositories that exist right now. Currently, Pinto repositories are not versioned which makes them much more difficult to migrate. It is not impossible, but that is outside the scope of this proposal.

Version Control Mechanism

One of Pinto's most powerful features is an integrated versioning system. This allows users to "branch" and "merge" the indexes in their repository in a manner similar to version control systems like Git and Subversion. This allows users to upgrade modules in an isolated, audited, and reversible way, thereby reducing risks from changes and increasing visibility into the evolution of their dependency stack.

Pinto's current implementation of the version control mechanism is incomplete and does not scale well. I'll be finishing & enhancing the version control mechanism by either extending the built-in system it already has, or replacing it with an external VCS that is accessed through language bindings (e.g. Alien::SVN or Git::Raw).

Inch-stones

For feature 1 (Schema Migration Mechanism)

  • Evaluate existing solutions for database schema migration.

  • Create or integrate database schema versioning and migration management code.

  • Create hooks for managing migration of non-database assets (e.g files, directories).

  • Implement and document command-line interface to migration hooks in App::Pinto.

For feature 2 (Version Control Mechanism)

  • Identify the key VCS features that are required (e.g. commit, branch, merge, log, diff).

  • Evaluate suitability of existing version control systems (notably libgit2).

  • Implement or integrate each of the aforementioned key VCS features in the Pinto core.

  • Implement and document command-line interfaces to those functions in App::Pinto.

Project Schedule

The features outlined in this proposal are estimated to take 3 to 4 person-weeks of effort. However, the release may include additional work that is not covered in this proposal, so it may take longer to actually complete. I'm available to start the work immediately and can commit about 20 hours per week to the project. I expect completion within 9 to 12 calendar weeks.

Both features covered by this proposal can be developed concurrently. I'll be reporting progress via bi-weekly updates to blogs.perl.org.

Completeness Criteria

Completeness can be evaluated when Pinto is released to the CPAN. The final implementation of the features outlined in this proposal will be described in the Changes file that accompanies the release. The details of the features can be further verified via the automated test cases that will also accompany the release.

Prior Art

Pinto is not the only tool in this space. CPAN::Mini, CPAN::Site, OrePAN, MyCPAN and others all have features that overlap with Pinto. They are all good tools that fit a particular niche. However, none of them are quite as robust and comprehensive as Pinto. For example, no other tool supports creating a single repository with multiple indexes like Pinto does. Also, Pinto is the only tool that is designed specifically to support team development and concurrent users.

More About Pinto

My presentation on Pinto at YAPC::NA 2012:

https://www.youtube.com/watch?v=oaBBVZFhJUk

A brief survey of Pinto's purpose and components:

https://metacpan.org/module/THALJEF/Pinto-Common-0.064/lib/Pinto/Manual/Introduction.pod

The tutorial I've written for Pinto:

https://metacpan.org/module/THALJEF/Pinto-Common-0.064/lib/Pinto/Manual/Tutorial.pod

Bio

I've been developing software (primarily in Perl) for more than 12 years. I am active in the Perl Community and have presented at YAPC and various Perl Monger meetings. I'm best known as the creator of Perl::Critic, the widely used static analyzer for Perl. I'm also the creator of Pinto, which was originally funded by Genentech in October of 2011. I have continued to work on Pinto ever since then.

3 Comments

UPDATE: The features outlined in this grant proposal have alaready been completed. However, the grant proposal still stands with these features instead:

  • Extend diff command to compare specific revisions
  • Create new command to merge stacks
Inch-stones for feature 1 (Extend diff command to compare specific revisions)

Determine command-line notation for specifying revisions

For example foo@af23e6 would mean stack "foo" at revision "af23e6". And/or foo@-1 would mean stack "foo" at the first revision prior to the HEAD.

Implement applicaiton logic to compare revisions

The code for creating the diff is already written, so this is mostly about wiring the command-line interface to the application back-end.

Inch-stones for feature 2 (Create new command to merge stacks)


Determine the semantics of merging stacks

It isn't obvious what it means to "merge" a stack. Michael Schwern has already proposed his vision here.

Implement automatic merging logic

Once the semantics of a merge are decided, then I can go ahead with implementing the "merge" command. Automatic merge strategies could include things like "accept newer" or "accept mine" or "accept theirs".

Implement conflict resolution mechanism

When automatic merging is impossible, the user needs some mechanism for resolving conflicts manually. Ideally, Pinto could provide an interactive mechanism for choosing.

The Perl community needs more well thought out and documented tools like Pinto. I think this is worth supporting.

I think this is well worth funding (and more so than my own proposal).

Leave a comment

About TPF

The Perl Foundation - supporting the Perl community since 2000. Find out more at www.perlfoundation.org.

Recent Comments

  • Neil Bowers: I think this is well worth funding (and more so read more
  • autarch.urth.org: The Perl community needs more well thought out and documented read more
  • Jeffrey Ryan Thalhammer: UPDATE: The features outlined in this grant proposal have alaready read more

About this Entry

This page contains a single entry by Alberto Simões published on May 4, 2013 6:38 PM.

Improving Perl 5: Grant Report for Month 18 was the previous entry in this blog.

2013Q2 GP: Review of Perl Web Frameworks is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

OpenID accepted here Learn more about OpenID
Powered by Movable Type 4.38