2013Q2 GP: Next Release of Pinto With Key Features
Sat, 04-May-2013 by
Alberto Simões
edit post
<![CDATA[
<dl>
<dt id="Name">Name:</dt>
<dd>
<p>Jeffrey Ryan Thalhammer</p>
</dd>
<dt id="Amount-Requested">Amount Requested:</dt>
<dd>
<p>$3000.00</p>
</dd>
</dl>
<h2 id="Synopsis">Synopsis</h2>
<p><a href="http://metacpan.org/module/Pinto">Pinto</a> is a turnkey solution for constructing and managing local CPAN-like repositories of Perl modules. This purpose of this grant proposal is to obtain funding for development of the next release of Pinto, which will include specific features described below.</p>
<p>The Pinto project is less than 2 years old, but it has already gained a modest user base and is potentially relevant to a wide audience within the Perl community (i.e. anyone that uses CPAN module). Funding development on Pinto will help the Perl community by providing them with better tools for managing their CPAN modules.</p>
<h2 id="Benefits-to-the-Perl-Community">Benefits to the Perl Community</h2>
<p>The CPAN ecosystem is one of the most valuable assets of the Perl Community. However, it is not stable because authors can freely add or remove modules. As a result, end-users cannot rely on the public CPAN to provide a consistent set of Perl module dependencies from one day to the next. Sudden "breakage" of dependencies is a common frustration among Perl users.</p>
<p>Furthermore, the inner workings of CPAN itself (i.e. PAUSE) are not readily accessible. The code is arcane, has minimal testing, and can't be installed in the usual fashion. So despite its enormous value to the Perl Community, the CPAN infrastructure is largely unusable to those wishing to locally distribute their own private modules using the standard Perl toolchain.</p>
<p>Pinto solves both of these problems with a complete, robust, and extensible application. Pinto users can easily create local CPAN-like repositories that are stable and will yield the exact same modules every time. Pinto also duplicates the indexing functionality of PAUSE, so they can use these repositories to locally distribute their private modules as well. Pinto even provides novel tools for managing upgrades to dependent modules.</p>
<p>These capabilities are beneficial to the Perl Community because:</p>
<ul>
<li><p>They make it safer to incorporate more CPAN modules into their work, thereby encouraging more use (and reuse) of CPAN modules.</p>
</li>
<li><p>They reduce the risk of breakage from upgrading CPAN modules, thereby enabling more frequent upgrades.</p>
</li>
<li><p>They facilitate sharing of private Perl modules within an organization, thereby promoting the growth of Perl.</p>
</li>
</ul>
<p>For more details about Pinto, see the "More About Pinto" section below.</p>
<h2 id="Deliverables">Deliverables</h2>
<p>The deliverable will be a release of Pinto (and possibly other attendant modules) that support the features described below in the "Project Details" section. All software will be released under the same terms as Perl itself and delivered via the CPAN.</p>
<h2 id="Deliverables">Deliverables</h2>
<p>The deliverable will be a release of Pinto (and possibly other attendant modules) that support the features described below in the "Project Details" section. All software will be released under the same terms as Perl itself and delivered via the CPAN.</p>
<h2 id="Project-Details">Project Details</h2>
<p>This proposal covers the addition of two key features to Pinto:</p>
<dl>
<dt id="Repository-Migration-Mechanism">Repository Migration Mechanism</dt>
<dd>
<p>Pinto will continue to evolve and future versions will undoubtedly introduce non-compatible changes. To grow and maintain its user base, Pinto needs a way to migrate existing repositories into a format that is compatible with future versions of Pinto. This migration may involve changes to the internal database schema as well as changes to file formats and directory structures.</p>
<p>To be clear, the migration mechanism will only work for Pinto repositories created with the *next* release of Pinto or later. It will not work for Pinto repositories that exist right now. Currently, Pinto repositories are not versioned which makes them much more difficult to migrate. It is not impossible, but that is outside the scope of this proposal.</p>
</dd>
<dt id="Version-Control-Mechanism">Version Control Mechanism</dt>
<dd>
<p>One of Pinto's most powerful features is an integrated versioning system. This allows users to "branch" and "merge" the indexes in their repository in a manner similar to version control systems like Git and Subversion. This allows users to upgrade modules in an isolated, audited, and reversible way, thereby reducing risks from changes and increasing visibility into the evolution of their dependency stack.</p>
<p>Pinto's current implementation of the version control mechanism is incomplete and does not scale well. I'll be finishing & enhancing the version control mechanism by either extending the built-in system it already has, or replacing it with an external VCS that is accessed through language bindings (e.g. Alien::SVN or Git::Raw).</p>
</dd>
</dl>
<h2 id="Inch-stones">Inch-stones</h2>
<p>For feature 1 (Schema Migration Mechanism)</p>
<ul>
<li><p>Evaluate existing solutions for database schema migration.</p>
</li>
<li><p>Create or integrate database schema versioning and migration management code.</p>
</li>
<li><p>Create hooks for managing migration of non-database assets (e.g files, directories).</p>
</li>
<li><p>Implement and document command-line interface to migration hooks in App::Pinto.</p>
</li>
</ul>
<p>For feature 2 (Version Control Mechanism)</p>
<ul>
<li><p>Identify the key VCS features that are required (e.g. commit, branch, merge, log, diff).</p>
</li>
<li><p>Evaluate suitability of existing version control systems (notably libgit2).</p>
</li>
<li><p>Implement or integrate each of the aforementioned key VCS features in the Pinto core.</p>
</li>
<li><p>Implement and document command-line interfaces to those functions in App::Pinto.</p>
</li>
</ul>
<h2 id="Project-Schedule">Project Schedule</h2>
<p>The features outlined in this proposal are estimated to take 3 to 4 person-weeks of effort. However, the release may include additional work that is not covered in this proposal, so it may take longer to actually complete. I'm available to start the work immediately and can commit about 20 hours per week to the project. I expect completion within 9 to 12 calendar weeks.</p>
<p>Both features covered by this proposal can be developed concurrently. I'll be reporting progress via bi-weekly updates to blogs.perl.org.</p>
<h2 id="Completeness-Criteria">Completeness Criteria</h2>
<p>Completeness can be evaluated when Pinto is released to the CPAN. The final implementation of the features outlined in this proposal will be described in the Changes file that accompanies the release. The details of the features can be further verified via the automated test cases that will also accompany the release.</p>
<h2 id="Prior-Art">Prior Art</h2>
<p>Pinto is not the only tool in this space. CPAN::Mini, CPAN::Site, OrePAN, MyCPAN and others all have features that overlap with Pinto. They are all good tools that fit a particular niche. However, none of them are quite as robust and comprehensive as Pinto. For example, no other tool supports creating a single repository with multiple indexes like Pinto does. Also, Pinto is the only tool that is designed specifically to support team development and concurrent users.</p>
<h2 id="More-About-Pinto">More About Pinto</h2>
<dl>
<dt id="My-presentation-on-Pinto-at-YAPC::NA-2012">My presentation on Pinto at YAPC::NA 2012:</dt>
<dd>
<p>https://www.youtube.com/watch?v=oaBBVZFhJUk</p>
</dd>
<dt id="A-brief-survey-of-Pintos-purpose-and-components">A brief survey of Pinto's purpose and components:</dt>
<dd>
<p>https://metacpan.org/module/THALJEF/Pinto-Common-0.064/lib/Pinto/Manual/Introduction.pod</p>
</dd>
<dt id="The-tutorial-Ive-written-for-Pinto">The tutorial I've written for Pinto:</dt>
<dd>
<p>https://metacpan.org/module/THALJEF/Pinto-Common-0.064/lib/Pinto/Manual/Tutorial.pod</p>
</dd>
</dl>
<h2 id="Bio">Bio</h2>
<p>I've been developing software (primarily in Perl) for more than 12 years. I am active in the Perl Community and have presented at YAPC and various Perl Monger meetings. I'm best known as the creator of <a href="http://metacpan.org/module/Perl::Critic">Perl::Critic</a>, the widely used static analyzer for Perl. I'm also the creator of Pinto, which was originally funded by Genentech in October of 2011. I have continued to work on Pinto ever since then.</p>
]]>
Category:
(none)
Comments (3)
UPDATE: The features outlined in this grant proposal have alaready been completed. However, the grant proposal still stands with these features instead:
Extend diff command to compare specific revisions
Create new command to merge stacks
Inch-stones for feature 1 (Extend diff command to compare specific revisions)
Determine command-line notation for specifying revisions
For example foo@af23e6 would mean stack "foo" at revision "af23e6". And/or foo@-1 would mean stack "foo" at the first revision prior to the HEAD.
Implement applicaiton logic to compare revisions
The code for creating the diff is already written, so this is mostly about wiring the command-line interface to the application back-end.
Inch-stones for feature 2 (Create new command to merge stacks)
Determine the semantics of merging stacks
It isn't obvious what it means to "merge" a stack. Michael Schwern has already proposed his vision
here.
Implement automatic merging logic
Once the semantics of a merge are decided, then I can go ahead with implementing the "merge" command. Automatic merge strategies could include things like "accept newer" or "accept mine" or "accept theirs".
Implement conflict resolution mechanism
When automatic merging is impossible, the user needs some mechanism for resolving conflicts manually. Ideally, Pinto could provide an interactive mechanism for choosing.
The Perl community needs more well thought out and documented tools like Pinto. I think this is worth supporting.
I think this is well worth funding (and more so than my own proposal).
