2009Q1 Grant Proposal: Pod Mangling Utility Improvements
Fri, 06-Feb-2009 by
Alberto Simões
edit post
<![CDATA[
<dl>
<dt><strong>Name:</strong>
<dd>
<p>Ricardo Signes</p>
</dd>
</li>
<dt><strong>Email:</strong>
<dd>
<p>[hidden email]</p>
</dd>
</li>
<dt><strong>Amount Requested:</strong>
<dd>
<p>$2000</p>
</dd>
</li>
</dl>
<p>
</p>
<h2>Synopsis</h2>
<p>This grant is for the completion of design and implementation of basic versions
of three pieces of software currently under development:</p>
<dl>
<dt><strong>Pod::Elemental</strong>
<dt><strong>Pod::Weaver</strong>
<dt><strong>Dist::Zilla::Plugin::PodWeaver</strong>
</dl>
<p>All three modules exist and have been in use by a number of people for some
time. The main thrust of the grant is to improve the usability and reliability
of Pod::Weaver. Pod::Weaver is one of the key components of Dist::Zilla. D::Z
was presented to good response at the Pittsburgh Perl Workshop in 2008.</p>
<h2>Benefits to the Perl Community</h2>
<p>Pod::Elemental and Pod::Weaver provide very powerful tools for POD
manipulation. They're designed to be easy to understand and extend, and should
make it much easier to write POD-reading and POD-munging tools.</p>
<p>The flagship example of such a tool, right now, is Dist::Zilla's Pod::Weaver
plugin. When writing a distribution that is will be munged by Pod::Weaver, the
author needn't write any POD save for that which actually documents his code.
This means no writing (or updating) of the LICENSE section, the NAME, the
VERSION, or the COPYRIGHT. Sections like METHODS or FUNCTIONS are built
automatically from <code>=method</code> commands. If the author decides he'd like to
radically change how he lays out his POD, this can be done by reconfiguring the
weaver, since the documentation itself doesn't have to change. The next time
the dist is built, everything is new.</p>
<p>This is summarized in this SlideShare presentation:</p>
<pre>
<a href="http://www.slideshare.net/rjbs/distzilla-presentation/">http://www.slideshare.net/rjbs/distzilla-presentation/</a></pre>
<p>I've been told by several people that "without Dist::Zilla, I would not have
released [my distribution] to the CPAN." Dist::Zilla makes it much easier to
release code, because it removes a lot of the boring overhead of building a
dist.</p>
<p>While <em>my</em> goal for this grant is to advance the state of Dist::Zilla,
Pod::Weaver has always been meant for operation as a standalone library. It
will be useful for anyone who wants to do work rewriting or maintaining POD
documents, as will Pod::Elemental, which provides a much simpler interface to
POD documents than Pod::Simple (at the cost of some specific features).</p>
<p>
</p>
<h2>Deliverables</h2>
<p>Pod::Elemental::Nester and Pod::Elemental::Document will be completed,
documented, and tested.</p>
<p>Pod::Weaver will be updated to munge Pod::Elemental::Documents instead of
queues of Pod::Elemental::Elements. It will be able to rewrite documents
without clobbering order. It will be documented and support external
configuration.</p>
<p>Dist::Zilla will integrate the new Pod::Weaver configuration to allow
Dist::Zilla::Plugin::PodWeaver to provide all the power of Pod::Weaver.</p>
<p>
</p>
<h2>Project Details</h2>
<p>Pod::Elemental is a library for handling POD documents (like this grant
application) as trees of elements. This is not an entirely straightforward
operation, as POD is primarily event-based and <em>most</em> of its notions of
containers or trees are assumed or implicit, rather than well defined.</p>
<p>Pod::Elemental is built atop Pod::Eventual, which handles reading the POD
events. Pod::Elemental already exists and is in use, but requires work to
correct design problems in the Nester and Document components, which convert
event streams to nested trees in two distinct ways.</p>
<p>Pod::Weaver is a framework for rewriting POD trees. For example, it can be
configured to find all <code>=method</code> trees and collect and rewrite them into
<code>=head2</code> events under a <code>=head1 METHOD</code> heading. Weavers make it very easy
to turn a document that contains only a few, nonstandard events into a
perfectly normal looking POD document, fleshed out with all the boilerplate the
most translators (and readers) would expect.</p>
<p>In addition to being blocked by Pod::Elemental's design needs, Pod::Weaver
needs a few of its planned Weaver plugins implemented before it can be safely
used on many POD trees. Among these, it needs plugins that focus on leaving
unknown content in place relative to rewritten content.</p>
<p>Both of the above libraries were written to support Dist::Zilla, a distribution
construction kit that replaces not <em class="file">h2xs</em> and Module::Starter, but rather
<code>make dist</code>. Among other things, it can be configured to add boilerplate POD
and translate minimal custom POD into verbose standard POD.</p>
<p>The Pod::Weaver plugin for Dist::Zilla will need some minor tweaks to take
advantage of the configurability of the new Pod::Weaver code to be written.</p>
<p>
</p>
<h2>Inch-stones</h2>
<ul>
<li><strong>a clear distinction laid out for ::Nester v. ::Document</strong>
<li><strong>extensive tests for Pod::Elemental, and Document implemented</strong>
<li><strong>Pod::Weaver operating on Document objects</strong>
<li><strong>the "Allow" Weaver, which leaves specific stuff in place, if found</strong>
<li><strong>the "Accordion" Weaver, which leaves generic stuff in place, if found</strong>
<p>These are more obnoxious than they sound, as they will require, at least,
rewriting all the existing Weavers to not assume they work linearly.</p>
</li>
<li><strong>Pod::Weaver::Config, for loading Pod::Weaver config from files</strong>
<p>This can hopefully be cribbed from Dist::Zilla's config code.</p>
</li>
<li><strong>Dist::Zilla::Plugin::PodWeaver 2 - with real config</strong>
<li><strong>[target of opportunity] Pod::Weaver dialects</strong>
<p>This is to allow things like Pod::WikiDoc to apply to portions of a POD
document before weaving occurs.</p>
</li>
</ul>
<p>
</p>
<h2>Project Schedule</h2>
<p>I predict the project will take between four and eight weekends of hard work,
plus some random work here and there when I have to dump ideas that strike me
in the shower. This may go up or down, based on luck, setbacks, or moments of
Satori, but I find it hard to imagine reaching a point where I would abandon
or massively delay this work.</p>
<p>
</p>
<h2>Bio</h2>
<p>I write Perl code to pay my bills, and that's what I've been doing for almost
ten years now. I have released a lot of code to the CPAN, and I've taken over
maintenance of almost as much. Well over 2,000 distributions that I didn't
write declare dependencies on my code, either directly or indirectly.</p>
<p>I wrote Dist::Zilla and Pod::Weaver because of the number of CPAN distributions
(and contained POD documents) that I maintain, and have worked to make it
possible for the lessons that I've learned to benefit those who haven't yet
gotten to painful levels of code maintenance, so that the CPAN can continue to
be the easiest way to reliably distribute open source code.</p>
]]>
Comments (8)
I hate writing (or copy & pasting) POD boilerplate. I've loved using Dist::Zilla's tools for eliminating the need for such boilerplate, and I'd love to see those tools made more robust.
This I think has the biggest bang for the buck to the current Perl(5) community. Developing more tools to make documentation easier can only help things improve in the long run, and Ricardo has a proven track record of developing useful and well build modules. If he's crazy enough to step into the world of POD, I say all we can do is say "Go forth brave warrior, we'll tell your children stories of your heroisim."
This definitely has potential, and I trust rjbs to write something sane. I think there's a lot to be said for funding people whose previous work is well-known to the community (but I would say that, wouldn't I?)
POD looks deceptively simple, but is really kind of strange when you dig down into it. It'd be nice to have better tools for POD munging (not just to make pretty docs, but for detailed manipulation). A DOM or SAX style interface would be nice if that's not too much trouble :)
And I agree with Chris Prather that Ricardo is good for crazy details.
There are many modules on CPAN to do with POD. How would this compare to Pod-POM "POD Object Model", for example?
There are a few key differences, at least. The most important, for me, is the fact that POM is more tied to "standard" POD commands and a single set of nesting rules. Pod::Elemental makes fewer hard and fast assumptions: you start with a stream of elements rather than a tree, and your rules for nesting them are mostly up to you. After all, while it might seem sensible that a "=over" after a "=head2" should be inside it, you might disagree. This becomes much more important when you extend the commands available to add things like "=method" or "=dialect"
Pod::POM's handling of unknown commands is, at least to me, confusing, though I didn't spent too long trying to understand it exactly. The tying together of views and parsing is where I started to feel this wasn't right for my work.
It's also built atop Pod::Eventual for getting the event streams from which the element streams are built, making it easier to replace that component as well.
One shortcoming (or, rather, one feature I've excluded as unneeded for my applications) is support for embedded sequences like B>. I may eventually add them with another layer, but I don't see a need for it, and it would certainly not be done as part of this grant work.
Finally, Pod::Weaver, which sits atop ::Elemental, is an entirely different animal. It is meant to slice up and rearrange or add to POD trees. It needed a flexible tree builder to be written to perform its goals, especially command types.
Sounds interesting, but I thing we need a simple example(with apples) of what are you intended to do).
I recommend perl community to see the slides
http://www.slideshare.net/rjbs/distzilla-presentation/
and form your own opinion.
