2009Q1 Grant Proposal: Pod Mangling Utility Improvements

8 Comments
Name:

Ricardo Signes

Email:

[hidden email]

Amount Requested:

$2000

Synopsis

This grant is for the completion of design and implementation of basic versions of three pieces of software currently under development:

Pod::Elemental
Pod::Weaver
Dist::Zilla::Plugin::PodWeaver

All three modules exist and have been in use by a number of people for some time. The main thrust of the grant is to improve the usability and reliability of Pod::Weaver. Pod::Weaver is one of the key components of Dist::Zilla. D::Z was presented to good response at the Pittsburgh Perl Workshop in 2008.

Benefits to the Perl Community

Pod::Elemental and Pod::Weaver provide very powerful tools for POD manipulation. They're designed to be easy to understand and extend, and should make it much easier to write POD-reading and POD-munging tools.

The flagship example of such a tool, right now, is Dist::Zilla's Pod::Weaver plugin. When writing a distribution that is will be munged by Pod::Weaver, the author needn't write any POD save for that which actually documents his code. This means no writing (or updating) of the LICENSE section, the NAME, the VERSION, or the COPYRIGHT. Sections like METHODS or FUNCTIONS are built automatically from =method commands. If the author decides he'd like to radically change how he lays out his POD, this can be done by reconfiguring the weaver, since the documentation itself doesn't have to change. The next time the dist is built, everything is new.

This is summarized in this SlideShare presentation:

  http://www.slideshare.net/rjbs/distzilla-presentation/

I've been told by several people that "without Dist::Zilla, I would not have released [my distribution] to the CPAN." Dist::Zilla makes it much easier to release code, because it removes a lot of the boring overhead of building a dist.

While my goal for this grant is to advance the state of Dist::Zilla, Pod::Weaver has always been meant for operation as a standalone library. It will be useful for anyone who wants to do work rewriting or maintaining POD documents, as will Pod::Elemental, which provides a much simpler interface to POD documents than Pod::Simple (at the cost of some specific features).

Deliverables

Pod::Elemental::Nester and Pod::Elemental::Document will be completed, documented, and tested.

Pod::Weaver will be updated to munge Pod::Elemental::Documents instead of queues of Pod::Elemental::Elements. It will be able to rewrite documents without clobbering order. It will be documented and support external configuration.

Dist::Zilla will integrate the new Pod::Weaver configuration to allow Dist::Zilla::Plugin::PodWeaver to provide all the power of Pod::Weaver.

Project Details

Pod::Elemental is a library for handling POD documents (like this grant application) as trees of elements. This is not an entirely straightforward operation, as POD is primarily event-based and most of its notions of containers or trees are assumed or implicit, rather than well defined.

Pod::Elemental is built atop Pod::Eventual, which handles reading the POD events. Pod::Elemental already exists and is in use, but requires work to correct design problems in the Nester and Document components, which convert event streams to nested trees in two distinct ways.

Pod::Weaver is a framework for rewriting POD trees. For example, it can be configured to find all =method trees and collect and rewrite them into =head2 events under a =head1 METHOD heading. Weavers make it very easy to turn a document that contains only a few, nonstandard events into a perfectly normal looking POD document, fleshed out with all the boilerplate the most translators (and readers) would expect.

In addition to being blocked by Pod::Elemental's design needs, Pod::Weaver needs a few of its planned Weaver plugins implemented before it can be safely used on many POD trees. Among these, it needs plugins that focus on leaving unknown content in place relative to rewritten content.

Both of the above libraries were written to support Dist::Zilla, a distribution construction kit that replaces not h2xs and Module::Starter, but rather make dist. Among other things, it can be configured to add boilerplate POD and translate minimal custom POD into verbose standard POD.

The Pod::Weaver plugin for Dist::Zilla will need some minor tweaks to take advantage of the configurability of the new Pod::Weaver code to be written.

Inch-stones

  • a clear distinction laid out for ::Nester v. ::Document
  • extensive tests for Pod::Elemental, and Document implemented
  • Pod::Weaver operating on Document objects
  • the "Allow" Weaver, which leaves specific stuff in place, if found
  • the "Accordion" Weaver, which leaves generic stuff in place, if found

    These are more obnoxious than they sound, as they will require, at least, rewriting all the existing Weavers to not assume they work linearly.

  • Pod::Weaver::Config, for loading Pod::Weaver config from files

    This can hopefully be cribbed from Dist::Zilla's config code.

  • Dist::Zilla::Plugin::PodWeaver 2 - with real config
  • [target of opportunity] Pod::Weaver dialects

    This is to allow things like Pod::WikiDoc to apply to portions of a POD document before weaving occurs.

Project Schedule

I predict the project will take between four and eight weekends of hard work, plus some random work here and there when I have to dump ideas that strike me in the shower. This may go up or down, based on luck, setbacks, or moments of Satori, but I find it hard to imagine reaching a point where I would abandon or massively delay this work.

Bio

I write Perl code to pay my bills, and that's what I've been doing for almost ten years now. I have released a lot of code to the CPAN, and I've taken over maintenance of almost as much. Well over 2,000 distributions that I didn't write declare dependencies on my code, either directly or indirectly.

I wrote Dist::Zilla and Pod::Weaver because of the number of CPAN distributions (and contained POD documents) that I maintain, and have worked to make it possible for the lessons that I've learned to benefit those who haven't yet gotten to painful levels of code maintenance, so that the CPAN can continue to be the easiest way to reliably distribute open source code.

8 Comments

I hate writing (or copy & pasting) POD boilerplate. I've loved using Dist::Zilla's tools for eliminating the need for such boilerplate, and I'd love to see those tools made more robust.

This I think has the biggest bang for the buck to the current Perl(5) community. Developing more tools to make documentation easier can only help things improve in the long run, and Ricardo has a proven track record of developing useful and well build modules. If he's crazy enough to step into the world of POD, I say all we can do is say "Go forth brave warrior, we'll tell your children stories of your heroisim."

This definitely has potential, and I trust rjbs to write something sane. I think there's a lot to be said for funding people whose previous work is well-known to the community (but I would say that, wouldn't I?)

POD looks deceptively simple, but is really kind of strange when you dig down into it. It'd be nice to have better tools for POD munging (not just to make pretty docs, but for detailed manipulation). A DOM or SAX style interface would be nice if that's not too much trouble :)

And I agree with Chris Prather that Ricardo is good for crazy details.

There are many modules on CPAN to do with POD. How would this compare to Pod-POM "POD Object Model", for example?

There are a few key differences, at least. The most important, for me, is the fact that POM is more tied to "standard" POD commands and a single set of nesting rules. Pod::Elemental makes fewer hard and fast assumptions: you start with a stream of elements rather than a tree, and your rules for nesting them are mostly up to you. After all, while it might seem sensible that a "=over" after a "=head2" should be inside it, you might disagree. This becomes much more important when you extend the commands available to add things like "=method" or "=dialect"

Pod::POM's handling of unknown commands is, at least to me, confusing, though I didn't spent too long trying to understand it exactly. The tying together of views and parsing is where I started to feel this wasn't right for my work.

It's also built atop Pod::Eventual for getting the event streams from which the element streams are built, making it easier to replace that component as well.

One shortcoming (or, rather, one feature I've excluded as unneeded for my applications) is support for embedded sequences like B>. I may eventually add them with another layer, but I don't see a need for it, and it would certainly not be done as part of this grant work.

Finally, Pod::Weaver, which sits atop ::Elemental, is an entirely different animal. It is meant to slice up and rearrange or add to POD trees. It needed a flexible tree builder to be written to perform its goals, especially command types.

Sounds interesting, but I thing we need a simple example(with apples) of what are you intended to do).

I recommend perl community to see the slides

http://www.slideshare.net/rjbs/distzilla-presentation/

and form your own opinion.

About TPF

The Perl Foundation - supporting the Perl community since 2000. Find out more at www.perlfoundation.org.

Recent Comments

  • quevlar: I recommend perl community to see the slides http://www.slideshare.net/rjbs/distzilla-presentation/ and read more
  • quevlar: Sounds interesting, but I thing we need a simple example(with read more
  • rjbs.manxome.org: There are a few key differences, at least. The most read more
  • Leon Brocard: There are many modules on CPAN to do with POD. read more
  • mpeters.myopenid.com: POD looks deceptively simple, but is really kind of strange read more
  • autarch.myopenid.com: This definitely has potential, and I trust rjbs to write read more
  • Chris Prather: This I think has the biggest bang for the buck read more
  • hdp: I hate writing (or copy & pasting) POD boilerplate. I've read more

About this Entry

This page contains a single entry by Alberto Simões published on February 6, 2009 5:25 PM.

2009Q1 Grant Proposal: Interactive Perl Interpreter was the previous entry in this blog.

2009Q1 Grant Proposal: Awareness and Utilization of Perl in Nigeria: A Survey is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

OpenID accepted here Learn more about OpenID
Powered by Movable Type 4.38