Grant Proposal: IO::All Redux

6 Comments

We have received the following grant application "IO::All Redux". Please leave feedback in the comments field by November 25th, 2014.

IO::All Redux

  • Name
    • Ingy döt Net
    • David Oswald
  • Amount Requested

    USD $4,000

Synopsis

Rewrite IO::All to match the needs of 10 years experience.

Benefits to the Perl Community

IO::All is a Perl module that attempts to make all Input/Output operations in Perl, as simple, normal, and uniform as possible. IO::All has been in existence since 2004. It is popular, useful and somewhat extensible, but has a number of inconsistencies, flaws and misgivings.

The proposal is to refactor andor rewrite the IO::All code to become the perfect way to encode almost all of Perl IO needs and file system needs. In other words:

  • All things are done safely
  • Unicode issues are all handled correctly
  • Options have the proper scope (lexicalpackageobject)
  • Can be used safely in production
  • Consistent method names (match prior art when possible)
  • Stays simple for 1-liners (small needs)
  • Internals are all plugin based rather than semi-hardcoded
  • All operations are checked for errors
  • Defaults are sane
  • Mo(o(se)) compatible OO
  • Weirder stuff like operator overloads will be off by default

IO::All has a community of maintainers on irc.perl.org/#io-all. These ideas are the result of working with them and talking about a rewrite for many years.

Deliverables

Using the Alt module strategy, we deliver an Alt-IO-All-Redux dist that installs a new IO::All. This is tested in beta until deemed safe to release properly as the new IO-All dist.

Current IO::All plugins are revised to the new strategy.

Project Details

See:

The style of the Inline Grant will be continued:

  • Regular Blogging/Tweeting
  • Public Pair Programming

Inch-stones

  • Create Alt-IO-All-Redux dist so that new code can be used and tested by the community immediately
  • Write tests for new behaviors
  • Refactor the code to match the new design
  • Port current IO::All extension modules to new design
  • Get people on #io-all IRC channel and beyond to test the rewrite
  • Release as IO-All when deemed ready

Project Schedule

This project will take 3 months and can be started immediately upon acceptance.

Completeness Criteria

Alt-IO-All-Redux passes all tests, is vetted by the community and is released to CPAN as the new IO-All dist.

Bio

Ingy döt Net wrote IO::All in 2004. Of his many CPAN modules, it is probably one of the 3 most popular.

David Oswald has been a Perl user for over a decade, is an author of several CPAN modules, and maintainer of more. David also runs Salt Lake Perl Mongers.

Ingy and David work well together and have decided to collaborate on a number of projects that will benefit Perl and Software Development.

6 Comments

I'm finding it hard to evaluate this, so I'll just list the thoughts that occurred when reading it:

  • $4000 is a non-trivial amount of money
  • The bullet list in the Benefits section is hard to argue with, but I don't get any sense of how well the existing distribution meets these.
  • How many dists rely on this dist? MetaCPAN says 96, and my slightly-out-of-date database says just over 400 total dependencies (direct + indirect).
  • What are the problems that the dependent dists are having that necessitate a rewrite?
  • Is there anyone saying they'd like to use IO::All in production, but feel they can't?
  • Pretty much every dist of mine on CPAN, when I look at it I think "*cough* that could do with a rewrite", but actually they're fine for the intended job, and could just do with some refactoring over time. Why is that not the case here?
  • Would be good to see some comments from current users of IO::All (the ones who aren't INGY :-)

Not trying to be arsey, but was having a hard time expanding on my initial thought of "Hmm..."

I've done some work on the guts of IO::All as well as added a variety of features lately. I'd really like this to happen and think that it would be a good benefit for the community in general.

Full disclosure: I'm a sometime-contributor to IO::All, and a fulltime user of it.

I am entirely of the opinion that this grant should be granted, since IO::All is the io library Perl needs.

Neil, to answer your concerns:

> The bullet list in the Benefits section is hard to argue with, but I don't get any sense of how well the existing distribution meets these.

IO::All is fairly old, and was mainly written with an eye to making the API work right. For the bullet points it fulfills pretty much 90% of each, but to get the remaining 10% necessitates large-scale refactoring of the guts of IO::All.

> How many dists rely on this dist? MetaCPAN says 96, and my slightly-out-of-date database says just over 400 total dependencies (direct + indirect).

I don't know about CPAN, but i do know that i use it in every piece of code of my work where i'd otherwise need to directly do an open(), which is a lot of production code; and i mandate junior devs working on projects with me use it, which usually results in their code being more reliable and more readable.

> What are the problems that the dependent dists are having that necessitate a rewrite?
> Is there anyone saying they'd like to use IO::All in production, but feel they can't?

A few points that keep others from using it, based on feedback i got by talking to people on IRC:

- homebrew OO system
- magic: automatic type detection when doing io($thing)
- magic: overloads of things like > and
- reliability and consistency: it does a lot of things, and most of them die usefully when something goes wrong, but nobody ever went through all of them to ensure they all do
- unicode: same as above

> Pretty much every dist of mine on CPAN, when I look at it I think "*cough* that could do with a rewrite", but actually they're fine for the intended job, and could just do with some refactoring over time. Why is that not the case here?

As mentioned, in this case refactoring means replacing all of the base code while keeping the API mostly as it is. There simply is no way to do this piece-a-meal.

There's been discussion on and off about rewriting IO::All to be cleaner and easier to extend. The intent, here, is to provide the swiss army chainsaw of IO modules, i.e. "what you use when Path::Tiny is more minimalist than you had in mind", which IO::All largely already is for my purposes, but for it to be so for people in general.

So, many of us are already using it in production, but believe that a rewrite to be more consistent, clear and comprehensible, and to make it easier for a true ecosystem to form around the distribution, would be a significant net win.

It'd be nice to be able to finally kill uses of e.g. Path::Class with a clear statement of "for simple stuff, there's Path::Tiny, for complex stuff, there's IO::All", and I believe this rewrite will put us within reaching distance of that.

I have to second what Neil said. There is potential here, but the proposal is too vague in both its problem definition and in its suggested solution.

"All things are done safely" and "Unicode issues are all handled correctly" sounds great, but what does that really mean? For example, are you planning to change the interface to make things safer and saner?

Generally I do recognize the list of issues I'd like to see fixed that Christian mentions.

Makoto asked me to write a followup reply to these comments.

I/O is one of the most core features of a programming language. I think it is a decent analogy to say that IO::All wants to be for open() what Moose is for bless(). Having a simple and consistent way to do all I/O and filesys operations is a good thing for Perl. IO::All is a great start, but it was done as a proof-of-concept in 2004, and needs to be redesigned. The thought on how to do it and release it in a way that doesn't break existing code has happened and been discussed on #io-all IRC channel for years now. I linked to a design overview in the proposal.

To reply directly to some of the points raised by neilb and leont:

1) Based on the experience of the Inline grant and the known amount of work and thought to do this, the pay rate of $4000 will come in at much less than minimum wage. In this case, the grant is a carrot to get something really good finished for Perl, and the rate is as low as I could possibly justify for other parties interested in my time.
2) Dists depending on IO::All is a wrong comparison. Almost all my Perl scripts use it. Few of my modules do, as I try to keeping dependencies low on my dists.
3) While not a complete spec, https://raw.githubusercontent.com/ingydotnet/io-all-pm/master/note/Design.swim is hopefully not _vague_.

I hope this helps. I'd like to thank mst, frew and Mithaldu for their supporting comments and work on IO::All over the years. More than getting this grant accepted, I really wish personally that this work was already done. I'd enjoy using it in my Perl coding every day.

Leave a comment

About TPF

The Perl Foundation - supporting the Perl community since 2000. Find out more at www.perlfoundation.org.

About this Entry

This page contains a single entry by Makoto Nozaki published on November 15, 2014 9:00 PM.

Grant Reports for October was the previous entry in this blog.

November 2014 Grant Proposal is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Pages

OpenID accepted here Learn more about OpenID
Powered by Movable Type 6.2.2