Grant Proposal: Standardization, Test Coverage, and Documentation of Perl 6 I/O Routines
Tue, 17-Jan-2017 by
Makoto Nozaki
edit post
<
found in `src/core/io_operators.pm`,
`src/core/IO.pm`, `src/core/IO/ArgFiles.pm`, `src/core/IO/Handle.pm`,
`src/core/IO/Notification.pm`, `src/core/IO/Path.pm`, `src/core/IO/Pipe.pm`,
`src/core/IO/Special.pm`, and `src/core/IO/Spec.pm` as well as its
subclasses found in `src/core/IO/Spec/` directory. Note that `IO::Socket` and
its subclasses are NOT in the scope of this grant proposal.
Deliverables:
- I/O Action Plan Report
- Commits in [rakudo](https://github.com/rakudo/rakudo/) implementing
the Action Plan.
- Commits in [Perl 6 Specification](https://github.com/perl6/roast)
providing full coverage for I/O routines.
- Commits in [Perl 6 Documentation](https://github.com/perl6/doc)
providing full coverage for undocumented I/O routines, as well as any
corrections for existing I/O documentation.
- Bonus deliverable: The "Map" of Rakudo Routines
- Bonus objective: whenever possible, I intend to fix any of the bugs
found by the new test coverage. Should a bug prove to be hard to fix,
the test exposing it will be fudged and a ticket for the bug will be filed.
## Project Details
- (Bonus deliverable): __The "Map" of Rakudo Routines__
This will be semi-automatically generated (by introspection) list of all
subroutines and publicly accessible object methods provided by Rakudo
implementation, together with information on what calling convention they
use and how they fail (e.g. returning some object, throwing, or returning
a `Failure` object). This data will guide the decision on how the calling convetion/failure mode for I/O routines should be standardized. I intend
to work on this deliverable while this grant proposal is deliberated. As
such, it's a bonus deliverable and will be completed regardless of whether
this grant is approved.
- __I/O Action Plan Report__
This will be a Markdown document detailing how the existing routines will
change, what the new routines will do, if any are to be added, and whether
any currently unspecced routines are to be removed. Parts of this document
will eventually be re-used as documention for currently undocumented
routines.
- The scope of all of the changes will be limited by the _6.c_ Perl 6
language specification, so none of the changes would have to be deferred
until _6.d_ language release.
- This document will be placed into
[rakudo](https://github.com/rakudo/rakudo/) repository when ready and
other core developers will be asked to reivew it and make comments on the
proposed changes.
- __Commits__
The commits to rakudo/roast/doc repositories will implement changes in
the Action Plan, and test and document any I/O routines that are still
left untested/undocumented after those changes.
## Inch-stones
These are the inch-stones I intend to follow:
### Routine Map
- Use Perl 6's introspection facilities to locate core subroutines and
classes in `CORE::` lexical scope. Further introspect those classes to obtain
`Method` objects. Further introspect the candidates and signatures of all of
these routines to obtain the calling convention list (e.g. we pass arguments
as adverbs or as positionals, etc)
- Further introspect the arguments and programmatically invoke the
routines with some probably-acceptable values to obtain the output range, in
particular failure modes. Some heuristics or manual massaging will be needed
here.
- Part of this work already exists in [a repo](https://github.com/zoffixznet/perl6-map). If the grant is approved, this repository will be
transfered to [Perl 6 GitHub organization](https://github.com/perl6/)
### I/O Action Plan Report
- Using the routine map, figure out a better interface for I/O routines
and then see what can be changed without breaking the 6.c language spec. Even
without that map, I already can see some targets for change: the routines that
take `$test` named arg as a predefined string of test letters in particular
order, as well as the inconsistency that most methods `fail()` while most
subroutines `throw()`. Based on a brief [conversation with
Larry Wall](https://irclog.perlgeek.de/perl6-dev/2016-12-28#i_13814034), the
likely change would be to make all of them consistently `fail()`.
- Inspect the [code of NewIO branch](https://github.com/rakudo/rakudo/compare/6bbb56f4c598ba0fef49ba9b11671df675019366...newio)
for any ideas that can be salvaged for 6.c Language. The work in this branch
was created by lizmat++ in 2014-2015, but due to unfortunate circumstances
never got merged to master prior to 6.c release. My understanding is it offers
a lot of improvements to our current IO, but based on brief perusal of the
code, I suspect it would have a lot of conflict with 6.c Language
Specification tests. Under this grant, I would like to see if any of the
improvements that work offers can be applied to current 6.c language.
- Using the [MoarVM coverage report
tool](http://perl6.party/post/Perl-6-Core-Hacking-Can-Has-Moar-Cover) (and by
grepping the [roast repo](https://github.com/perl6/roast/)), find
routines that are entirely unspecced and decide whether they
should stay or be removed. For example, the currently unspecced `indir()`
routine is a likely candidate for staying, while I've seen some calls on IRC
to remove the unspecced `tmpdir()`.
- I forcee the largest part of this grant work to be the writing of
the tests. Here are some examples of the rough current coverage state of IO
routines:
[src/core/io\_operators.pm](http://perl6.wtf/src_core_io_operators.pm.coverage.html),
[IO::Argfiles](http://perl6.wtf/src_core_IO_ArgFiles.pm.coverage.html),
[IO::Special](http://perl6.wtf/src_core_IO_Special.pm.coverage.html),
[IO::Spec::Win32](http://perl6.wtf/src_core_IO_Spec_Win32.pm.coverage.html),
and [IO::Handle](http://perl6.wtf/src_core_IO_Handle.pm.coverage.html). The
routines whose names are on red lines indicate they're untested in roast
(this report was generated on Oct 7 and as I recall only IO::ArgFiles.lines and
IO::Handle.seek received any further tests since then).
- A lot of the decisions made by the Action Plan will be able to close
many of the tickets opened by _brian d foy_ while gathering info on our
I/O routines that is going to be included in _Learning Perl 6_ book and the
concerns raised in the tickets will be used as input for the Action Plan as
well. (Some of them are:
[RT#130460](https://rt.perl.org/Public/Bug/Display.html?id=130460),
[RT#130456](https://rt.perl.org/Public/Bug/Display.html?id=130456),
[RT#130454](https://rt.perl.org/Public/Bug/Display.html?id=130454),
[RT#130455](https://rt.perl.org/Public/Bug/Display.html?id=130455),
[RT#130489](https://rt.perl.org/Public/Bug/Display.html?id=130489),
[RT#130490](https://rt.perl.org/Public/Bug/Display.html?id=130490)).
- Write down the action plan in Markdown format, make it available
in Rakudo repository, and invite other core members in `#perl6-dev` IRC
channel to comment on it.
- After a 1 week review period, update the Action Plan to reflect
any received feedback, and proceed to implement it.
### Commits
The commits will implement the Action Plan. Roast commits will be based on
the changes to routines as well as the report generated by the coverage tool.
And the doc commits will be done by manually searching and reading exiting
documentation and amending it as needed.
(N.B.: I'm aware the coverage tool is currently busted by a commit that changed
filenames in `.file` method for core routines; however MasterDuke++ promised
to fix it, and if they won't be able to find time to do so, the fix should be
simple enough that I'd fix the tool myself).
## Project Schedule
I already began work on the routine map generator and will complete it by
the time the decision on this grant proposal is available. After that,
I expect to spend 2 weeks preparing the I/O Action Plan Report, 1 week for its
review by other core members, and 2 weeks for its full implementation
(including tests and docs). I also allow for extra 2 weeks for any unforseen
delays in any of the steps.
If my understanding of [the date when the decision on this grant would be
ready](http://news.perlfoundation.org/2017/01/call-for-grant-proposals-jan-2.html) is correct, I intend to finish the work before the end of March, 2017.
## Completeness Criteria
- [rakudo repository](https://github.com/rakudo/rakudo/tree/nom/docs)
will contain the IO Action Plan document and it will be fully implemented.
- All of the I/O routines will have tests in
[roast](https://github.com/perl6/roast/) and documented on
[docs.perl6.org](https://docs.perl6.org). If any of the currently implemented
but unspecced routines are decided against being included in Perl 6 Language,
their implementation will no longer be available in Rakudo.
- The test coverage tool will report all I/O routines as covered and
the information will be visible on [perl6.wtf](http://perl6.wtf) (_Perl 6's
Wonderful Test Files_) website. Note: due to current experimental status of the
coverage tool, its report may still show some lines or conditionals untested
despite them actually being tested; however, it _will_ show the lines where
routines' names are specified as covered.
## Sidenote Comments
The grant amount requested may be low compared to the described amount of work
because I'd like to still view part of my time on this work as donated to
Perl 6. Those who see me on IRC may notice I bounced around the ideas in this
proposal before.
The grant will let me finish this work much sooner and in a more complete
state than I would be able to otherwise.
## Bio
I'm a 30-year old Canadian who lived near Toronto for the past 14 years.
I also spent some years of my life living in USA (New Jersey) and, before that,
in Russia (Siberia), where I was born.
I started with Perl 5 about 12 years ago and have since released over
200 Perl 5 modules on CPAN and for the last 10 years held a single job,
large part of which is web development with Perl 5.
In the fall of 2015, I switched my focus to Perl 6 and to date released 34
Perl 6 modules and delivered a couple of Perl 6 presentations at the Toronto
Perl Mongers meetings.
Around July, 2016 I joined the Rakudo Perl 6 Core Development Team. I also have
been Rakudo's release manager every month since the 2016.06 release.
My notable deliverables to the Perl 6 Community involve the creation of the
web app driving [modules.perl6.org](https://modules.perl6.org); nearly total
automation of the Rakudo's release process, including the development of
[perl6.fail](http://perl6.fail) web app for RT interfacing and release status
tracking; and writing all of the tutorials on
[perl6.party](http://perl6.party) website.
To date, I have authored __461 commits__ to
[Rakudo](https://github.com/rakudo/rakudo/) and __1,823 commits__ to
repositories in [Perl 6 GitHub organization](https://github.com/perl6/)
(__523__ of which have been to the
[Perl 6 Specification](https://github.com/perl6/roast/) repository).
]]>
Comments (15)
+1
I'm all in favor of this grant.
Coding with filesystem is always double work (check code and system ACL, locks, flows etc...).
It would be LTA for perl6 to let I/O and filesystem coding half way on the road.
Not to say the current I/O system is broken, it just needs polishing before problems pile-up.
+1 For Zoffix
Awesome! I'm all in favor of this.
One of the comments about Perl 6 that we often get at a booth / conference, by people who say it is a bit of a stumbling block for them before they would use Perl 6, is that I/O in Perl 6 needs to be improved, extended, better documented. If that happens thanks to this grant, I am all for it, so yes, ++.
Perl 6 should have some module ships with the "core" like Perl's File::Find::Rule which standarizing file traversing and making this task very easy and somthing fun to do .
I'm in favor of this grant .
+1
Now I guess I have to follow through on that promise to look at the coverage tool...
Gets my vote. I/O, both in test coverage and implementation, could very much use the careful attention this grant proposes, and Zoffix has established a track record of quality contributions to Perl 6.
+1
+1
Zoffix has been a very prolific contributor, and is the right person for the task. And yes, it needs doing.
+1
Awesome! +1 of course!
+1
Ditto what Moritz said.
+1
+1
And I second Moritz's comments.
