2008Q2 Grant Proposal - Test::Builder 2
Thu, 01-May-2008 by
Alberto Simões
edit post
<![CDATA[
* **Name:** Michael G Schwern
* **Project Title:** Test::Builder2
* **Synopsis:** Write a new, incompatible version of Test::Builder (in a new namespace) to improve the architecture and support desired features which Test::Builder does not or cannot. Make sure that both Test::Builder and Test::Builder2 based testing modules can work together.
**Name:**
Michael G Schwern
**Project Title:**
Test::Builder2
**Synopsis:**
Write a new, incompatible version of Test::Builder (in a new namespace) to improve the architecture and support desired features which Test::Builder does not or cannot. Make sure that both Test::Builder and Test::Builder2 based testing modules can work together.
**Benefits to the Perl Community:**
Test::Builder underpins 80% of the tests on CPAN. http://www.nntp.perl.org/group/perl.qa/2008/01/msg10172.html
Its limitations become everyone's limitations. It's done a very good job adapting the last six years, and testing has become more sophisticated in that time, but age and backwards compatibility holds things back. There are a number of desired features which Test::Builder cannot support, such as end-of-test actions, without radically altering how tests are built.
Another desire is to make Test::Builder "subclassable" to override its behaviors. This is currently not possible and a simple subclass is unsafe. A more sophisticated system will be necessary and requires some careful thought.
**Deliverables**
* Split up the Test::Builder object into an aggregate of objects.
* Split up global shared Test resources into individual objects
* The test counter
* The output filehandles
* The plan
* ...
* Split up localizable behaviors into objects
* Allow individual test modules to locally override Test::Builder2 behaviors
* Allow test modules to globally override Test::Builder2 behaviors
* How the plan works
* Allow hooks for global beginning and end of test functions.
* Ensure multiple hooks "stack"
* die on fail
* debug on fail
* ...
* Hooks for global beginning and end of test actions
* Example: A safer Test::NoWarnings
* Example: Don't cleanup intermediate files on failure so they can be debugged
* Allow for test output other than TAP
* Allow another Test::Builder-like module to work in the same process as Test::Builder (for example, sharing the counter).
* Rewrite Test::Builder in terms of Test::Builder2.
**Project Details**
Much can be learned from chromatic's Perl 6 implementation of Test::Builder.
Because this would be an incompatible API change, it would have to go into a new namespace. Probably Test::Builder2.
There's a good chance much of the changes having to do with turning Test::Builder2 into an aggregate will be able to be backported to Test::Builder. Some are necessary, such as sharing single plan, output and test counter objects.
**Project Schedule**
I will be availble to work on this project after June 20th.
There are three major stages...
* Test::Builder2 alpha release
* Test::Builder2 stable release
* Test::Builder ported to Test::Builder2
Much of the work on Test::Builder2 will be figuring out what people want to do with it and what the API will be. The code itself is not that hard. An alpha release can be coded quickly to begin getting feedback and have the API worked with.
Stabilizing the API and feature set will require a long iterative feedback process with test module authors and many alpha releases and much discussion. This is necessary so as not to wall off the Test::Builder2 API into supporting a bad design decision. As such, there is a long period built into the schedule before the first stable release.
Porting Test::Builder to Test::Builder2 is the final declaration that Test::Builder2 is stable and usable. This effectively "throws the switch" moving CPAN over to Test::Builder2.
* First alpha release: Two weeks from the start
* Feature complete alpha release: One month after first alpha.
* First stable release: At least two months after complete alpha.
* Feature complete stable release: At least one month after first stable.
* Test::Builder -> Test::Builder2: At least one month after first stable.
If the project starts June 20th, the "at best" schedule is...
* First alpha release: Beginning of July
* Feature complete alpha: Beginning of August
* First stable: Beginning of October
* Feature complete stable: Beginning of November
* Test::Builder -> Test::Builder2: Beginning of November
I must reiterate, most of the time in the schedule is to gain feedback, form use cases and allow the API to gel.
**Bio**
I wrote Test::More and maintain Test::Builder. I've been at the center of Perl testing for the last six or seven years.
I work as an independent contractor and have cleared my summer schedule to be able to provide at least 20 hours of work a week on TPF grants.
**Amount Requested**
$3000
]]>
Comments (3)
This is also crucial work to allow the average Perl user to take advantage of the significant advances made or coming-soon in TAP. Totally worth it.
Yes, yes,yes! I can't say how useful this will be for the advancement of Perl testing. And who better to do it than Schwern?
This particular project is perhaps my highest priority (though Perl-On-A-Stick is pretty awesome). I am repeatedly hit with issues surrounding Test::Builder. Several times I've had to monkeypatch it to get interesting things done and I've met quite a number of other people who do very large-scale testing who need the sort of fine-grained control you simply can't get with Test::Builder.
Additionally, to really get proper "enterprise support" for testing, the new features available in Test::Builder 2 are crucial.
