I think Test::Stream is an important project for the Perl community, and I think this is a great grant to fund.
As much as I appreciate the intent to "Take Test::More forward", I'm still very fearful about the code not having anywhere near as much oversight as it needs.
That is, very few people are actually reviewing the code.
I myself attempted to review it, and I was constantly at a disadvantage due to the pace.
As such, I feel uncomfortable with the plan as-is, because it doesn't make any guarantees about quality.
And we need a more comprehensive strategy that focuses on the review aspects of the code before we throw it into core and unleash it on the world.
Yes, we've got testing, and testing for ages, but testing only compliments code review, it does not replace it.
The rate of change in the codebase to meet the quality required to actually transition needs to slow down, and basically stop for anything other than pure bug fixes and code-review concerns for a long time before we underwrite the universe with it.
I'm not opposing any funding, but I do feel the absence of any kind of review plan for something this important is planning to do the least important things and ignoring the most important.
Sadly, we can't just magically create reviewers who are willing to review the code and dissect it.
I am concerned to see "Test-Stream in bleadperl" as a deliverable for this grant, as there are prerequisites to that that aren't necessarily covered in this grant.
I would prefer to see the project split up into more components, including a review of the proposed API, as well as an architectural review of the implementation. As it is, the Test-Stream distribution has too great a scope to be integrated into core, and should be split into multiple distributions.
Kent's comments are significant, but I still think the project should be funded. Integration into Perl could be delayed to delivery of stable code.
I'm already using Test::Stream and find its error reporting style extremely useful.
Chad's work is a much needed improvement to the existing test infrastructure. This is especially evident in the mechanisms for testing the test modules written with Test::Stream. Yes the project needs stabilization and oversight but those are things that are part of the objectives of the grant. Of all the module-level projects I've seen come past this committee process, I think this is among the most clearly worth funding and supporting in other ways that this grant can facilitate.
there seems to be a misunderstanding on what 'in perl blead' means. 'in perl blead' in this case does not mean in the next version of perl to be released. It means the code is put into the blead branch so it can get heavy smoking, and lots of eyes on it looking for problems. This has already been done once before, last time it was deemed not ready, so it was not put into the next version on perl. That can happen again.
The goal in this proposal is to go through the process of getting this checked again now that it is more mature.
I'm very concerned about backward compatibility. Something with changes as large as "I'm removing cmp_ok() and changing behavior of is()" seems pretty large. I know that that's changed a bit, but my point is still there.
This is great work, but the chances for breakage are huge. I agree with Karen's direction of "don't make blead part of this grant" while still funding the work.
Andy's comment highlights some confusion. I am not removing or changing anything in Test::More. What I have done is written a new testing infrastructure that solves the problems of Test::Builder. On top of this new infrastructure I have built a new toolset that is intended to be a successor to Test::More, which means it has differences, some tools are new, some are gone, and some work different. The key is that these differences are in a new namespace, they do not effect Test::More.
This grant request is for making Test::Builder use the new infrastructure I have written. This work is important for 3 reasons, 1) it allows tools built on both infrastructures to work together. 2) it allows old tools to take advantage of the new benefits. 3) being in blead ensures it gets more testing on more platforms to ensure nothing breaks.
If the perl blead work is removed from the grant then there is effectively no grant request anymore. I already have a dev release on cpan with the Test::Builder guts altered. All I have left are putting it into blead, fixing bugs that finds, and completing the punch list, which contains several items requiring a merge into blead (or at least the patch, if not an acceptance of the patch). For example, I need to be sure there are no significant test output differences between blead with and without my changes, that requires a complete and working blead patch. Same for the punchlist item asking me to ensure performance is acceptible.
There is fear about letting my work into blead. An earlier version of the work was already put into blead. It was removed before a stable release because it was not ready. Merging into blead unblocks a lot of important work, but in no way forces anyone to release it as core next perl release.
Test-Simple is one of the core pieces of CPAN and has been lagging behind for a very long time. Chad has been pushing it forward and he does it openly and transparently.
I think this is one of the most important grants we can provide that has a net-worth for Perl. It affects the stability of CPAN, its testing spirit, the ability to move forward and to address problems we're already trying to work around.
I would like to see this grant approved.