+1
I'm sure that diakopter is the right person to do it.
I am a long-time user of Perl 5, and have followed Perl 6 for all these years.
The latest progress on Rakudo Perl 6 is tremendous. This proposal by diakopter is the Grand Unification - if he do it, even if it takes longer than 2 months, it will be money well spent.
+1, because this is IMO The Only Way To Do It(TM), even if that does not sound very perlish. My set goal is it to use Perl 6 at work this year, and with his work, I am sure, this can be done.
+1
This proposal, if well accomplished, is going to be of tremendous help for Perl 6 to become useful more quickly, and for Perl 5 to remain relevant for a longer time.
+1
diakopter has done some seriously good work on moarvm (about 700 commits so far), and has also put serious thought into the matter, so he is about as qualified as one can be for such a task.
I don't think I have to explain to anybody how much both p5 and p6 would profit from such a thing if done right, so I'm all for it.
+1
This is a very exciting proposal.
The Perl6 effort on the JVM has been fantastic to see and now we can have Moar! :-)
Perl5 interoperability will enable companies to build upon their investment in Perl 5 and provide an adoption route for Perl 6.
Go for it diakopter++.
This proposal will see an important and needed task tackled, and I think it comes at a good time. Rakudo's architecture has stabilized a good deal by now, and this comes at the point where MoarVM is in a sweet spot of supporting non-trivial things, but still early on enough to be malleable in the face of this project.
I know that diakopter++ has spent a lot of time considering and researching this topic ahead of making a grant application. And, as the bio states, he has also been involved with MoarVM from the start and responsible for much of what exists today. Overall, there's nobody in a better position to do this work than diakopter.
So, a +1 from me. I'd like to see this work encouraged through funding it, and of course would be delighted to see it succeed. I will be happy to provide any needed input as it proceeds too!
Wooooow that is a lot of money! I have not been looking at the Moar VM for very long, but does it run any Perl-6 code right now?
Does it make more sense to delay this proposal until the Moar VM has proven that it can run Perl-6 code well? That is a lot of money to spend if the Moar VM does not work out very well. Maybe there can be code that someone can salvage in that case, but it still seems like an expensive thing to figure out.
I know my opinion does not matter very much but I am worried that spending all of this money and time right now on something that is unproven will be a big risk. Does it make sense to use the Parrot VM or the Java VM instead, because those can both run Perl-6 code right now?
If there was any project I'd want the Perl community to fund, it's embedding libperl in moarvm, and diakopter looks to me like the right guy to build it out.
But then again, I'd want to see him or other folk incentivized to do any and all work that needs a serious block of focused energy to more quickly get to a solid 6.0.0. I'm concerned that the $100k of Hague money intended to get P6 to 6.0.0 is about spent and we're still, imo, at the current rate of progress, more than a year from 6.0.0. (Fwiw I've thought and said that since 2009, and I'm currently thinking it's more like 2 years away. Imo the only thing that can accelerate it is broad community awareness that several community leaders say it's time to accelerate it.)
I'm thinking there could be a crowdtilt campaign with matching funds from the TPF. Maybe triple matching (one dollar from the Hague dev fund, one from the Hague marketing fund, and another from the general p5 fund). Maybe crowdtilt themselves would contribute another dollar, making it 5x. (Maybe more perl using corporations would chip in to take it even further. Set a cap of $1k max per corporation and I can see 5 stepping in to make this a 10x campaign.)
Then if the campaign can raise $2k from individual contributions (or $1k if there's 10x), we've got the $10k. (Maybe anything in excess of this target flows back to TPF's general fund, so TPF might even come out of this with more money that it starts off with.) This way the TPF and the perl community can send a very strong message of upcoming reunification, and it will have only cost the TPF $4k at most, and the Hague dev fund just $2k, and the perl community can demonstrate support for upcoming P5/P6 reunification.
The JVM efforts and bringing the CPAN content's forward are a breath of fresh air. The kind of work that needs to be done seems hard, and the risk of not supporting efforts like this is bigger than the investment.
Carlos,
Your opinion does matter, and your question/objection is a good one.
MoarVM's future is definitely unclear if you're unfamiliar with the current status of Parrot (especially compared to MoarVM). One thing contributing to this confusion is the fact MoarVM was only recently revealed, when it has in fact been under *intense* development for more than a year prior to that. Here are some facts about MoarVM and Parrot's status w.r.t. Rakudo that should promote some of this tribal knowledge to general awareness.
1. First, MoarVM has many more active developers than Parrot. The last remaining active Parrot developer has publicly said he wants to take Parrot in a new direction (notably, one that doesn't support Rakudo Perl 6). MoarVM has at least 4 extremely active contributors, and another several active contributors.
2. Second, Parrot has technical limitations preventing further expansion of Rakudo Perl 6's capabilities using the Parrot backend, which preclude it from being a good platform on which to prototype Rakudo's libperl interop.
a. Parrot's concurrency features (in their current form) are devastatingly unusable by Rakudo:
1) Its barriers (to intercept accessor and mutator PMC opcalls and redirect those calls to the thread that allocated the PMC) are only on the PMC v-tables, and not on the 6model v-tables, which are what Rakudo uses, so that mechanism would need redesigned in light of 6model, of which the author of Parrot's threads clearly wasn't aware.
2) Currently, only parrot's main (first/original) thread can spawn new threads, and its child threads cannot interact with each other via shared objects, so objects returned to the main thread from child threads should not be accessible from another child thread [... but there is no built-in protection against this]. Claims this is easy to fix are less than veracious.
3) Since only the allocating thread can access or mutate its shared objects, no object can be accessed by more than one thread at once, so there can be no such thing as lock-free shared objects, since every object is thread-safe, due to requiring a lock for all operations, including reading! However, it is even worse than this: all objects created by the main thread *share the same lock*, since only one of them can be accessed at once. This manifests practically as a limitation that if you want to have multiple child threads utilize shared objects, the only work that can be done concurrently is work that doesn't access any shared objects. This relegates Parrot threads to the same effectiveness domain as JavaScript's web-workers.
4) Parrot's garbage collector has no way of knowing when a given object can be freed, because it does not know whether or not it has passed that pointer to another thread (deeply accessible/nested), if there are no more references to that pointer in its own thread. Parrot's thread design attempts to mitigate this by keeping track of which objects it *explicitly* sends to other threads... but it fails to keep track of any objects reachable from *that* object (at the time of passing it), if that pointer is a collection/container object with links of its own, but then the link to the other object was broken, but is still held as a reference by another object, in that other thread. This effectively means that no memory can ever be safely freed by Parrot, and thus it has no garbage collector. How someone thought one could invent a safe concurrent garbage collector with only a few hours work is pretty much unfathomable, especially when one is discovering an entirely unheard-of method of doing something that normally takes thousands of hours of work by professionals with decades of related experience, instead of being designed for a junior undergrad semester project.
b. Parrot has no asynchronous I/O, and no one has any plans to incorporate it, contrary to the Parrot Design Documents, which are written as if it already exists. MoarVM has resources committed to furnishing this by this year.
c. Parrot has no knowledge of anything having to do with signals (and again, no one has claimed they are available, willing, or able to provide them); MoarVM will have them soon.
d. Parrot has no interaction with (or control of streams of) child processes, except that it can read stdout from a child process (and again, no one has claimed they are available, willing, or able to provide anything beyond this). MoarVM has somewhat more control (via APR currently), but when APR is replaced (probably by some variant of libuv), we'll enable capabilities equivalent to Perl 5's child process extensions.
e. Parrot has no atomic operations; MoarVM does.
f. Parrot's potential for optimizing Perl 6 regexes is very limited due to its reliance on libicu and its limited API. MoarVM already has its own implementation of the Unicode Character Database, and so doesn't have this limitation.
g. Parrot has no JIT (and again, no one has claimed they are available, willing, or able to work on this). MoarVM plans one within a year (and again, more than several qualified resources are lined up to do it).
h. Parrot takes ages to build compared to MoarVM (and not because it provides more features), which constrains its developer productivity dramatically.
3. Even parrot devs (3 recently core/chief ones, afaik) have said MoarVM can be viewed as a continuation of Parrot's ideals and goals... implying they view Parrot's ability to further those goals has reached its end [recently, if not a long time ago].
4. At this moment, MoarVM passes over 90% of NQP's test suite (NQP defines the opcode set for Rakudo backends and a Perl-6 like language with many of its features). The JVM backend accomplished this only a few short weeks ago, and now passes over 99% of the Perl 6 test suite that Rakudo on Parrot passes, which bodes well for MoarVM's ability to catch up quickly.
As stated in the grant application, much of the work is already applicable to the same features on the JVM backend; in fact I considered proposing funding that instead of using the MoarVM backend, but I chose MoarVM when it became clear Rakudo would likely be caught up in time.
Regarding the amount of money, it is about 3/4 of the remaining uncommitted funds and 1/2 of the remaining total funds. Much (or all?) of the $100k portion of the Hague Grant was for fund-raising activities, and that still remains. Do you have any ideas for how to spend that, other than hiring someone full-time for a year to fund-raise (which was previously proposed and rejected)? [I can't think of anything right now, but I can think of plenty to do once Perl 6 has a marketable product.]
I appreciate your question; I hope this answer has helped clear up some of the cognitive dissonance.
LOL you didn't answer my question! You gave a laundry list of things that Parrot doesn't do and that you hope the Moar VM will do in the future.
I have also embedded libperl, and I didn't need asynchronous I/O, JIT, or threads to make the design work. Maybe I will offer to do this work for $8000 USD and save the TPF some money.
Carlos,
$10000 isn't that much. If what you really care about is saving TPF some money, you may could sponsor TPF $2000, or you can do this interop work for free. Note that the work will be not that easy(mostly will be harder), because that need many changes to MoarVM itself in a spiral development mode, that means diakopter++ may spend more time than two months. Actually, if things go well, the value of the grant work will be much more than $10000.
"you didn't answer my question" - I answered each of your three questions:
You asked whether it runs Perl-6 code; I said: 90% of NQP's test suite, and NQP is a large subset of Perl 6.
You suggested waiting until MoarVM runs Perl-6 code well; I answered with an estimate of when it would catch up to Rakudo-JVM. But even if it didn't, did you forget "run Perl 6 code well" is included in (a prerequisite of, in fact) the grant being completed, and that TPF doesn't pay the grant until it's completed? There have been plenty of proposals that were accepted, but not completed, and therefore not paid.
You suggested it would be better to use the Parrot VM or the Java VM instead, so I explained why Parrot wasn't a good choice (since Rakudo is blocked from progressing on it), and also why I didn't choose the JVM.
"I have also embedded libperl" - If you embedded libperl in a Rakudo implementation, with the degree of interoperability proposed, and you're willing to contribute it, feel free to submit a pull request; we'll gladly review it.
"and I didn't need asynchronous I/O, JIT, or threads to make the design work." - I didn't say I needed those things to make the design work. I was saying Parrot has no plan to implement those things, so its long-term viability as a Rakudo backend is in question, so it's not a good choice for this project.
+1
I have followed MoarVM since before it was revealed, and also been aware of diakopter's plans to get good, reliable interop on top of the VM. I know he has spent a lot of time thinking things through, discussing with various other people, and planning his approach.
I believe diakopter is the person to do this work, and I am excited to see this grant proposal. It has my full support.
10.000K for blitzkost on MoarVM? Why not. If we have the money, use it.
But beware that it will not lead to a faster perl5, just better p5 integration into p6. As blitzkost did for parrot/rakudo.
There are several false points in the discussion/limitations on parrot threads, but I'm happy that the current opinion on parrot threads are finally public somewhere. And this has nothing to do with this proposal.
+1
+1
Making this work would remove a big barrier to greater Perl6 adoption, and I heavily support the concept.
John Napiorkowski
+1
I think the proposed schedule is optimistic, but that's certainly no reason to preclude funding this proposal.
And if anyone can do the work, it's diakopter. He has the knowledge and experience necessary to accomplish the goal of attaining perl 5 interoperability on moarvm. So, again, +1 for funding.
Whatever you say dude! I think you told me why you want to work on your vanity VM instead of working on something that already works.
I think I liked this proposal better when it was called Blizkost, but I can see now that Perl-6 is one long farce of throwing away projects before they are done because it is always more fun to start something new. I am glad it is not my money TPF is going to throw away here.
I think that this work is worth funding.
I am somewhat troubled that as-is the grant proposal will mean that diakopter might not get paid despite delivering code that meets all the goals. That would be silly, and should be avoided.
Firstly, is the intent to pay all $10,000 when the project completes? Or is the plan to have a halfway payment of $5000 after (projected) one month's work? Because I'm doubtful that the 16 items listed on "feature details/schedule" are of equal size, and in a linear order. Hence I don't think that items 8 and 9 mark the halfway point. What does?
Secondly, is it specifically required to support Perl 5 all the way back to 5.8.3 to complete the grant, or is that just a "nice to have"? There have been a lot of bugs fixed in the 9½ years since 5.8.3 was released, and it may turn out that some of them are essential to making interworking work. I remain of the opinion that it's more important to get completeness, correctness and breadth of support on current Perl 5 (be it 5.18.x or even 5.20.x) than divert resources to working around bugs and problems in older versions. Support for older versions of Perl 5 is nice to have, but shouldn't come at the cost of reducing functionality on current versions.
I don't think that restricting support to more recent versions will hurt uptake. Until distributions pick up MoarVM to package, people wanting to use it will have to build it from source, which means that they are also capable of building a current Perl 5 from source. By the time MoarVM is packaged by distributions, those distributions will also be packing a current Perl 5.
Thirdly, are all the 16 detailed features required to be delivered and working in order for the grant to be paid? If it turns out to be too hard to get some working correctly in the timeframe, but the rest of the features are working and the project delivers production grade interoperability suitable for a wide range of uses, has the grant failed? I think not, but it's not clear from the proposal how to assess this.
Is there an acceptance test for the project? Something like having a web framework running in Perl 6 accessing databases via DBIx::Class and DBI running on Perl 5? Calls from Perl 6 into methods written in Perl 5, passing callbacks to Perl 6 routines (and vice versa). Demonstrating Perl 6 classes derived from Perl 5 classes in turn derived from Perl 6 classes?
Finally I think that MoarVM is the right VM to target first. As both MoarVM and the Perl 5 interpreter are written in C, it will avoid additional problems in having to bridge from Java to C, which would complicate targeting the JVM. I don't think that targeting Parrot first is a good idea, as it's not clear whether Parrot will even be supporting Perl 6 in the future. See the thread starting at http://lists.parrot.org/pipermail/parrot-dev/2013-June/007489.html (Although it must be noted that to date there is no consensus on the direction Parrot will take.)
+1
I feel strongly this is an important direction for Perl 6, and have a great deal of confidence that diakopter is the right one to pull it off.