Grant Proposal: Perl 6 Performance and Reliability Engineering

20 Comments

Jonathan Worthington has submitted the following grant proposal under the Perl 6 Core Development Fund.

Before we vote on this proposal we would like to get feedback and endorsements from the Perl community. Please leave feedback in the comments or send email with your comments to karen at perlfoundation.org.

Name: Jonathan Worthington (jnthn)

Project Title: Perl 6 Performance and Reliability Engineering

Synopsis:

Improve both runtime and compiler performance of Rakudo Perl 6, with a focus on the MoarVM backend. At the same time, increase the robustness of the compiler, VM, and built-ins library.

Benefits to Perl 6 Development:

The Perl 6 Christmas release has eliminated "lack of a stable language" as an adoption blocker. The module ecosystem is steadily growing and documentation completeness improving, thanks to this language stability. Performance is now a key adoption blocker, and so improving performance is important to enabling Perl 6 to be used more widely. It's also important that this performance drive does not come at the cost of reliability. Rather, VM-level crashes - already relatively rare - should be further reduced, and high-impact bugs in the compiler and built-ins should be addressed.

Deliverable Elements / Project Details:

On the performance front, a huge range of improvements are possible throughout the stack - that is, in the VM, compiler, and built-ins. Of note, MoarVM has a fairly sophisticated dynamic optimization framework, but numerous optimizations have yet to be implemented since performance has not been the primary focus. Here are some examples of optimizations that I expect to work on over the course of the grant (including extensions):

  • More aggressive lowering of lexical variables to locals in Rakudo's optimizer, which can decrease memory use and enable/simplify numerous other optimizations
  • Making object accessors and object construction cheaper
  • Decreasing the cost of invocation (that is, calling subs/methods/closures)
  • Decreasing the GC and cloning costs of closures, and making the MoarVM dynamic optimizer far more aware of them
  • Better JIT compilation of Int math operations
  • Smarter inlining, especially of simple control-flow-free functions
  • Optimizing out box/unbox sequences, as well as taking/using native references
  • Performing GC finalization on a background thread, to decrease pause times
  • Performing dynamic optimization on a background thread, so it does not pause code execution
  • Making await non-blocking, to greatly improve scalability of async code
  • Improving ThreadPoolScheduler, so it picks a smarter number of threads
  • Improving I/O performance
  • Better optimization and code-gen for tokens and regexes
  • Decreasing the number of times characters are visited during parsing
  • Implementing escape analysis, to reduce GC overhead
  • Looking into more effective parallel GC synchronization mechanisms

Since I intend to look to Perl 6 users' examples of unperformant programs to drive my work, the exact prioritization of these tasks will likely not match what is listed above, and other more pressing performance issues will likely be discovered and addressed along the way also.

It is also important we are able to analyze performance and understand it. Right now, there is an instrumenting profiler. That's useful in so far as it produces detailed information, capturing every single call and allocation, along with statistics on inlining, optimization, and JIT compilation. However, for anything besides micro-benchmarks, it produces vast amounts of data - into the hundreds of megabytes. That is to say, it's more useful for the kind of in-depth analysis a VM engineer wants to do having isolated a performance issue than it is for finding the hotspots in a larger application. It also isn't so useful for getting a good picture of how memory is being used. So, I propose to build as part of this grant:

  • A sampling profiler, which is better suited to analyzing applications. This would likely become the default, user-facing profiler.
  • A heap profiler, for getting a better overview of what's in memory.

On the reliability track, a lot of the focus will go on picking off the most serious or impacting bug reports from the RT queue. Key candidates are:

  • Anything involving a segfault, bus error, etc.
  • Anything where the compiler reports an internal error
  • Memory leaks
  • Regressions that impact modules in the ecosystem
  • Bugs that trip Perl 6 users up regularly

I will also write regression tests to cover fixed issues.

Project Schedule:

I intend to work 40-50 hours a month on this grant. Working on performance and reliability is, by nature, open-ended: there's always more that can be done. Therefore, so long as I feel the topics this grant addresses are the best way I can serve the Perl 6 project, I will request extensions of this grant every 2-3 months. Each time I will report what has been achieved, and indicate where I expect to focus next. This way, the wider community can also provide their views on whether the focus is correct on a regular basis also.

Bio.:

I'm the current architect for both MoarVM and the Rakudo Perl 6 compiler. In MoarVM, I designed and implemented large parts of the dynamic optimization infrastructure, mentored its JIT implementation, and built the current instrumenting profiler (in the space of 2-3 days!) In Rakudo, I've been a key contributor to many parts of the codebase, implementing numerous language features and fixing hundreds of bugs. I also played a leading role in the design of Perl 6's concurrency features. Between the Rakudo, NQP and MoarVM repositories I have made just short of 12,000 commits at the time of writing.

Endorsed by: Moritz Lenz

Amount Requested:

$50 USD / Hour * initial 200 hours = $10,000 USD

Suggestions for Grant Manager: Will Coleda

20 Comments

I wholeheartedly endorse this proposal, the perfomance is a perceived barrier to adoption for many applications and actively addressing it will give rise to more adoption and thus and increase in contributors and hopefully therefore further improvements.

It can only be a good thing.

By all means please approve this grant.

In the US, a senior software architect can expect a median hourly rate of around $79/hour. There is of course considerable variability based on skill sets, experience, location, and industry.

Understanding that the grantee lives and works in Prague, where the market rate is considerably lower... The low hourly rate for a lead architect still sets an odd precedent which could carry over to earnings expectations for Perl 6 developers.

The grantee should either be given considerably more per hour, or some explanation should be provided which explains why the grantee is working at that hourly rate.

This is a good idea. Jonathan has unmatched insight into the working of both MoarVM and rakudo as well as an unmatched track record. The performance of perl6 programs in real-world cases can probably still improve dramatically, but this requires a focused effort. It seems likely that this grant would enable such an effort and ultimately allow perl6 to become competitive in performance.

I'm definitely in favour of this grant. Jonathan Worthington is definitely the right person for this task, and the task is worth being done.

I myself have spent some time in the past working on performance improvements across rakudo, nqp, and moarvm. Jonathan's input has always been very helpful and discussions with him about potential future features/changes always brought a lot of insight, in my opinion.

This all sounds great to me. Having spent a fair bit of time on Perl 6, it's clear to me that the language design itself is in good shape, and it's quite usable on that front. However, performance needs a lot of improvement, and I can't see myself using it for serious production work until that happens.

Jonathan has a proven track record of getting things done.
While the gaps in the ecosystem can be worked around by using libraries from other language (for example using things such as Inline::Perl5), the speed of Rakudo is something that needs to improved.

Performance is the major if not only barrier to production use of Perl 6 for me in the field of bioinformatics. Especially around the areas of IO and string/list manipulation.

I believe this grant is going to be instrumental in the adoption of Perl 6 to a wider audience that demands a minimum level of performance for computationally intensive tasks. Jonathan is an obvious sure bet for improvement in this area, having already done so much.

I really hope this work proposal is accepted, as Perl 6 has a lot to offer if the performance was improved to the point of approaching parity with other major high level languages. Even if this is the first incremental step towards that goal, it sends a clear message that performance can get there and is being actively supported.

I would like to see it is accepted. Thank Jonathan for working on Perl 6.

Personally I am more interested in reliability improvements. Even though current performance is less than awesome, it is still good enough for the vast majority of the tasks (though I am aware that lots of people are not going to agree with this statement).

Whenever I actually face performance issues I always try to change the code in such a way that it processes the data in parallel. Now that's where the reliability is very problematic. For example, 「hyper」 is pretty much broken, and any code involving 「start」 blocks will eventually crash with “double free or corruption”. There are also some kind of memory leaks that are only becoming apparent during high performance tasks.

Perhaps it would make sense to pay a bit more attention to reliability right now and leave performance improvements to the second part of this year?

That being said, as long as jnthn is involved, the progress will be significant. jnthn's work has always been one of the major driving forces for Perl 6 development.

+1 on everything above. Please, approve this grant

ditto Matt Oates's comment. I use Perl 6 almost every day for scripting, but I still need to pull out Perl 5 when processing huge data files. Jonathan Worthington will do well in advancing Perl 6 performance.

Jonathan, in collaboration with the perl6 dev- team, created a beautiful and modern language, fully capable and fun to work with. The technical expertise of Jonathan is above doubt.

The language released at Christmas delivered the features to make it successful. However, as acknowledged by the team, most things were optimized for validating the many innovations the language brought to the table. Speed, many will agree, is lacking for many types of applications. Further adoption will --imho-- depend on the optimization work on speed (next to a mature CPAN like ecosystem/infrastructure)

On a more personal note, I'll like to add that Jonathan is very approachable person (e.g. by irc), always ready to discuss bugs, explain less obvious design decisions or just give pointers to new users. Also, he's a talented and enthusiastic speaker always ready to participate in Perl events. His talk about Perl 6 at FOSDEM was very successful in reaching a new audience. Any investment in Jonathan's time creates additional --and fundamental-- non-technical added value.

Claudio

The performance is crucial for the language to become popular. For many developers that + the seg faults is the barrier for now.
+1++ for the grant !

Thanks to all who have provided feedback so far. I'll take a moment to address a couple of the comments.

First, with regard to Garrett's comments about the hourly rate, it's useful to put the grant application into context.

I first got involved in Perl 6 because I wanted something interesting and compiler/VM related to do in my free time, and wanted to give back to the Perl community since I used Perl extensively in many years of freelance jobs during my school/university years. For a number of years, I was in a position to donate notable amounts of time (case in point: MoarVM was developed without any funding for its first couple of years). Up until my 2015 grants, all grants I took were not aimed at providing me a good hourly rate (they didn't). Really, they meant I could continue working less-than-full-time at my $dayjob so I could do Perl 6 things, but still afford to do the traveling I enjoyed (often taking in Perl conferences and meeting loads of great people along the way).

More recently, it's been valuable to the Perl 6 effort for me to be able to spend more time on Perl 6, and at the same time my circumstances have changed a bit. It's thus been useful to make Perl 6 compiler/VM development contribute a bit more to my income, so I can sustain a decent level of involvement. Thus why I now seek grants with an hourly rate instead.

Anyway, to be very clear, my choice of rate for this grant is not in any way a reflection of what I consider myself "worth" in a typical consulting scenario, and companies interested in having me serve them as an architecture consultant or developer can expect to pay a good amount more. :-) Rather, the rate here is sufficient to make a sensible contribution to my salary, given it's just one part of what I do in a month. It's also aligned with the Perl 5 Core Maint Fund rate and so easy from a community/political angle. Further, in being below the market rate for such work, it reflects that I still wish to view my Perl 6 involvement as a case where I'm "giving something", and so still to some degree a volunteer, like nearly everyone else who has been working on the Perl 6 implementation. So, it reflects my own situation and preferences, not a market valuation.

With regards to Aleks-Daniel's note on concurrency reliability issues, I'm only too aware we're less mature in that area (and I think it really is a maturity issue; the code there is newer that much of the rest of the compiler/VM/built-ins and some of it was developed under time pressure). Improvements there are very much on my radar on the reliability track, which will proceed in parallel with the performance one. I think working on the two together will be more efficient; it means I can get a chunk of the system "in my head" and give it a good auditing for both reliability and performance problems.

Hope this helps!

Jonathan

Jonathan gets my vote! I think having him doing the speed-up work is really timely. One of the major darts thrown at Perl 6 by some old monks at <http://perlmonks.org> is its speed versus Perl 5. I hope in the process of this grant work that the process of getting an explicitly compiled version of a script will get fixed, too (although that may not fall within the scope of this grant request).

"Make it run" and "make it right" do come before "make it fast." I have every confidence that Jonathan will act accordingly, balancing performance and reliability work appropriately. I really can't imagine him neglecting a reliability problem in order to just chase speed -- not with the sense of craftsmanship he has already demonstrated.

+1

A definite +1

Performance is the biggest adoption obstacle for Perl 6 and jnthn is perfect for the job, and has proved self to be very capable in the past.

Go, Jnthn, go ...

Perhaps a bit late, but ...

This grant should absolutely be approved. As the creator of perl6-bench, I am acutely aware of performance limitations of the Rakudo/MoarVM stack at present. And as a Rakudo user and early stress tester of the concurrency subsystem, I find the current performance and crash bugs limit the use cases to which I can apply my favorite language. I really want to see that change.

Jonathan is (and has been for years) the best person in the community for deep, core changes to the Rakudo/MoarVM stack -- and performance/reliability work is nothing if not that.

That the language itself is stable means, that it's possible to get familiar with Perl 6 right now. This is of course a big step forward.

But both runtime and memory consumption needs vast improvements for major adoption. Jonathan Worthington is of course the right person for the job. No doubt about that.

I would like to ask for different updates (at least different sections of the reports) for Rakudo vs MoarVM enhancements, as I see this grant -
http://news.perlfoundation.org/2016/02/ian-hague-perl-6-grant-applica.html - to be a very interesting proposition.

The dream of having a performant Perl 6 in the browser...

Leave a comment

About TPF

The Perl Foundation - supporting the Perl community since 2000. Find out more at www.perlfoundation.org.

About this Entry

This page contains a single entry by Karen Pauley published on February 15, 2016 10:34 AM.

QA Hackathon Grant Successful was the previous entry in this blog.

Maintaining the Perl 5 Core: Report for Months 26 & 27 is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Pages

OpenID accepted here Learn more about OpenID
Powered by Movable Type 6.2.2