Hague Grant Completed by Jonathan Worthington
Sat, 07-Jul-2012 by
Karen Pauley
edit post
<![CDATA[
_Jonathan Worthington writes:_
This is the final report for my Hague Grant on "Meta-model Improvements and Natively Typed Attributes":http://news.perlfoundation.org/2010/07/hague-grant-application-meta-m.html. There was also an "intermediate progress report":http://news.perlfoundation.org/2011/05/interim-grant-report-meta-mode.html. At the point of the intermediate progress report, a new version of NQP had been delivered. Its objects were implemented in terms of a new meta-object system, developed with the needs of Perl 6 in mind. However, at that point Rakudo had not started to use it. This report covers that part of the work. In this report I will use D1, D2, etc. to refer to Deliverable 1, Deliverable 2, etc. from the original "grant application":http://news.perlfoundation.org/2010/07/hague-grant-application-meta-m.html.
Rakudo has evolved and improved enormously during the course of this grant. This is partly thanks to the work of this grant, but also as a consequence of work that has built upon it. In many cases the work done not only meets the goals of the grant but goes far beyond them. Furthermore, the MOP work delivered under this grant is also being studied by those working on the Perl 5 MOP project.
Rakudo today has all of its "meta-objects":https://github.com/rakudo/rakudo/tree/nom/src/Perl6/Metamodel implemented in NQP (D2, D3), using the new meta-object system designed and implemented under this grant, which is often known by the name "6model" (D1). Each meta-object implements the semantics of a kind of type: classes, grammars, roles, subsets, enumerations and so forth. These meta-object implementations are in turn factored in terms of roles, thanks to NQP having been given role support during the course of the grant. The NQP role support has also been used by Patrick Michaud in development of an "updated grammar engine":https://github.com/perl6/nqp/blob/master/src/QRegex/Cursor.nqp, which has in turn resulted in many improvements to Rakudo's grammar support.
The point of a meta-object protocol is not only to enable clean implementation of language features, but also to allow extending the semantics of the language. These extensions can be written in Perl 6 - that is, in the language itself. The ability to extend a language using the language itself is known as meta-circularity. The most useful modules using this ability so far are Grammar::Tracer and "Grammar::Debugger":https://github.com/jnthn/grammar-debugger, which add debugging support to grammars in pure Perl 6. As well as being a good demonstration of meta-programming in Perl 6, the modules have been genuinely useful to those writing Perl 6 grammars. More examples of meta-programming can be found in my "Meta-programming in Perl 6 talk":http://jnthn.net/papers/2012-gpw-meta-programming.pdf [pdf]. The specification has a section on "custom meta-objects":http://perlcabal.org/syn/S12.html#Custom_Meta-objects also (D5); this will continue to be fleshed out over time.
The grant also called for the implementation of natively typed attributes. This deliverable has been achieved in both NQP and Rakudo (D4). In NQP, they have been used to slim down the "Cursor":https://github.com/perl6/nqp/blob/master/src/QRegex/Cursor.nqp#L4 and "Match":https://github.com/perl6/nqp/blob/master/src/QRegex/Cursor.nqp#L436 data structures. Furthremore, Rakudo has also gained natively typed lexical variables. As well as being available to user-space programs, they have also been used to optimize various built-ins in the "core setting":https://github.com/rakudo/rakudo/blob/nom/src/core/Str.pm#L54.
Test coverage for the metamodel and other deliverables is available throughout the existing testing frameworks (D6). Testing of 6model and role support is provided in NQP's "test suite":https://github.com/perl6/nqp/tree/master/t. Testing of OO programming is performed extensively through the S06, S12, and S14 portions of the "roast":https://github.com/perl6/roast/ (spectest) test suite.
During the course of the work, pre-compilation support and BEGIN handling have also been improved greatly. This partly involved the implementation of "bounded serialization":http://6guts.wordpress.com/2012/02/10/bounded-serialization-better-regexes-and-better-errors/, which dramatically improved the startup time of Rakudo as well as the load time of pre-compiled modules. It also means that BEGIN time compuations are properly persisted over the compile-time/runtime boundary.
The richer type system support also led to development of an "optimizer":https://github.com/rakudo/rakudo/blob/nom/src/Perl6/Optimizer.pm; this work was presented in a "blog post":http://6guts.wordpress.com/2011/10/15/an-optimizer-lands-bringing-native-operators/ as well as a "talk":http://jnthn.net/papers/2011-tcpw-optimization.pdf [pdf] given at a couple of workshops. The optimizer supports optimizations such as inlining and compile-time resolution of multiple dispatch. The analysis done by the optimizer can also detect various program errors that will lead to innevitable failure at runtime.
While the optimizer can greatly improve performance in certain situations, the work done in this grant has led to an overall performance increase even without the optimizer. The effect of these improvements on a mandelbrot program was analysed early on in a "blog post":http://perlgeek.de/blog-en/perl-6/how-fast-is-nom.html by Moritz Lenz; "further improvements":http://6guts.wordpress.com/2011/09/17/this-is-not-enough/ led to running this program 19 times faster than was possible in Rakudo as it was before this grant was started. As another data point, the list/iterator model developed in Rakudo using the primitives provided by this grant is 30x faster than Rakudo previously had, despite being written in Perl 6 rather than PIR.
Also worth mentioning is that by implementing representation polymorphism, the new object system has allowed for greatly improved native call support; it is now possible to pass arrays and structs to C from Perl 6 code using the "NativeCall module":https://github.com/jnthn/zavolaj/blob/master/README.markdown. The improved native call support has been used in modules such as "DBIish":https://github.com/perl6/DBIish/, which now supports MySQL, Postgres and Sqlite.
Rakudo is a great deal further forward today than it was before this grant started. While this is of course thanks to the efforts of many contributors, the work done under this grant has been key to the progress than has been made. In many senses, the main success of this grant has not just been in achieving the deliverables themselves, but in the many things that have been built around and on top of them. Critically, Rakudo is in a strong position to continue improving stability, performance and usability, as well as add additional missing features - all important to being useful to a wider range of potential users.
]]>
Comments (0)
