RakuAST Grant Report for May 2021

Regarding his RakuAST Grant, Jonathan Worthington reports some awesome progress for May 2021.

Side note: due to a bug in the Markdown rendering in this blog post, the less-than and greater-than characters in code blocks are double-encoded. Please refer to this gist for a correctly rendered version.

Anyway, here is Jonathan's report:


During May I focused on filling out the regex part of RakuAST, with the result that the majority of the constructs found in Raku regexes now have AST nodes and can be compiled by the RakuAST-based compiler frontend. Of note, I did AST nodes for:

  • Declarations in regexes (:my $foo)
  • Code-based assertions (and)
  • The \e, \f, \h, \r, \t, \v, and \0 escape sequences
  • Character classes, including the common <[a..z_]> style, Unicode properties (<:l>), rules &lt;+rulename&gt;, and the combination of these using + and -
  • Regex code evaluation (&lt;{ ... }&&gt;)
  • Variable interpolation, both as data ($x) and in an assertion syntax as code (&lt;$code&gt;)
  • Internal modifiers (:i, :m, etc.)
  • Calls to lexical rules (&lt;&rulename&gt;), including passing of arguments

Furthermore, I corrected a mistake in handling of backtracking modifiers, and added a missing action method in the new compiler frontend that blocked regexes passing arguments from compiling.

Away from the regex language itself, I:

  • Implemented a mechanism for an AST node to specify it would like to participate in CHECK time and report problems
  • Implemented construction and collection of typed exceptions, both at CHECK time, but also so that the new compiler frontend can produce them. Probably some things that are done in the compiler frontend might want to become CHECK-time things in the AST nodes, since they can happen in synthetically constructed programs too. Those that are purely syntactic are often not possible to represent in the AST.
  • Added RakuAST nodes for token/rule declarations, so that grammars can now be expressed and compiled in RakuAST (without protoregexes so far, however)
  • Correctly set the scope on routine declarations
  • Added missing operator properties for bitshifts, so they can be compiled

All of this work won a rather underwhelming 17 extra fully passing spectest files; as has been previously noted, the test files are quite good at depending on a lot of language features.

Category: Grants , Raku Development

Comments (1)


For one, I am a great believer in investing in moving the core language to new levels and AST/macros is a very broad area for improvement and further raku awesomeness. I would say "keep up the good work" - and maybe lay a trail as for how the roast tests could/should be disaggregated....


Sign in to add comment