February 2019 Archives

I am pleased to announce that The Perl Foundation has been selected as a Google Summer of Code 2019 mentor organization.

You can still submit new ideas or propose yourself as a mentor. To add ideas, or to become a mentor, see our GitHub repo.

Students can connect with us at our Google Summer of Code page.

I appreciate the hard work made by the community volunteers, especially JJ Merelo, Mark Keating and Patrick Böker.

Maintaining Perl 5 (Tony Cook): January 2019 Grant Report

This is a monthly report by Tony Cook on his grant under Perl 5 Core Maintenance Fund. We thank the TPF sponsors to make this grant possible.

Approximately 54 tickets were reviewed, and 11 patches were

[Hours]         [Activity]
  8.88          #108276 review
                #108276 check over committed changes, look to re-work, ask
                list about PERL_OP_PARENT
                #108276 cleanup PERL_OP_PARENT detritus
                #108276 review old patches, re-work, testing
                #108276 more testing, work on optimize_op()
                #108276 research, cleanup, more testing. Comment with
  0.58          #121033 research, comment
  1.98          #123724 rebase, retesting, bump versions, apply to blead,
                make public
  0.05          #126191 check for activity and close
  2.69          #127521 review code, POSIX, work on implementation
                #127521 local commit with notes
  1.72          #130367 rebase, write new tests, testing, apply to blead
                #130367 perldelta
  1.90          #130585 debugging
                #130585 debugging
  0.32          #131506 research and comment
                #131506 close
  0.07          #131931 link to stack-not-refcounted meta ticket
  0.08          #131955 check for activity and close
  0.52          #132158 rebase, retest and apply to blead
  1.35          #132338 research, comment
  0.12          #132583 same and add the form sub-parse fix to backports
                (the other is in 5.28 already)
  3.76          #132777 prep, review code, research, work on some docs
                #132777 testing, more docs, comment with patch
  0.27          #133030 re-check, comment
  0.27          #133153 comment
  0.62          #133504 research and comment
  0.55          #133522 review patch, testing, apply to blead
  1.88          #133524 work up a fix and apply to blead
  3.88          #133575 prep and start gcc 8.2.0 build
                #133575 work up a patch, testing, comment with patch
                #133575 consider comment changes, research, testing, apply
                to blead
  0.60          #133590 try to work with metaconfig, report upstream to
                the metaconfig wizards
  0.77          #133721 review, write test, testing, apply test and fix to
  0.13          #133740 review new patch and comment
  0.13          #133744 review and comment
  0.10          #133746 review and close
  0.10          #133750 review and reject
  1.00          #133751 comment with patch
                #133751 retest, apply to blead
  0.55          #133752 follow up
                #133752 got a response privately due to rt.perl.org
                issues, forwarded request to perlbug-admin and closed
  0.25          #133753 (sec) fix subject, comment
                #133753 (sec) testing, comment
  1.82          #133754 test setup, testing, research
  2.08          #133755 research, produce a patch and comment
                #133755 fix a typo, test, apply to blead
                #133755 comment and close
  0.25          #133756 review and comment briefly
  0.61          #133760 research and comment
                #133760 research (read the Configure source) and comment
  0.72          #133765 review, start prep to test, review code, comment
                #133765 research, add commit to votes file
  0.53          #133771 review patch and comment
                #133771 review new patch and comment
  0.97          #133776 review and comment
                #133776 review and comment
  3.32          #133782 diagnose, work on a fix, testing
                #133782 retest patch against blead, apply to blead
                #133782 re-test, fix ticket number in patch, apply to
  0.38          #133787 research (more Configure) and comment
  0.33          #133788 research history of APIs, remove M flag and apply
                to blead
  1.07          #133789 debugging, comment
  0.25          #133806 bump $IO::VERSION, testing
  2.35          add t/ file leak checks and TMPDIR leak checks
  0.48          look for possibly fixed sub-parse tokenizer bugs, find a
                couple and comment, bookmark for later closing
  0.72          look for some more possibly fixed tokenizer bugs
  2.82          look into detecting shmem leaks reported by Tux,
  0.17          review maint votes, remove one already picked
  1.07          utf8-readline: build fixes, debugging
  1.50          utf8-readline: clean up, consider re-working options
  0.62          utf8-readline: debug failing tests
  2.33          utf8-readline: debugging
  2.52          utf8-readline: debugging, fixes for buffer parsing tests
  0.32          utf8-readline: more debugging failing tests
  0.10          utf8-readline: rebase, review
  2.80          utf8-readline: track down a fix warning handling
 65.25 hours total

Jonathan writes:

January was a busy and productive month for my Perl 6 grant work.

Back in November, I was working on allowing us to lower lexical variables into locals. This is possible when they are not closed over, and allows for generation of more efficient code, which is in turn much easier for backends - MoarVM, the JVM, and JavaScript - to deal with in their own optimizers. It can also shorten the lifetimes of objects, leading to better memory performance. I completed and merged the almost-complete work to be able to lower normal variable declarations (such as my $a = 42). This will combine well with the ongoing work on escape analysis, since it will be able to eliminate many allocations of Scalar containers.

The next step was to take on the lowering of $_. In Perl 6.d, we made this a normal lexical rather than a dynamic lexical (available through CALLER::) in order to allow further analysis and optimization. Relatively few Perl 6 users were aware of its dynamic nature anyway, and fewer still making use of it. With the topic variable of course showing up in numerous common idioms, it would be a pity to then have people avoid them in the name of performance, due to the dynamic nature of $_ impeding analysis. Thus the change in 6.d.

This month I worked on implementing the most immediate optimization that this enabled: being able to also lower $_ into a local. In a loop like for 1..10_000_000 { something() }, previously we would have to allocate an Int object for every single iteration. Since $_ was dynamic, we could not be sure that something() - or anything it in turned called - would not access it. Under the new semantics, we can lower it into a local, and then dead code analysis can see that the boxing is useless and discard it, saving a memory allocation every iteration. Even were it used, for example in an array index, we now have opened the door to being able to use escape anslysis in the future to also avoid the allocation.

This work had some amount of fallout, and turned up some rather dubious semantics around regexes matching against the topic variable when in boolean or sink context. This included some curious “at a distance” behavior, where things either worked by a happy accident, or could potentially match against completely unexpected data further down the callstack thanks to the thing they were expected to match against being undefined! I proposed a cleaner set of behaviors here and, with a lack of objections, implemented them.

Along with these optimizations, I also implemented more aggressive scope flattening in Rakudo’s static optimizer. When we can show that a particular scope’s existence would not be missed - such as the body of an if statement which doesn’t have any variables in it that get closed over - we can flatten it into the surrouding block. This is a bit cheaper at runtime and leads to more chances to do lexical to local lowering, which - as already mentioned - is the gateway to further optimizations.

Back over in MoarVM, I continued my work on escape analysis and scalar replacement. I took a small detour to implement retention of deoptimization information in specialized bytecode, so we can recover it when inlining. This will allow us to more aggressively optimize code that we inline. I didn’t yet enable the full set of optimizations we might do there, but I was able to enable dead instruction elimination, which can save some work (for example, if we see a return value is unused, we can strip out the work that produces it). This more detailed data was also required for the analysis algorithm that works out which object allocations that we eliminate as part of a scalar repalcement need to be restored when deoptimizing. I got most of that worked out, with a few issues left before it could be merged. (Spoiler: those were resolved in early February and it was merged.)

The lexical to local lowering optimization in Rakudo results in lower quality information being available to debuggers, and with the scope of it greatly expanding with my recent work this would have become quite noticeable. I implemented a new mechanism to retain symbol mappings in order to counteract that.

Further to this work, I worked on 8 other smaller issues.

15:50   Lexical to local lowering, including of $_ where possible
07:59   More aggressive block flattening
05:33   Keep deopt information when inlining, allowing more dead code elimination
07:59   Implement deoptimization of scalar-replaced allocations
00:56   Fix sometimes-missing redeclaration error
01:47   Make exceptions doing X::Control be thrown as control exceptions
01:06   Fix hang when a Channel with a reactive subscription was closed and then drained.
00:48   Look into a crash in t/04-nativecall/06-struct.t, comment on issue
01:59   Track down a runaway memory use spesh plugin bug and fix it
00:42   Fix inaccurate reporting of line numbers in simple routines
06:04   Preserve symbol names for debug in lexical to local lowering
00:39   Fix reporting of Failure where exception has no message method
03:18   Analyze and fix an optimizer bug with when inside a loop

Total: 54:39

Of which were covered by previous grant period: 48:30
Of which were covered by current grant period: 6:09
Remaining funding-approved hours on current grant period: 160:51
Remaining community-approved hours on current grant period: 326:51

Dave Rolsky taught classes at YAPC/TPC for a number of years and he was compensated for it. As Dave is a new member of the TPF Board and TPF organizes TPC, there is a potential conflict of interest. Dave wishes to continue his class.

The typical rate paid by students is in the range of $150-$175 ($165 in 2018). TPC takes a portion of it to cover expense such as venue and the remaining part goes to Dave. Other costs such as transport and accommodation were paid by Dave.

Dave also participates in TPC as a volunteer, working on the website for 2019. Dave is not involved in talk or class selection, nor is he in any Slack channel/email list/etc., where this will be discussed.

If you have an opinion on whether TPC should have his class or not, or if you have a plan to teach at TPC which may conflict with Dave's class, please comment here or email me at [email protected]

Brrt (Bart Wiegmans) is untangling the intricacies of floating point expressions so close to the metal in his MoarVM JIT Compiler grant. He provides some of his insights in his recent blog post. Here is his brief report (edited slightly by me):

Last month, I've been busy with floating point support. Getting that ready involved two things:

  • Getting the type system for the expression template precompiler ready. This works now, I'm happy with the result.

  • Fixing the register allocator to support it. This works as well, but isn't fully tested.

  • Ensured that we convert the register to x86-compatible form in the register allocator rather than in the emitter, which means that we can often select better registers.

The next thing on my list is irregular instruction handling.


About TPF

The Perl Foundation - supporting the Perl community since 2000. Find out more at www.perlfoundation.org.

About this Archive

This page is an archive of entries from February 2019 listed from newest to oldest.

January 2019 is the previous archive.

March 2019 is the next archive.

Find recent content on the main index or look in the archives to find all content.


OpenID accepted here Learn more about OpenID
Powered by Movable Type 6.2.2