2010Q1 Grant Proposal: perl core memory improvements
Sat, 06-Feb-2010 by
Alberto Simões
edit post
<![CDATA[
<h1>perl core memory improvements</h1>
<dl>
<dt><strong>Name:</strong></dt>
<dd>
<p>Jim Cromie</p>
</dd>
<dt><strong>Email:</strong></dt>
<dd>
<p>[hidden email]</p>
</dd>
<dt><strong>Amount Requested:</strong></dt>
<dd>
<p>How much is your project worth? $3000</p>
</dd>
</dl>
<p>
</p>
<h2>Synopsis</h2>
<p>Memory allocation enhancements in core (sv.c).</p>
<p>Perl's variable namespace model is very flexible, users can:</p>
<pre>
- create vars, in any package, or in my scope, by naming them;
- give them complex values: my $foo = [ 1, { a => 2}, 3 ];
- share/assign/shallow-copy them: $main::bar = $foo;
- crosslink or self ref them: $a[2] = [$a[2], $a[1]];
- other hairy stuff</pre>
<p>This user data is all built on-demand from an inventory of sv-parts
which is kept on the interpreter's freelists (sv_root, PL_body_roots).
These are refilled periodically by S_more_bodies, which gets-an-arena,
slices it into sv-parts, and threads them onto the freelist.</p>
<p>This can result in user data spread across memory like a spiderweb in
a corner; its hard to clean the corner without destroying the web.
IOW, it makes memory reclaim "hard", and probably ineffective. As a
result I think, perl core has never really seen the need/benefit to
bother reclaiming arenas.</p>
<p>One important workload however could benefit; Storable::freeze() uses
a ptr-table to track SVs that it has %seen, but its PTEs hang off the
interpreter until process termination. For a long-running process,
this is clearly suboptimal.</p>
<h2>Benefits to the Perl Community</h2>
<p>1st, theres this in perltodo:</p>
<pre>
use less 'memory'
Investigate trade offs to switch out perl's choices on memory
usage. Particularly perl should be able to give memory back.</pre>
<pre>
This task is incremental - even a little bit of work on it will help.</pre>
<p>This is deep core work, benefits accrue to users of 5.14, which is
eventual target. Since the interfaces changed are internal, it may be
possible to get it into 5.12.x.</p>
<p>Currently, Storable::freeze() uses ptr-tables to track seen SVs as it
freezes them, so that it honors shared linkages. Doing this on large
datasets will allocate a huge ptr-table, which when freed, releases
all those PTEs back to the interpreter-global freelist, where they
hang uselessly until process death (or interpreter shutdown).</p>
<p>The work proposed below appears to provide a workable mechanism to
implement the private-arenas that Tim Bunce expressed want/need for,
with Nicholas Clark's comments, here:</p>
<p><a href="http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2009-12/msg00821.html">http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2009-12/msg00821.html</a></p>
<p>By my 1st reads, Tim wants to coax a set of SV allocations to be taken
out of separate arenas, to protect them from others. Nick outlined a
solution that largely fits with my earlier revision of this grant
proposal (Aug, 09), but added a discussion of savestacks, and implied
(to me, at any rate) a need for a robust underlying mechanism,
prompting this revision of the proposal.</p>
<p>General benefits will likely flow from finding out what nytprof needs,
and figuring out how to provide it :-D</p>
<p>
</p>
<h2>Deliverables, Project Details</h2>
<p>here are the major elements</p>
<p>
</p>
<h3>Private Arenas</h3>
<pre>
The short version:
- adapt get-arenas(sig): sv_type arg2 -> (void*) reqid
and track allocs by the reqid
- propagate that to S_more_bodies and its macro wrappers
- add release-arenas() stub 1st</pre>
<p>With get-arenas(reqid), we can track arenas by its users, with
S_more_bodies we can extend that tracking to the interpreter's svtype
consumers individually. With unique tracking of arena users, we can
offer release-arenas(reqid), and since we're an internal sub-system
interface, expect them to use it properly.</p>
<p>
</p>
<h4>Design Benefits</h4>
<p><code>S_more_bodies()</code> outer-users (disregarding the macro-wrapper) keep
their current interface, the arenas provisioned by it for each sv_type
are transparently tracked, and can soon be reclaimed.</p>
<p>get-arena/release-arena give a balanced api for clients to manage
slabs of memory themselves. The api is minimal, allowing and
requiring simply that callers of get-arena(reqid) do:</p>
<pre>
- call release-arenas(reqid) when done with mem.
- know theyre not sharing parts of the arenas when done.
- dont abuse the reqids of others, ref your own object.
- users can create and abandon arenas (be careful!)</pre>
<p>With this, users hacking in core can allocate many slabs, of various
sizes, using just one reqid, assemble them with pointers into
arbitrary structures, and when done, know that they're all cleaned up
together. Users may also use multiple reqids to simplify their memory
reclaim operations.</p>
<p>It should also be flexible and efficient enough for use by XS
libraries, given their tolerance for newness.</p>
<p>
</p>
<h3>Private-Arenas 1st user: ptr-tables</h3>
<p>Given the Storable use case, this has potential merit; being
parsimonious with PTE mem by default will work for some users.</p>
<p>But for less specific cases, the global PTE freelist probably wins a
performance contest; the malloc demand is intrinsically less when PTEs
are reused, not only <code>freeze()</code> uses ptr-tables, and its only
pathological cases that would even cause notice.</p>
<p>Nonetheless, it provides a test-case for 1st use of the new interface,
and an alternate ptr-table implementation, possibly providing support
for 'use less memory'</p>
<p>Note that with the stubbed release_arenas, we only pretend to free the
private allocations; this may cause problems in <code>make test</code>, but the
overall demand for ptr-tables is quite limited (iirc the big user,
t/re/regexp_qr_embed_thr.t creates ~2000 ptr-tables), and on 1GB
machines, we may not run out of memory.</p>
<p>This has some probitive value for OOM handling also, especially in a
setrlimit()d sandbox.</p>
<p>
</p>
<h4>Design Benefit</h4>
<p>private arenas in ptr-tables provides a concrete basis to consider
other resource reclaim strategies, narrowly 1st, but perhaps also
broadly for other potential users.</p>
<p>When ptr_table_free is called, we know that:</p>
<pre>
- we start with an empty, private PTE freelist, fill it as needed
- pt-store consumes PTEs from private PTE freelist
- all PTEs in the table came from our arenas
- all PTEs cleared back to it are from our arenas
- no other users of those PTEs exist
- all our arenas have our reqid</pre>
<p>With this, we should be able to just whack the whole table (by finding
and freeing the arenas with the reqid), skipping all the rethreading
to the global freelist, and immediately releasing the memory back to
the system. This sounds possibly useful later.</p>
<p>
</p>
<h3>release_arenas(reqid)</h3>
<p>Ive separated this deliverable because private-arenas in ptr-tables
can be mostly validated without it (using the stub) and because in
some respects its our 1st new feature, where the previous focus was on
refactoring the existing code to accommodate the feature.</p>
<p>The 1st test of this code will be in perl_destruct</p>
<pre>
release_arenas(&PL_body_roots[$_]) foreach @sv_types;
release_arenas(&PL_sv_root);</pre>
<p>Then we call it from ptr_table_clear.</p>
<p>
</p>
<h3>support for NYTProf</h3>
<p>Given the recent p5p traffic 12/20 (link above), I think this path to
private arenas helps; it adds support needed beneath the fancy
freelist pushing-and-popping briefly described there. What nytprof
needs will take further study.</p>
<p>
</p>
<h3>register_arena_consumer</h3>
<p>The design thus far does nothing to protect (or even advise) of reqid
trampling between 2 users, <code>get_arena()</code> implicitly allows callers to
start new reservations with the given ID, which allows sharing amongst
knowing users. Formal registration will provide at least advisory
protection. This could be done with a flag too.</p>
<p>
</p>
<h2>Semi-Deliverables</h2>
<p>These have real merit in my estimation, but are rather speculative,
and I'm reluctant to call them committable deliverables. I think
think they help illustrate the potential of the above work.</p>
<p>
</p>
<h3>use less memory, pte</h3>
<p>One way to nudge this rock foward is to plug in a 2nd (private)
ptr-table-* function set, addressing the Storable::freeze use case.</p>
<p>I suspect however that freelist pushing and popping, along with
<code>get_arenas()</code> and <code>release_arenas()</code>, will ultimately be a better tool
than this specialized fix for PTEs, but it serves as a point of
discussion (strawman); we dont even have decent terminology yet, let
alone a few paths forward.</p>
<p>
</p>
<h2>use my_arenas</h2>
<p>Storable::thaw() might want to put the perl-data it vivifies into a
constrained region of memory, as this may improve processor cache
performance, especially with their modern prefetch systems. So would
perl routines, such as parsers, data generators, etc.</p>
<pre>
# Doing it lexically would be nice;
get_tight_hash {
my $var;
use my_arenas depth => 1, 'xs';
return { Storable::thaw($packet) };
}</pre>
<p>Here, my_arenas seeks to capture only SVs in the contained xs scope
(the thaw), and those in {} composition. depth => 1 sounds safest wrt
the spiderweb problem, N might be nice if it makes sense (depth=>0
makes me nervous). I also suppose that xs might somehow be different
than just depth => 1.</p>
<p>This doesnt attempt to migrate perl data into a container; that would
be tantamount to lifting the spiderweb without damaging it, and is out
of scope here. But this may shed some light in a dimly lit corner.</p>
<p>
</p>
<h2>Inch-stones</h2>
<p>The deliverables above are largely self explanatory, but will also
include responding and resolving issues; they're then largely defined
by porters and particularly pumpkings.</p>
<p>Tim Bunce, given his interest for nytprof, will hopefully offer
guidance as to what he needs, Id treat those as immediate goals.</p>
<p>There are no doubt numerous knock-on effects to the rest of core,
some of these will be in-scope, though I hope not all.</p>
<p>setrlimit()d sandbox, oom tests. work this into fresh_perl, maybe
wrap this as <code>sandboxed_perl()</code>.</p>
<p>p5p discussion, review, responses, revisions, variations, etc.</p>
<p>
</p>
<h2>Project Schedule</h2>
<p>1-2 months</p>
<p>
</p>
<h2>Bio</h2>
<p>Ive been hacking in perl for a while</p>
<pre>
[jimc@groucho perl-git]$ git log blead | grep Cromie | wc -l
102</pre>
<pre>
Ive also hacked in pertinent parts of core, ext/ code:
- added arena-sets into the arena allocator
- reworked the body-allocator around S_more_bodies
- helped refactor sv_upgrade (Nick did the heavy lifting)
- added struct body_details (says the blamelog)
- extended B::Concise feature set
- implemented OptreeCheck and tests using it</pre>
]]>
Comments (1)
I believe that the proposed approach is wrong to the point of being counterproductive. Specifically, it won't help NYTProf at all, and it's likely to slow Storable down.
As I've already sent my reasoning to perl5-porters, I'll just give a link to it: http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2010-02/msg00014.html
5 days later, and there has still been no reply to it.
