Improving Perl 5: Grant Report for Month 3

No Comments

Nicholas Clark writes:

A fair chunk of November was spent on disentangling pod/buildtoc

Originally pod/buildtoc was a script run by the pumpking to rebuild pod/buildtoc.pod, and pod/buildtoc.pod was shipped in the distribution. In 2003 I extended it to add code to regenerate the lists of Pod files in various Makefiles, which were repetitive to maintain manually, and contained several errors where particular files were omitted. The assumption was that pod/buildtoc would be run by the pumpking, against a built tree. This all made sense then.

Subsequently the build process was changed to build pod/buildtoc.pod, instead of shipping it, meaning that pod/buildtoc is run at build time.

Meanwhile the release process has been documented and regularised, and the stage where one would add a new perldelta would ideally happen against a clean build tree. But pod/buildtoc would seem to demand a build tree for its consistency checks.

Also, we've switched the direction of the symlink, such that pod/perldelta.pod is now always a file, and pod/perl5155delta.pod is the symlink to it, deleted by the 'distclean' target in the Makefile. But adding a new version's perldelta changes the name of the symlink, turns the pathname that was a symlink into a file, updates Makefile.SH, but doesn't change the makefile actually run for distclean. So confusion could happen. It would be much better to do this part against a clean tree.

So I refactored pod/buildtoc to split it into 3 parts - a small library to parse pod.lst and run consistency checks, a script run by the release manager to update Makefiles, that runs against a clean tree, leaving pod/buildtoc to just run at build time, building pod/perltoc.pod

Steve Hay used it in the 5.15.5 release, and all reports are that it worked quite nicely. [Certainly, I haven't woken up to find a Camel's head in my bed, which is a good sign :-)]

After the 5.15.5 release I amended pod/buildtoc to also build pod/roffitall. This is a shell script to convert all the man pages to printed documentation, which previously was shipped in the distribution tarball, and kept up to date manually. Or not. Now it's built with a current list of man pages. However, an open bug (RT #103202) needs to be resolved before it can correctly generate the table of contents.

Thanks to Rainer Tammer providing me with access to an AIX system, we did finally get to the bottom of the AIX stdbool.h problem. It turned out to be a bug in xlc (the default AIX compiler), which caused it to generate bad code. Much pain and gnashing of teeth was worked around with this minimal tweak:

commit 18f5643bfee3e4c35fe1b510ad2e37dcb28b9efc
Author: Nicholas Clark
Date: Tue Nov 15 19:44:18 2011 +0100

Tweak the cBOOL() macro to avoid problems with the AIX compiler.
(cherry picked from commit 0cebf65582f924952bfee1472749d442d51e43e6)

diff --git a/handy.h b/handy.h
index 2c2311a..8777644 100644
--- a/handy.h
+++ b/handy.h
@ -115,9 +115,10 @ Null SV pointer. (No longer available when C is defined.)

/* a simple (bool) cast may not do the right thing: if bool is defined
* as char for example, then the cast from int is implementation-defined
+ * (bool)!!(cbool) in a ternary triggers a bug in xlc on AIX
*/

-#define cBOOL(cbool) ((bool)!!(cbool))
+#define cBOOL(cbool) ((cbool) ? (bool)1 : (bool)0)

/* Try to figure out func or FUNCTION equivalent, if any.
* XXX Should really be a Configure probe, with HAS__FUNCTION__

I also discovered and fixed a subtle parallel make race condition on AIX (and presumably OS/2) in the process of getting the bisect runner to build on AIX. (Figuring out this make bug turned out to be 90% of the work needed). The bisect runner should now also be useful to the vast majority of OS X users. It now detects a case sensitive file system, limiting itself to perl-5.005 or later, and patches the OS X hints file appropriately to get
perl-5.6.0 and perl-5.000 to build. I hope this makes it more useful to more people.

The largest and most "exciting" part of last week was spent single stepping in gdb to determine the cause of a global destruction "panic" related to weak references, that Karl Williamson accidentally stumbled upon whilst innocently refactoring completely unrelated code. It took the better part of the 30th November to locate and understand the cause, which means that the analysis and (hopeful solution) will have to wait until next month's
report. Stay tuned... :-)

A more detailed breakdown summarised from the weekly reports. In these:

16 hex digits refer to commits in http://perl5.git.perl.org/perl.git
RT #... is a bug in https://rt.perl.org/rt3/
CPAN #... is a bug in https://rt.cpan.org/Public/
BBC is "bleadperl breaks CPAN" - Andreas König's test reports for CPAN modules

HoursActivity
7.25AIX parallel build failure
0.50BBC, CPAN #72133
0.50OpenBSD and MirBSD smoke failure
0.50Porting/bisect-runner.pl (--force-regen)
7.25Porting/bisect-runner.pl (refactoring for maintainability)
0.25Porting/bisect-runner.pl (start,end for validate)
0.25RT #103006
0.25RT #104504
1.50RT #18514
1.50RT #81332
1.00Simplifying S_parse_body()
0.25Solaris -xO3 failure
6.50bisect.pl (AIX)
1.50bisect.pl (OS X for HFS+)
0.50bisect.pl (fatal build regression)
1.50bisect.pl (feedback from Ricardo)
8.50bisect.pl (fixing parallel make bugs)
2.75buildtoc regressions
1.00debugger tests
1.50delete vs splice on arrays
0.50fc
2.25global destruction backref panic (analysis)
6.75global destruction backref panic (diagnosis)
1.00intrpvar.h
0.50makedef.pl
6.00parallel make race condition for SDBM_File
1.00perltodo
23.25pod/buildtoc
3.25process, scalability, mentoring
35.00reading/responding to list mail
0.75release process
7.25roffitall
roffitall (RT #103202)
3.25sitecustomize/relocatableinc RT #104112
6.75stdbool.h
1.00t/test.pl
1.25use v5.16;

144.50 hours total

Leave a comment

About TPF

The Perl Foundation - supporting the Perl community since 2000. Find out more at www.perlfoundation.org.

About this Entry

This page contains a single entry by Karen published on December 13, 2011 9:42 AM.

Fixing Perl5 Core Bugs: Report for Month 21 was the previous entry in this blog.

2011Q4 Grant Proposal: Turkish Localization of the 5.14.1 Perl Documentation is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

OpenID accepted here Learn more about OpenID
Powered by Movable Type 4.38