2008Q2 Grant Proposal - Make localtime() and gmtime() Work Past 2038
Thu, 01-May-2008 by
Alberto Simões
edit post
<![CDATA[
* **Name:** Michael G Schwern
* **Project Title:** Make localtime() and gmtime() Work Past 2038
* **Synopsis:** Prevent the Y2038 bug from effecting Perl users. This means make localtime() and gmtime() work for times beyond 2038. Do this regardless of the state of the system C libraries. A working solution, in C, already exists. This grant is to finish the portability issues, write additional tests and change perl to use the repaired C library.
**Name:**
Michael G Schwern
**Project Title:**
Make localtime() and gmtime() Work Past 2038
**Synopsis:**
Prevent the Y2038 bug from effecting Perl users. This means make localtime() and gmtime() work for times beyond 2038. Do this regardless of the state of the system C libraries.
A working solution, in C, already exists. This grant is to finish the portability issues, write additional tests and change perl to use the repaired C library.
**Benefits to the Perl Community:**
Outlined here in detail. http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-02/msg00040.html
* Many applications and modules make use of gmtime() and localtime() directly or indirectly to display dates or do date math.
* Many applications have to work on dates 30 years or more in the future.
* The problem will only get worse with time.
* The C time libraries are being very slow to fix this well known problem.
* Not everyone can just upgrade to 64 bit hardware.
* Even DateTime is effected by this when it works with Unix timestamps.
Perl has traditionally shielded programmers from the limitations of C. The idea that time ends at 2038 is a ridiculous notion to anyone but a Unix C programmer. When localtime() returns the wrong date programmers will blame Perl, not C, resulting in another ding against Perl.
We can instead turn this into a PR win for Perl. Similar to the publicity for being one of the safest Y2K languages, Perl can be one of the first Y2038 safe languages.
**Deliverables:**
64 bit localtime/gmtime
* Write C versions of localtime() and gmtime() which work with times beyond 2038 regardless of the limits of the system C libraries.
* Test for negative times.
* If it can be made to work, make it work.
* If not, provide a proper error message.
* Fix any performance issues so the process is O(1) (currently parts of the code are O(n)).
* Make gmtime64_r() properly report an EOVERFLOW error when the year is too large to be held by tm.tm_year.
* Fix any portability issues. [See http://svn.schwern.org/svn/y2038/README.txt]
Provide a patch for Perl which...
* Adapts these libraries for perl and change pp_gmtime() and pp_localtime() to use them.
* Add additional tests to localtime(), gmtime() and related core modules (Time::Local) for beyond 2038.
* Ensure cross-platform compatibility including any necessary Configure probing.
* Use the existing system libraries should they be 64 bit clean.
* Works with bleadperl.
* Works with 5.10.
* [BONUS] Backport to 5.8.
**Project Details:**
Perl will have to bundle it's own 64 bit clean localtime() and gmtime() implementations.
There is one OS specific component to localtime() which will need to be duplicated or gotten by other means and that is how to adjust for the current time zone. The scalar value of localtime() and gmtime() is not effected by locales. This problem has been solved. http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-02/msg00259.html
The usual sorts of system compatibility problems will need to be resolved. A discussion of the issues has been started here. Much of the information is already gathered by Configure. Andy Dougherty has said he'll work on the rest. http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-02/msg00277.html
localtime() and gmtime() will still be limited by the size of Perl scalars. For 32 bit systems this means NV limits which is about 2**47 or about 4 million years in the future. With 64 bit integer scalars it should be able to go all the way out to the limit of a signed long long or 2**63 which is something like 300 billion years. However, since we're still using the 32 bit tm struct and tm.tm_year is a signed int so the practical limit on 32 bit systems is about 2 billion years in the future which I'm happy with.
Permission has been gotten from the original author of the 64 bit localtime code to use and modify it in Perl without a binary attribution clause.
**Project Schedule**
I will be available to work on the project after June 20th (post YAPC::NA).
The first, speculative phase is already complete. This involved writing the 64 bit clean localtime/gmtime C functions which work on 32 bit machines and shoving it into the perl core.
* Write a 64 bit localtime/gmtime which works on 32 bit machines. [COMPLETE See http://svn.schwern.org/svn/y2038/]
* First cut patch adapting pp_gmtime() and pp_localtime() [COMPLETE]
Second phase is cleanup of the patch with perl to make it play nice with Configure and not leak memory, etc. This will require the assistance of p5p and those who know Configure, perl and C.
* Make the patch play nice with Configure.
* Fix any known and revealed cross-platform issues.
* Fix any known and revealed bugs.
* Extend existing core time tests to check for dates beyond 2038 and fix any revealed bugs.
At this point it works well enough to be shipped in bleapderl. This is a long back and forth process requiring lots of input from many people on many different machines. A lot of time will be spent waiting for feedback. It is anticipated this will take at least one
month, probably two.
Third phase is to improve the C functions.
* Test for negative times.
* If it can be made to work, make it work.
* If not, provide a proper error message.
* Fix any performance issues so the process is O(1) (currently parts of the code are O(n)).
* Make gmtime64_r() properly report an EOVERFLOW error when the year is too large to be held by tm.tm_year.
These parts are relatively straight forward and should be solved in two weeks. They can be done in paralell with the 2nd phase.
Finally, backport into 5.10 and 5.8. This should be trivial.
The project is considered complete when maintperl contains a 64 bit localtime.
The earliest date for completion is July 26th. Latest is August 18th. It is not a coincidence that these correspond with the end of OSCON and YAPC::Europe. They provide opportunities to get some intense hacking done on any hanging issues that might be holding up the process.
**Bio**
Although my C skills are far from ideal for solving this problem, I seem to be the guy who cared enough about this problem to push it through and write a complete implementation. I work as an independent contractor and will be working half time for the summer in anticipation of working on grant work.
Andy Dougherty and Jan Dubois have expressed interest. Andy knows Configure and Jan knows cross-platform porting.
**Amount Requested:**
Total: $1500
$1000 for myself to write tests, preliminary perl core code patches and ensure the remaining deliverables get finished.
Because my expertise is not up to snuff in some places, I would request another $500 to be dispersed to folks helping out, mostly with the Configure and porting issues and necessary cleanup, to ensure the work gets done. Should this money not be necessary it will be returned.
]]>
Comments (7)
This is critical work, and Schwern already showed he was interested and pretty able by taking the lead on this on p5p.
I concur with rjbs.
This is the exact type of project that TPF grants are best for, IMO -- important work that benefits everyone but is too complicated to just fall out of developer itchiness.
This is a great idea, I find it very important, too,
and it has to be done anyway someday, so I'd be glad
to see a grant for this.
This is a no-brainer. Yes, please do this. The epoch limit is going to start biting people in the ass very soon, as things like 30-year mortgage calculations start blowing up.
This fits in nicely with Perl's philosophy of just making things work for numbers (at least, Perl can handle >32-bit ints on a 32-bit platform).
Sounds good. Looks like an important work.
Regards,
Shlomi Fish
CMIIW, is this has been done by activestate in activeperl 5.10 build 1001 ?
activeperl 5.10 changelog
