Grant Application: Raku Ecosystem
Tue, 23-Jan-2024 by
Saif Ahmed
edit post
< is proposing a project to develop a Raku Ecosystem written in Raku. This gets rid of a dependency on other languages and proprietary code to create a more sustainable environment. Tony has a number of [Raku projects](https://github.com/tony-o?tab=repositories&q=&type=&language=raku&sort=) including the very important [Fez](https://github.com/tony-o/raku-fez), working with `niner`, `vrurg`, and `ugexe`.
<hr>
### Raku Ecosystem Infrastructure API written in Raku
**Author**: Anthony O'Dell
**Country**: USA
**Date Submitted**: 22 Jan 2024
<img src="_ra-rel-mnt_/simplecas/fetch_content/dd1e1502cab5d5bed742159522782a6266699f5d/Raku Infrastructure.png" style="max-width:100%;" />
### Proposal
To create an open source infrastructure around raku ecosystems. This proposal to write this code in raku and make it available to the general public. The current
state is that the underlying code is written in go and using proprietary AWS lambdas to make everything flow smoothly. The go lambdas are due to be deprecated in early January and this flow will become unmaintanable but usable in its current form.
This grant will deliver the following:
1. A raku ecosystem repository via Artistic-2.0 with tony-o retaining the original copyright
1. The repository would live inside of an RSC org's repo
1. A runnable ecosystem software written in raku
### Benefits to Raku
This proposal would benefit the raku community by providing a means of running a raku compatible ecosystem with strict auth, user management, group & role management. This would be a major lift for the raku ecosystems as it would decrease the reliance on shoe-horned solutions such as CPAN/PAUSE
and manually maintained list of git repositories. This would also make raku more appealing to organizations considering raku as it would allow for centralized and private module ecosystems for private or proprietary code using already published, publicly available tools such as fez or mi6.
As the solution becomes public this also allows for collaboration with raku developers in fixing bugs, introducing new features, and improving security/reliability of an ecosystem.
### Schedule
This schedule will flow as my time allows as I currently have a day job and this happens around that schedule but will take less than a calendar quarter from the start of work.
1. Create ERD for application entities (~5 hours)
1. This is will help enable future developers to build and develop against this repo
3. Create general configuration modules that allow versatility in ecosystems (~20 hours)
1. This aims to provide some versatility for companies or users that want to configure their private ecosystems
5. User management modules (~20 hours)
1. CRUD operations for users
2. Email operations for password resets and dist related notifications
7. Group/role management modules (~30 hours)
1. CRUD operations for groups
2. Email operations for recovery, member management, and dist related notifications
3. Member management functions to include:
1. Role changes
1. User invites
1. Membership management (user leaving)
9. Dist management modules (~40 hours)
1. META6 verification models
1. Dist storage and management
11. Create ingestion modules for dists (~109 hours)
1. Processing dist uploads/deletes
2. Serializing into one ecosystem archive
3. Providing binary format indexes for searching/exact modules
12. Ecosystem META6.json & binary index endpoints (~1 hour)
### Amount Requested
This amount does not include operating costs and there is no intent to seek operating costs for the `zef:` ecosystem.
220 hours * $60USD = $13,200
]]>
Comments (3)
Not sure about Raku, as I'm not using it. As a Perl user I would like the grants to be focused mostly on Perl core (C code and the shipped modules), at least primarily and for few years (esp. during the new "class" era). Perl comes with almost every Unix-like distribution, and improving it could significantly contribute to reviving and expanding its use in production where those systems are used extensively.
The work in this proposal would be extremely beneficial to the Raku ecosystem – it would provide the final missing piece for Raku to have a genuinely modern and ergonomic library story, without relying on external services like CPAN or GitHub.
I won't go into the technical details here – the proposal itself does a good job of laying those out, and other Rakoons are far more versed in the details than I am. But I'm slightly concerned that the proposal doesn't fully explain the big picture, especially to those who haven't been following the behind-the-scenes details of how Raku software is delivered.
So here's the big picture:
• Raku software is typically installed using the Raku package manager, Zef (https://github.com/ugexe/zef). For example, `zef install JSON::Fast`.
• Zef can install Raku software that has been uploaded to a variety of different "ecosystems" – originally, for public modules, mostly CPAN; later, also a (GitHub-backed) p6c ecosystem. This functionality also supports private/custom ecosystems.
• For a variety of reasons, neither CPAN nor p6c/git are a great fit for Raku – they (understandably!) don't support the features that Raku needs for proper versioning/authentication. (And, of course, using them ties Raku to platforms it doesn't control). See https://deathbyperl6.com/faq-zef-ecosystem for details.
• To solve ^^^^, Raku is transitioning to the Fez ecosystem, which is built to provide what Raku needs.
• Fez has two parts: First, a front-end CLI client that lets programmers register themselves and upload authenticated code, e.g., with `fez upload`. This front-end is written in Raku; its code is at https://github.com/tony-o/raku-fez
• Fez also needs a back-end to handle the authentication, store the code, and make it searchable by Zef. Right now, this back-end is (despite running wonderfully in production) basically a proof-of-concept version – it's written in Go, isn't public, and doesn't provide many of the advanced features that are possible when the front-end and back-end can work together (better group management, more efficient downloads, etc.).
With that context, you can probably see where this grant proposal fits in: the work covered by this grant would rebuild the back end in Raku and would add many of those advanced features. It would effectively complete Raku's transition to Fez as its primary public ecosystem; it would improve the Fez ecosystem in the short term; and, by opening up the back-end code, it would set Fez up for continued improvement. I really hope it gets funded!
I am in support of this grant. Having the ecosystem code open sourced and ported from Go to Raku would itself be a huge boon. It would allow the community to have more insight into policy (such as namespace ownership, meta data validation). People interested in contributing to improving the Raku ecosystem are likely to be more interested in contributing Raku code they can run locally rather than a cloud lamda written in an unrelated language, and so I think this proposal would continue to pay dividends as time goes on.
