Hi there. Before you spend any time working on https://github.com/ganezdragon/tree-sitter-perl you may wish to know that we already looked at that a while ago and found it so terrible we wrote a far better one.
You might already find this satisfactory - or if not we'd welcome more input into improving it: https://github.com/tree-sitter-perl/tree-sitter-perl/
You mentioned one perl extension for VScode, and while that one may be problematic--I haven't used it--there are (at least) two others. The one I use is Perl Navigator, at https://marketplace.visualstudio.com/items?itemName=bscan.perlnavigator, and I have found it to be absolutely amazing. It replaces several other extensions I was using, and I've not had any trouble installing or using it. Additionally, on the repo, there are instructions for installing it in Sublime, Neovim, Kate, and Emacs, so it is multi-editor friendly.
And, as leonerd points out, there's a good tree-sitter implementation that exists already; no need to reinvent wheels for that part, at least.
Your proposal sounds new and shiny and theoretically wonderful, but I'm not seeing the benefits for that cost.
Perl navigator also uses microsoft's LSP, but for parsing the code, it uses perl - and what it does is, it checks syntax, applies perlcritic, etc, by using perl modules.
The main difference between Perl navigator and this grant's implementation is the way the code is parsed. Our implementation is going to parse and give the entire syntax tree, and also would be able to update the syntax tree on every keystroke. Refer the introduction part of this tool - https://tree-sitter.github.io/tree-sitter/
And, if we aren't dependent on Perl for parsing perl. This could be used for online IDE's as well, which most of it runs on JavaScript - frontend. The applications are endless!
Personally when I have used perl navigator in our Org, it doesn't quite understand our code for navigation. We have used this extension though - https://marketplace.visualstudio.com/items?itemName=cfgweb.vscode-perl.
This extension uses ctags for navigation. And its pretty fast for us.
This grant proposal is very muddled. It proposes to build a whole ide but then wants to build another language server?
In addition to PerlNavigator there are also already two LSP's on metacpan.
The tree sitter for perl already has an implementation.
If there is a feature gap in an existing implementation (applying to lsp's and tree sitter) it would be better to focus the grant application on closing that feature gap.
Retrospective payment for existing work also seems like an unusual request.
Hi @ganezdragon, I agree that Tree-sitter is an excellent feature, one that a variety of editors are looking to incorporate. I'm the maintainer of the Perl Navigator and would like to share some thoughts.
The Perl Navigator currently operates without any non-core Perl dependencies. In vscode, it can be installed directly from the marketplace and should just work, accommodating Perl versions as far back as 5.8. While Perltidy and Perlcritic are widely appreciated add-ons, they are not required to use the Navigator.
We've also integrated a browser-based mode within the Navigator, allowing for operation without any reliance on Perl. Although this is in need of more development, it can be currently be used on github.dev and vscode.dev. However, due to the lack of Perl, this will always be severely limited in features; you'll never be able to navigate to core dependencies, nor run perltidy or perlcritic. If you wanted to collaborate, tree-sitter would be a great addition to the Navigator, especially for the browser-based mode to improve the navigation and outline features. This is generally how Microsoft supports languages as well: they use compilers and native tools when available, and use tree-sitter in the browser. For reference, see https://marketplace.visualstudio.com/items?itemName=ms-vscode.anycode, the Microsoft tree-sitter option for running in the browser. Tree-sitter is of course limited, and Microsoft displays a warning when relying on the anycode tree-sitter based LSPs of "Partial Mode - Language Support for this file is inaccurate".
If the Perl Navigator is unable to parse some of the code at Athena, I'd appreciate if you can file an issue. By heavily leveraging the Perl compiler, it should be able to parse all valid Perl code and resolve all "use" dependencies. There are some cases like "require" statements, and various edge cases that limit the scope, but it would be nice to address all of these issues over time. Regarding ctags, the Navigator itself contains a ctag style approach originally based on perl-tags-vim. I'd like to expand this tagging to cover any code not currently being parsed (for both the browser and desktop versions of the LSP). I'd love to collaborate, and have already received and resolved github issues from two other perl developers at Athena. Let me know if you're interested.