Grant Proposal: General Perl OpenAPI Validator / Interpreter

Category: Grants

Comments (16)


https://metacpan.org/pod/JSON::Validator # has experimental support for OpenAPI v3 spec. Is maintained and widely used.


A valid observation but one of the driving forces behind creating a reference module for OpenAPI was to make it absolutely un-linked from any major framework; the module you have linked appears to require mojolicious - which would would invalidate the point of it being a reference implementation. The later scope for this particular proposal is that web frameworks (and other) may extend it freely for their own space.


JSON::Validator uses the Mojolicious distribution because of the utility modules (Mojo::JSON, Mojo::Util, Mojo::URL, Mojo::Base, etc) and it *probably* uses those because Mojolicious itself has a minimal non-core dependency chain. It is in no way coupled to the actual web framework other than in a couple of tests. If you're going to build something here you're probably going to be importing the likes of Cpanel::JSON::XS, some sort of URL module, some sort of OO framework, amongst others, so you're already pulling in many dependencies. I think it's a non-issue if the code itself is relatively well factored and loosely coupled to any framework - I pulled out the OAuth2 server stuff from my Mojolicious plugin for this very reason, even though it still pulls in Mojolicious to do some testing. I guess my major qualm here is not having n different OpenAPI related distributions on CPAN, given there's already a mature distribution that is close to having v3 of the spec supported. Just look at improving/completing the existing implementation to make it reference level . All of this stuff is moving so fast that having multiple implementations is a bad idea, one or more of those will get left behind very quickly.


Sorry for the word soup above, it seems the comments here don't honour any formatting.


re: word soup Indeed I noticed that while attempting my reply :) perhaps that would be one to suggest TPF work on. Anyhow, The initial thoughts was to use JSON::MaybeXS for most of the heavy lifting, simply as relying on XS modules in some cases such as when wanting to fatpack packet projects into a single script. We are thinking of not pulling in any OO modules and instead focusing on using a standard blessed object tree, though that is not finalised yet and is what the proposed analysis section is for. As for why to create the reference module from scratch instead of improving JSON::Validator is to allow an absolutely on point and specific module that deals only with OpenAPI v3(and possible v2) with no drift what so ever.


We are however looking for more people to get involved with making the OpenAPI module as on-point and 100% coverage viable as possible, it is indeed a target for it to be one of the few modules that does cover all aspects of the swagger spec, you appear to have a high motivation into achieving such a goal for perl - would you be interested in actually helping it become a reality if this grant passes?


I'd be interested but I am spread very thin at the moment and it's unlikely I have the time unless my employer can allow me the odd hour here and there. That *might* be a possibility. FWIW we are using the JSON::Validator module in our stack at the moment, so my motivation would be more biased towards it and getting our stack on OpenAPI v3 (and to clarify: I have no leaning towards any particular framework).


More word soup... I recently used this module in context of an OpenAPI v2 service and was not a great fan of its interface, or at least of the documentation surrounding it (since I came across an interaction between its interface and the interface of modules it uses, I found it hard to do what I wanted). Secondarily, when I tried doing OpenAPI v3 stuff (admittedly some time ago) this module didn't show up in my searches, and nor did any other module that was DWIM enough - and understandable enough - for my purposes. I daresay that if we delve into this module and find that it implements a lot of what we need, the outcome of the first stage - the specification - could be to make a more user-friendly API over the top of this module. This said, as mentioned by Paul, we were intending to have a low-dependency distribution; I hadn't realised this module has so many dependencies itself, which means it might not be suitable for our target. Decisions like this would be made during the first stage: perhaps, in fact, we put time into completing this experimental feature, and removing those dependencies. In any case. looking at a module called JSON::Validator when you want to do OpenAPI stuff doesn't seem like the obvious thing to do, and doesn't leave room for expansion into other things that are not simply validation.


My apologies for the word soup as well ;)


From https://metacpan.org/pod/Mojolicious::Plugin::OpenAPI: "Mojolicious::Plugin::OpenAPI is Mojolicious::Plugin that add routes and input/output validation to your Mojolicious application based on a OpenAPI (Swagger) specification. This plugin supports both version 2.0 and 3.x, though 3.x might have some missing features." We use that quite successfully for OpenAPI v3 validatin in our Mojo based projects. Looking at the source this is based on https://metacpan.org/pod/JSON::Validator which in turn says: "JSON::Validator can load JSON schemas in multiple formats: Plain perl data structured (as shown in "SYNOPSIS"), JSON or YAML. The JSON parsing is done with Mojo::JSON, while YAML files requires YAML::PP or YAML::XS." "This module comes with some JSON specifications bundled, so your application don't have to fetch those from the web. These specifications should be up to date, but please submit an issue if they are not." "Here is the list of the bundled specifications: ... Swagger / OpenAPI specification, version 2 ... OpenAPI specification, version 3 ... This specification is still EXPERIMENTAL." While OpenAPI v3 being experimental, I still think it should at least be evaluated if it would not be worthwhile to fix that instead of starting a whole new module. The same holds true for the dependency on some Mojo modules, which seem to be mostly (if not only) "support modules" not really related to the framework aspects of Mojolicious as already described above. For example, Mojo::JSON basically uses JSON::PP or (optionally) Cpanel::JSON::XS.


Oops, more word soup ... is there any documentation somewhere for how formatting works here?


If something new needs to be implemented, I would suggest https://modules.raku.org/dist/JSON::Schema and https://modules.raku.org/dist/OpenAPI::Schema::Validate for inspiration (knowing the SW design skills of author).


json-schema.org puts out a new draft specification version about once a year. Draft 2020-XX is due very soon, coinciding with the release of OpenAPI v3.1 which will depend on this draft. Note that OpenAPI v3.0.3, the latest published specification so far, uses parts of JSON Schema draft-04 but also makes some of its own modifications on top. OpenAPI v3.1 will be fully harmonized with the JSON Schema specification, requiring much less additional code on top. I'm the author of JSON::Schema::Draft201909 (https://metacpan.org/release/JSON-Schema-Draft201909). I wrote this implementation because JSON::Validator only supports up to draft 07 of the json-schema.org specification and I needed newer features in my application. I've also joined the json-schema.org working groups, reviewing new parts of the specification as it is written and also contributing to its test suite (https://github.com/json-schema-org/json-schema-spec and https://github.com/json-schema-org/JSON-Schema-Test-Suite respectively). I have also been in contact with the two authors of the JSON::Schema module on cpan (no longer maintained since JSON Schema draft-03) with the plan to take over this namespace as soon as the authors are comfortable that all the previous features are forward-compatible with my module; I've already taken over the cpan module Test::JSON::Schema::Acceptance (https://metacpan.org/release/Test-JSON-Schema-Acceptance) by the same authors which contains the updated acceptance test suites from json-schema.org for guaranteeing specification compliance. Regarding this grant request: if it is the intent to write a wholly new JSON Schema evaluation implementation, the schedule is unreastically optimistic. To get to this point implementing JSON::Schema::Draft201909 has taken approximately six calendar months, and at least one month x 40 hour weeks of developer time. However, pivoting to supporting Draft 2020-XX of the JSON Schema (when it is finalized) should be fairly quick as I am quite familiar with the changes involved and have architected the evaluator with these changes in mind. Additionally, I would suggest that any new OpenAPI implementation should target v3.1, not v3.0, which is quite old by now and has accumulated a very large list of issues which are being addressed by the new draft revision. This new version requires much less support atop JSON Schema evaluators -- in fact most of the additional code that I envision would be required is exactly the type of code that this grant request says it wants to avoid -- coupling into specific web frameworks. There really isn't any other work to be done - it's all a matter of "fetch the API specs for this website", "evaluate the incoming requests against the schemas", "issue an error response when validation fails", and also potentially "automatically generate my API endpoints from the specification document". (This is also what Mojolicious::Plugin::OpenAPI does for openapi spec v2 and v2.) - Karen Etheridge, ether@cpan.org


formatted version of my above comments: https://gist.github.com/karenetheridge/c8bd55475971c51822a42270f15bea24


Good day all, I have recently been having a chat with a few of the commentators on irc.freenode.net #perl-openapi, such as the author of JSON::Validator and a few others. I believe I have isolated some concerns with this proposal and some issues with how I have described some aspects of it, so with that said I will be hopefully working with laurence and alistair this weekend to make an amendment to the plan with some more specifics. I will also be running this plan via the openapi IRC channel to hopefully get as complete of a proposal as absolutely possible that shows with absolute clarity the milestones and intended end product.


Please see the updated version of this proposal and comment there: https://news.perlfoundation.org/post/updated_grant_proposal_general_perl_openapi_validator


Sign in to add comment