API for validating RO-Crate metadata only

[douglowe]

I've prepared an API call for validating the metadata within the RO-Crate JSON file.

Because of the roc-validator requirements this currently creates an RO-Crate directory which only contains the JSON file, and runs the base RO-Crate validation process without local file checks. Hopefully we might be given a way in future to pipe the metadata straight into the tool. The method here (asking for JSON data, rather than using a file upload), should make switching more seamless when that occurs.

Limitations:

• API could be more RESTful, but I'll address that in the next branch
• I've only checked this for the base RO-Crate profile (1.1). We might need to include custom exclusion commands if/when we want to expand to cover other profiles.

[alexhambley]

Looks good!

One thing I noticed looking at it now is that I think it might be a good idea to define separate schemas for endpoints? The reason is that right now validate_data(partial=True) means that the API fields are optional, but then we're throwing KeyErrors if they're missing, which doesn't quite make sense.

Also the validate_data / validate_json schema classes should be camel case! ;)

[douglowe]

Only some of the API fields are optional - though perhaps the checks for KeyErrors aren't needed as the required fields should throw an error anyway?

[douglowe]

We could do with some tests for the API, so that it's clearer how it responds (and should respond). I'll do some reading up and add a couple here.

[alexhambley]

Only some of the API fields are optional - though perhaps the checks for KeyErrors aren't needed as the required fields should throw an error anyway?

My point was that partial=True technically makes all fields optional, even those marked as required=True in the schema. I think it overrides the schema logic.

Edit: The current logic would work for something like a PATCH request, but I think doing something like:

class ValidateByIdSchema(Schema):
    crate_id = String(required=True)
    webhook_url = String(required=True)
    profile_name = String(required=False)

would perhaps make more sense?

[OliverWoolland]

Look decent to me - only question I have is if you guys are planning to add some tests?

[douglowe]

We definitely should, and will, add some tests. I'll start with some for this new functionality, and then we can expand these for the rest of the code base.

[douglowe]

With considerable help from ChatGPT I've setup an initial integration test for the metadata API. This uses @pytest.fixture for spinning up docker compose stack - which makes more testable locally (I think). I've retained this function in the GH action script for continuity (and just built the containers first, to avoid delays).

Next step will be the addition of unit tests for the functions within the code.

[OliverWoolland]

Looking good @douglowe! Do I get the sense you're having... fun?

Not used it myself but have considered https://pypi.org/project/pytest-docker-compose/ as an alternative to the subprocess calls - would be interested in your opinion

(don't seem to be able to link to a subheading on pypi but it was the "Running Integration Tests" section that caught my eye")

[douglowe]

I did definitely enjoy getting to do some coding.

https://pypi.org/project/pytest-docker-compose/ looks very interesting. But the last updates were 3 years ago, and it only supports python 3.6. I couldn't get pip to install it for python 3.11, so I think we'll have to skip it :(

[douglowe]

There is a fork of the above package: https://pypi.org/project/pytest-docker-compose-v2/#description

But the documentation is a bit old it seems - there's functions in the https://pypi.org/project/pytest-docker/ it recommends using that don't exist anymore.

For the moment, as these tests work, I'll stick with this code.

[OliverWoolland]

Sorry I didn't spot this earlier - all looks sound to me and I see the action is passing so all good for me!