|
| 1 | +# FAIR Testing Resource Vocabulary (FTR) |
| 2 | +[](https://www.repostatus.org/#active) |
| 3 | + |
| 4 | +Repository to track the requirements and specifications of FAIR assessment results, metrics, benchmarks and scoring algorithms. |
| 5 | + |
| 6 | +**Permanent identifier:** [https://w3id.org/ftr#](https://w3id.org/ftr#) (click to see documentation and examples) |
| 7 | + |
| 8 | +**Authors**: Daniel Garijo, Mark Wilkinson, Pablo Alarcón, Esteban Gonzalez-Guardia, Tomasz Miksa, Lukas Arnhold, Allyson Lister, Elli Papadopoulou, Leonidas Pispiringas |
| 9 | + |
| 10 | +**Contributors**: Rober Huber, Neil Chue Hong, Clement Jonquet, Wim Hugo |
| 11 | + |
| 12 | +**Source document**: [https://docs.google.com/document/d/1HusredfHgymRg2ub4L0GnVSRV8IWZvFJyMkE6POejpc/edit?usp=sharing](https://docs.google.com/document/d/1HusredfHgymRg2ub4L0GnVSRV8IWZvFJyMkE6POejpc/edit?usp=sharing), with contributions from an initial modeling by Robert Huber and a [diagram](https://owncloud.tuwien.ac.at/index.php/s/VaGxqnf5MxfDtzz#/files_mediaviewer/dmp-dqv.png) authored by Lukas Arnhold. |
| 13 | + |
| 14 | +**Ongoing document**: [Assessment components glossary and description](https://docs.google.com/document/d/1_dFj5bi6JKlcGZt40BEzVWUg6NjG1CBzGnCWAaad6NU/edit?usp=sharing) and [Metadata elements for FAIR assessment](https://docs.google.com/spreadsheets/d/1QKOoy-YJLNgnQywe6wOTTCBqCg0YuRWmIT-xFoB3ZCo/edit?usp=sharing) |
| 15 | + |
| 16 | + |
| 17 | +## Core test result representation |
| 18 | +We distinguish the following main concepts: |
| 19 | +- **TestResult**: Output of running a test over a resource. A test result also should contain provenance metadata about the process followed to create it. `TestResult`is represented as an extension of `prov:Entity`. A test result points to the corresponding test through the `ftr:outputFromTest` property. |
| 20 | +- **TestResultSet**: A set of FAIR test results, together with their respective metadata. Common metadata may describe the set. For example, if all results where run by a request to the same API. |
| 21 | +- **Test**: Service, formed by an API and associated piece of code that implements a Metric, and is executed (by a FAIR assessment tool), retrieving a particular and standardised result. |
| 22 | +- **TestExecutionActivity**: The action carried out by an agent of calling an API in which a test (or set of tests) were run. The result of this activity is either a `TestResult` or a `TestResultSet`. |
| 23 | +- **Metric**: Narrative description that a Test must wholly implement. Metrics may be domain-agnostic or not. Metrics do not include technical details on their implementation. |
| 24 | +- **Dimension**: Represents criteria relevant for an assessment (e.g., one of the FAIR principles). Each quality dimension must have one or more metric to measure it (definition adapted from [W3C Data Quality Vocabulary](https://www.w3.org/TR/vocab-dqv/#dqv:Dimension)). |
| 25 | +- **Benchmark**: Benchmarks are community-specific groupings of a set of Metrics that provides a narrative of those particular ways in which that community defines FAIR for assessment purposes. |
| 26 | +- **ScoringAlgorithm**: Piece of code that contextualises the sum of all test results for a given benchmark, into a final quantitative assessment result. |
| 27 | +- **BenchmarkScore**: Score obtained after executing a scoring algorithm over a set of test results. The benchmark score includes a value, a log and a link to the test results used to obtain the score. |
| 28 | + |
| 29 | + |
| 30 | + |
| 31 | +## Context |
| 32 | +If you want to use a JSON-LD serialization to implement FTR, we recommend using the [ftr contex](https://w3id.org/ftr/context). It will help you making your JSON-LD documents easier to read (see some [examples](https://w3id.org/ftr#desc) in the specification too). |
| 33 | + |
| 34 | +## OpenAPI template |
| 35 | +If you want to implement a REST API following the FTR specification, you may use the [OpenAPI template](https://github.com/OSTrails/FAIR_testing_resource_vocabulary/blob/main/development/api/open_api_description.yaml) provided in this repository. See [an example implementation](https://w3id.org/foops/api)) |
| 36 | + |
| 37 | +## Requirements and Competency Questions |
| 38 | +The proposed vocabulary is derived from a set of competency questions, available at the `cqs` folder. CQs come from: |
| 39 | +* Requirements based on the experience of the authors (as FAIR assessment tool developers) |
| 40 | +* The [open document](https://docs.google.com/document/d/1HusredfHgymRg2ub4L0GnVSRV8IWZvFJyMkE6POejpc/edit?usp=sharing) where experts from various projects regarding FAIR assessment have gathered. |
| 41 | + |
| 42 | +## Examples |
| 43 | +Please see the [latest specification draft for examples](https://w3id.org/ftr#desc) of tests, metrics, benchmarks and scoring algorithms. |
| 44 | + |
| 45 | +## Reference implementations |
| 46 | +The FTR specification has been implemented through four different reference implementations: |
| 47 | +- FAIR Champion: https://tools.ostrails.eu/champion |
| 48 | +- FOOPS!: https://w3id.org/foops (see API [here](https://w3id.org/foops/api)) |
| 49 | +- Research Software FAIRness Checks (RSFC): https://w3id.org/rsfc/ (see API [here](https://w3id.org/rsfc/api)) |
| 50 | +- FAIR for Research Objects (FAIROs): https://w3id.org/FAIROS/ |
0 commit comments