Merge pull request #11753 from IQSS/11747-review-dataset-type

Review Datasets

Author: scolapasta

doc/release-notes/11747-review-dataset-type.md

@@ -0,0 +1,24 @@
+## Highlights
+
+### Review Datasets
+
+Dataverse now supports review datasets, a type of dataset that can be used to review resources such as other datasets in the Dataverse installation itself or various resources in external data repositories. APIs and a new "review" metadata block (with an "Item Reviewed" field) are in place but the UI for this feature will only available in a future version of the new React-based [Dataverse Frontend](https://github.com/IQSS/dataverse-frontend). See also the [guides](https://dataverse-guide--11753.org.readthedocs.build/en/11753/api/native-api.html#add-dataset-type), #11747, #12015, #11887, #12115, and #11753.
+
+## Other Features Added
+
+- Citation Style Language (CSL) output now includes "type:software" or "type:review" when those dataset types are used. See the [guides](https://dataverse-guide--11753.org.readthedocs.build/en/11753/api/native-api.html#get-citation-in-other-formats) and #11753.
+
+## Updated APIs
+
+- The Change Collection Attributes API now supports `allowedDatasetTypes`. See the [guides](https://dataverse-guide--11753.org.readthedocs.build/en/11753/api/native-api.html#change-collection-attributes), #12115, and #11753.
+
+## Bugs Fixed
+
+- 500 error when deleting dataset type by name. See #11833 and #11753.
+- Dataset Type facet works in JSF but not the SPA. See #11758 and #11753.
+
+## Backward Incompatible Changes
+
+### Dataset Types Must Be Allowed, Per-Collection, Before Use
+
+In previous releases of Dataverse, as soon as additional dataset types were added (such as "software", "workflow", etc.), they could be used by all users when creating datasets (via API only). As of this release, on a per-collection basis, superusers must allow these dataset types to be used. See #12115 and #11753.

doc/sphinx-guides/source/_static/api/dataset-create-review.json

@@ -0,0 +1,99 @@
+{
+  "datasetType": "review",
+  "datasetVersion": {
+    "license": {
+      "name": "CC0 1.0",
+      "uri": "http://creativecommons.org/publicdomain/zero/1.0"
+    },
+    "metadataBlocks": {
+      "citation": {
+        "fields": [
+          {
+            "typeName": "title",
+            "value": "Review of Percent of Children That Have Asthma",
+            "typeClass": "primitive",
+            "multiple": false
+          },
+          {
+            "value": [
+              {
+                "authorName": {
+                  "value": "Wazowski, Mike",
+                  "typeClass": "primitive",
+                  "multiple": false,
+                  "typeName": "authorName"
+                }
+              }
+            ],
+            "typeClass": "compound",
+            "multiple": true,
+            "typeName": "author"
+          },
+          {
+            "value": [
+              {
+                "datasetContactEmail": {
+                  "value": "mwazowski@mailinator.com",
+                  "typeClass": "primitive",
+                  "multiple": false,
+                  "typeName": "datasetContactEmail"
+                }
+              }
+            ],
+            "typeClass": "compound",
+            "multiple": true,
+            "typeName": "datasetContact"
+          },
+          {
+            "value": [
+              {
+                "dsDescriptionValue": {
+                  "value": "This is a review of a dataset.",
+                  "typeClass": "primitive",
+                  "multiple": false,
+                  "typeName": "dsDescriptionValue"
+                }
+              }
+            ],
+            "typeClass": "compound",
+            "multiple": true,
+            "typeName": "dsDescription"
+          },
+          {
+            "value": [
+              "Medicine, Health and Life Sciences"
+            ],
+            "typeClass": "controlledVocabulary",
+            "multiple": true,
+            "typeName": "subject"
+          },
+          {
+            "value": {
+              "itemReviewedUrl": {
+                "value": "https://datacommons.org/tools/statvar#sv=Percent_Person_Children_WithAsthma",
+                "typeClass": "primitive",
+                "multiple": false,
+                "typeName": "itemReviewedUrl"
+              },
+              "itemReviewedType": {
+                "value": "Dataset",
+                "typeClass": "controlledVocabulary",
+                "multiple": false,
+                "typeName": "itemReviewedType"
+              },
+              "itemReviewedCitation": {
+                "value": "\"Statistical Variable Explorer - Data Commons.\" Datacommons.org, 2026, datacommons.org/tools/statvar#sv=Percent_Person_Children_WithAsthma. Accessed 9 Mar. 2026.",
+                "typeClass": "primitive",
+                "multiple": false,
+                "typeName": "itemReviewedCitation"
+              }
+            },
+            "typeClass": "compound",
+            "multiple": false,
+            "typeName": "itemReviewed"
+          }
+        ]
+      }
+    }
+  }
+}

doc/sphinx-guides/source/admin/dataverses-datasets.rst

@@ -109,6 +109,64 @@ If the :AllowedCurationLabels setting has a value, one of the available choices
 
 Individual datasets can be configured to use specific curationLabelSets as well. See the "Datasets" section below.
 
+.. _review-datasets-setup:
+
+Configure a Collection for Review Datasets
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+:ref:`review-datasets-user` are a specialized type of dataset that can be used to review resources (such as datasets) in the Dataverse installation itself or resources in external data repositories.
+
+Review datasets require some setup, as described below.
+
+Load the Review Metadata Block
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+First, download the Review metadata block tsv file from :ref:`experimental-metadata`.
+
+Then, load the block and update Solr. See the following sections of :doc:`metadatacustomization` for details:
+
+- :ref:`load-tsv`
+- :ref:`update-solr-schema`
+
+Create and Enable Custom "Rubric" Metadata Blocks
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The Review metadata block gives you a few basic fields common to all reviews such as the URL of the item being reviewed.
+
+You probably will want to create your own metadata blocks specific to the resources you are reviewing, your own "rubric". See :doc:`metadatacustomization` for details on creating and enabling custom metadata blocks.
+
+Instead of creating a new custom metadata block from scratch (if you simply want to evaluate the feature, for example), you can use the metadata blocks at https://github.com/IQSS/dataverse.harvard.edu
+
+After loading the block, don't forget to update the Solr schema!
+
+Create a Review Dataset Type
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Review datasets are built on the :ref:`dataset-types` feature. Dataset types can only be created via API so follow the steps under :ref:`api-add-dataset-type`. Copy and paste from below or download :download:`review.json <../../../../scripts/api/data/datasetTypes/review.json>` and pass it to the API.
+
+.. literalinclude:: ../../../../scripts/api/data/datasetTypes/review.json
+   :language: json
+
+Create a Collection for Reviews and Configure Permissions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Follow the normal steps:
+
+- :ref:`create-dataverse`.
+- :ref:`dataverse-permissions`.
+
+Allow the Review Dataset Type for the Collection
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Non-dataset types, such as the "review" type, are only available when a collection admin has enabled them, via API.
+
+Using the API :ref:`collection-attributes-api`, change the ``allowedDatasetTypes`` attribute so that it includes "review". If you only want to allow reviews, you can pass just ``review``. If you want to allow multiple dataset types, you can pass a comma-separated list, such as ``review,dataset``.
+
+Invite Users to Create Review Datasets
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+At this point, users should be able to create review datasets via API, if you gave them permission on the collection. You can point them to :ref:`creating-a-review-dataset` for details.
+
 Datasets
 --------
 

doc/sphinx-guides/source/admin/metadatacustomization.rst

@@ -444,6 +444,8 @@ Please note that metadata fields share a common namespace so they must be unique
 
 We'll use this command again below to update the Solr schema to accomodate metadata fields we've added.
 
+.. _load-tsv:
+
 Loading TSV files into a Dataverse Installation
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

doc/sphinx-guides/source/api/changelog.rst

@@ -10,6 +10,7 @@ This API changelog is experimental and we would love feedback on its usefulness.
 v6.9
 ----
 
+- When creating datasets that contain a datasetType, that datasetType must be allowed at the collection level. This can be accomplished by passing ``allowedDatasetTypes`` to the :ref:`collection-attributes-api` API.
 - The POST /api/admin/makeDataCount/{id}/updateCitationsForDataset processing is now asynchronous and the response no longer includes the number of citations. The response can be OK if the request is queued or 503 if the queue is full (default queue size is 1000).
 - The way to set per-format size limits for tabular ingest has changed. JSON input is now used. See :ref:`:TabularIngestSizeLimit`.
 - In the past, the settings API would accept any key and value. This is no longer the case because validation has been added. See :ref:`settings_put_single`, for example.

doc/sphinx-guides/source/api/native-api.rst

@@ -1015,8 +1015,8 @@ You should expect an HTTP 200 ("OK") response and JSON indicating the database I
 
 .. _api-create-dataset-with-type:
 
-Create a Dataset with a Dataset Type (Software, etc.)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Create a Dataset with a Dataset Type (Software, Workflow, Review, etc.)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 By default, datasets are given the type "dataset" but if your installation had added additional types (see :ref:`api-add-dataset-type`), you can specify the type.
 
@@ -1070,8 +1070,8 @@ Before calling the API, make sure the data files referenced by the ``POST``\ ed
 
 .. _import-dataset-with-type:
 
-Import a Dataset with a Dataset Type (Software, etc.)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Import a Dataset with a Dataset Type (Software, Workflow, Review, etc.)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 By default, datasets are given the type "dataset" but if your installation had added additional types (see :ref:`api-add-dataset-type`), you can specify the type.
 
@@ -1185,6 +1185,7 @@ The following attributes are supported:
 * ``affiliation`` Affiliation
 * ``filePIDsEnabled`` ("true" or "false") Restricted to use by superusers and only when the :ref:`:AllowEnablingFilePIDsPerCollection <:AllowEnablingFilePIDsPerCollection>` setting is true. Enables or disables registration of file-level PIDs in datasets within the collection (overriding the instance-wide setting).
 * ``requireFilesToPublishDataset`` ("true" or "false") Restricted to use by superusers. Defines if Dataset needs files in order to be published.  If not set the determination will be made through inheritance by checking the owners of this collection. Publishing by a superusers will not be blocked.
+* ``allowedDatasetTypes`` Restricted to use by superusers. By default "dataset" is implied. Pass a comma-separated list of dataset types (e.g. "dataset,software"). You cannot unset this attribute so if you want to delete a dataset type, set ``allowedDatasetTypes`` to a dataset type you won't be deleting. See also :ref:`dataset-types`.
 
 See also :ref:`update-dataverse-api`.
 
@@ -3966,11 +3967,13 @@ Usage example:
 
   curl -H "Accept:application/json" "$SERVER_URL/api/datasets/:persistentId/versions/$VERSION/citation?persistentId=$PERSISTENT_IDENTIFIER&includeDeaccessioned=true"
 
+.. _get-citation-in-other-formats:
+
 Get Citation In Other Formats
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Dataverse can also generate dataset citations in "EndNote", "RIS", "BibTeX", and "CSLJson" formats.
-Unlike the call above, which wraps the result in JSON, this API call sends the raw format with the appropriate content-type (EndNote is XML, RIS and BibTeX are plain text, and CSLJson is JSON). ("Internal" is also a valid value, returning the same content as the above call as HTML).
+Dataverse can also generate dataset citations in "EndNote", "RIS", "BibTeX", and "CSL" formats.
+Unlike the call above, which wraps the result in JSON, this API call sends the raw format with the appropriate content-type (EndNote is XML, RIS and BibTeX are plain text, and CSL is JSON). ("Internal" is also a valid value, returning the same content as the above call as HTML).
 This API call adds a format parameter in the API call which can be any of the values listed above.
 
 Usage example:
@@ -3994,6 +3997,7 @@ Usage example:
 
   curl "$SERVER_URL/api/datasets/:persistentId/versions/$VERSION/citation/$FORMAT?persistentId=$PERSISTENT_IDENTIFIER&includeDeaccessioned=true"
 
+The type under CSL can vary based on the dataset type, with "dataset", "software", and "review" as supported values. See also :ref:`dataset-types`.
 
 Get Citation by Preview URL Token
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -4246,30 +4250,46 @@ The fully expanded example above (without environment variables) looks like this
 Add Dataset Type
 ^^^^^^^^^^^^^^^^
 
-Note: Before you add any types of your own, there should be a single type called "dataset". If you add "software" or "workflow", these types will be sent to DataCite (if you use DataCite). Otherwise, the only functionality you gain currently from adding types is an entry in the "Dataset Type" facet but be advised that if you add a type other than "software" or "workflow", you will need to add your new type to your Bundle.properties file for it to appear in Title Case rather than lower case in the "Dataset Type" facet.
+The default dataset type is "dataset" and ships with Dataverse.
+
+Only superusers can add additional dataset types. Once added, they can only be used if a collection has been configured to allow them (see ``allowedDatasetTypes`` under :ref:`collection-attributes-api`).
+
+Here's an example of all available fields when creating a dataset type:
+
+.. literalinclude:: ../../../../scripts/api/data/datasetTypes/datasetTypeAllFields.json
+   :language: json
 
-With all that said, we'll add a "software" type in the example below. This API endpoint is superuser only. The "name" of a type cannot be only digits. Note that this endpoint also allows you to add metadata blocks and available licenses for your new dataset type by adding "linkedMetadataBlocks" and/or "availableLicenses" arrays to your JSON.
+Here's a description of each field:
+
+- ``name`` (required): Machine-readable name. Cannot be only digits.
+- ``displayName`` (required): Human-readable name.
+- ``description``: A description.
+- ``linkedMetadataBlocks``: Linking a dataset type with one or more metadata blocks results in additional fields from those blocks appearing in the output from the :ref:`list-metadata-blocks-for-a-collection` API endpoint. Use the machine-readable names of the blocks. See :ref:`api-link-dataset-type` for details.
+- ``availableLicenses``: Limits the dataset type to certain licenses. For example, a "software" dataset type could be limited to "MIT" and "Apache-2.0". See :ref:`dataset-types-set-available-licenses` for details.
+
+Download the :download:`datasetTypeAllFields.json <../../../../scripts/api/data/datasetTypes/datasetTypeAllFields.json>` file show above, edit it to suit your needs, and use it in the following command.
 
 .. code-block:: bash
 
   export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
   export SERVER_URL=https://demo.dataverse.org
-  export JSON='{"name":"software","linkedMetadataBlocks":["codeMeta20"],"availableLicenses":["MIT", "Apache-2.0"]}'
 
-  curl -H "X-Dataverse-key:$API_TOKEN" -H "Content-Type: application/json" "$SERVER_URL/api/datasets/datasetTypes" -X POST -d $JSON
+  curl -H "X-Dataverse-key:$API_TOKEN" -H "Content-Type: application/json" "$SERVER_URL/api/datasets/datasetTypes" -X POST --upload-file datasetTypeAllFields.json
 
 The fully expanded example above (without environment variables) looks like this:
 
 .. code-block:: bash
 
-  curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -H "Content-Type: application/json" "https://demo.dataverse.org/api/datasets/datasetTypes" -X POST -d '{"name":"software","linkedMetadataBlocks":["codeMeta20"],"availableLicenses":["MIT", "Apache-2.0"]}'
+  curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -H "Content-Type: application/json" "https://demo.dataverse.org/api/datasets/datasetTypes" -X POST --upload-file datasetTypeAllFields.json
+
+Note that adding certain dataset types will result in a value other than "Dataset" being sent to DataCite (if you use DataCite), see :ref:`dataset-types-datacite` for details.
 
 .. _api-delete-dataset-type:
 
 Delete Dataset Type
 ^^^^^^^^^^^^^^^^^^^
 
-Superuser only.
+Superuser only. Note that if a collection has the type listed as an allowed dataset type, you will be unable to delete the dataset type until you first use the :ref:`collection-attributes-api` to change ``allowedDatasetTypes`` to a dataset type (or dataset types) that you are not trying to delete.
 
 .. code-block:: bash
 
@@ -4315,6 +4335,8 @@ To update the blocks that are linked, send an array with those blocks.
 
 To remove all links to blocks, send an empty array.
 
+.. _dataset-types-set-available-licenses:
+
 Set Available Licenses for a Dataset Type
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 

doc/sphinx-guides/source/user/appendix.rst

@@ -17,6 +17,8 @@ The Dataverse Project is committed to using standard-compliant metadata to ensur
 metadata can be mapped easily to standard metadata schemas and be exported into JSON
 format (XML for tabular file metadata) for preservation and interoperability.
 
+.. _supported-metadata:
+
 Supported Metadata
 ~~~~~~~~~~~~~~~~~~
 
@@ -34,6 +36,8 @@ Detailed below are what metadata schemas we support for Citation and Domain Spec
 - Journal Metadata (`see .tsv <https://github.com/IQSS/dataverse/blob/master/scripts/api/data/metadatablocks/journals.tsv>`__): based on the `Journal Archiving and Interchange Tag Set, version 1.2 <https://jats.nlm.nih.gov/archiving/tag-library/1.2/chapter/how-to-read.html>`__.
 - 3D Objects Metadata (`see .tsv <https://github.com/IQSS/dataverse/blob/master/scripts/api/data/metadatablocks/3d_objects.tsv>`__).
 
+.. _experimental-metadata:
+
 Experimental Metadata
 ~~~~~~~~~~~~~~~~~~~~~
 
@@ -43,6 +47,7 @@ Unlike supported metadata, experimental metadata is not enabled by default in a
 - Computational Workflow Metadata (`see .tsv <https://github.com/IQSS/dataverse/blob/master/scripts/api/data/metadatablocks/computational_workflow.tsv>`__): adapted from `Bioschemas Computational Workflow Profile, version 1.0 <https://bioschemas.org/profiles/ComputationalWorkflow/1.0-RELEASE>`__ and `Codemeta <https://codemeta.github.io/terms/>`__.
 - Archival Metadata (`see .tsv <https://github.com/IQSS/dataverse/blob/master/scripts/api/data/metadatablocks/archival.tsv>`__): Enables repositories to register metadata relating to the potential archiving of the dataset at a depositor archive, whether that be your own institutional archive or an external archive, i.e. a historical archive.
 - Local Contexts Metadata (`see .tsv <https://github.com/gdcc/dataverse-external-vocab-support/blob/main/packages/local_contexts/cvocLocalContexts.tsv>`__): Supports integration with the `Local Contexts <https://localcontexts.org/>`__ platform, enabling the use of Traditional Knowledge and Biocultural Labels, and Notices. For more information on setup and configuration, see :doc:`../installation/localcontexts`.
+- Review Metadata (`see .tsv <https://github.com/IQSS/dataverse/blob/master/scripts/api/data/metadatablocks/review.tsv>`__): For :ref:`review-datasets-user`.
 
 Please note: these custom metadata schemas are not included in the Solr schema for indexing by default, you will need
 to add them as necessary for your custom metadata blocks. See "Update the Solr Schema" in :doc:`../admin/metadatacustomization`.