Add CSL-JSON import endpoint for correct entry-type mapping (#16151)

koppor · claude · subhramit · web-flow · commit 47f441ec05fc · 2026-07-07T12:27:09.000Z
* Add CSL-JSON import endpoint for correct entry-type mapping Extend `POST /libraries/{id}/entries` to accept CSL-JSON (`application/vnd.citationstyles.csl+json`), so browser translators (and any CSL-JSON producer) can hand items to JabRef and get the correct entry type instead of a flat @Article. Rather than a bespoke Zotero-itemType -> BibTeX map (rejected by ADR 0064), reuse the merged citation-js-based CSL mapping (`CSLItemTypeDefinitions`, PR #15946): extract the CSL-JSON -> BibEntry core from `ZoteroCitationMarkParser.toBibEntry` into a new public `parseCslJsonItems(String)` that accepts a bare CSL item or an array. `EntriesResource.addCslJson` serialises the mapped entries to BibTeX and appends them via the existing `AppendBibTeXToLibrary` command, preserving duplicate handling and group assignment. Entries carry no citation key so JabRef generates keys on import. Freshly built entries are written with reformat=true (empty parsed serialisation otherwise writes nothing). Tests: type/field mapping and array/object/blank/invalid handling in `ZoteroCitationMarkParserTest`; endpoint conversion + 400 paths in `EntriesResourceTest`. Related to #15875 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(changelog): link CSL-JSON endpoint entry to PR #16151 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Drop dead null guards in parseCslJsonItems The isJsonArray/isJsonObject guards already ensure Gson returns non-null, so the != null checks were unreachable; removing them also satisfies the no-null-check style rule. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Add CSL-JSON demo requests to import.http Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Guard CSL-JSON import against explicit null fields CSL-JSON that sets optional properties explicitly to null ("type": null, "author": null, "issued": null, "issued": {"date-parts": null}) crashed the import path with an NPE (Gson overwrites the field defaults with null, and the immutable CSL mapping tables reject a null key lookup), surfacing as HTTP 500. Normalise type/author/issued and guard date-parts before use. Add a parameterised regression test covering each null shape. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Use Markdown Javadoc for the group parameter of addCslJson Replace the traditional @param tag with a Markdown sentence, per the project's Markdown-Javadoc guideline for new docs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Table-ise the null-field test with @CsvSource Use @CsvSource with a `|` delimiter (JSON keeps its commas) so the null-field cases read as a table of input -> expected entry type, and clean up the provider imports. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Apply OpenRewrite cleanup to ZoteroCitationMarkParserTest Replace list.get(0) with list.getFirst() per SequencedCollection recipe. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Update jablib/src/main/java/org/jabref/logic/openoffice/ZoteroCitationMarkParser.java Co-authored-by: Subhramit Basu <subhramit.bb@live.in> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Subhramit Basu <subhramit.bb@live.in>
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -11,6 +11,7 @@ Note that this project **does not** adhere to [Semantic Versioning](https://semv
 
 ### Added
 
+- The HTTP import endpoint (`POST /libraries/{id}/entries`) now accepts CSL-JSON (`application/vnd.citationstyles.csl+json`), mapping each item to the correct entry type (e.g. conference paper, book chapter, thesis) via the citation-js-based mapping. [#16151](https://github.com/JabRef/jabref/pull/16151)
 - We added a new "Main" tab to the entry editor showing all fields of an entry in a single scrollable list, with one-click chips for adding optional fields and a free-form box for adding arbitrary fields. Identifiers, files and links, bibliometrics, comments, and meta fields (groups, owner, timestamps, special fields) live in collapsible sections — collapsed when empty — each offering chips for its unset fields. [#12711](https://github.com/JabRef/jabref/issues/12711)
 - We added auto-detection import for drag-and-dropped library files. [#15391](https://github.com/JabRef/jabref/issues/15391)
 - We added a preview style selection bar which shows the current preview style and allows to select a specific style without cycling through all of them. [#15820](https://github.com/JabRef/jabref/pull/15820)
diff --git a/jablib/src/main/java/org/jabref/logic/openoffice/ZoteroCitationMarkParser.java b/jablib/src/main/java/org/jabref/logic/openoffice/ZoteroCitationMarkParser.java
@@ -1,6 +1,7 @@
 package org.jabref.logic.openoffice;
 
 import java.util.ArrayList;
+import java.util.Collections;
 import java.util.List;
 import java.util.Map;
 import java.util.NoSuchElementException;
@@ -13,10 +14,11 @@
 import org.jabref.model.entry.Date;
 import org.jabref.model.entry.field.Field;
 import org.jabref.model.entry.field.StandardField;
-import org.jabref.model.entry.types.EntryType;
 
 import com.google.gson.Gson;
+import com.google.gson.JsonElement;
 import com.google.gson.JsonParseException;
+import com.google.gson.JsonParser;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -52,6 +54,40 @@ public static List<BibEntry> parse(String referenceMarkName) {
         }
     }
 
+    /// Parses a bare CSL-JSON payload (a single CSL item object, or an array of them) into
+    /// [BibEntry] instances, reusing the citation-js-based CSL item-type/field mapping (ADR 0064).
+    ///
+    /// Unlike [#parse(String)] this expects the CSL items directly (as produced by a Zotero /
+    /// citation-js CSL-JSON export), not a Zotero reference-mark wrapper. The produced entries have
+    /// no citation key, so callers importing them let JabRef generate keys from its pattern.
+    ///
+    /// @return the parsed list of entries or an empty list when the input is blank or not parsable CSL JSON.
+    public static List<BibEntry> parseCslJsonItems(String cslJson) {
+        if (StringUtil.isBlank(cslJson)) {
+            return List.of();
+        }
+
+        try {
+            JsonElement root = JsonParser.parseString(cslJson);
+            List<ZoteroCitationData.ItemData> items = new ArrayList<>();
+            // The isJsonArray/isJsonObject guards ensure Gson returns a non-null result here.
+            if (root.isJsonArray()) {
+                Collections.addAll(items, GSON.fromJson(root, ZoteroCitationData.ItemData[].class));
+            } else if (root.isJsonObject()) {
+                items.add(GSON.fromJson(root, ZoteroCitationData.ItemData.class));
+            }
+
+            List<BibEntry> entries = new ArrayList<>();
+            for (ZoteroCitationData.ItemData itemData : items) {
+                toBibEntry(itemData).ifPresent(entries::add);
+            }
+            return entries;
+        } catch (JsonParseException | NumberFormatException | NoSuchElementException e) {
+            LOGGER.debug("Could not parse CSL JSON items", e);
+            return List.of();
+        }
+    }
+
     private static Optional<String> extractCSLJSON(String referenceMarkName) {
         int jsonStart = referenceMarkName.indexOf('{');
         int jsonEnd = referenceMarkName.lastIndexOf('}');
@@ -63,14 +99,22 @@ private static Optional<String> extractCSLJSON(String referenceMarkName) {
     }
 
     private static Optional<BibEntry> toBibEntry(ZoteroCitationData.CitationItemData citationItem) {
-        ZoteroCitationData.ItemData itemData = citationItem.itemData;
-
-        EntryType entryType = CSLItemTypeDefinitions.getEntryType(itemData.type);
-        BibEntry entry = new BibEntry(entryType);
-        entry.withCitationKey("Zotero-" + (citationItem.id));
-        setAuthors(itemData.author).ifPresent(authors -> entry.withField(StandardField.AUTHOR, authors));
-        setDate(entry, itemData.issued);
-        for (Map.Entry<String, Field> fieldMapping : CSLItemTypeDefinitions.getFieldMappings(itemData.type, itemData).entrySet()) {
+        return toBibEntry(citationItem.itemData)
+                .map(entry -> entry.withCitationKey("Zotero-" + citationItem.id));
+    }
+
+    private static Optional<BibEntry> toBibEntry(ZoteroCitationData.ItemData itemData) {
+        // Gson replaces the field defaults with null when the JSON sets these keys explicitly to
+        // null (e.g. "type": null, "author": null, "issued": null), so normalise before use. The
+        // CSL mapping tables are immutable maps, which reject a null key lookup with an NPE.
+        String type = itemData.type == null ? "" : itemData.type;
+        BibEntry entry = new BibEntry(CSLItemTypeDefinitions.getEntryType(type));
+        List<ZoteroCitationData.AuthorData> authors = itemData.author == null ? List.of() : itemData.author;
+        setAuthors(authors).ifPresent(value -> entry.withField(StandardField.AUTHOR, value));
+        if (itemData.issued != null) {
+            setDate(entry, itemData.issued);
+        }
+        for (Map.Entry<String, Field> fieldMapping : CSLItemTypeDefinitions.getFieldMappings(type, itemData).entrySet()) {
             setField(entry, fieldMapping.getValue(), itemData.getFieldValue(fieldMapping.getKey()));
         }
 
@@ -92,7 +136,7 @@ private static Optional<String> setAuthors(List<ZoteroCitationData.AuthorData> a
     }
 
     private static void setDate(BibEntry entry, ZoteroCitationData.IssuedData issuedData) {
-        if (issuedData.dateParts.isEmpty()) {
+        if ((issuedData.dateParts == null) || issuedData.dateParts.isEmpty()) {
             return;
         }
 
diff --git a/jablib/src/test/java/org/jabref/logic/openoffice/ZoteroCitationMarkParserTest.java b/jablib/src/test/java/org/jabref/logic/openoffice/ZoteroCitationMarkParserTest.java
@@ -15,9 +15,12 @@
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.params.ParameterizedTest;
 import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.CsvSource;
 import org.junit.jupiter.params.provider.MethodSource;
+import org.junit.jupiter.params.provider.ValueSource;
 
 import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
 
 class ZoteroCitationMarkParserTest {
 
@@ -340,4 +343,92 @@ private static Stream<CSLTestCase> parseCSLJSON() {
                 )
         );
     }
+
+    @Test
+    void parseCslJsonItemsAcceptsBareItemObject() {
+        List<BibEntry> entries = ZoteroCitationMarkParser.parseCslJsonItems("""
+                {
+                  "type": "paper-conference",
+                  "title": "Conference paper",
+                  "container-title": "Proceedings title"
+                }
+                """);
+
+        assertEquals(1, entries.size());
+        BibEntry entry = entries.getFirst();
+        assertEquals(StandardEntryType.InProceedings, entry.getType());
+        assertEquals(Optional.of("Conference paper"), entry.getField(StandardField.TITLE));
+        assertEquals(Optional.of("Proceedings title"), entry.getField(StandardField.BOOKTITLE));
+    }
+
+    @Test
+    void parseCslJsonItemsLeavesCitationKeyEmptyForImport() {
+        List<BibEntry> entries = ZoteroCitationMarkParser.parseCslJsonItems("""
+                {"type": "article-journal", "title": "T"}
+                """);
+
+        assertEquals(Optional.empty(), entries.getFirst().getCitationKey());
+    }
+
+    @Test
+    void parseCslJsonItemsAcceptsArrayOfItems() {
+        List<BibEntry> entries = ZoteroCitationMarkParser.parseCslJsonItems("""
+                [
+                  {"type": "book", "title": "A book"},
+                  {"type": "thesis", "title": "A thesis"}
+                ]
+                """);
+
+        assertEquals(2, entries.size());
+        assertEquals(StandardEntryType.Book, entries.getFirst().getType());
+        assertEquals(StandardEntryType.Thesis, entries.get(1).getType());
+    }
+
+    @Test
+    void parseCslJsonItemsMapsFieldsPerType() {
+        List<BibEntry> entries = ZoteroCitationMarkParser.parseCslJsonItems("""
+                {
+                  "type": "article-journal",
+                  "title": "An article",
+                  "container-title": "A journal",
+                  "author": [{"family": "Doe", "given": "Jane"}],
+                  "issued": {"date-parts": [["2021", 5]]}
+                }
+                """);
+
+        BibEntry entry = entries.getFirst();
+        assertEquals(StandardEntryType.Article, entry.getType());
+        assertEquals(Optional.of("A journal"), entry.getField(StandardField.JOURNALTITLE));
+        assertEquals(Optional.of("Doe, Jane"), entry.getField(StandardField.AUTHOR));
+        assertEquals(Optional.of("2021"), entry.getField(StandardField.YEAR));
+    }
+
+    @ParameterizedTest
+    @ValueSource(strings = {"", "   ", "not json", "{ broken"})
+    void parseCslJsonItemsReturnsEmptyForBlankOrInvalid(String input) {
+        assertTrue(ZoteroCitationMarkParser.parseCslJsonItems(input).isEmpty());
+    }
+
+    /// CSL-JSON that explicitly sets optional composite properties to null must not throw
+    /// (Gson overwrites the field defaults with null on explicit `"key": null`, and the immutable
+    /// CSL mapping tables reject a null key lookup). `|` is the column separator so the JSON commas
+    /// stay intact.
+    @ParameterizedTest
+    @CsvSource(delimiter = '|', textBlock = """
+            # CSL JSON                                                                | entry type
+            {"type": "article-journal", "title": "T", "author": null}                | Article
+            {"type": "article-journal", "title": "T", "issued": null}                | Article
+            {"type": "article-journal", "title": "T", "issued": {"date-parts": null}} | Article
+            {"type": null, "title": "T"}                                             | Misc
+            """)
+    void parseCslJsonItemsHandlesExplicitNullFields(String json, String expectedEntryType) {
+        List<BibEntry> entries = ZoteroCitationMarkParser.parseCslJsonItems(json);
+
+        assertEquals(1, entries.size());
+        BibEntry entry = entries.getFirst();
+        assertEquals(expectedEntryType, entry.getType().getDisplayName());
+        assertEquals(Optional.of("T"), entry.getField(StandardField.TITLE));
+        assertEquals(Optional.empty(), entry.getField(StandardField.AUTHOR));
+        assertEquals(Optional.empty(), entry.getField(StandardField.YEAR));
+    }
 }
diff --git a/jabsrv/src/main/java/org/jabref/http/server/resources/EntriesResource.java b/jabsrv/src/main/java/org/jabref/http/server/resources/EntriesResource.java
@@ -20,6 +20,7 @@
 import org.jabref.logic.exporter.BibWriter;
 import org.jabref.logic.importer.FetcherException;
 import org.jabref.logic.importer.util.MediaTypes;
+import org.jabref.logic.openoffice.ZoteroCitationMarkParser;
 import org.jabref.logic.preferences.CliPreferences;
 import org.jabref.logic.util.strings.StringUtil;
 import org.jabref.model.database.BibDatabaseContext;
@@ -84,6 +85,48 @@ public void addBibtex(@PathParam("id") String id, @QueryParam("group") @Nullable
                 .orElseGet(() -> new UiCommand.AppendBibTeXToLibrary(bibtex, group))));
     }
 
+    /// Parses CSL-JSON (a single CSL item or an array of them, as produced by a Zotero /
+    /// citation-js CSL-JSON export) into BibTeX entries and appends them to the selected library.
+    ///
+    /// The CSL item type and fields are mapped to Bib(La)TeX via the citation-js-based mapping
+    /// selected in [ADR 0064](docs/decisions/0064-use-citation-js-mapping.md), so a conference paper,
+    /// book chapter, thesis, report, etc. yields the correct entry type instead of a flat @article.
+    /// The produced entries carry no citation key; JabRef generates keys on import.
+    ///
+    /// The optional `group` query parameter additionally files the imported entries under a
+    /// top-level group of that name, creating it if it does not exist.
+    @POST
+    @Consumes(MediaTypes.CITATIONSTYLES_JSON)
+    public void addCslJson(@PathParam("id") String id, @QueryParam("group") @Nullable String group, String cslJson) throws IOException {
+        if (StringUtil.isBlank(cslJson)) {
+            throw new BadRequestException("CSL-JSON data must not be empty.");
+        }
+        if (!uiMessageHandler.isGuiConnected()) {
+            throw new BadRequestException("Only possible in GUI mode.");
+        }
+        Optional<java.nio.file.Path> targetLibrary = resolveTargetLibrary(id);
+
+        List<BibEntry> entries = ZoteroCitationMarkParser.parseCslJsonItems(cslJson);
+        if (entries.isEmpty()) {
+            throw new BadRequestException("Could not parse any CSL-JSON item from the given data.");
+        }
+
+        StringWriter rawEntries = new StringWriter();
+        BibWriter bibWriter = new BibWriter(rawEntries, "\n");
+        BibEntryWriter entryWriter = new BibEntryWriter(
+                new FieldWriter(preferences.getFieldPreferences()),
+                entryTypesManager);
+        for (BibEntry entry : entries) {
+            // Freshly built entries have no meaningful parsed serialization, so force a reformat
+            // instead of writing the (empty) original source back out.
+            entryWriter.write(entry, bibWriter, BibDatabaseMode.BIBTEX, true);
+        }
+
+        uiMessageHandler.handleUiCommands(List.of(targetLibrary
+                .map(library -> new UiCommand.AppendBibTeXToLibrary(library, rawEntries.toString(), group))
+                .orElseGet(() -> new UiCommand.AppendBibTeXToLibrary(rawEntries.toString(), group))));
+    }
+
     /// Parses a plain-text bibliography reference into a BibTeX entry and appends it to the
     /// currently selected library.
     ///
diff --git a/jabsrv/src/test/import.http b/jabsrv/src/test/import.http
@@ -99,6 +99,47 @@ TI  - Targeted RIS import
 PY  - 2021
 ER  -
 
+### Import CSL-JSON to the active library (`current`)
+
+// Consumes application/vnd.citationstyles.csl+json. The CSL item type is mapped to the correct
+// entry type via the citation-js-based mapping (ADR 0064), so this becomes an @InProceedings with
+// booktitle = {Proc. of Test}, not a flat @article. Entries get no citation key; JabRef generates one.
+
+POST http://localhost:23119/libraries/current/entries
+Content-Type: application/vnd.citationstyles.csl+json
+
+{
+  "type": "paper-conference",
+  "title": "A conference paper added via CSL-JSON",
+  "container-title": "Proc. of Test",
+  "author": [
+    { "family": "Doe", "given": "Jane" }
+  ],
+  "issued": { "date-parts": [["2021", 5]] },
+  "DOI": "10.1000/demo"
+}
+
+### Import a CSL-JSON array (mixed item types) to a specific library
+
+// An array adds several entries in one request; each item type maps independently
+// (book -> @Book, chapter -> @InBook, thesis -> @Thesis with publisher -> institution).
+
+POST http://localhost:23119/libraries/{{libraryId}}/entries
+Content-Type: application/vnd.citationstyles.csl+json
+
+[
+  { "type": "book", "title": "A book", "publisher": "Test Press", "issued": { "date-parts": [["2020"]] } },
+  { "type": "chapter", "title": "A chapter", "container-title": "The book", "page": "10-20" },
+  { "type": "thesis", "title": "A thesis", "publisher": "Test University" }
+]
+
+### Import CSL-JSON and also assign it to a group (created if missing)
+
+POST http://localhost:23119/libraries/current/entries?group=import-test-group
+Content-Type: application/vnd.citationstyles.csl+json
+
+{ "type": "webpage", "title": "A web page", "URL": "https://example.org" }
+
 ###
 
 # Creation — error cases
@@ -116,6 +157,13 @@ POST http://localhost:23119/libraries/{{libraryId}}/entries
 Content-Type: application/x-bibtex
 
 
+### Add unparseable CSL-JSON -> 400 Bad Request
+
+POST http://localhost:23119/libraries/current/entries
+Content-Type: application/vnd.citationstyles.csl+json
+
+not json
+
 ## Cached plain-citation lookup, then add
 
 // Two-step flow so the (possibly expensive) parse happens once:
diff --git a/jabsrv/src/test/java/org/jabref/http/server/EntriesResourceTest.java b/jabsrv/src/test/java/org/jabref/http/server/EntriesResourceTest.java
@@ -62,6 +62,12 @@ private Response postBibtex(String id, String bibtex) {
                 .post(Entity.entity(bibtex, MediaTypes.APPLICATION_BIBTEX));
     }
 
+    private Response postCslJson(String id, String cslJson) {
+        return target("/libraries/" + id + "/entries")
+                .request()
+                .post(Entity.entity(cslJson, MediaTypes.CITATIONSTYLES_JSON));
+    }
+
     /// Rebinds the standalone null object ([UiMessageHandler#NONE], not GUI-connected) and restarts
     /// the test container so the resource sees it. Mirrors {@link ServerTest#setAvailableLibraries}.
     private void useStandaloneMode() throws Exception {
@@ -115,9 +121,41 @@ void importEmptyBibtexReturns400() {
         Mockito.verifyNoInteractions(uiMessageHandler);
     }
 
+    /// CSL-JSON is mapped to the correct entry type (paper-conference -> @InProceedings) and container
+    /// field (container-title -> booktitle) before being appended as BibTeX.
+    @Test
+    void importCslJsonConvertsEntryType() {
+        Response response = postCslJson(TestBibFile.GENERAL_SERVER_TEST.id, """
+                {"type":"paper-conference","title":"A paper","container-title":"Proc. of Test"}
+                """);
+
+        assertEquals(Response.Status.NO_CONTENT.getStatusCode(), response.getStatus());
+        UiCommand.AppendBibTeXToLibrary command = (UiCommand.AppendBibTeXToLibrary) captureSingleCommand();
+        assertEquals(java.util.Optional.of(TestBibFile.GENERAL_SERVER_TEST.path), command.library());
+        assertTrue(command.bibtex().toLowerCase(java.util.Locale.ROOT).contains("@inproceedings"), command.bibtex());
+        assertTrue(command.bibtex().contains("Proc. of Test"), command.bibtex());
+    }
+
+    @Test
+    void importEmptyCslJsonReturns400() {
+        Response response = postCslJson(TestBibFile.GENERAL_SERVER_TEST.id, "   ");
+
+        assertEquals(Response.Status.BAD_REQUEST.getStatusCode(), response.getStatus());
+        Mockito.verifyNoInteractions(uiMessageHandler);
+    }
+
+    @Test
+    void importUnparseableCslJsonReturns400() {
+        Response response = postCslJson(TestBibFile.GENERAL_SERVER_TEST.id, "not json");
+
+        assertEquals(Response.Status.BAD_REQUEST.getStatusCode(), response.getStatus());
+        Mockito.verify(uiMessageHandler, Mockito.never()).handleUiCommands(Mockito.any());
+    }
+
     static Stream<Arguments> standaloneEndpoints() {
         return Stream.of(
                 Arguments.of(MediaTypes.APPLICATION_BIBTEX, "@article{a, title={t}}"),
+                Arguments.of(MediaTypes.CITATIONSTYLES_JSON, "{\"type\":\"book\",\"title\":\"t\"}"),
                 Arguments.of(MediaType.TEXT_PLAIN, "Smith, J. (2020). A title."),
                 Arguments.of(MediaType.APPLICATION_OCTET_STREAM, "some data"));
     }
