Skip to content

AI Refactoring#726

Closed
InAnYan wants to merge 255 commits into
JabRef:mainfrom
InAnYan:refactor/ai-1
Closed

AI Refactoring#726
InAnYan wants to merge 255 commits into
JabRef:mainfrom
InAnYan:refactor/ai-1

Conversation

@InAnYan
Copy link
Copy Markdown
Member

@InAnYan InAnYan commented Apr 24, 2026

Related issues and pull requests

Closes https://github.com/JabRef/jabref-issue-melting-pot/issues/698

Relevant documentation change: JabRef/user-documentation#625

PR Description

A big refactoring PR of the AI features. Also has new features.

Main changes:

  • Refactored UI to use the MVVM pattern.
  • Added regenerate button to chat messages.
  • Added the "AI Chat Status" window to show the state of an AI chat, ingested files, chatted entries and miscalleneus actions.
  • Added an abstraction layer for document splitter algorithms.
  • Added an abstraction layer for summarization algorithms.
  • Added a "full-document" summarization algorithm which just sends the full paper to the user message.
  • Added an abstraction layer for token estimation algorithms.
  • Added an abstraction layer for "answer engines" (it controls how RAG is performed).
  • Added a "full text" document engine: the full paper is sent to the AI instead of embeddings (customizable).
  • Added some starting code to allow to customize AI parameters per session (for default values for summarization algorithm and answer engine is stored in AI preferences, but they can be customized in place).
  • Added abilities to cancel AI requests (summarization, chatting).
  • Added AI requirements (yes, it's a bit extra and low-priority, but I liked the idea of requirements tracking and wanted to see how it would work and look in practice).
  • Made the task and storage logic more clear (TrackedBackgroundTask, *TaskAggregator, InMemoryChache).
  • Added unit tests for many clases.
  • Migrated to other schema of storage for summaries, chat history, and embeddings. As a result wrote a migration.
  • (Controversial) added an AI library ID that is a part of database metadata. I admit I should've discussed this with the maintainers team, but it solves some architectural problems of storing AI artifacts.
  • Ocasinally added some helpers for JavaFX observables.
  • Chat messages now have a timestamp.
  • Added comments for some classes whose purpose is not understood on the first sight.

Preferences:
java_UP6kzObEUv
java_uZF4kDsdKw

Summarization:
java_8LOLlTuFZw

java_oh943SJSop

AI chat:
java_cXZPF3HcJa

java_UkpuHcJqox

Steps to test

Test summarization:

  1. Click on an entry with a linked file.
  2. Go to "AI Summary" tab.
  3. Wait for summary generation.
  4. Observe the summary.
  5. Try clicking on "Regenerate (custom)" and choose other summarization algorithm.

Test chatting:

  1. Click on an entry with a linked file.
  2. Go to "AI Chat" tab.
  3. Click on info button to open the chat status window.
  4. See that you have the current chat model, selected answer engine, look at the entries used in chat (more interesting in the group chat), check the status of ingestion.
  5. Close and chat with AI.
  6. Hover on an AI message and find the "rotation" button. Click on it to regenerate the response.
  7. Go to the AI chat status window and change the answer engine.
  8. Chat and compare the results.
  9. Close JabRef.
  10. Open again JabRef, the same library and entry.
  11. See that the chat history is preserved.

Test chat/summary migration:

  1. Run this PR.
  2. Open a library with entries you have chatted before.
  3. Open logs and see that there should be a log about "Successfully migrated chat messages/summaries".
  4. Open entries and see that you have the same chats and summaries.

Checklist

  • I own the copyright of the code submitted and I license it under the MIT license
  • I manually tested my changes in running JabRef (always required)
  • I added JUnit tests for changes (if applicable)
  • I added screenshots in the PR description (if change is visible to the user)
  • [/] I added a screenshot in the PR description showing a library with a single entry with me as author and as title the issue number
  • I described the change in CHANGELOG.md in a way that can be understood by the average user (if change is visible to the user)
  • I checked the user documentation for up to dateness and submitted a pull request to our user documentation repository

AI Usage Disclosure

While refactoring and adding new features I have used the AI tools like GitHub Copilot, IntelliJ Junie a lot. All AI generated code was carefully reviewed, edited and tested by me, and I take the full responsibility of the AI results.

…gic.ai.preferences`; update package structure and adjust imports across the codebase.
…ic package structure; update imports and restructure hierarchy accordingly.
…ion components into dedicated classes and interfaces.
}

private void setupItemReferenceListener() {
if (itemsProperty.get() != null) {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't be null ever since this is only called in the constructor, right?

InAnYan and others added 9 commits May 4, 2026 21:23
…abRef#15672)

Bumps [com.github.javaparser:javaparser-core](https://github.com/javaparser/javaparser) from 3.28.0 to 3.28.1.
- [Release notes](https://github.com/javaparser/javaparser/releases)
- [Changelog](https://github.com/javaparser/javaparser/blob/master/changelog.md)
- [Commits](javaparser/javaparser@javaparser-parent-3.28.0...javaparser-parent-3.28.1)

---
updated-dependencies:
- dependency-name: com.github.javaparser:javaparser-core
  dependency-version: 3.28.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…JabRef#15674)

Bumps [com.github.javaparser:javaparser-symbol-solver-core](https://github.com/javaparser/javaparser) from 3.28.0 to 3.28.1.
- [Release notes](https://github.com/javaparser/javaparser/releases)
- [Changelog](https://github.com/javaparser/javaparser/blob/master/changelog.md)
- [Commits](javaparser/javaparser@javaparser-parent-3.28.0...javaparser-parent-3.28.1)

---
updated-dependencies:
- dependency-name: com.github.javaparser:javaparser-symbol-solver-core
  dependency-version: 3.28.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…#15663)

* Mark bst package as nonnull by default

* Enhance comment and fix abbreviation

* Replace apache commons

* Remove trivial comment

* Ease error level for clear log info

* Empty citation key

* Enhance null safety for BstVMContext

* Revert "Ease error level for clear log info"

This reverts commit f4fcf7a.

* Cleanups and fix test

* Fix comment

* Reintroduced null checks to harden

---------

Co-authored-by: Carl Christian Snethlage <calixtus@users.noreply.github.com>
Co-authored-by: Oliver Kopp <kopp.dev@gmail.com>
…JabRef#15673)

Bumps [com.github.ben-manes.caffeine:caffeine](https://github.com/ben-manes/caffeine) from 3.2.3 to 3.2.4.
- [Release notes](https://github.com/ben-manes/caffeine/releases)
- [Commits](ben-manes/caffeine@v3.2.3...v3.2.4)

---
updated-dependencies:
- dependency-name: com.github.ben-manes.caffeine:caffeine
  dependency-version: 3.2.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Copy link
Copy Markdown
Member

@calixtus calixtus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On @koppor vouching approved

@koppor
Copy link
Copy Markdown
Member

koppor commented May 4, 2026

@jjohannes - we get

[0](https://github.com/JabRef/jabref-koppor/actions/runs/25341431051/job/74299395885?pr=726#step:4:1181)
Execution failed for task ':jabgui:checkModuleDirectivesScope' (registered by plugin class 'org.gradlex.javamodule.dependencies.JavaModuleDependenciesPlugin').
> /home/runner/work/jabref-koppor/jabref-koppor/jabgui/src/main/java/module-info.java
  
  Please add the following requires directives:
      requires org.jspecify;

but we have requires transitive org.jspecify; - am I overlooking somethinghere?

Siedlerchr and others added 5 commits May 4, 2026 22:58
* New translations jabref_en.properties (French)

[ci skip]

* New translations jabref_en.properties (Italian)

[ci skip]

* New translations jabref_en.properties (Portuguese, Brazilian)

[ci skip]
…odel#parse() (JabRef#15576)

* Fix empty column name causing WARN in MainTableColumnModel#parse()

fixes JabRef#15571

* Update CHANGELOG for JabRef#15571

* Fix null check and filter null columns in MainTableColumnFactory

* Remove filter and annotate createColumn() with @nullable

* Move @nullable annotation to createColumn() instead of createColumns()

---------

Co-authored-by: Carl Christian Snethlage <50491877+calixtus@users.noreply.github.com>
* Fix requirements

* Fix requirements tracing defects

Resolve duplicate req~ui.dialogs.confirmation.naming~1 by linking
ui-recommendations.md to the canonical definition in ux.md. Drop
unmet "Needs: impl" from the two aspirational search requirements
and add an impl marker for req~ux.disabled-vs-hidden~1 in
OpenUrlAction (the exact example from the requirement).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Christoph <siedlerkiller@gmail.com>
* including SearchRxiv integration for SLR and enhancements to citation fetching and PDF downloads.

* Add share functionality for SearchRxiv

* introduce buildStudy method for creating Study instances and enhance share functionality for SearchRxiv

* adding a button for exporting search queries to SearchRxiv and improving layout for validation messages.

* Implement SearchRxivExporter for exporting study search queries in JSON format, adhering to the search-query library's specifications.

* adding export options for search queries in JSON format

* Add unit tests for SearchRxivExporter, verifying file creation and JSON structure for study exports

* Revert unrelated changes, keep only SearchRxiv integration files

* Sync remaining unrelated files with upstream main

* Fix localization

* Fix localization: match tooltip key with FXML

* Fix localization: add missing key and match tooltip with FXML

* Fix CHANGELOG, restore some missing

* Extract noCatalogEnabled to ActionHelper

* Address review comment

* Use FileUtil.getValidFileName for SearchRxiv export filenames

Replace ad-hoc regex-based filename cleaning and arbitrary 20-char
truncation with the existing FileUtil.getValidFileName helper, so the
SearchRxiv exporter follows the same illegal-character handling and
filesystem length limit as the rest of JabRef.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Clean filename components and add parameterized test

FileUtil.getValidFileName only cleans illegal characters when it also
truncates (length > 255), so short filenames like "IEEE-a/b:c*d-0.json"
slipped through unchanged and broke Path.resolve. Apply
FileNameCleaner.cleanFileName per component, then keep
FileUtil.getValidFileName as a length safeguard. Add a parameterized
test covering illegal chars, blank query fallback, and a regular query.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Compare exported JSON as a tree in a single assertion

Replace the field-by-field assertions with a single JsonNode tree
equality check against an inline expected document. Tree equality is
order- and whitespace-independent, and any unexpected extra field
(such as "title") fails naturally.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Use Markdown link in buildJson Javadoc description

Convert the inline HTML link in the buildJson description to Markdown
syntax. The @see tags are kept as HTML since the codebase uniformly
uses HTML there and JEP 467 only guarantees reference-link form for
@see, not inline links.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Link to SearchRxiv in CHANGELOG entry

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Use @CsvSource textBlock for filename parameterized test

Replace the @MethodSource Stream<Arguments> with @CsvSource using a
text block. Single-quoted values preserve whitespace and special
characters; surrounding spaces are trimmed by the CSV parser, so the
columns can be padded for visual alignment.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Drop redundant BooleanExpression wrap in noCatalogEnabled

Bindings.createBooleanBinding already returns a BooleanBinding (which
extends BooleanExpression), so wrapping it in BooleanExpression.booleanExpression
is a no-op.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Carl Christian Snethlage <50491877+calixtus@users.noreply.github.com>
Co-authored-by: Oliver Kopp <kopp.dev@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
@jjohannes
Copy link
Copy Markdown

@koppor I am afraid you are running into this: gradlex-org/java-module-dependencies#46

If a project is only an application, and not a java-library, Gradle does not offer an api (== requires transitive) scope. The rational is that a pure application is not something anything else depends on. Because of the linked issue, a requires transitive entry is ignored in such cases and that's why the advice from the check task is incomplete.

Fix is to replace requires transitive org.jspecify; with requires org.jspecify;.

In any case, the plugin should behave better - see linked issue.

renovate-bot and others added 7 commits May 5, 2026 13:18
…f#15681)

Bumps org.glassfish.jaxb:jaxb-runtime from 4.0.7 to 4.0.8.

---
updated-dependencies:
- dependency-name: org.glassfish.jaxb:jaxb-runtime
  dependency-version: 4.0.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…f#15682)

Bumps [com.konghq:unirest-modules-gson](https://github.com/Kong/unirest-java) from 4.8.1 to 4.9.0.
- [Release notes](https://github.com/Kong/unirest-java/releases)
- [Changelog](https://github.com/Kong/unirest-java/blob/main/CHANGELOG.md)
- [Commits](Kong/unirest-java@v4.8.1...v4.9.0)

---
updated-dependencies:
- dependency-name: com.konghq:unirest-modules-gson
  dependency-version: 4.9.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [com.dlsc.gemsfx:gemsfx](https://github.com/dlsc-software-consulting-gmbh/GemsFX) from 4.0.3 to 4.0.4.
- [Release notes](https://github.com/dlsc-software-consulting-gmbh/GemsFX/releases)
- [Commits](dlsc-software-consulting-gmbh/GemsFX@v4.0.3...v4.0.4)

---
updated-dependencies:
- dependency-name: com.dlsc.gemsfx:gemsfx
  dependency-version: 4.0.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…5683)

Bumps com.konghq:unirest-java-core from 4.8.1 to 4.9.0.

---
updated-dependencies:
- dependency-name: com.konghq:unirest-java-core
  dependency-version: 4.9.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* New translations jabref_en.properties (French)

[ci skip]

* New translations jabref_en.properties (Arabic)

[ci skip]

* New translations jabref_en.properties (Italian)

[ci skip]
@InAnYan
Copy link
Copy Markdown
Member Author

InAnYan commented May 6, 2026

Moved to: JabRef#15688

@InAnYan InAnYan closed this May 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants