Skip to content

refactor: merge API and yaci-indexer into a single "rosetta-app" module #731

Description

@matiwinnetou

refactor: merge API and yaci-indexer into a single "rosetta-app" module

Ref: #731


Problem

Today cardano-rosetta-java runs as two Spring Boot services:

  • yaci-indexer — owns the JPA entities + repositories for block, transaction, address_utxo, metadata_reference_nft, ft_offchain_metadata, and all other yaci-store tables. This is the write side — it consumes Cardano node events and persists them via yaci-store's storage modules.
  • api — the Rosetta-spec HTTP endpoints. Reads the same tables the indexer writes, but because it doesn't pull in yaci-store's storage jars, it re-defines its own JPA entities and repositories that map to the same column names.

This means every yaci-store schema used by a Rosetta endpoint is duplicated:

  • AddressUtxoEntity (api) ↔ yaci-store's utxo module entity
  • BlockEntity / TxnEntity / TxInputEntity (api) ↔ yaci-store's blocks module entities
  • PoolRegistrationEntity / PoolDelegationEntity / WithdrawalEntity (api) ↔ yaci-store staking module
  • TokenMetadataEntity / TokenLogoEntity / MetadataReferenceNftEntity (api) ↔ yaci-store assets-ext entities — added in refactor: replace REST token registry with DB-based assets-ext reads #730
  • And so on for epoch params, governance, etc.

Plus duplicate *Repository interfaces and, in some cases, duplicate custom native-SQL queries (e.g. the CIP-68 window-function batch query added in #730 exists both downstream in MetadataReferenceNftRepositoryCustom and upstream in yaci-store-assets-ext).


Why we can kill the duplication

yaci-store supports read-only mode via store.read-only-mode=true — listeners/processors/cron jobs are skipped, repositories are still registered, storage reader beans are still created. That's exactly the shape the Rosetta API needs.

With that knob, a single Spring Boot application — rosetta-app — can:

  • Run yaci-store in write mode (sync blocks from the node, populate all tables) on the indexer node(s)
  • Run the same application in read-only mode on the API node(s) — consuming yaci-store's Cip26StorageReader, Cip68StorageReader, AddressUtxoStorageReader, etc. directly without its own entity/repository layer

No behavioral difference for operators — the mode is a single config flag. Horizontal scaling is unchanged (you still run N api replicas + M indexer replicas, they just share the same jar).


Benefits

  1. No more duplicate JPA entities. Schema changes in yaci-store stop being "downstream code must be manually updated" — they flow through the jar dependency.
  2. No more duplicate queries. The CIP-68 batch window-function query in refactor: replace REST token registry with DB-based assets-ext reads #730 was written twice (custom fragment downstream + @Query upstream). Post-merge we reuse upstream's findBySubjects directly.
  3. Single build, single image. Operators deploy one rosetta-app jar with two config profiles instead of two separate artifacts.
  4. Smaller surface for drift bugs. Right now a rename or column-type change in yaci-store-assets-ext can silently break the API's custom entities until runtime — there's no compile-time or build-time guardrail. After the merge, compilation against the yaci-store jar catches it.
  5. Simpler CI. One module to test, not two.

Naming

The merged module will be called rosetta-app — a single Spring Boot application that runs either in Rosetta HTTP read mode (default) or in yaci-store write mode (indexer profile).


What PR #746 already completed (groundwork for this merge)

PR #746 moved index creation natively into yaci-indexer and established clean
ownership boundaries. The following is already done:

  • indexes package in yaci-indexer owns the full index lifecycle:
    IndexService, PgIndexService, NoOpIndexService, IndexCatalog,
    IndexesEndpoint, IndexStallIndicator
  • index-applier sidecar deleted — Docker Compose file, Helm Job + ConfigMap
    templates, apply-indexes.sh shell script, jq/yq from Postgres Dockerfile
  • Single db-indexes.yaml in yaci-indexer drives both index creation and the
    api's index readiness check (kept identical via IndexCatalogParityTest)

Index service migration (specific steps at merge time)

The api has its own parallel index monitoring stack that duplicates logic already
in the yaci-indexer's indexes package. At merge time, consolidate as follows:

1. Wire IndexService into SyncStatusService

// BEFORE — api's SyncStatusService
@RequiredArgsConstructor
public class SyncStatusService {
    private final IndexCreationMonitor indexCreationMonitor;
    ...
    boolean indexesNotReady = indexCreationMonitor.isCreatingIndexes();
}

// AFTER — direct injection of IndexService
@RequiredArgsConstructor
public class SyncStatusService {
    private final IndexService indexService;
    ...
    boolean indexesNotReady = indexService.getState() != IndexLifecycleState.READY;
}

getIndexCreationProgress() (used for detailed status) maps to
indexService.getIndexStatus()List<IndexItemStatus>, which is richer than
the old List<IndexCreationProgress> (4 always-null numeric fields).

2. Delete from api

Class / file Replacement
IndexCreationMonitor (interface) IndexService
PostgreSQLIndexCreationMonitor PgIndexService
H2IndexCreationMonitor NoOpIndexService
RosettaIndexConfig IndexCatalog
api/.../config/db-indexes.yaml Single copy in yaci-indexer
IndexCatalogParityTest No longer two files to compare

Also remove the spring.config.import line from api's application.yaml:

# REMOVE this line
- classpath:config/db-indexes.yaml

3. Profile alignment

Both modules already use the same profile names — no changes needed:

Profile yaci-indexer (keep) api (delete)
h2, test-integration NoOpIndexService H2IndexCreationMonitor
everything else PgIndexService PostgreSQLIndexCreationMonitor

Scope of work

  • Create the new rosetta-app module by merging api and yaci-indexer
  • Add com.bloxbean.cardano:yaci-store-* dependencies to rosetta-app
  • Introduce a Spring profile indexer that enables write-mode yaci-store; default rosetta-app profiles set store.read-only-mode=true
  • Index service migration (detail above):
    • Wire IndexService into SyncStatusService
    • Delete IndexCreationMonitor + both implementations
    • Delete RosettaIndexConfig from api (getIndexCommands() was already dead code)
    • Delete api/src/main/resources/config/db-indexes.yaml
    • Delete IndexCatalogParityTest
  • Replace api-side *Entity + *Repository classes with references to yaci-store's equivalents, module by module:
    • account / address_utxo → yaci-store-utxo
    • block / transaction → yaci-store-blocks + yaci-store-transaction
    • staking (pool, delegation, withdrawal) → yaci-store-staking
    • governance → yaci-store-governance
    • epoch params → yaci-store-epoch
    • token metadata (CIP-26 / CIP-68) → yaci-store-assets-ext (blocked on refactor: replace REST token registry with DB-based assets-ext reads #730 release)
  • Move yaci-indexer's application.properties contents into rosetta-app under the indexer profile
  • Delete the api and yaci-indexer modules
  • Update Docker Compose / Helm charts to run one rosetta-app image with two different SPRING_PROFILES_ACTIVE values (indexer vs default)
  • Update the README deployment docs
  • Regression test: full sync → full Rosetta API test pass against the merged rosetta-app binary

Dependencies


Related

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions