feat: decouple ChangeLogAccumulator from Hibernate Session [DHIS2-21378]

[enricocolasante]

What changes

ChangeLogAccumulator

• flushAll(EntityManager) → flushAll(Connection).
  The class was already pure JDBC internally; it only used Hibernate to unwrap a Connection via session.doWork(). That indirection is removed.
  EntityManager and org.hibernate.Session imports are gone. The class now has zero Hibernate dependencies.

AbstractTrackerPersister

• Injects DataSource as a constructor field (via @RequiredArgsConstructor).
• At the start of persist(), obtains the connection once with DataSourceUtils.getConnection(dataSource) and releases it in a finally block via DataSourceUtils.releaseConnection(conn, dataSource).
  Within the active @Transactional context in DefaultTrackerBundleService.commit(), DataSourceUtils returns the connection already bound to the Spring-managed transaction — the same connection Hibernate's doWork was providing — so transaction integrity is maintained.
• Passes conn to changeLogs.flushAll(conn) at both flush points (FlushMode.OBJECT inside the entity loop, FlushMode.AUTO after it).
• SQLException from the FlushMode.AUTO flush path is caught at the outer level and wrapped in PersistenceException (unchecked), triggering Spring's transactional rollback. SQLException from the FlushMode.OBJECT path is caught by the per-entity inner catch, consistent with the existing atomic/non-atomic branching.

Concrete persisters (TrackedEntityPersister, EnrollmentPersister, TrackerEventPersister, SingleEventPersister, RelationshipPersister)

• Each constructor gains a DataSource parameter forwarded to super(...).

Performance

Four compare runs of a load test only on the tracker importer

25659662711
25659660683
25656873606
25656346626

Median Response Time (p50) (ms)

Requests	baseline	req/s	candidate	req/s	Diff (ms)	Change
Login	122	0.38	117	0.38	-5	⬇️ -4.1%
MNCH import	1,071	1.91	1,054	1.92	-18	⬇️ -1.6%
Child Programme import	497	1.91	506	1.92	+10	⬆️ +1.9%
ANC import	384	1.91	380	1.92	-4	⬇️ -1.2%

95th Percentile Response Time (p95) (ms)

Requests	baseline	req/s	candidate	req/s	Diff (ms)	Change
Login	230	0.38	247	0.38	+17	⬆️ +7.3%
MNCH import	1,271	1.91	1,330	1.92	+59	⬆️ +4.7%
Child Programme import	552	1.91	555	1.92	+3	⬆️ +0.6%
ANC import	457	1.91	441	1.92	-15	⬇️ -3.3%

⬇️ = faster, ⬆️ = slower

Open question

In non-atomic mode with FlushMode.OBJECT, a SQLException from changeLogs.flushAll(conn) leaves the PostgreSQL connection in an aborted-transaction state. Subsequent SQL on the same connection fails with "current transaction is aborted", so a single flush failure silently cascades to all remaining entities in the batch. This becomes more significant in Phase 3 when EntityWriteBatch.flush(conn) is added alongside changeLogs.flushAll(conn).
When a SQLException is thrown, does it mean that there is a bug in the query? Should we just return 500?

What does not change

• No behaviour change for callers.
• DefaultTrackerBundleService and the @Transactional boundary are unchanged.

Next steps

• Quick fix ✅ formatDate: SimpleDateFormat → static final DateTimeFormatter ─
• Phase 0 ✅ Remove EntityManager from TrackerPersister interface and all method parameters - feat: decouple ChangeLogAccumulator from Hibernate Session [DHIS2-21378] #23836
• Phase 1 ✅ Decouple ChangeLogAccumulator from Hibernate Session; obtain the
  transaction-bound Connection via DataSourceUtils instead of session.doWork()
• Phase 2 — Extract FileResource assignment update out of AbstractTrackerPersister
  into FileResourceStore
• Phase 3 — Introduce EntityWriteBatch; accumulate and batch-flush TEAV writes
  (multi-row INSERT / unnest UPDATE / tuple DELETE)
• Phase 4 — Pre-allocate IDs from PostgreSQL sequences; stage top-level entity
  inserts and updates in EntityWriteBatch; remove all entityManager.flush() calls
• Phase 5 — Replace updateTrackedEntitiesLastUpdated named query with
  NamedParameterJdbcTemplate; remove last Hibernate dependency from
  DefaultTrackerBundleService
• Phase 6 — Implement JDBC flush methods one entity type at a time
  (TrackedEntity → Enrollment → TrackerEvent/SingleEvent → Relationship)

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
2 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[muilpp]

When a SQLException is thrown, does it mean that there is a bug in the query? Should we just return 500?

IMO, it's either a bug in the query or infrastructure. It shouldn't be because of the data because at that point it's been validated already, so I think returning a 500 is fine.
That said, will be implement the save points mentioned in this comment?

[enricocolasante]

When a SQLException is thrown, does it mean that there is a bug in the query? Should we just return 500?

IMO, it's either a bug in the query or infrastructure. It shouldn't be because of the data because at that point it's been validated already, so I think returning a 500 is fine. That said, will be implement the save points mentioned in this comment?

I don't think so. That is just one consideration based on the answer of the open question. I agree with you that a 500 should be fine.