chore: [ANDROSDK-2128] Replace Auto-Increment `_id` Keys with UIDs and Composite Primary Keys by marcamsn · Pull Request #2438 · dhis2/dhis2-android-sdk

marcamsn · 2025-07-09T05:36:28Z

Description
This PR removes all legacy _id INTEGER PRIMARY KEY AUTOINCREMENT columns and migrates each table to use meaningful TEXT UIDs or composite unique constraints. It includes migration scripts (v175), refactors SQL definitions, updates model builders and toDomain() mappings, adapts tests, and bumps the database version to 175.

Highlights

Migration Script: Drop old _id columns, redefine PKs, update snapshot.sql
Code Updates: Remove id(Long) from builders and core interfaces; refactor persistence stores and handlers
Store Behavior: Switch from direct insert to updateOrInsert in certain stores (e.g., ForeignKeyViolationStore now backed by an ObjectWithoutUidStore)
Ordering: Introduce sortOrder in TrackerVisualizationDimension and VisualizationDimensionItem models to preserve row order lost with autoincrement
Indexes: Removes event_data_element and tracked_entity_instance_attribute indexes since they're primary keys now.
Tests: Adapt unit & integration tests

Note

⚠️ This is a dangerous migration: row ordering may change, affecting list-return order expectations. Please review this PR with extra care.

…DROSDK-2128

…DROSDK-2128 # Conflicts: # core/src/main/java/org/hisp/dhis/android/persistence/itemfilter/ItemFilterDB.kt

…mary key definitions

…ssed PKs

… are correctly defined

…e related logic

…atorLink

…tUidStore

…ticsPeriodBoundaryType in primary key

…y key and enhance event filter assertions

… primary key

…aseObject

…d delete ObjectShould.java

…DROSDK-2128 # Conflicts: # core/src/test/java/org/hisp/dhis/android/core/map/layer/internal/microsoft/BingServerResponseShould.kt

vgarciabnz

I have not reviewed the migration yet, I want to do it tomorrow. In the migration, The only thing I would consider is that primary keys that create new UNIQUE constraints might fail during the migration (probably because an existing inconsistent data). I will try to be permissive with those errors and just ignore them.

…. SonarQube issue

marcamsn · 2025-07-10T06:33:20Z

@vgarciabnz , about your comment:

I have not reviewed the migration yet, I want to do it tomorrow. In the migration, The only thing I would consider is that primary keys that create new UNIQUE constraints might fail during the migration (probably because an existing inconsistent data). I will try to be permissive with those errors and just ignore them.

Maybe we can just use INSERT OR IGNORE INTO instead of the INSERT INTO to skip any rows that violate our new PK/UNIQUE constraints in 175.sql.

…tDataElement, CustomIntentAttribute, and TrackerJobObject tables

…Type and uid columns

…nd closingDate in primary key

…EntityAttribute and dataElement in primary key

…here methods

…umns

… update displayDescription instead of event

…imary key order

…DROSDK-2128

sonarqubecloud · 2025-07-21T13:47:30Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
87.4% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

marcamsn added 26 commits June 19, 2025 11:50

[ANDROSDK-2128] Remove id from CoreObject and BaseObject.Builder

1f5359e

[ANDROSDK-2128] Remove id from sharedTests

5248882

[ANDROSDK-2128] Adapt unit tests

c6beae4

[ANDROSDK-2128] Remove integration tests id usages

49d71ed

[ANDROSDK-2128] Merge remote-tracking branch 'origin/develop' into AN…

52575c2

…DROSDK-2128

[ANDROSDK-2128] Merge remote-tracking branch 'origin/develop' into AN…

f6fb3e5

…DROSDK-2128 # Conflicts: # core/src/main/java/org/hisp/dhis/android/persistence/itemfilter/ItemFilterDB.kt

[ANDROSDK-2128] Remove ItemFilterDB

d42b231

[ANDROSDK-2128] Add migration removing _id

c682ea9

[ANDROSDK-2128] Add not null to TEXT primary keys

e4be3d5

[ANDROSDK-2128] Fix inconsistencies

6f53269

[ANDROSDK-2128] Refactor SQL table creation statements to improve pri…

976b1ab

…mary key definitions

[ANDROSDK-2128] Move drops to the end of migration, and add toBeDiscu…

68e14c5

…ssed PKs

[ANDROSDK-2128] Update snapshot.sql

ae86336

[ANDROSDK-2128] Bump database version to 175

149964e

[ANDROSDK-2128] Remove id from persistence toDomain methods.

832b229

[ANDROSDK-2128] Refactor SQL table definitions to ensure primary keys…

615daa4

… are correctly defined

[ANDROSDK-2128] Add sortOrder to VisualizationDimensionItem and updat…

f34646e

…e related logic

[ANDROSDK-2128] Remove sortOrder from ProgramStageSectionProgramIndic…

482026d

…atorLink

[ANDROSDK-2128] Remove sqlite_sequence from cursor results in test

ec1d1ee

[ANDROSDK-2128] Refactor ForeignKeyViolationStore to use ObjectWithou…

23fff3a

…tUidStore

[ANDROSDK-2128] Update AnalyticsPeriodBoundary table to include analy…

ab2b6fb

…ticsPeriodBoundaryType in primary key

[ANDROSDK-2128] Add sortOrder to TrackerVisualizationDimension

ebd83f9

[ANDROSDK-2128] Update TrackerJobObject table to use composite primar…

ee13a40

…y key and enhance event filter assertions

[ANDROSDK-2128] Update ProgramTempOwner table to include 'created' in…

e6ecce1

… primary key

[ANDROSDK-2128] KtlintFormat

629db1a

[ANDROSDK-2128] Pmd

c7c5a1b

marcamsn requested review from taridepaco and vgarciabnz July 9, 2025 05:51

marcamsn added 2 commits July 9, 2025 08:12

[ANDROSDK-2128] Api dump

eb72118

[ANDROSDK-2128] Remove unused import

51e4199

marcamsn added 4 commits July 9, 2025 09:32

[ANDROSDK-2128] Refactor classes to implement CoreObject instead of B…

3dbcf13

…aseObject

[ANDROSDK-2128] Rename BaseObjectKotlinxShould to CoreObjectShould an…

42e2c7e

…d delete ObjectShould.java

[ANDROSDK-2128] Merge remote-tracking branch 'origin/develop' into AN…

4729e24

…DROSDK-2128 # Conflicts: # core/src/test/java/org/hisp/dhis/android/core/map/layer/internal/microsoft/BingServerResponseShould.kt

[ANDROSDK-2128] Override map_from_json_string method

ae59d0b

vgarciabnz reviewed Jul 9, 2025

View reviewed changes

[ANDROSDK-2128] Refactor Base Builder classes to implement interfaces…

f2264f9

…. SonarQube issue

marcamsn and others added 14 commits July 10, 2025 08:44

[ANDROSDK-2128] Modify primary keys for RelationshipItem, CustomInten…

5216440

…tDataElement, CustomIntentAttribute, and TrackerJobObject tables

[ANDROSDK-2128] Api dump

093eef2

[ANDROSDK-2128] Use insert or ignore for this migration

56ac786

[ANDROSDK-2128] Refactor D2Error table structure by removing resource…

9e3c7b3

…Type and uid columns

[ANDROSDK-2128] Update DataInputPeriod table to include openingDate a…

418c123

…nd closingDate in primary key

[ANDROSDK-2128] Update TrackerImportConflict table to include tracked…

beb06ca

…EntityAttribute and dataElement in primary key

[ANDROSDK-2128] Refactor TrackerImportConflict to use updateOrInsertW…

c132190

…here methods

[ANDROSDK-2128] Fix D2Error insert to remove resourceType and uid col…

62f97eb

…umns

[ANDROSDK-2128] Update TrackerImportConflictStoreIntegrationShould to…

41c396a

… update displayDescription instead of event

[ANDROSDK-2128] Update DataSetOrganisationUnitLink table to switch pr…

510ff09

…imary key order

[ANDROSDK-2128] Merge remote-tracking branch 'origin/develop' into AN…

a5c5e5c

…DROSDK-2128

[ANDROSDK-2128] fix EventSync and AnalyticsDhisVisualization issues

4de4c6f

[ANDROSDK-2128] update browserstack device

00bb1b6

[ANDROSDK-2128] Add data_extraction rules to instrumentedTest

0c183a2

vgarciabnz approved these changes Jul 22, 2025

View reviewed changes

marcamsn merged commit 6e4c161 into develop Jul 22, 2025
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore: [ANDROSDK-2128] Replace Auto-Increment `_id` Keys with UIDs and Composite Primary Keys#2438

chore: [ANDROSDK-2128] Replace Auto-Increment `_id` Keys with UIDs and Composite Primary Keys#2438
marcamsn merged 47 commits into
developfrom
ANDROSDK-2128

marcamsn commented Jul 9, 2025 •

edited

Loading

Uh oh!

vgarciabnz left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

marcamsn commented Jul 10, 2025

Uh oh!

sonarqubecloud Bot commented Jul 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

marcamsn commented Jul 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vgarciabnz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

marcamsn commented Jul 10, 2025

Uh oh!

sonarqubecloud Bot commented Jul 21, 2025

Quality Gate passed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

marcamsn commented Jul 9, 2025 •

edited

Loading