Skip to content

Commit f3eea67

Browse files
authored
refactor(address-service): DOMA-13038 move heuristics backfill into migrations (#7332)
* docs(address-service): DOMA-13038 add versioning section and reorganize migration guide for 2.0.0 release Add version 2.0.0 to README with migration guide reference. Restructure migration documentation with clearer breaking changes table, upgrade order section, and step-by-step instructions. Move migration scripts from bin/ to bin/migrate/1.x.x-to-2.0.0/ directory structure. Rename MIGRATION-heuristics.md to migration-guide.md. * docs(address-service): reorganize migration scripts and docs Remove version 2.0.0 section and migration guide reference from README. Simplify models documentation with clearer descriptions of Address, AddressSource, AddressHeuristic, and AddressInjection. Add duplicate resolution section explaining possibleDuplicateOf flag and ResolveAddressDuplicateService. * refactor(address-service): DOMA-13038 extend migration to handle all heuristic type prefixes in Address.key Update migration 0009 to add/remove prefixes for all heuristic types (fallback, coordinates, google_place_id, fias_id) instead of only fallback. Add exclusion filters in up migration to skip already-prefixed keys. Update down migration to strip all heuristic prefixes using CASE expression with substring operations. * refactor(address-service): DOMA-13038 simplify migration fingerprint identifiers in 0009 fallback prefix migration Remove timestamp prefix from sender fingerprint values in up and down migrations, changing from "20260311122102-0009_manual_add_fallback_prefix" to "0009_add_fallback_prefix" for cleaner migration tracking. * refactor(address-service): DOMA-13038 remove deletedAt filter from migration 0009 to include soft-deleted addresses * refactor(address-service): DOMA-13038 remove unused @open-condo/apollo-server-client dependency * refactor(address-service): DOMA-13038 convert create-address-heuristics script to database migration Move create-address-heuristics.js from bin/ to migrations/20260311162944-0010_manual_create_heuristics.js as a proper Keystone migration. Replace Knex query builder calls with raw SQL queries for better control and performance. Remove --dry-run flag and loadKeystoneContext() since migrations run in a controlled environment. Inline heuristic type and provider constants instead of importing from domains. * feat(address-service): DOMA-13038 add migrate:unlock script to package.json * refactor(address-service): DOMA-13038 optimize heuristics migration with cursor-based pagination and spatial bucketing Replace offset-based pagination with cursor-based pagination using (createdAt, id) composite key for better performance on large datasets. Implement spatial bucketing for coordinate heuristic lookups using grid-based indexing (COORDINATE_TOLERANCE-sized buckets) with 3x3 neighbor search instead of linear array scan. Batch possibleDuplicateOf updates into single SQL statement using * refactor(address-service): DOMA-13038 remove deletedAt filter from migration 0010 to include soft-deleted addresses * docs(address-service): DOMA-13038 clarify migration documentation with timing estimates and automatic backfill details * refactor(address-service): DOMA-13038 add comprehensive JSDoc documentation and fix duplicate detection to prefer highest-reliability conflict * docs(address-service): DOMA-13038 update migration timing estimates based on actual performance data * refactor(address-service): DOMA-13038 replace lodash get with optional chaining in migration 0010 * refactor(address-service): DOMA-13038 replace parameterized queries with inline SQL literals in migration 0010 * refactor(address-service): DOMA-13038 simplify migration 0009 down function to only handle fallback prefix removal
1 parent 12c3c48 commit f3eea67

14 files changed

Lines changed: 976 additions & 1100 deletions

apps/address-service/README.md

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,20 @@
11
# Description
22
It's a separate service for address processing.
33

4-
Each property is presented as the Address model instance. Each address may be found by different strings, so several AddressSource model instances may belong to each address.
4+
Each property is represented by an `Address` record. Different provider responses and user inputs can still point to the same physical building, so the service stores multiple lookup sources and heuristics for matching, deduplication, and manual duplicate resolution.
55

66
## Models
7-
- **Address**. A model containing data on the particular building's address.
8-
- **AddressSource**. A model containing data on the particular building's address origin.
7+
- **Address**. Normalized building address data. Can be flagged with `possibleDuplicateOf` when the service detects a likely duplicate.
8+
- **AddressSource**. Stores source strings that were used to resolve an address.
9+
- **AddressHeuristic**. Stores provider-generated identifiers such as `fias_id`, `coordinates`, `google_place_id`, and `fallback` for cross-provider address matching.
910
- **AddressInjection**. Addresses that do not exist in external providers.
1011

12+
## Duplicate resolution
13+
14+
Potential duplicates are flagged on `Address.possibleDuplicateOf`.
15+
16+
Administrators can review them and resolve them through `ResolveAddressDuplicateService`, which supports duplicate dismissal and merge flows.
17+
1118
## Endpoints
1219

1320
### /suggest

0 commit comments

Comments
 (0)