STD_HIV_DATAMART - widen NBS177/NBS167 columns + page-aware codeset

[ericnagel]

Description

This change fixes decoding and truncation issues affecting follow-up referral and notification fields throughout the HIV reporting pipeline.

Previously, notification follow-up values could be stored as numeric-only codes (for example, 6 or 7) and several columns were too narrow to store the full decoded descriptions. This prevented downstream consumers from seeing the complete business meaning of the data and introduced truncation risk for longer HIV values such as:

7 - Partner Not Notified

Changes Implemented

1. Increased column widths to prevent truncation

The following columns were widened from varchar(15) to varchar(100):

STD_HIV_DATAMART

• FL_FUP_ACTUAL_REF_TYPE
• FL_FUP_NOTIFICATION_PLAN

nrt_investigation_case_management

• fl_fup_actual_ref_type
• fl_fup_notification_plan_cd

D_CASE_MANAGEMENT

• Corresponding case management columns were widened to ensure no downstream bottleneck remained.

A guarded Liquibase migration (261-alter_std_hiv_case_management_widths-001.sql) performs the required schema updates for existing environments.

2. Updated HIV follow-up value decoding

Both Actual Referral Type (FL_FUP_ACTUAL_REF_TYPE) and Notification Plan (FLD_FOLL_UP_NOTIFICATION_PLAN) in D_CASE_MANAGEMENT are now derived by decoding the stored code against the Notification Actual Method (HIV) value set.

This behavior was reviewed and confirmed with EPI Emily Holman.

3. Full decoded values now flow through the reporting pipeline

FL_FUP_NOTIFICATION_PLAN and FL_FUP_ACTUAL_REF_TYPE in STD_HIV_DATAMART now contain the full decoded text rather than numeric-only values.

Examples:

Previous Value	New Value
6	6 - Refused
7	7 - Partner Not Notified

This provides complete business context to downstream reporting consumers and aligns reporting output with expected HIV follow-up terminology.

4. DM_INV_HIV width adjustment

DM_INV_HIV.FL_FUP_ACTUAL_REF_TY has been increased to a length of 25 characters.

Unlike the other reporting tables, this column was not expanded to 100 characters due to the size and performance considerations associated with the DM_INV_HIV table. A length of 25 characters is sufficient to accommodate the currently defined HIV notification-method descriptions, including:

7 - Partner Not Notified

Validation

Verified in RDB_MODERN:

• STD_HIV_DATAMART.FL_FUP_ACTUAL_REF_TYPE length = 100
• STD_HIV_DATAMART.FL_FUP_NOTIFICATION_PLAN length = 100
• nrt_investigation_case_management follow-up columns length = 100
• D_CASE_MANAGEMENT follow-up columns length = 100
• DM_INV_HIV.FL_FUP_ACTUAL_REF_TY length = 25

Verified output now contains fully decoded values such as:

6 - Refused
7 - Partner Not Notified

rather than numeric-only codes.

RDB

D_CASE_MANAGEMENT

FL_FUP_NOTIFICATION_PLAN_CD	FLD_FOLL_UP_NOTIFICATION_PLAN	ACT_REF_TYPE_CD	FL_FUP_ACTUAL_REF_TYPE
6	6	7	7

STD_HIV_DATAMART

FL_FUP_NOTIFICATION_PLAN	FL_FUP_ACTUAL_REF_TYPE
6	7

RDB_MODERN

D_CASE_MANAGEMENT

FL_FUP_NOTIFICATION_PLAN_CD	FLD_FOLL_UP_NOTIFICATION_PLAN	ACT_REF_TYPE_CD	FL_FUP_ACTUAL_REF_TYPE
6	6 - Refused	7	7 - Partner Not Notified

STD_HIV_DATAMART

FL_FUP_NOTIFICATION_PLAN	FL_FUP_ACTUAL_REF_TYPE
6 - Refused	7 - Partner Not Notified

Testing

All existing functional tests pass after updates made to various expected.json files.

Related Issue

APP-619

Additional Notes

Added a dedicated HIV fixture that verifies decoding of a long value such as:

7 - Partner Not Notified

Checklist

• I have ensured that the pull request is of a manageable size, allowing it to be reviewed within a single session.
• I have reviewed my changes to ensure they are clear, concise, and well-documented.
• I have updated the documentation, if applicable.
• I have added or updated test cases to cover my changes, if applicable.

[ericbuckley]

what about floats or datetimes, do we need when statements to handle those?

[ericnagel]

Yes, probably, and I can add them but this modification was a major side-quest to the original issue. I'm not sure how far down that path to go - and I'm not sure how I'd test types that are not part of this process.