Add EMF config that enables adding extra properties to the EMF record

[mzurita-amz]

Description

Adds EMF configuration to the DataPrepper config schema. This config supports adding additional properties to the EMF record by specifying the following in the config:

emf:
    additional_properties: 
        test_property: "test_value"
        environment": "integration_test"

This will generate an EMF record with the following:

{
  "_aws": {
    "Timestamp": 1762919187031,
    "CloudWatchMetrics": [
      {
        "Namespace": "DataPrepper",
        "Metrics": [
          {"Name": "test-pipeline.s3-source.s3ObjectsFailed.count", "Unit": "Count"},
          // ... other metrics
        ],
        "Dimensions": [[]]
      }
    ]
  },
  "test_property": "test_value",           // ✅ OUR ADDITIONAL PROPERTY!
  "environment": "integration_test",       // ✅ OUR ADDITIONAL PROPERTY!
  "test-pipeline.s3-source.s3ObjectsFailed.count": 1.0,
  "test-pipeline.s3-source.s3ObjectsSucceeded.count": 0.0,
  // ... other metric values
}

Issues Resolved

N/A

Check List

• New functionality includes testing.
• [-] New functionality has a documentation issue. Please link to it in this PR.
  • [-] New functionality has javadoc added
• Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[dlvenable]

Thanks @mzurita-amz for this metrics improvement! I have a few comments.

[dlvenable]

This is probably better to rename to emf_metrics to help make it clearer to somebody reading the config.

[mzurita-amz]

Addressed in next rev

[dlvenable]

Can we remove this now?

[mzurita-amz]

ADdressed in next rev

[dlvenable]

What other options do we have to verify this?

I see that these tests do not currently test the flush() method. But, maybe that would be ideal.

1. Create a mock for ISink.
2. Perform a flush.
3. Extract the MetricsContext provided to your mock.
4. Assert the property exists on getProperty
5. Assert there is no dimension in getDimensions

[dlvenable]

This works so far, though it is rough code and doesn't completely test it.

    @Test
    void testAdditionalPropertiesAreAddedToMetricsLogger_v2() {
        final Map<String, String> additionalProperties = Map.of("key1", "value1", "key2", "value2");
        final EMFLoggingRegistryConfig config = new EMFLoggingRegistryConfig() {
            @Override
            public String get(String key) {
                return null;
            }

            @Override
            public Map<String, String> additionalProperties() {
                return additionalProperties;
            }
        };

        Environment environment = mock(Environment.class);

        ISink sink = mock(ISink.class);
        when(environment.getSink()).thenReturn(sink);

        final EMFLoggingMeterRegistry registryWithProperties = new EMFLoggingMeterRegistry(
                config, environment, clock);

        registryWithProperties.gauge("test_gauge", 1.0);

        registryWithProperties.publish();

        ArgumentCaptor<MetricsContext> contextArgumentCaptor = ArgumentCaptor.forClass(MetricsContext.class);
        verify(sink).accept(contextArgumentCaptor.capture());

        MetricsContext actualMetricsContext = contextArgumentCaptor.getValue();

        assertThat(actualMetricsContext.getProperty("key1"), equalTo("value1"));
        assertThat(actualMetricsContext.getProperty("key2"), equalTo("value2"));
    }

[mzurita-amz]

ADdressed in next rev. Made the test so it checks specific properties and its values.

[dlvenable]

Please use random keys and values for testing.

[mzurita-amz]

Addressed next revision

[viquer]

NIT: Don't know if we want to return a copy to avoid mutating the map outside of the object.

[dlvenable]

Or wrap in an unmodifiableMap.

return Collections.unmodifiableMap(dataPrepperConfiguration.getEmfAdditionalProperties());

[viquer]

Where can we verify that?

[mzurita-amz]

Changed this test in next revision

[dlvenable]

Thanks @mzurita-amz !

[dlvenable]

Or wrap in an unmodifiableMap.

return Collections.unmodifiableMap(dataPrepperConfiguration.getEmfAdditionalProperties());

[viquer]

The small original comment still stands (return a copy, unmodifiable copy, of the properties), but I would not block on that for now.

[viquer]

maybe it's better to throw an exception if we don't really expect this to be called. A new IllegalStateException("This method should never be called on normal circumstances")

[mzurita-amz]

this method is used internally

[dlvenable]

Thank you @mzurita-amz for this improvement!