update

Author: jchrostek-dd

integration-tests/README.md

@@ -4,91 +4,60 @@ This directory contains integration tests for the Datadog Lambda Extension.
 
 The general flow is:
 1. Deploy test setup using CDK.
-2. Invoke lambda function.
-3. Call Datadog to get telemetry data. 
-4. Check telemtry data based on test requirements.
+2. Invoke lambda functions.
+3. Wait ~10 minutes for data to propagate to Datadog.
+3. Call Datadog to get telemetry data and check the data based on test requirements.
 
 For simplicity, integraiton tests are setup to only test against ARM runtimes.
 
-### Local Development
+## Guidelines
 
-1. **Datadog API Keys**: Set environment variables
+### Naming
+* An `identifier` is used to differentiate the different stacks. For local development, the identifier is automatically set using the command `whoami` and parses the user's first name. For gitlab pipelines, the identifier is the git commit short sha.
+* Stacks should be named `integ-<identifier>-<stack name>`
+* Lambda functions should be named `<stack-id>-<function name>`
+
+### Stacks
+* Stacks automatically get deployed as part of the gitlab integration pipeline. Stack should be setup to not retain resources. A helper function `createLogGroup` exists with `removalPolicy: cdk.RemovalPolicy.DESTROY`. 
+
+
+## Local Development
+
+### Prerequisites Set env variables
+**Datadog API Keys**: Set environment variables
    ```bash
    export DD_API_KEY="your-datadog-api-key"
    export DD_APP_KEY="your-datadog-app-key"
-   ```
-
-2. **AWS Secrets Manager**: You need a secret ARN for the Datadog API key that Lambda functions will use
-   ```bash
    export DATADOG_API_SECRET_ARN="arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:YOUR_SECRET"
    ```
 
-## Local Testing Workflow
-
 ### 1. Build and Deploy Extension Layer
 
-First, publish your extension layer with a unique suffix:
+First, publish your extension layer. 
 
 ```bash
-cd /path/to/datadog-lambda-extension
-
-# Example: scripts/publish_bottlecap_sandbox.sh <suffix>
-TODO
+./scripts/publish_local.sh
 ```
 
+This will create and publish `Datadog-Extension-ARM-<your name>`.
+
 ### 2. Deploy Test Stacks
 
-Deploy the CDK stacks that create Lambda functions for testing:
+Deploy the CDK stacks that create Lambda functions for testing.
 
 ```bash
-cd integration-tests
-./scripts/deploy.sh ExampleTestStack yourname
+./scripts/deploy.sh base-node
 ```
 
+This will create `integ-<your name>-baes-node`. The stacks will use the lambda extension created in the previous step.
+
 ### 3. Run Integration Tests
 
 Run Jest tests that invoke Lambda functions and verify Datadog telemetry:
 
 ```bash
-export SUFFIX="yourname" 
-export DD_API_KEY="your-datadog-api-key"
-export DD_APP_KEY="your-datadog-app-key"
-
-# Run tests
 npm test
 ```
-
-**Note**: Tests wait 10 minutes after Lambda invocation to allow telemetry to appear in Datadog. The total test timeout is ~11.6 minutes per test.
-
-
-## GitLab CI/CD Pipeline
-
-The integration tests run automatically on merge requests:
-
-### Pipeline Stages
-
-1. **`publish integration layer (arm64)`**
-   - Publishes an arm64 extension layer with suffix: `${CI_COMMIT_SHORT_SHA}`
-   - Layer name: `Datadog-Extension-<commit-hash>`
-   - Saves layer ARN as artifact
-
-2. **`integration-deploy`**
-   - Deploys all CDK stacks with suffix: `${CI_COMMIT_SHORT_SHA}`
-   - Uses the layer ARN from previous step
-   - Sets `SUFFIX=${CI_COMMIT_SHORT_SHA}`
-
-3. **`integration-test`**
-   - Runs Jest test suite
-   - Uses `SUFFIX=${CI_COMMIT_SHORT_SHA}` to find deployed functions
-   - DD_API_KEY and DD_APP_KEY retrieved from AWS SSM Parameter Store
-   - Generates JUnit XML and HTML reports
-
-4. **`integration-cleanup-stacks`**
-   - Destroys all stacks with the commit hash suffix
-   - Runs even if tests fail (`when: always`)
-
-5. **`integration-cleanup-layers`**
-   - Deletes all layer versions with the commit hash suffix
-   - Runs even if tests fail (`when: always`)
+**Note**: Tests wait 10 minutes after Lambda invocation to allow telemetry to appear in Datadog. 
 
 

integration-tests/lib/base-python-stack.ts

@@ -11,7 +11,7 @@ export class BasePythonStack extends cdk.Stack {
     const pythonFunction = new lambda.Function(this, pythonFunctionName, {
       runtime: lambda.Runtime.PYTHON_3_12,
       architecture: lambda.Architecture.ARM_64,
-      handler: '/opt/python/lib/python3.12/site-packages/datadog_lambda/handler.handler',
+      handler: 'datadog_lambda.handler.handler',
       code: lambda.Code.fromAsset('./lambda/base-python'),
       functionName: pythonFunctionName,
       timeout: cdk.Duration.seconds(30),

integration-tests/scripts/local_deploy.sh

@@ -30,7 +30,7 @@ fi
 
 echo "Using extension layer: $EXTENSION_LAYER_ARN"
 
-FULL_STACK_NAME="IntegrationTests-$IDENTIFIER-$STACK_NAME"
+FULL_STACK_NAME="integ-$IDENTIFIER-$STACK_NAME"
 echo "Deploying stack: $FULL_STACK_NAME"
 
 # Build and deploy