-
Notifications
You must be signed in to change notification settings - Fork 0
[SVLS-8757] Improve cloud environment detection logic #39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 3 commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -7,6 +7,8 @@ | |||||||||||||||||
| import java.nio.file.Path; | ||||||||||||||||||
| import java.nio.file.Paths; | ||||||||||||||||||
| import java.nio.file.StandardCopyOption; | ||||||||||||||||||
| import java.util.ArrayList; | ||||||||||||||||||
| import java.util.List; | ||||||||||||||||||
| import java.util.Map; | ||||||||||||||||||
| import org.slf4j.Logger; | ||||||||||||||||||
| import org.slf4j.LoggerFactory; | ||||||||||||||||||
|
|
@@ -16,6 +18,7 @@ enum CloudEnvironment { | |||||||||||||||||
| AZURE_FUNCTION, | ||||||||||||||||||
| AZURE_SPRING_APP, | ||||||||||||||||||
| GOOGLE_CLOUD_RUN_FUNCTION_1ST_GEN, | ||||||||||||||||||
| GOOGLE_CLOUD_RUN_FUNCTION_2ND_GEN, | ||||||||||||||||||
| UNKNOWN | ||||||||||||||||||
| } | ||||||||||||||||||
|
|
||||||||||||||||||
|
|
@@ -50,23 +53,39 @@ public static boolean isLinux() { | |||||||||||||||||
| } | ||||||||||||||||||
|
|
||||||||||||||||||
| public static CloudEnvironment getEnvironment() { | ||||||||||||||||||
| Map<String, String> env = System.getenv(); | ||||||||||||||||||
| return getEnvironment(System.getenv()); | ||||||||||||||||||
| } | ||||||||||||||||||
|
|
||||||||||||||||||
| static CloudEnvironment getEnvironment(Map<String, String> env) { | ||||||||||||||||||
| List<CloudEnvironment> detected = new ArrayList<>(); | ||||||||||||||||||
|
|
||||||||||||||||||
| if (env.get("FUNCTIONS_EXTENSION_VERSION") != null && | ||||||||||||||||||
| env.get("FUNCTIONS_WORKER_RUNTIME") != null) { | ||||||||||||||||||
| return CloudEnvironment.AZURE_FUNCTION; | ||||||||||||||||||
| if (env.get("FUNCTIONS_EXTENSION_VERSION") != null | ||||||||||||||||||
| && env.get("FUNCTIONS_WORKER_RUNTIME") != null) { | ||||||||||||||||||
| detected.add(CloudEnvironment.AZURE_FUNCTION); | ||||||||||||||||||
| } | ||||||||||||||||||
|
|
||||||||||||||||||
| if (env.get("ASCSVCRT_SPRING__APPLICATION__NAME") != null) { | ||||||||||||||||||
| return CloudEnvironment.AZURE_SPRING_APP; | ||||||||||||||||||
| detected.add(CloudEnvironment.AZURE_SPRING_APP); | ||||||||||||||||||
| } | ||||||||||||||||||
|
Comment on lines
55
to
69
|
||||||||||||||||||
|
|
||||||||||||||||||
| if (env.get("FUNCTION_NAME") != null && | ||||||||||||||||||
| env.get("GCP_PROJECT") != null) { | ||||||||||||||||||
| return CloudEnvironment.GOOGLE_CLOUD_RUN_FUNCTION_1ST_GEN; | ||||||||||||||||||
| if (env.get("K_SERVICE") != null && env.get("FUNCTION_TARGET") != null) { | ||||||||||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It looks like the compared python logic checks 1st gen and returns 1st gen if all four gen1 + gen2 env vars are present due to order of if/else. Would it make more sense to follow that behavior? Python could be detecting multiple environments as well, it just isn't doing anything about it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also, does compat-go also expect all four variables or do you think this line is a typo? https://github.com/DataDog/datadog-serverless-compat-go/blob/main/datadogserverlesscompat/datadog_serverless_compat.go#L56
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm true, it would be better to return 1st gen if all four env vars are present. I'm not sure if that ever happens, but if it does, it would be less confusing if the customer looks at the debug log. I'll update the order to check 1st gen first! 7e122c5 To your second question, it does seem like a typo (unless someone tested and saw those env vars in a 1st gen function but the observation wasn't documented), but the outcome would be the same in all of these - as long as the environment isn't |
||||||||||||||||||
| // Set by Google Cloud Functions for newer runtimes | ||||||||||||||||||
| detected.add(CloudEnvironment.GOOGLE_CLOUD_RUN_FUNCTION_2ND_GEN); | ||||||||||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This new branch marks Useful? React with 👍 / 👎.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||||||||||||||||||
| } else if (env.get("FUNCTION_NAME") != null && env.get("GCP_PROJECT") != null) { | ||||||||||||||||||
| // Set by Google Cloud Functions for older runtimes | ||||||||||||||||||
|
||||||||||||||||||
| // Set by Google Cloud Functions for newer runtimes | |
| detected.add(CloudEnvironment.GOOGLE_CLOUD_RUN_FUNCTION_2ND_GEN); | |
| } else if (env.get("FUNCTION_NAME") != null && env.get("GCP_PROJECT") != null) { | |
| // Set by Google Cloud Functions for older runtimes | |
| // This Cloud Functions/Cloud Run-style signature is not exclusive to 2nd gen. | |
| detected.add(CloudEnvironment.GOOGLE_CLOUD_RUN_FUNCTION_2ND_GEN); | |
| } else if (env.get("FUNCTION_NAME") != null && env.get("GCP_PROJECT") != null) { | |
| // Legacy Google Cloud Functions markers used as a fallback when the above signature is absent. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm reaching out to Google to confirm why these environment variables were injected for a Gen 1 function!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They said:
Your function is still a 1st Gen function and has not been automatically converted to a 2nd Gen function. It retains all the capabilities of the 1st Gen runtime, and that is why it correctly appears under the 1st Gen page in the Google Cloud console.
Let me clarify the distinction between the function itself and the environment it runs in. As part of a broad infrastructure upgrade, Google is migrating the underlying environment for Cloud Functions to run on the same modern infrastructure as Cloud Run.
So, while your function's code and capabilities remain 1st Gen, the environment it executes in has been updated. This new environment uses a different set of automatically populated environment variables (e.g., K_SERVICE instead of FUNCTION_NAME). This is why you are seeing the new variables, even though your function is still 1st Gen.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you think it'd make sense to track gen2/unknown uses of the compat layer, similarly to the telemetry we are thinking about adding for serverless-init? https://datadoghq.atlassian.net/browse/SVLS-8970 (possibility is a separate question)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That would be interesting if we are able to track that! I can bring it up in the next standup or ask Piyali
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| package com.datadog; | ||
|
|
||
| import java.lang.reflect.Method; | ||
| import java.util.HashMap; | ||
| import java.util.Map; | ||
| import org.junit.jupiter.api.Test; | ||
| import org.junit.jupiter.params.ParameterizedTest; | ||
| import org.junit.jupiter.params.provider.CsvSource; | ||
| import org.slf4j.Logger; | ||
| import org.slf4j.helpers.NOPLogger; | ||
|
|
||
| import static org.junit.jupiter.api.Assertions.*; | ||
|
|
||
| class ServerlessCompatAgentTest { | ||
|
|
||
| @ParameterizedTest | ||
| @CsvSource({ | ||
| "TRACE, TRACE", | ||
| "DEBUG, DEBUG", | ||
| "INFO, INFO", | ||
| "WARN, WARN", | ||
| "ERROR, ERROR", | ||
| "CRITICAL, ERROR", | ||
| "OFF, null" | ||
| }) | ||
| void testInitLogger(String ddLogLevel, String expectedSlf4jLevel) throws Exception { | ||
| Method initLoggerMethod = ServerlessCompatAgent.class.getDeclaredMethod("initLogger", String.class); | ||
| initLoggerMethod.setAccessible(true); | ||
| Logger logger = (Logger) initLoggerMethod.invoke(null, ddLogLevel); | ||
|
|
||
| if ("OFF".equals(ddLogLevel)) { | ||
| assertTrue(logger instanceof NOPLogger); | ||
| } else { | ||
| assertEquals(expectedSlf4jLevel, | ||
| System.getProperty("org.slf4j.simpleLogger.defaultLogLevel")); | ||
| } | ||
| } | ||
|
|
||
| @Test | ||
| void getEnvironment_returnsUnknownWhenNoVarsSet() { | ||
| assertEquals(CloudEnvironment.UNKNOWN, | ||
| ServerlessCompatAgent.getEnvironment(new HashMap<>())); | ||
| } | ||
|
|
||
| @Test | ||
| void getEnvironment_detectsAzureFunction() { | ||
| Map<String, String> env = new HashMap<>(); | ||
| env.put("FUNCTIONS_EXTENSION_VERSION", "~4"); | ||
| env.put("FUNCTIONS_WORKER_RUNTIME", "java"); | ||
| assertEquals(CloudEnvironment.AZURE_FUNCTION, | ||
| ServerlessCompatAgent.getEnvironment(env)); | ||
| } | ||
|
|
||
| @Test | ||
| void getEnvironment_requiresBothAzureFunctionVars() { | ||
| Map<String, String> env = new HashMap<>(); | ||
| env.put("FUNCTIONS_EXTENSION_VERSION", "~4"); | ||
| assertEquals(CloudEnvironment.UNKNOWN, | ||
| ServerlessCompatAgent.getEnvironment(env)); | ||
| } | ||
|
|
||
| @Test | ||
| void getEnvironment_detectsAzureSpringApp() { | ||
| Map<String, String> env = new HashMap<>(); | ||
| env.put("ASCSVCRT_SPRING__APPLICATION__NAME", "my-app"); | ||
| assertEquals(CloudEnvironment.AZURE_SPRING_APP, | ||
| ServerlessCompatAgent.getEnvironment(env)); | ||
| } | ||
|
|
||
| @Test | ||
| void getEnvironment_detectsGcpOlderRuntimes() { | ||
| Map<String, String> env = new HashMap<>(); | ||
| env.put("FUNCTION_NAME", "my-function"); | ||
| env.put("GCP_PROJECT", "my-project"); | ||
| assertEquals(CloudEnvironment.GOOGLE_CLOUD_RUN_FUNCTION_1ST_GEN, | ||
| ServerlessCompatAgent.getEnvironment(env)); | ||
| } | ||
|
|
||
| @Test | ||
| void getEnvironment_requiresBothGcpOlderRuntimesVars() { | ||
| Map<String, String> env = new HashMap<>(); | ||
| env.put("FUNCTION_NAME", "my-function"); | ||
| assertEquals(CloudEnvironment.UNKNOWN, | ||
| ServerlessCompatAgent.getEnvironment(env)); | ||
| } | ||
|
|
||
| @Test | ||
| void getEnvironment_detectsGcpNewerRuntimes() { | ||
| Map<String, String> env = new HashMap<>(); | ||
| env.put("K_SERVICE", "my-service"); | ||
| env.put("FUNCTION_TARGET", "handler"); | ||
| assertEquals(CloudEnvironment.GOOGLE_CLOUD_RUN_FUNCTION_2ND_GEN, | ||
| ServerlessCompatAgent.getEnvironment(env)); | ||
| } | ||
|
|
||
| @Test | ||
| void getEnvironment_returnsUnknownWhenMultipleEnvironmentsDetected() { | ||
| Map<String, String> env = new HashMap<>(); | ||
| env.put("FUNCTIONS_EXTENSION_VERSION", "~4"); | ||
| env.put("FUNCTIONS_WORKER_RUNTIME", "java"); | ||
| env.put("FUNCTION_NAME", "my-function"); | ||
| env.put("GCP_PROJECT", "my-project"); | ||
| assertEquals(CloudEnvironment.UNKNOWN, | ||
| ServerlessCompatAgent.getEnvironment(env)); | ||
| } | ||
|
|
||
| @Test | ||
| void getEnvironment_bareFunctionNameDoesNotTriggerGcpDetection() { | ||
| Map<String, String> env = new HashMap<>(); | ||
| env.put("FUNCTION_NAME", "my-function"); | ||
| env.put("FUNCTIONS_EXTENSION_VERSION", "~4"); | ||
| env.put("FUNCTIONS_WORKER_RUNTIME", "java"); | ||
| assertEquals(CloudEnvironment.AZURE_FUNCTION, | ||
| ServerlessCompatAgent.getEnvironment(env)); | ||
| } | ||
| } |
This file was deleted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
getEnvironment()behavior changed significantly (multi-detection handling + new Google signature), but there are no unit tests covering these branches. Since the repo already has JUnit tests forServerlessCompatAgent, consider extracting the detection logic into a package-private helper that accepts an envMap<String,String>so tests can cover each detection path and the "multiple environments" case.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added unit tests in c359922