Lambda ARM (Apple M1): AWS SAM Golang lambda debugging does not work

[ashprao]

Description:

After creating a plain vanilla AWS SAM application with sam init for golang, and then using the AWS Toolkit provided code lens in VS Code, to create an AWS SAM template file based debug configuration, I get the following error when trying to debug the lambda code.

Running the code without debugging works.

Note:
I had raised this issue in aws/aws-toolkit-vscode as aws/aws-toolkit-vscode#2020. There was a suggestion in that thread that I should raise an issue here.

Steps to reproduce:

1. Create a new AWS SAM application using Go with sam init.
2. Select the Hello world template.
3. In the main.go file, create a debug configuration from the code lens provided by the AWS Toolkit and associated with the handler.
4. Set a breakpoint and start debugging.

Observed result:

2021-09-11 10:55:02 [INFO]: The Delve repo was not found in your GOPATH. Downloading in a temporary directory...
2021-09-11 10:55:03 [INFO]: Preparing to debug locally: Lambda "hello-world"
2021-09-11 10:55:03 [INFO]: Building SAM application...
2021-09-11 10:55:03 [INFO]: Running command: (not started) [/opt/homebrew/bin/sam build --build-dir /tmp/aws-toolkit-vscode/vsctkeYZ6BB/output --template /Users/user1/dev/godev/aws/lambdas/hello-world/hello-world/app___vsctk___template.yaml --base-dir /Users/user1/dev/godev/aws/lambdas/hello-world/hello-world]
2021-09-11 10:55:03 [INFO]: Building codeuri: /Users/user1/dev/godev/aws/lambdas/hello-world/hello-world runtime: go1.x metadata: {} functions: ['helloworld']

2021-09-11 10:55:03 [INFO]: Running GoModulesBuilder:Build

2021-09-11 10:55:03 [INFO]: 
Build Succeeded

2021-09-11 10:55:03 [INFO]: 
Built Artifacts  : ../../../../../../../private/tmp/aws-toolkit-vscode/vsctkeYZ6BB/output
Built Template   : ../../../../../../../private/tmp/aws-toolkit-vscode/vsctkeYZ6BB/output/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke -t ../../../../../../../private/tmp/aws-toolkit-vscode/vsctkeYZ6BB/output/template.yaml
[*] Deploy: sam deploy --guided --template-file ../../../../../../../private/tmp/aws-toolkit-vscode/vsctkeYZ6BB/output/template.yaml
    

2021-09-11 10:55:04 [INFO]: Build complete.
2021-09-11 10:55:04 [INFO]: Starting SAM application locally
2021-09-11 10:55:05 [INFO]: AWS.running.command
Invoking hello-world (go1.x)
Requested to skip pulling images ...

Mounting /tmp/aws-toolkit-vscode/vsctkeYZ6BB/output/helloworld as /var/task:ro,delegated inside runtime container
START RequestId: 7c546e9e-8558-4fde-8dd6-fc6946034850 Version: $LATEST
API server listening at: [::]:5858
2021-09-11T00:55:07Z info layer=debugger launching process with args: [/var/task/hello-world]
2021-09-11 10:55:07 [INFO]: Waiting for SAM application to start...
could not launch process: fork/exec /var/task/hello-world: function not implemented
2021/09/11 00:55:07 exit status 1
time="2021-09-11T00:55:07.765" level=error msg="Init failed" InvokeID= error="Runtime exited with error: exit status 1"
time="2021-09-11T00:55:07.773" level=error msg="INIT DONE failed: Runtime.ExitError"
2021-09-11 10:55:08 [INFO]: Attaching debugger to SAM application...

Command stopped: "sam local invoke"

From the output listed above, it seems to me that when using the debugger, it thinks that the function is not implemented.
2021-09-11 10:55:07 [INFO]: Waiting for SAM application to start...
could not launch process: fork/exec /var/task/hello-world: function not implemented

Other considerations

• Go debugging works for vanilla go apps (non lambdas) on M1 based Apple Macs.
• Go lambdas run successfully with sam local invoke on M1 based Apple Macs.
• Go lambdas can be successfully debugged on x86 based Apple Macs.

Expected result:

• Lambda debugging working with breakpoints being hit.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: macOS Big Sur 11.6 (M1) / Darwin arm64 20.6.0
2. sam --version: 1.31.0
3. AWS region: any (currently working in ap-south-1)
4. VS Code version: 1.16
5. AWS Toolkit extension version: 1.29.0
6. Go version: 1.17.1
7. dlv and dlv-dap version: 1.7.1 Build: $Id: 3bde2354aafb5a4043fd59838842c4cd4a8b6f0b $
8. Docker version: 20.10.8, build 3967b7d

[ashprao]

Hello, just checking in to see if there is any update on this. My self interest here is to be able to work with Go based Lambdas on my M1 Mac and not rely on an intel based system, which is what I am doing now for just those Lambdas we use Go for. At the least it would be good to get some understanding regarding what is the cause for this issue from the aws-sam-cli team point of view?

[hawflau]

Hey @ashprao, sorry for the late response. I confirmed that Go lambdas can be successfully debugged on x86 based Apple Macs. But I personally don't have a M1 Mac, I'll ask a team member to help with this.

[ashprao]

Hello @hawflau, Thanks for getting to this. Would love to be able fully switch to my M1 based Mac for all of my development. :-)

To add more context. I did try debugging after updating, AWS CLI, AWS SAM, VS Code, VS Code AWS Toolkit to the latest versions and I am still facing the issue.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

OS: macOS Big Sur 11.6 (M1) / Darwin arm64 20.6.0
aws --version: aws-cli/2.2.45 Python/3.8.8 Darwin/20.6.0 exe/x86_64 prompt/off
sam --version: SAM CLI, version 1.33.0
AWS region: any (currently working in ap-south-1)
VS Code version: 1.61.0 (Universal)
AWS Toolkit extension version: 1.31.0
Go version: 1.17.1
dlv and dlv-dap version: 1.7.1 Build: $Id: 3bde2354aafb5a4043fd59838842c4cd4a8b6f0b $
Docker version: 20.10.8, build 3967b7d

[ashprao]

Hello @hawflau, just checking in to see if your colleagues have had any success in reproducing this, and if so, whether there is a resolution or workaround that can be made available in the near term.

[ashprao]

Hi @hawflau, apologies for the second ping on this. Any luck with having this reproduced? If this is an issue with my setup only, then I would be happy to get some pointers to figure out how to resolve it? But I am hoping that this is a bug that would have an ETA for a fix. Thanks in advance.