Skip to content

Commit 3e1740e

Browse files
lucasmcdonald3lucasmcdonald3
authored
feat!: updates to the S3 Encryption Client (#70)
* Updates to the S3 Encryption Client ## Breaking Changes The S3 Encryption Client now requires key committing algorithm suites by default. See migration guide: https://docs.aws.amazon.com/amazon-s3-encryption-client/latest/developerguide/go-v4-migration.html
1 parent 6cdbb4a commit 3e1740e

106 files changed

Lines changed: 19733 additions & 281 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci_test_examples.yml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
name: Test Migration Examples
2+
3+
on:
4+
workflow_call:
5+
secrets:
6+
CI_AWS_ACCOUNT_ID:
7+
required: true
8+
9+
jobs:
10+
test-migration-examples:
11+
runs-on: ubuntu-latest
12+
13+
permissions:
14+
id-token: write
15+
contents: read
16+
17+
env:
18+
TEST_BUCKET: s3ec-go-github-test-bucket
19+
TEST_KEY: migration-test-ci
20+
TEST_KMS_KEY: arn:aws:kms:us-west-2:370957321024:alias/S3EC-Go-Github-KMS-Key
21+
TEST_REGION: us-west-2
22+
23+
steps:
24+
- uses: actions/checkout@v4
25+
with:
26+
submodules: recursive
27+
28+
- name: Set up Go
29+
uses: actions/setup-go@v4
30+
with:
31+
go-version: '1.24'
32+
33+
- name: Configure AWS Credentials
34+
uses: aws-actions/configure-aws-credentials@v4
35+
with:
36+
role-to-assume: arn:aws:iam::${{ secrets.CI_AWS_ACCOUNT_ID }}:role/${{ vars.CI_AWS_ROLE }}
37+
role-session-name: S3EC-Github-Go-CI-Tests
38+
aws-region: ${{ vars.CI_AWS_REGION }}
39+
40+
- name: Test Migration Examples
41+
run: |
42+
# Test comprehensive migration examples
43+
cd examples/migration/v3-to-v4
44+
go mod tidy
45+
go test -v

.github/workflows/ci_test_go.yml renamed to .github/workflows/ci_test_go_v3.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ jobs:
1616
strategy:
1717
matrix:
1818
os: [ubuntu-latest, macos-latest]
19-
go-version: ['1.20', '1.21']
19+
go-version: ['1.24', '1.25']
2020
steps:
2121
- uses: actions/checkout@v3
2222

‎.github/workflows/ci_test_go_v4.yml‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
name: Go Tests
2+
3+
on:
4+
workflow_call:
5+
secrets:
6+
CI_AWS_ACCOUNT_ID:
7+
required: true
8+
9+
jobs:
10+
unix-tests:
11+
name: Unix Tests
12+
runs-on: ${{ matrix.os }}
13+
permissions:
14+
id-token: write
15+
contents: read
16+
strategy:
17+
matrix:
18+
os: [ubuntu-latest, macos-latest]
19+
go-version: ['1.24', '1.25']
20+
steps:
21+
- uses: actions/checkout@v3
22+
23+
- name: Configure AWS Credentials
24+
uses: aws-actions/configure-aws-credentials@v4
25+
with:
26+
role-to-assume: arn:aws:iam::${{ secrets.CI_AWS_ACCOUNT_ID }}:role/${{ vars.CI_AWS_ROLE }}
27+
role-session-name: S3EC-Github-Go-CI-Tests
28+
aws-region: ${{ vars.CI_AWS_REGION }}
29+
30+
- name: Set up Go
31+
uses: actions/setup-go@v4
32+
with:
33+
go-version: ${{ matrix.go-version }}
34+
35+
- name: Install golint
36+
run: go install golang.org/x/lint/golint@latest
37+
38+
- name: Go Test
39+
run: |
40+
# CDK example tests make go test ./... fail
41+
# so go package by package
42+
cd v4/client
43+
go test *.go -v
44+
cd ../internal
45+
go test *.go -v
46+
cd ../materials
47+
go test *.go -v
48+
49+
- name: Test Vectors
50+
run: |
51+
cd v4/testvectors
52+
export AWS_REGION=${{ vars.CI_AWS_REGION }}
53+
export BUCKET=${{ vars.CI_S3_BUCKET }}
54+
export AWS_KMS_ALIAS=${{ vars.CI_KMS_KEY_ALIAS }}
55+
export AWS_ACCOUNT_ID=${{ secrets.CI_AWS_ACCOUNT_ID }}
56+
go test *.go -v

‎.github/workflows/daily_ci.yml‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,15 @@ on:
66
- cron: "00 15 * * 1-5"
77

88
jobs:
9-
daily-ci-go-test:
10-
uses: ./.github/workflows/ci_test_go.yml
9+
daily-ci-go-v3-test:
10+
uses: ./.github/workflows/ci_test_go_v3.yml
11+
secrets:
12+
CI_AWS_ACCOUNT_ID: ${{ secrets.CI_AWS_ACCOUNT_ID }}
13+
daily-ci-go-v4-test:
14+
uses: ./.github/workflows/ci_test_go_v4.yml
15+
secrets:
16+
CI_AWS_ACCOUNT_ID: ${{ secrets.CI_AWS_ACCOUNT_ID }}
17+
daily-ci-go-migration-examples-test:
18+
uses: ./.github/workflows/ci_test_examples.yml
1119
secrets:
1220
CI_AWS_ACCOUNT_ID: ${{ secrets.CI_AWS_ACCOUNT_ID }}

‎.github/workflows/pull.yml‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,15 @@ on:
55
pull_request:
66

77
jobs:
8-
pr-ci-go-test:
9-
uses: ./.github/workflows/ci_test_go.yml
8+
pr-ci-go-v3-test:
9+
uses: ./.github/workflows/ci_test_go_v3.yml
1010
secrets:
11-
CI_AWS_ACCOUNT_ID: ${{ secrets.CI_AWS_ACCOUNT_ID }}
11+
CI_AWS_ACCOUNT_ID: ${{ secrets.CI_AWS_ACCOUNT_ID }}
12+
pr-ci-go-v4-test:
13+
uses: ./.github/workflows/ci_test_go_v4.yml
14+
secrets:
15+
CI_AWS_ACCOUNT_ID: ${{ secrets.CI_AWS_ACCOUNT_ID }}
16+
pr-ci-go-migration-examples-test:
17+
uses: ./.github/workflows/ci_test_examples.yml
18+
secrets:
19+
CI_AWS_ACCOUNT_ID: ${{ secrets.CI_AWS_ACCOUNT_ID }}

‎.github/workflows/push.yml‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,15 @@ on:
77
- main
88

99
jobs:
10-
push-ci-go-test:
11-
uses: ./.github/workflows/ci_test_go.yml
10+
push-ci-go-v3-test:
11+
uses: ./.github/workflows/ci_test_go_v3.yml
1212
secrets:
13-
CI_AWS_ACCOUNT_ID: ${{ secrets.CI_AWS_ACCOUNT_ID }}
13+
CI_AWS_ACCOUNT_ID: ${{ secrets.CI_AWS_ACCOUNT_ID }}
14+
push-ci-go-v4-test:
15+
uses: ./.github/workflows/ci_test_go_v4.yml
16+
secrets:
17+
CI_AWS_ACCOUNT_ID: ${{ secrets.CI_AWS_ACCOUNT_ID }}
18+
push-ci-go-migration-examples-test:
19+
uses: ./.github/workflows/ci_test_examples.yml
20+
secrets:
21+
CI_AWS_ACCOUNT_ID: ${{ secrets.CI_AWS_ACCOUNT_ID }}

‎CHANGELOG.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,39 @@
1+
## 4.0.0 (2025-12-16)
2+
3+
### Features
4+
5+
* Updates to the S3 Encryption Client
6+
7+
See migration guide from 3.x to 4.x: [link](https://docs.aws.amazon.com/amazon-s3-encryption-client/latest/developerguide/go-v4-migration.html)
8+
9+
### ⚠ BREAKING CHANGES
10+
11+
* The S3 Encryption Client now uses key committing algorithm suites by default.
12+
* Removed `S3EncryptionClientV3` in favor of the upgraded `S3EncryptionClientV4`
13+
* Updated expectations for custom implementations of the `CryptographicMaterialsManager` interface.
14+
* Custom implementations of the interface's `GetEncryptionMaterials` method MUST set the `AlgorithmSuite` field on the returned `EncryptionMaterials`.
15+
* The provided `DefaultCryptographicMaterialsManager`'s `GetEncryptionMaterials` method and the provided `NewEncryptionMaterials` method set this field from the `AlgorithmSuite` provided in the `req EncryptionMaterialsRequest`.
16+
* If the custom implementation wraps the provided `DefaultCryptographicMaterialsManager.GetEncryptionMaterials` method or calls the provided `NewEncryptionMaterials` method, it's likely that no code updates are required. The provided logic has been updated with this change.
17+
* Custom implementations of the interface's `DecryptMaterials` method MUST set the `KeyCommitment` field on the returned `CryptographicMaterials`.
18+
* The provided `DefaultCryptographicMaterialsManager`'s `DecryptMaterials` method and the provided `DecryptMaterials` method set this field from the `KeyCommitment` provided in the `req DecryptMaterialsRequest`.
19+
* If the custom implementation wraps the provided `DefaultCryptographicMaterialsManager.DecryptMaterials` method or calls the provided `NewDecryptionMaterials` method, it's likely that no code updates are required. The provided logic has been updated with this change.
20+
* Updated expectations for custom implementations of the `Keyring` interface.
21+
* Custom implementations of the interface's `OnDecrypt` method MUST set the `KeyCommitment` field on the returned `CryptographicMaterials`.
22+
* The provided `KmsKeyring`'s `OnDecrypt` method and the provided `commonDecrypt` method set this field from the `KeyCommitment` provided in the `materials DecryptionMaterials`.
23+
* If the custom implementation wraps the provided `KmsKeyring.OnDecrypt` method or calls the provided `commonDecrypt` method, it's likely that no code updates are required. The provided logic has been updated with this change.
24+
25+
### Fixes
26+
27+
* Fixed an issue where nonces of invalid lengths could cause a panic during decryption.
28+
29+
## 3.2.0 (2025-12-16)
30+
31+
### Features
32+
33+
* Updates to the S3 Encryption Client
34+
35+
See migration guide from 3.x to 4.x: [link](https://docs.aws.amazon.com/amazon-s3-encryption-client/latest/developerguide/go-v4-migration.html)
36+
137
## 3.1.0 (2024-11-11)
238

339
### Features

‎README.md‎

Lines changed: 12 additions & 108 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
1-
# Amazon S3 Encryption Client for Go V3
1+
# Amazon S3 Encryption Client for Go V4
22

33
[![Go Build status](https://github.com/aws/amazon-s3-encryption-client-go/actions/workflows/go-test.yml/badge.svg?branch=main)](https://github.com/aws/amazon-s3-encryption-client-go/actions/workflows/go-test.yml) [![Apache V2 License](https://img.shields.io/badge/license-Apache%20V2-blue.svg)](https://github.com/aws/amazon-s3-encryption-client-go/blob/main/LICENSE)
44

55
This library provides an S3 client that supports client-side encryption.
6-
`amazon-s3-encryption-client-go` is the v3 of the Amazon S3 Encryption Client for the Go programming language.
6+
`amazon-s3-encryption-client-go` is the v4 of the Amazon S3 Encryption Client for the Go programming language.
77

8-
The v3 encryption client requires a minimum version of `Go 1.20`.
8+
The v4 encryption client requires a minimum version of `Go 1.24`.
99

1010
Check out the [release notes](https://github.com/aws/amazon-s3-encryption-client-go/blob/main/CHANGELOG.md) for information about the latest bug
1111
fixes, updates, and features added to the encryption client.
@@ -24,7 +24,7 @@ following in the AWS SDKs and Tools Shared Configuration and Credentials Referen
2424

2525
### Go version support policy
2626

27-
The v3 Encryption Client follows the upstream [release policy](https://go.dev/doc/devel/release#policy)
27+
The v4 Encryption Client follows the upstream [release policy](https://go.dev/doc/devel/release#policy)
2828
with an additional six months of support for the most recently deprecated
2929
language version.
3030

@@ -33,7 +33,7 @@ address critical security issues.**
3333

3434
## Getting started
3535
To get started working with the S3 Encryption Client set up your project for Go modules, and retrieve the client's dependencies with `go get`.
36-
This example shows how you can use the v3 encryption client to make a `PutItem` request using a KmsKeyring.
36+
This example shows how you can use the v4 encryption client to make a `PutItem` request using a KmsKeyring.
3737

3838
###### Initialize Project
3939
```sh
@@ -43,7 +43,7 @@ $ go mod init encryptionclient
4343
```
4444
###### Add SDK Dependencies
4545
```sh
46-
$ go get github.com/aws/amazon-s3-encryption-client-go/v3
46+
$ go get github.com/aws/amazon-s3-encryption-client-go/v4
4747
```
4848

4949
###### Write Code
@@ -61,8 +61,8 @@ import (
6161
"github.com/aws/aws-sdk-go-v2/service/s3"
6262

6363
// Import the materials and client package
64-
"github.com/aws/amazon-s3-encryption-client-go/v3/client"
65-
"github.com/aws/amazon-s3-encryption-client-go/v3/materials"
64+
"github.com/aws/amazon-s3-encryption-client-go/v4/client"
65+
"github.com/aws/amazon-s3-encryption-client-go/v4/materials"
6666
)
6767

6868
func main() {
@@ -80,7 +80,7 @@ func main() {
8080
s3Client := s3.NewFromConfig(cfg)
8181
kmsClient := kms.NewFromConfig(cfg)
8282

83-
// Create the keyring and &CMM-long; (&CMM-short;)
83+
// Create the keyring and CMM
8484
cmm, err := materials.NewCryptographicMaterialsManager(materials.NewKmsKeyring(kmsClient, kmsKeyArn, func(options *materials.KeyringOptions) {
8585
options.EnableLegacyWrappingAlgorithms = false
8686
}))
@@ -103,108 +103,12 @@ func main() {
103103

104104
## Migration
105105

106-
This version of the library supports reading encrypted objects from previous versions.
106+
This version of the library supports reading encrypted objects from previous versions with extra configuration.
107107
It also supports writing objects with non-legacy algorithms.
108108
The list of legacy modes and operations will be provided below.
109109

110-
### Examples
111-
#### V2 KMS to V3
112-
113-
The following example demonstrates how to migrate a version v2 application that uses
114-
the `NewKMSContextKeyGenerator` kms-key provider with a material
115-
description and `AESGCMContentCipherBuilderV2` content cipher to
116-
version v3 of the S3 Encryption Client for Go.
117-
118-
```go
119-
func KmsContextV2toV3GCMExample() error {
120-
bucket := LoadBucket()
121-
kmsKeyAlias := LoadAwsKmsAlias()
122-
123-
objectKey := "my-object-key"
124-
region := "us-west-2"
125-
plaintext := "This is an example.\n"
126-
127-
// Create an S3EC Go v2 encryption client
128-
// using the KMS client from AWS SDK for Go v1
129-
sessKms, err := sessionV1.NewSession(&awsV1.Config{
130-
Region: aws.String(region),
131-
})
132-
133-
kmsSvc := kmsV1.New(sessKms)
134-
handler := s3cryptoV2.NewKMSContextKeyGenerator(kmsSvc, kmsKeyAlias, s3cryptoV2.MaterialDescription{})
135-
builder := s3cryptoV2.AESGCMContentCipherBuilderV2(handler)
136-
encClient, err := s3cryptoV2.NewEncryptionClientV2(sessKms, builder)
137-
if err != nil {
138-
log.Fatalf("error creating new v2 client: %v", err)
139-
}
140-
141-
// Encrypt using KMS+Context and AES-GCM content cipher
142-
_, err = encClient.PutObject(s3V1.PutObjectInput{
143-
Bucket: aws.String(bucket),
144-
Key: aws.String(objectKey),
145-
Body: bytes.NewReader([]byte(plaintext)),
146-
})
147-
if err != nil {
148-
log.Fatalf("error calling putObject: %v", err)
149-
}
150-
fmt.Printf("successfully uploaded file to %s/%s\n", bucket, key)
151-
152-
// Create an S3EC Go v3 client
153-
// using the KMS client from AWS SDK for Go v2
154-
ctx := context.Background()
155-
cfg, err := config.LoadDefaultConfig(ctx,
156-
config.WithRegion(region),
157-
)
158-
159-
kmsV2 := kms.NewFromConfig(cfg)
160-
cmm, err := materials.NewCryptographicMaterialsManager(materials.NewKmsKeyring(kmsV2, kmsKeyAlias))
161-
if err != nil {
162-
t.Fatalf("error while creating new CMM")
163-
}
164-
165-
s3V2 := s3.NewFromConfig(cfg)
166-
s3ecV3, err := client.New(s3V2, cmm)
167-
168-
result, err := s3ecV3.GetObject(ctx, s3.GetObjectInput{
169-
Bucket: aws.String(bucket),
170-
Key: aws.String(objectKey),
171-
})
172-
if err != nil {
173-
t.Fatalf("error while decrypting: %v", err)
174-
}
175-
```
176-
177-
#### Enable legacy decryption modes
178-
The `enableLegacyUnauthenticatedModes` flag enables the S3 Encryption Client to decrypt
179-
encrypted objects with a fully supported or legacy encryption algorithm.
180-
Version V3 of the S3 Encryption Client uses one of the fully supported wrapping algorithms and the
181-
wrapping key you specify to encrypt and decrypt the data keys. The
182-
`enableLegacyWrappingAlgorithms` flag enables the S3 Encryption Client to decrypt
183-
encrypted data keys with a fully supported or legacy wrapping algorithm.
184-
185-
```go
186-
cmm, err := materials.NewCryptographicMaterialsManager(materials.NewKmsKeyring(kmsClient, kmsKeyArn, func(options *materials.KeyringOptions) {
187-
options.EnableLegacyWrappingAlgorithms = true
188-
})
189-
190-
if err != nil {
191-
t.Fatalf("error while creating new CMM")
192-
}
193-
194-
client, err := client.New(s3Client, cmm, func(clientOptions *client.EncryptionClientOptions) {
195-
clientOptions.EnableLegacyUnauthenticatedModes = true
196-
})
197-
198-
if err != nil {
199-
// handle error
200-
}
201-
```
202-
203-
### Legacy Algorithms and Modes
204-
#### Content Encryption
205-
* AES/CBC
206-
#### Key Wrap Encryption
207-
* KMS (without context)
110+
* [3.x to 4.x Migration Guide](https://docs.aws.amazon.com/amazon-s3-encryption-client/latest/developerguide/go-v4-migration.html)
111+
* [2.x to 3.x Migration Guide](https://docs.aws.amazon.com/amazon-s3-encryption-client/latest/developerguide/go-v3-migration.html)
208112

209113
## Security
210114

0 commit comments

Comments
 (0)