Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -237,8 +237,17 @@ public void testParallelScanExceptionHandling() {
assertNotNull(ase.getErrorCode());
assertNotNull(ase.getErrorType());
assertNotNull(ase.getMessage());
} catch (java.util.concurrent.RejectedExecutionException e) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to let it fail for visibility of the failure

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are we gaining from this visibility apart from having to retry? Is there any action item on failure?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is RejectedExecutionException expected failure from this test?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. In parallel run two threads are scheduled to make call to DDB. If first one fails before other thread is even submitted, the executor shuts down and second submit fails leading to RejectedExecutionException.

Since this is a failure scenario, the first thread will always fail. In that case, it depends on whether the second thread is scheduled or not which is out of our control.

// This is expected when the thread pool shuts down after the first
// segment fails with AmazonServiceException, causing the second
// segment's task to be rejected.
} catch (Exception e) {
fail("Should have seen the AmazonServiceException");
fail(
"Should have seen AmazonServiceException or RejectedExecutionException, but got: " +
e.getClass().getName() +
": " +
e.getMessage()
);
}
}

Expand Down

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: just having random PK will largely reduce flaky test.

Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0
package software.amazon.cryptography.examples;

import java.util.concurrent.ThreadLocalRandom;

/**
* Shared utilities for examples. The TEST_SUFFIX provides a unique suffix
* per JVM process so that parallel CI matrix jobs (different Java versions / OS)
* do not overwrite each other's items in the shared DynamoDB test table.
*
* Each example uses a hardcoded partition key like "rawEcdhKeyringItem".
* In CI, multiple JVMs run in parallel against the same table. Without
* isolation, one JVM can overwrite another's item, causing decrypt failures.
* By appending a random suffix, each JVM writes to its own partition key
* (e.g., "rawEcdhKeyringItem-482913") while keeping cross-example reads
* working within the same JVM.
*/
public class ExampleUtils {

private static final String TEST_SUFFIX = String.valueOf(
ThreadLocalRandom.current().nextInt(100000, 999999)
);

/**
* Returns a partition key value unique to this JVM process.
* Example: "rawEcdhKeyringItem" becomes "rawEcdhKeyringItem-482913"
*/
public static String uniquePk(String base) {
return base + "-" + TEST_SUFFIX;
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that partition key is random which will reduce flaky CI. This will end up creating huge DDB. So, we should put clean up code to remove the item created by examples. I would send the pk name from test then add clean up code in test to keep actual examples clean.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are multiple solutions to clean up.
But for now, I think we punt that issue, and accept these changes.
Addressing CI clean up is an issue best left for another day.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me attempt a afterClass in each file. If that takes too long, we can park that and come back with another cleanup story.

Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
import software.amazon.cryptography.dbencryptionsdk.dynamodb.model.DynamoDbTableEncryptionConfig;
import software.amazon.cryptography.dbencryptionsdk.dynamodb.model.DynamoDbTablesEncryptionConfig;
import software.amazon.cryptography.dbencryptionsdk.structuredencryption.model.CryptoAction;
import software.amazon.cryptography.examples.ExampleUtils;
import software.amazon.cryptography.materialproviders.IKeyring;
import software.amazon.cryptography.materialproviders.MaterialProviders;
import software.amazon.cryptography.materialproviders.model.CreateAwsKmsMrkDiscoveryMultiKeyringInput;
Expand Down Expand Up @@ -166,7 +167,10 @@ public static void ClientSupplierPutItemGetItem(
final HashMap<String, AttributeValue> item = new HashMap<>();
item.put(
"partition_key",
AttributeValue.builder().s("clientSupplierItem").build()
AttributeValue
.builder()
.s(ExampleUtils.uniquePk("clientSupplierItem"))
.build()
);
item.put("sort_key", AttributeValue.builder().n("0").build());
item.put(
Expand All @@ -191,7 +195,10 @@ public static void ClientSupplierPutItemGetItem(
final HashMap<String, AttributeValue> keyToGet = new HashMap<>();
keyToGet.put(
"partition_key",
AttributeValue.builder().s("clientSupplierItem").build()
AttributeValue
.builder()
.s(ExampleUtils.uniquePk("clientSupplierItem"))
.build()
);
keyToGet.put("sort_key", AttributeValue.builder().n("0").build());

Expand Down Expand Up @@ -289,7 +296,10 @@ public static void ClientSupplierPutItemGetItem(
new HashMap<>();
onlyReplicaKeyKeyToGet.put(
"partition_key",
AttributeValue.builder().s("awsKmsMrkMultiKeyringItem").build()
AttributeValue
.builder()
.s(ExampleUtils.uniquePk("awsKmsMrkMultiKeyringItem"))
.build()
);
onlyReplicaKeyKeyToGet.put(
"sort_key",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
import software.amazon.cryptography.dbencryptionsdk.dynamodb.enhancedclient.CreateDynamoDbEncryptionInterceptorInput;
import software.amazon.cryptography.dbencryptionsdk.dynamodb.enhancedclient.DynamoDbEnhancedClientEncryption;
import software.amazon.cryptography.dbencryptionsdk.dynamodb.enhancedclient.DynamoDbEnhancedTableEncryptionConfig;
import software.amazon.cryptography.examples.ExampleUtils;
import software.amazon.cryptography.examples.ExampleUtils;
import software.amazon.cryptography.materialproviders.IKeyring;
import software.amazon.cryptography.materialproviders.MaterialProviders;
import software.amazon.cryptography.materialproviders.model.CreateAwsKmsMrkMultiKeyringInput;
Expand Down Expand Up @@ -153,7 +155,7 @@ public static void PutItemGetItem(String kmsKeyId, String ddbTableName) {
// The item will be encrypted client-side according to your
// configuration above before it is sent to DynamoDb.
final SimpleClass4 item = new SimpleClass4();
item.setPartitionKey("EnhancedPutGetExample");
item.setPartitionKey(ExampleUtils.uniquePk("EnhancedPutGetExample"));
item.setSortKey(0);
item.setAttribute1("encrypt and sign me!");
item.setAttribute2("sign me!");
Expand All @@ -167,7 +169,7 @@ public static void PutItemGetItem(String kmsKeyId, String ddbTableName) {
// original item.
final Key key = Key
.builder()
.partitionValue("EnhancedPutGetExample")
.partitionValue(ExampleUtils.uniquePk("EnhancedPutGetExample"))
.sortValue(0)
.build();

Expand All @@ -181,7 +183,8 @@ public static void PutItemGetItem(String kmsKeyId, String ddbTableName) {

// retrieve the same record via a Query
PageIterable<SimpleClass4> items = table.query(
QueryConditional.keyEqualTo(k -> k.partitionValue("EnhancedPutGetExample")
QueryConditional.keyEqualTo(k ->
k.partitionValue(ExampleUtils.uniquePk("EnhancedPutGetExample"))
)
);
List<SimpleClass4> itemList = new ArrayList<SimpleClass4>();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@
import software.amazon.cryptography.dbencryptionsdk.dynamodb.enhancedclient.DynamoDbEnhancedClientEncryption;
import software.amazon.cryptography.dbencryptionsdk.dynamodb.enhancedclient.DynamoDbEnhancedTableEncryptionConfig;
import software.amazon.cryptography.dbencryptionsdk.dynamodb.itemencryptor.DynamoDbItemEncryptor;
import software.amazon.cryptography.examples.ExampleUtils;
import software.amazon.cryptography.examples.ExampleUtils;
import software.amazon.cryptography.materialproviders.IKeyring;
import software.amazon.cryptography.materialproviders.MaterialProviders;
import software.amazon.cryptography.materialproviders.model.CreateAwsKmsMrkMultiKeyringInput;
Expand Down Expand Up @@ -163,7 +165,7 @@ public static void PutItemGetItem(String kmsKeyId, String ddbTableName) {
// configuration above before it is sent to DynamoDb.
final SimpleViaLombok.SimpleViaLombokBuilder itemBuilder =
SimpleViaLombok.builder();
itemBuilder.partitionKey("LombokPutGetExample");
itemBuilder.partitionKey(ExampleUtils.uniquePk("LombokPutGetExample"));
itemBuilder.sortKey(0);
itemBuilder.attribute1("encrypt and sign me!");
itemBuilder.attribute2("sign me!");
Expand All @@ -176,7 +178,7 @@ public static void PutItemGetItem(String kmsKeyId, String ddbTableName) {
// original item.
final Key key = Key
.builder()
.partitionValue("LombokPutGetExample")
.partitionValue(ExampleUtils.uniquePk("LombokPutGetExample"))
.sortValue(0)
.build();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
import software.amazon.cryptography.dbencryptionsdk.dynamodb.enhancedclient.CreateDynamoDbEncryptionInterceptorInput;
import software.amazon.cryptography.dbencryptionsdk.dynamodb.enhancedclient.DynamoDbEnhancedClientEncryption;
import software.amazon.cryptography.dbencryptionsdk.dynamodb.enhancedclient.DynamoDbEnhancedTableEncryptionConfig;
import software.amazon.cryptography.examples.ExampleUtils;
import software.amazon.cryptography.examples.ExampleUtils;
import software.amazon.cryptography.materialproviders.IKeyring;
import software.amazon.cryptography.materialproviders.MaterialProviders;
import software.amazon.cryptography.materialproviders.model.CreateAwsKmsMrkMultiKeyringInput;
Expand Down Expand Up @@ -168,21 +170,21 @@ public static void TransactWriteItems(String kmsKeyId, String ddbTableName) {
// The item will be encrypted client-side according to your
// configuration above before it is sent to DynamoDb.
final SimpleClass item1 = new SimpleClass();
item1.setPartitionKey("EnhancedPutGetExample1");
item1.setPartitionKey(ExampleUtils.uniquePk("EnhancedPutGetExample1"));
item1.setSortKey(0);
item1.setAttribute1("item1 encrypt and sign me!");
item1.setAttribute2("item1 sign me!");
item1.setAttribute3("item1 ignore me!");

final SimpleClass2 item2 = new SimpleClass2();
item2.setPartitionKey("EnhancedPutGetExample2");
item2.setPartitionKey(ExampleUtils.uniquePk("EnhancedPutGetExample2"));
item2.setSortKey(0);
item2.setAttribute4("item2 encrypt and sign me!");
item2.setAttribute5("item2 sign me!");
item2.setAttribute3("item2 ignore me!");

final SimpleClass3 item3 = new SimpleClass3();
item3.setPartitionKey("EnhancedPutGetExample3");
item3.setPartitionKey(ExampleUtils.uniquePk("EnhancedPutGetExample3"));
item3.setSortKey(0);
item3.setAttribute6("item3 encrypt and sign me!");
item3.setAttribute2("item3 sign me!");
Expand All @@ -202,15 +204,15 @@ public static void TransactWriteItems(String kmsKeyId, String ddbTableName) {
// The item will be decrypted client-side, and you will get back the
// original item.
final SimpleClass key1 = new SimpleClass();
key1.setPartitionKey("EnhancedPutGetExample1");
key1.setPartitionKey(ExampleUtils.uniquePk("EnhancedPutGetExample1"));
key1.setSortKey(0);

final SimpleClass2 key2 = new SimpleClass2();
key2.setPartitionKey("EnhancedPutGetExample2");
key2.setPartitionKey(ExampleUtils.uniquePk("EnhancedPutGetExample2"));
key2.setSortKey(0);

final SimpleClass3 key3 = new SimpleClass3();
key3.setPartitionKey("EnhancedPutGetExample3");
key3.setPartitionKey(ExampleUtils.uniquePk("EnhancedPutGetExample3"));
key3.setSortKey(0);

final TransactGetItemsEnhancedRequest getRequest =
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@
import software.amazon.cryptography.dbencryptionsdk.dynamodb.enhancedclient.SignOnlyTag;
import software.amazon.cryptography.dbencryptionsdk.dynamodb.itemencryptor.DynamoDbItemEncryptor;
import software.amazon.cryptography.examples.ConfigUtils;
import software.amazon.cryptography.examples.ExampleUtils;
import software.amazon.cryptography.examples.ExampleUtils;
import software.amazon.cryptography.materialproviders.IKeyring;
import software.amazon.cryptography.materialproviders.MaterialProviders;
import software.amazon.cryptography.materialproviders.model.CreateAwsKmsMrkMultiKeyringInput;
Expand Down Expand Up @@ -212,7 +214,9 @@ public static void PutItemGetItem(String kmsKeyId, String ddbTableName) {
// The item will be encrypted client-side according to your
// configuration above before it is sent to DynamoDb.
final SimpleClass item = new SimpleClass();
item.setPartitionKey("TableSchemaBuilderPutGetExample");
item.setPartitionKey(
ExampleUtils.uniquePk("TableSchemaBuilderPutGetExample")
);
item.setSortKey(0);
item.setAttribute1("encrypt and sign me!");
item.setAttribute2("sign me!");
Expand All @@ -224,7 +228,7 @@ public static void PutItemGetItem(String kmsKeyId, String ddbTableName) {
// original item.
final Key key = Key
.builder()
.partitionValue("TableSchemaBuilderPutGetExample")
.partitionValue(ExampleUtils.uniquePk("TableSchemaBuilderPutGetExample"))
.sortValue(0)
.build();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
import software.amazon.cryptography.dbencryptionsdk.dynamodb.model.DynamoDbTableEncryptionConfig;
import software.amazon.cryptography.dbencryptionsdk.dynamodb.model.DynamoDbTablesEncryptionConfig;
import software.amazon.cryptography.dbencryptionsdk.structuredencryption.model.CryptoAction;
import software.amazon.cryptography.examples.ExampleUtils;
import software.amazon.cryptography.materialproviders.IKeyring;
import software.amazon.cryptography.materialproviders.MaterialProviders;
import software.amazon.cryptography.materialproviders.model.CreateAwsKmsEcdhKeyringInput;
Expand Down Expand Up @@ -339,7 +340,10 @@ private static void GetItemWithKeyring(
final HashMap<String, AttributeValue> keyToGet = new HashMap<>();
keyToGet.put(
"partition_key",
AttributeValue.builder().s("awsKmsEcdhKeyringItem").build()
AttributeValue
.builder()
.s(ExampleUtils.uniquePk("awsKmsEcdhKeyringItem"))
.build()
);
keyToGet.put("sort_key", AttributeValue.builder().n("0").build());

Expand Down Expand Up @@ -456,7 +460,10 @@ private static void PutGetItemWithKeyring(
final HashMap<String, AttributeValue> item = new HashMap<>();
item.put(
"partition_key",
AttributeValue.builder().s("awsKmsEcdhKeyringItem").build()
AttributeValue
.builder()
.s(ExampleUtils.uniquePk("awsKmsEcdhKeyringItem"))
.build()
);
item.put("sort_key", AttributeValue.builder().n("0").build());
item.put(
Expand All @@ -481,7 +488,10 @@ private static void PutGetItemWithKeyring(
final HashMap<String, AttributeValue> keyToGet = new HashMap<>();
keyToGet.put(
"partition_key",
AttributeValue.builder().s("awsKmsEcdhKeyringItem").build()
AttributeValue
.builder()
.s(ExampleUtils.uniquePk("awsKmsEcdhKeyringItem"))
.build()
);
keyToGet.put("sort_key", AttributeValue.builder().n("0").build());

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
import software.amazon.cryptography.dbencryptionsdk.dynamodb.model.DynamoDbTableEncryptionConfig;
import software.amazon.cryptography.dbencryptionsdk.dynamodb.model.DynamoDbTablesEncryptionConfig;
import software.amazon.cryptography.dbencryptionsdk.structuredencryption.model.CryptoAction;
import software.amazon.cryptography.examples.ExampleUtils;
import software.amazon.cryptography.materialproviders.IKeyring;
import software.amazon.cryptography.materialproviders.MaterialProviders;
import software.amazon.cryptography.materialproviders.model.CreateAwsKmsRsaKeyringInput;
Expand Down Expand Up @@ -201,7 +202,10 @@ public static void KmsRsaKeyringGetItemPutItem(
final HashMap<String, AttributeValue> item = new HashMap<>();
item.put(
"partition_key",
AttributeValue.builder().s("awsKmsRsaKeyringItem").build()
AttributeValue
.builder()
.s(ExampleUtils.uniquePk("awsKmsRsaKeyringItem"))
.build()
);
item.put("sort_key", AttributeValue.builder().n("0").build());
item.put(
Expand All @@ -226,7 +230,10 @@ public static void KmsRsaKeyringGetItemPutItem(
final HashMap<String, AttributeValue> keyToGet = new HashMap<>();
keyToGet.put(
"partition_key",
AttributeValue.builder().s("awsKmsRsaKeyringItem").build()
AttributeValue
.builder()
.s(ExampleUtils.uniquePk("awsKmsRsaKeyringItem"))
.build()
);
keyToGet.put("sort_key", AttributeValue.builder().n("0").build());

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
import software.amazon.cryptography.dbencryptionsdk.dynamodb.model.DynamoDbTableEncryptionConfig;
import software.amazon.cryptography.dbencryptionsdk.dynamodb.model.DynamoDbTablesEncryptionConfig;
import software.amazon.cryptography.dbencryptionsdk.structuredencryption.model.CryptoAction;
import software.amazon.cryptography.examples.ExampleUtils;
import software.amazon.cryptography.materialproviders.IKeyring;
import software.amazon.cryptography.materialproviders.MaterialProviders;
import software.amazon.cryptography.materialproviders.model.CreateAwsKmsMrkDiscoveryMultiKeyringInput;
Expand Down Expand Up @@ -157,7 +158,10 @@ public static void MultiMrkDiscoveryKeyringGetItemPutItem(
final HashMap<String, AttributeValue> item = new HashMap<>();
item.put(
"partition_key",
AttributeValue.builder().s("awsKmsMrkDiscoveryMultiKeyringItem").build()
AttributeValue
.builder()
.s(ExampleUtils.uniquePk("awsKmsMrkDiscoveryMultiKeyringItem"))
.build()
);
item.put("sort_key", AttributeValue.builder().n("0").build());
item.put(
Expand Down Expand Up @@ -250,7 +254,10 @@ public static void MultiMrkDiscoveryKeyringGetItemPutItem(
final HashMap<String, AttributeValue> keyToGet = new HashMap<>();
keyToGet.put(
"partition_key",
AttributeValue.builder().s("awsKmsMrkDiscoveryMultiKeyringItem").build()
AttributeValue
.builder()
.s(ExampleUtils.uniquePk("awsKmsMrkDiscoveryMultiKeyringItem"))
.build()
);
keyToGet.put("sort_key", AttributeValue.builder().n("0").build());

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
import software.amazon.cryptography.dbencryptionsdk.dynamodb.model.DynamoDbTableEncryptionConfig;
import software.amazon.cryptography.dbencryptionsdk.dynamodb.model.DynamoDbTablesEncryptionConfig;
import software.amazon.cryptography.dbencryptionsdk.structuredencryption.model.CryptoAction;
import software.amazon.cryptography.examples.ExampleUtils;
import software.amazon.cryptography.materialproviders.IKeyring;
import software.amazon.cryptography.materialproviders.MaterialProviders;
import software.amazon.cryptography.materialproviders.model.AesWrappingAlg;
Expand Down Expand Up @@ -203,7 +204,10 @@ public static void MultiKeyringGetItemPutItem(
final HashMap<String, AttributeValue> item = new HashMap<>();
item.put(
"partition_key",
AttributeValue.builder().s("multiKeyringItem").build()
AttributeValue
.builder()
.s(ExampleUtils.uniquePk("multiKeyringItem"))
.build()
);
item.put("sort_key", AttributeValue.builder().n("0").build());
item.put(
Expand All @@ -230,7 +234,10 @@ public static void MultiKeyringGetItemPutItem(
final HashMap<String, AttributeValue> keyToGet = new HashMap<>();
keyToGet.put(
"partition_key",
AttributeValue.builder().s("multiKeyringItem").build()
AttributeValue
.builder()
.s(ExampleUtils.uniquePk("multiKeyringItem"))
.build()
);
keyToGet.put("sort_key", AttributeValue.builder().n("0").build());

Expand Down Expand Up @@ -298,7 +305,10 @@ public static void MultiKeyringGetItemPutItem(
new HashMap<>();
onlyAesKeyringKeyToGet.put(
"partition_key",
AttributeValue.builder().s("multiKeyringItem").build()
AttributeValue
.builder()
.s(ExampleUtils.uniquePk("multiKeyringItem"))
.build()
);
onlyAesKeyringKeyToGet.put(
"sort_key",
Expand Down
Loading
Loading