|
1 | | -Coming soon! |
| 1 | +You will connect Elementary Cloud to Athena for syncing the Elementary schema (created by the [Elementary dbt package](/cloud/onboarding/quickstart-dbt-package)). |
| 2 | + |
| 3 | +<Snippet file="cloud/integrations/create-user-operation.mdx" /> |
| 4 | + |
| 5 | +## AWS Setup |
| 6 | + |
| 7 | +### 1. Create Required IAM Policy |
| 8 | + |
| 9 | +First, you'll need to create an IAM policy with the following permissions: |
| 10 | +- **AthenaPermissions**: Allows executing and retrieving query results from Athena |
| 11 | +- **GluePermissions**: Enables reading metadata about databases and tables |
| 12 | +- **S3AccessForStagingBuckets**: Provides full access to store Athena query results |
| 13 | +- **S3AccessForElementarySchema**: Grants read-only access to your elementary schema |
| 14 | + |
| 15 | +Here is an example of a JSON policy: |
| 16 | +```json |
| 17 | +{ |
| 18 | + "Version": "2012-10-17", |
| 19 | + "Statement": [ |
| 20 | + { |
| 21 | + "Sid": "AthenaPermissions", |
| 22 | + "Effect": "Allow", |
| 23 | + "Action": [ |
| 24 | + "athena:StartQueryExecution", |
| 25 | + "athena:GetQueryExecution", |
| 26 | + "athena:GetQueryResults" |
| 27 | + ], |
| 28 | + "Resource": "*" |
| 29 | + }, |
| 30 | + { |
| 31 | + "Sid": "GluePermissions", |
| 32 | + "Effect": "Allow", |
| 33 | + "Action": [ |
| 34 | + "glue:GetDatabase", |
| 35 | + "glue:GetDatabases", |
| 36 | + "glue:GetTable", |
| 37 | + "glue:GetTables", |
| 38 | + "glue:GetTableVersions", |
| 39 | + "glue:GetPartition", |
| 40 | + "glue:GetPartitions" |
| 41 | + ], |
| 42 | + "Resource": "*" |
| 43 | + }, |
| 44 | + { |
| 45 | + "Sid": "S3AccessForStagingBuckets", |
| 46 | + "Effect": "Allow", |
| 47 | + "Action": [ |
| 48 | + "s3:GetObject", |
| 49 | + "s3:PutObject", |
| 50 | + "s3:DeleteObject", |
| 51 | + "s3:ListBucket", |
| 52 | + "s3:GetBucketLocation" |
| 53 | + ], |
| 54 | + "Resource": [ |
| 55 | + "arn:aws:s3:::your-query-results-bucket", |
| 56 | + "arn:aws:s3:::your-query-results-bucket/*" |
| 57 | + ] |
| 58 | + }, |
| 59 | + { |
| 60 | + "Sid": "S3AccessForElementarySchema", |
| 61 | + "Effect": "Allow", |
| 62 | + "Action": [ |
| 63 | + "s3:GetObject", |
| 64 | + "s3:ListBucket" |
| 65 | + ], |
| 66 | + "Resource": [ |
| 67 | + "arn:aws:s3:::your-elementary-schema-bucket", |
| 68 | + "arn:aws:s3:::your-elementary-schema-bucket/*" |
| 69 | + ] |
| 70 | + } |
| 71 | + ] |
| 72 | +} |
| 73 | +``` |
| 74 | + |
| 75 | +### 2. Choose Authentication Method |
| 76 | + |
| 77 | +Elementary supports two authentication methods for connecting to Athena: |
| 78 | + |
| 79 | +#### Option 1: AWS Role Authentication (Recommended) |
| 80 | + |
| 81 | +This is the recommended approach as it provides better security and follows AWS best practices. [Learn more about AWS IAM roles](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html). |
| 82 | + |
| 83 | +1. **Create an IAM Role**: |
| 84 | + - Go to AWS IAM Console |
| 85 | + - Create a new role |
| 86 | + - Select "Another AWS account" as the trusted entity |
| 87 | + - Enter Elementary's AWS account ID: `743289191656` |
| 88 | + - (Optional but recommended) Enable "Require external ID" and set a value |
| 89 | + - Attach the policy created in step 1 |
| 90 | + |
| 91 | +2. **Note down the following information**: |
| 92 | + - Role ARN |
| 93 | + - External ID (if you enabled it) [Learn more about external IDs](https://aws.amazon.com/blogs/security/how-to-use-external-id-when-granting-access-to-your-aws-resources/). |
| 94 | + |
| 95 | +#### Option 2: Access Key Authentication |
| 96 | + |
| 97 | +This method is less secure as it requires permanent credentials. We recommend using AWS Role authentication instead. |
| 98 | + |
| 99 | +1. **Create an IAM User**: |
| 100 | + - Go to AWS IAM Console |
| 101 | + - Create a new user, that will be used by elementary to query athena |
| 102 | + - Enable programmatic access |
| 103 | + - Attach the policy created in step 1 |
| 104 | + |
| 105 | +2. **Note down the following information**: |
| 106 | + - AWS Access Key ID of the new elementary athena user |
| 107 | + - AWS Secret Access Key of the new elementary athena user |
| 108 | + |
| 109 | +## Elementary Configuration |
| 110 | + |
| 111 | +### Connection Settings |
| 112 | + |
| 113 | +Regardless of the authentication method you choose, you'll need to provide: |
| 114 | + |
| 115 | +- **Region**: The AWS region where your Athena instance is located |
| 116 | +- **Database**: The name of the database where your Elementary schema exist. |
| 117 | +- **Schema**: The name of your Elementary schema. Usually [schema name]_elementary |
| 118 | +- **S3 Staging Directory**: The S3 path where Athena query results will be stored |
| 119 | +- **Workgroup**: (Optional) Your Athena workgroup name |
| 120 | + |
| 121 | +### Authentication Details |
| 122 | + |
| 123 | +Based on your chosen authentication method: |
| 124 | + |
| 125 | +#### If using AWS Role Authentication: |
| 126 | +- Select "AWS Role" as the authentication method |
| 127 | +- Enter your role ARN |
| 128 | +- Enter your external ID (if you enabled it) |
| 129 | + |
| 130 | +#### If using Access Key Authentication: |
| 131 | +- Select "Access Key" as the authentication method |
| 132 | +- Enter your AWS Access Key ID |
| 133 | +- Enter your AWS Secret Access Key |
0 commit comments