Skip to content

Commit ccf1db7

Browse files
bethcgrokroskar
andauthored
docs: Azure Blob Storage Connection (#4488)
* Added documentation for access to s3 bucket * Improved guidelines Azure Blob storage * Solve formatting issues * chore: resize images for readability * Resolved prettier style issues * chore: formatting * docs: add note about SAS links expiring --------- Co-authored-by: Rok Roškar <rok.roskar@sdsc.ethz.ch> Co-authored-by: Rok Roškar <roskarr@ethz.ch>
1 parent 8a69776 commit ccf1db7

12 files changed

Lines changed: 154 additions & 2 deletions

docs/README.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ builds and deploys when changes are made in the repo.
4848
![image 10](./image-10.png)
4949
</p>
5050
```
51-
The available container sizes are `image-container-s` (55% width), `image-container-m` (70% width), and `image-container-l` (85% width).
51+
The available container sizes are `image-container-s` (55% width), `image-container-m` (70% width), and `image-container-l` (85% width), and `image-container-xl` (100% width).
5252
- Image properties that are controllable by CSS can be modified in the documentation CSS file at `docs/src/css/custom.css`. Currently, we use it to add shadows to all images and to define the container sizes for images:
5353

5454
```css
@@ -59,7 +59,8 @@ builds and deploys when changes are made in the repo.
5959

6060
.image-container-s,
6161
.image-container-m,
62-
.image-container-l {
62+
.image-container-l,
63+
.image-container-xl {
6364
margin: auto;
6465
text-align: center; /* Center the image horizontally when its width is narrower than the max-width*/
6566
}
@@ -75,6 +76,10 @@ builds and deploys when changes are made in the repo.
7576
.image-container-l {
7677
max-width: 85%;
7778
}
79+
80+
.image-container-l {
81+
max-width: 100%;
82+
}
7883
```
7984

8085
- When adding a video to a page, put it in the same folder as the page. You then need to import it in the markdown file using its relative path and put the imported name in the `src` attribute of the `video` tag. For example:
Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
---
2+
title: Azure Blob Storage
3+
---
4+
5+
# How to connect to Azure Blob Storage
6+
7+
:::info
8+
9+
This guide assumes Storage Account-level permissions. If you had Azure IAM (Identity) rights, you could use "Service Principals" or "Entra ID" logins, which remains outside the scope of this documentation.
10+
11+
:::
12+
13+
## Public access
14+
15+
This set-up allows you to provide access to data publicly without requiring authentication with **read-only** access. For setting this up, you just need your Azure storage account name and the container name. Bear in mind that access is controlled entirely at container level.
16+
17+
### Azure portal configuration
18+
19+
1. **Account level:** Go to **Settings > Configuration** and set **Allow Blob anonymous access** to **Enabled**.
20+
21+
<p class="image-container-xl">
22+
![Azure portal: Allow Blob anonymous access setting](./azure-portal-public-account-config.png)
23+
</p>
24+
25+
2. **Container level:** Go to **Data storage > Containers**, select your container, and click **Change access level**.
26+
- Select **Container**, which allows anonymously listing files and downloading.
27+
28+
<p class="image-container-xl">
29+
![Azure portal: Change access level dialog with Container option selected](./azure-portal-public-container-access.png)
30+
</p>
31+
32+
### Set up in Renku
33+
34+
1. Under **Data** section click on **+** button
35+
2. Go to the tab **+ Create a data connector** and select **azureblob**.
36+
3. Click on **Next**
37+
38+
<p class="image-container-s">
39+
![image.png](./add-azureblob-10.png)
40+
</p>
41+
42+
4. Set the following parameters in **Connection information**:
43+
1. **Source path**: the `container_name` as specified in the storage account you are using (e.g. `my-container`). You can also mount a sub-folder by appending it to the bucket name with a slash, e.g. `my-container/sub-folder`.
44+
2. **Account Name**: the storage account name, e.g. `elisabettestsa`
45+
46+
5. Click on **Test connection** and if succeeds, click **Next**
47+
48+
<p class="image-container-s">
49+
![image.png](./azureblob-20.png)
50+
</p>
51+
52+
6. On the last page, fill in the final details for your data connector, namely:
53+
1. **Name**: pick any name for the data connector (e.g. `data`)
54+
2. **Owner:** select the namespace of the data connector (e.g. the user's, project's or a group's).
55+
3. **Visibility:** decide whether the data connector should be Public or Private.
56+
4. **Read-only**: keep it as read-only access.
57+
58+
7. Click on **+ Add connector**.
59+
60+
<p class="image-container-s">
61+
![image.png](./azureblob-30.png)
62+
</p>
63+
64+
:::warning
65+
66+
If you set up an Azure blob for public access from a public project, anyone who
67+
can see your project may be able to pull data from the storage account. This
68+
could incurr egress costs on your Azure subsciption.
69+
70+
## Restricted access
71+
72+
Use this option to share data securely with your collaborators and stakeholders, controlling their access rights (e.g. read-only, write and delete files).
73+
74+
### Azure portal configuration
75+
76+
1. Go to **Data storage > Containers > [Your container] > Settings > Shared access tokens**. Select the desired permissions (e.g. **Read** and **List** for read-only access, select on top **Add**, **Create** and **Write** for read and write access, avoid **Delete** if you want an append/upload-only environment).
77+
78+
<p class="image-container-xl">
79+
![Azure portal: Shared access tokens permissions panel](./azure-portal-restricted-sas-permissions.png)
80+
</p>
81+
82+
2. Click on the button **Generate SAS token and URL**.
83+
3. Copy the generated **SAS URL** field.
84+
85+
<p class="image-container-xl">
86+
![Azure portal: Generated SAS URL field to copy](./azure-portal-restricted-sas-url.png)
87+
</p>
88+
89+
### Set up in Renku
90+
91+
Proceed to steps 1 to 3 from the [Public access set-up](#set-up-in-renku).
92+
93+
4. In **Connection information**, click on **Show full options list** and set the following parameters:
94+
1. **Source path**: the storage account name, e.g. `elisabettestsa`. You can also mount a sub-folder by appending it to the bucket name with a slash, e.g. `my-container/sub-folder`.
95+
2. **sas_url**: the `blob_sas_url` as generated in your storage account.
96+
97+
Continue with steps 5 to 7 as presented in the [public access set-up](#set-up-in-renku).
98+
99+
:::warning
100+
101+
You will need to share the sas_url value with your collaborators since they will be requested to enter the credentials. This will be stored as a user secret. Note also that SAS links expire and there is no automated process for renewing them, so you will need to generate a new one manually.
102+
103+
:::
104+
105+
:::info
106+
107+
The most advanced way to manage external access in Azure is through controlled access with revocation, which allows you to revoke access immediately without changing account keys. You need to create the policy under **Containers > [Your Container] > Settings > Access policy** and click **+ Add policy**. Define a name and the permissions. When you are creating the shared access token, select the policy you just created under **Stored access policy**. If you delete the policy in Azure, access to the container will be instantly revoked.
108+
109+
:::
110+
111+
## Full access to your storage account
112+
113+
This set-up allows you to have access to your full Azure storage account, with the access rights that you decide to set-up.
114+
115+
### Azure portal configuration
116+
117+
1. Go to **Security + networking > Shared access signature**. Select **Blob** service, **Service/Container/Object** types, and check **ONLY Read** and **List**.
118+
119+
<p class="image-container-xl">
120+
![Azure portal: Shared access signature page with Blob service and Read/List permissions selected](./azure-portal-full-access-sas.png)
121+
</p>
122+
123+
### Set up in Renku
124+
125+
Proceed to steps 1 to 3 from the [Public access set-up](#set-up-in-renku).
126+
127+
4. Set the following parameters in **Connection information**:
128+
1. **Source path**: the `container_name` as specified in the storage account you are using (e.g. `my-container`). You can also mount a sub-folder by appending it to the bucket name with a slash, e.g. `my-container/sub-folder`.
129+
2. **Account Name**: the storage account name, e.g. `elisabettestsa`
130+
131+
5. Click on **Test connection** and if succeeds, click **Next**
132+
133+
<p class="image-container-s">
134+
![image.png](./azureblob-20.png)
135+
</p>
136+
137+
6. On the last page, fill in the final details for your data connector, namely:
138+
1. **Name**: pick any name for the data connector (e.g. `data`)
139+
2. **Owner:** select the namespace of the data connector (e.g. the user's, project's or a group's).
140+
3. **Visibility:** decide whether the data connector should be Public or Private.
141+
4. **Read-only**: do not uncheck this box, or the data connector will not work properly.
142+
143+
7. Click on **+ Add connector**.
172 KB
Loading
995 KB
Loading
329 KB
Loading
304 KB
Loading
368 KB
Loading
326 KB
Loading
327 KB
Loading
245 KB
Loading

0 commit comments

Comments
 (0)