Skip to content

Commit 44d11c5

Browse files
jochenehretjochenehret
committed
Use "storage-cli" implementation in use-s3-blobstore ops file
1 parent 9080d71 commit 44d11c5

2 files changed

Lines changed: 209 additions & 17 deletions

File tree

‎operations/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ This is the README for Ops-files. To learn more about `cf-deployment`, go to the
1616
| [`use-gcs-blobstore-access-key.yml`](use-gcs-blobstore-access-key.yml) | Enables access key credentials for Google blobstore. | Requires `use-external-blobstore.yml`. Introduces [new variables](example-vars-files/vars-use-gcs-blobstore-access-key.yml) for access key/secret and bucket names. | **NO** |
1717
| **AWS** | | | |
1818
| [`aws.yml`](aws.yml) | Overrides the loggregator endpoint port to 4443. | It is required to have a separate port from the standard HTTPS port (443) for loggregator traffic in order to use "classic" AWS ELBs. Newer "Application Load Balancers" and "Network Load Balancers" (as setup by [bbl](https://github.com/cloudfoundry/bosh-bootloader) >= v7.0.0) should not require this port override, so no need to use this ops-file if you're using the newer load balancer. | **YES** |
19-
| [`use-s3-blobstore.yml`](use-s3-blobstore.yml) | Configures external blobstore to use Amazon S3. | Requires `use-external-blobstore.yml`. Introduces [new variables](example-vars-files/vars-use-s3-blobstore.yml) for s3 credentials and bucket names. | **YES** |
19+
| [`use-s3-blobstore.yml`](use-s3-blobstore.yml) | Configures external blobstore to use Amazon S3. Uses the new "storage-cli" implementation instead of the deprecated "fog" library. | Requires `use-external-blobstore.yml`. Introduces [new variables](example-vars-files/vars-use-s3-blobstore.yml) for s3 credentials and bucket names. | **YES** |
2020
| **Azure** | | **\* Not validated or supported by the Release Integration team** | |
2121
| [`azure.yml`](azure.yml) | Sets gorouter's `frontend_idle_timeout` to value appropriate for Azure load balancers. | Any value below 240 should work. | **NO** |
2222
| [`use-azure-storage-blobstore.yml`](use-azure-storage-blobstore.yml) | Configures external blobstore to use Azure Storage. Uses the new "storage-cli" implementation instead of the deprecated "fog" library. | Requires `use-external-blobstore.yml`. Introduces [new variables](example-vars-files/vars-use-azure-storage-blobstore.yml) for Azure credentials and container names. | **NO** |

‎operations/use-s3-blobstore.yml‎

Lines changed: 208 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,53 +1,245 @@
11
---
2+
# Note: You must apply "use-external-blobstore.yml" before applying this ops file.
3+
4+
# ========= api =========
25
- type: replace
3-
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/buildpacks/fog_connection
4-
error: "Please apply 'use-external-blobstore.yml' before applying 'use-s3-blobstore.yml'."
5-
value: &blobstore-properties
6-
provider: AWS
6+
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/buildpacks/blobstore_type?
7+
value: storage-cli
8+
- type: replace
9+
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/buildpacks/blobstore_provider?
10+
value: AWS
11+
- type: replace
12+
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/buildpacks/connection_config?
13+
value:
14+
bucket_name: ((buildpack_directory_key))
715
aws_access_key_id: ((blobstore_access_key_id))
816
aws_secret_access_key: ((blobstore_secret_access_key))
917
region: ((aws_region))
18+
upload_concurrency: 10
19+
upload_part_size: 16777216
20+
download_concurrency: 10
21+
download_part_size: 16777216
22+
- type: remove
23+
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/buildpacks/fog_connection
1024

1125
- type: replace
26+
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/droplets/blobstore_type?
27+
value: storage-cli
28+
- type: replace
29+
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/droplets/blobstore_provider?
30+
value: AWS
31+
- type: replace
32+
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/droplets/connection_config?
33+
value:
34+
bucket_name: ((droplet_directory_key))
35+
aws_access_key_id: ((blobstore_access_key_id))
36+
aws_secret_access_key: ((blobstore_secret_access_key))
37+
region: ((aws_region))
38+
upload_concurrency: 10
39+
upload_part_size: 16777216
40+
download_concurrency: 10
41+
download_part_size: 16777216
42+
- type: remove
1243
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/droplets/fog_connection
13-
value: *blobstore-properties
1444

1545
- type: replace
46+
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/packages/blobstore_type?
47+
value: storage-cli
48+
- type: replace
49+
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/packages/blobstore_provider?
50+
value: AWS
51+
- type: replace
52+
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/packages/connection_config?
53+
value:
54+
bucket_name: ((app_package_directory_key))
55+
aws_access_key_id: ((blobstore_access_key_id))
56+
aws_secret_access_key: ((blobstore_secret_access_key))
57+
region: ((aws_region))
58+
upload_concurrency: 10
59+
upload_part_size: 16777216
60+
download_concurrency: 10
61+
download_part_size: 16777216
62+
- type: remove
1663
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/packages/fog_connection
17-
value: *blobstore-properties
1864

1965
- type: replace
66+
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/resource_pool/blobstore_type?
67+
value: storage-cli
68+
- type: replace
69+
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/resource_pool/blobstore_provider?
70+
value: AWS
71+
- type: replace
72+
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/resource_pool/connection_config?
73+
value:
74+
bucket_name: ((resource_directory_key))
75+
aws_access_key_id: ((blobstore_access_key_id))
76+
aws_secret_access_key: ((blobstore_secret_access_key))
77+
region: ((aws_region))
78+
upload_concurrency: 10
79+
upload_part_size: 16777216
80+
download_concurrency: 10
81+
download_part_size: 16777216
82+
- type: remove
2083
path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/resource_pool/fog_connection
21-
value: *blobstore-properties
2284

85+
# ========= cc-worker =========
86+
- type: replace
87+
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/buildpacks/blobstore_type?
88+
value: storage-cli
2389
- type: replace
90+
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/buildpacks/blobstore_provider?
91+
value: AWS
92+
- type: replace
93+
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/buildpacks/connection_config?
94+
value:
95+
bucket_name: ((buildpack_directory_key))
96+
aws_access_key_id: ((blobstore_access_key_id))
97+
aws_secret_access_key: ((blobstore_secret_access_key))
98+
region: ((aws_region))
99+
upload_concurrency: 10
100+
upload_part_size: 16777216
101+
download_concurrency: 10
102+
download_part_size: 16777216
103+
- type: remove
24104
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/buildpacks/fog_connection
25-
value: *blobstore-properties
26105

27106
- type: replace
107+
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/droplets/blobstore_type?
108+
value: storage-cli
109+
- type: replace
110+
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/droplets/blobstore_provider?
111+
value: AWS
112+
- type: replace
113+
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/droplets/connection_config?
114+
value:
115+
bucket_name: ((droplet_directory_key))
116+
aws_access_key_id: ((blobstore_access_key_id))
117+
aws_secret_access_key: ((blobstore_secret_access_key))
118+
region: ((aws_region))
119+
upload_concurrency: 10
120+
upload_part_size: 16777216
121+
download_concurrency: 10
122+
download_part_size: 16777216
123+
- type: remove
28124
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/droplets/fog_connection
29-
value: *blobstore-properties
30125

31126
- type: replace
127+
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/packages/blobstore_type?
128+
value: storage-cli
129+
- type: replace
130+
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/packages/blobstore_provider?
131+
value: AWS
132+
- type: replace
133+
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/packages/connection_config?
134+
value:
135+
bucket_name: ((app_package_directory_key))
136+
aws_access_key_id: ((blobstore_access_key_id))
137+
aws_secret_access_key: ((blobstore_secret_access_key))
138+
region: ((aws_region))
139+
upload_concurrency: 10
140+
upload_part_size: 16777216
141+
download_concurrency: 10
142+
download_part_size: 16777216
143+
- type: remove
32144
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/packages/fog_connection
33-
value: *blobstore-properties
34145

35146
- type: replace
147+
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/resource_pool/blobstore_type?
148+
value: storage-cli
149+
- type: replace
150+
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/resource_pool/blobstore_provider?
151+
value: AWS
152+
- type: replace
153+
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/resource_pool/connection_config?
154+
value:
155+
bucket_name: ((resource_directory_key))
156+
aws_access_key_id: ((blobstore_access_key_id))
157+
aws_secret_access_key: ((blobstore_secret_access_key))
158+
region: ((aws_region))
159+
upload_concurrency: 10
160+
upload_part_size: 16777216
161+
download_concurrency: 10
162+
download_part_size: 16777216
163+
- type: remove
36164
path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/resource_pool/fog_connection
37-
value: *blobstore-properties
38165

166+
# ========= scheduler (clock) =========
167+
- type: replace
168+
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/buildpacks/blobstore_type?
169+
value: storage-cli
170+
- type: replace
171+
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/buildpacks/blobstore_provider?
172+
value: AWS
39173
- type: replace
174+
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/buildpacks/connection_config?
175+
value:
176+
bucket_name: ((buildpack_directory_key))
177+
aws_access_key_id: ((blobstore_access_key_id))
178+
aws_secret_access_key: ((blobstore_secret_access_key))
179+
region: ((aws_region))
180+
upload_concurrency: 10
181+
upload_part_size: 16777216
182+
download_concurrency: 10
183+
download_part_size: 16777216
184+
- type: remove
40185
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/buildpacks/fog_connection
41-
value: *blobstore-properties
42186

43187
- type: replace
188+
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/droplets/blobstore_type?
189+
value: storage-cli
190+
- type: replace
191+
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/droplets/blobstore_provider?
192+
value: AWS
193+
- type: replace
194+
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/droplets/connection_config?
195+
value:
196+
bucket_name: ((droplet_directory_key))
197+
aws_access_key_id: ((blobstore_access_key_id))
198+
aws_secret_access_key: ((blobstore_secret_access_key))
199+
region: ((aws_region))
200+
upload_concurrency: 10
201+
upload_part_size: 16777216
202+
download_concurrency: 10
203+
download_part_size: 16777216
204+
- type: remove
44205
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/droplets/fog_connection
45-
value: *blobstore-properties
46206

47207
- type: replace
208+
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/packages/blobstore_type?
209+
value: storage-cli
210+
- type: replace
211+
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/packages/blobstore_provider?
212+
value: AWS
213+
- type: replace
214+
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/packages/connection_config?
215+
value:
216+
bucket_name: ((app_package_directory_key))
217+
aws_access_key_id: ((blobstore_access_key_id))
218+
aws_secret_access_key: ((blobstore_secret_access_key))
219+
region: ((aws_region))
220+
upload_concurrency: 10
221+
upload_part_size: 16777216
222+
download_concurrency: 10
223+
download_part_size: 16777216
224+
- type: remove
48225
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/packages/fog_connection
49-
value: *blobstore-properties
50226

51227
- type: replace
52-
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/resource_pool/fog_connection
53-
value: *blobstore-properties
228+
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/resource_pool/blobstore_type?
229+
value: storage-cli
230+
- type: replace
231+
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/resource_pool/blobstore_provider?
232+
value: AWS
233+
- type: replace
234+
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/resource_pool/connection_config?
235+
value:
236+
bucket_name: ((resource_directory_key))
237+
aws_access_key_id: ((blobstore_access_key_id))
238+
aws_secret_access_key: ((blobstore_secret_access_key))
239+
region: ((aws_region))
240+
upload_concurrency: 10
241+
upload_part_size: 16777216
242+
download_concurrency: 10
243+
download_part_size: 16777216
244+
- type: remove
245+
path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/resource_pool/fog_connection

0 commit comments

Comments
 (0)