Skip to content

Commit a68dfc0

Browse files
SwitchTV-BenBettridgeSwitchTV-BenBettridge
committed
Merge branch 'master' of github.com:Switch-TV/s3proxy into feature/overlay-middleware
2 parents e01852f + 59094ba commit a68dfc0

42 files changed

Lines changed: 3840 additions & 289 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/dependabot.yml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# To get started with Dependabot version updates, you'll need to specify which
2+
# package ecosystems to update and where the package manifests are located.
3+
# Please see the documentation for all configuration options:
4+
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
5+
6+
version: 2
7+
updates:
8+
- package-ecosystem: "maven" # See documentation for possible values
9+
directory: "/" # Location of package manifests
10+
schedule:
11+
interval: "monthly"

‎.github/workflows/ci-main.yml‎

Lines changed: 181 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,181 @@
1+
name: Main CI
2+
on:
3+
push:
4+
branches:
5+
- "master"
6+
- "develop"
7+
pull_request:
8+
branches:
9+
- "*"
10+
jobs:
11+
meta:
12+
runs-on: ubuntu-latest
13+
outputs:
14+
dockerhub-publish: ${{ steps.dockerhub-publish.outputs.defined }}
15+
registry: ghcr.io/${{ github.repository }}/container:${{ fromJSON(steps.docker_action_meta.outputs.json).labels['org.opencontainers.image.version'] }}
16+
container_tags: ${{ steps.docker_action_meta.outputs.tags }}
17+
container_labels: ${{ steps.docker_action_meta.outputs.labels }}
18+
container_buildtime: ${{ fromJSON(steps.docker_action_meta.outputs.json).labels['org.opencontainers.image.created'] }}
19+
container_version: ${{ fromJSON(steps.docker_action_meta.outputs.json).labels['org.opencontainers.image.version'] }}
20+
container_revision: ${{ fromJSON(steps.docker_action_meta.outputs.json).labels['org.opencontainers.image.revision'] }}
21+
container_base: ${{ fromJSON(steps.docker_action_meta.outputs.json).tags[0] }}
22+
new_release_version: ${{ steps.version.outputs.new_release_version }}
23+
steps:
24+
- name: Checkout
25+
uses: actions/checkout@v3
26+
with:
27+
submodules: false
28+
persist-credentials: false
29+
- id: dockerhub-publish
30+
if: "${{ env.MY_KEY != '' }}"
31+
run: echo "::set-output name=defined::true"
32+
env:
33+
MY_KEY: ${{ secrets.DOCKER_PASS }}
34+
- uses: actions/setup-node@v2
35+
with:
36+
node-version: "14"
37+
- name: Docker meta
38+
id: docker_action_meta
39+
uses: docker/metadata-action@v4.0.1
40+
with:
41+
images: ghcr.io/${{ github.repository }}/container
42+
flavor: |
43+
latest=false
44+
tags: |
45+
type=sha,format=long
46+
type=sha
47+
type=semver,pattern={{version}},value=${{ steps.version.outputs.new_release_version }}
48+
type=semver,pattern={{major}},value=${{ steps.version.outputs.new_release_version }}
49+
type=semver,pattern={{major}}.{{minor}},value=${{ steps.version.outputs.new_release_version }}
50+
type=ref,event=branch
51+
type=ref,event=pr
52+
type=ref,event=tag
53+
labels: |
54+
org.opencontainers.image.licenses=Apache-2.0
55+
runTests:
56+
runs-on: ubuntu-latest
57+
needs: [meta]
58+
steps:
59+
- uses: actions/checkout@v2
60+
with:
61+
submodules: "recursive"
62+
63+
# These steps are quick and will work or if fail only because of external issues
64+
- uses: actions/setup-java@v2
65+
with:
66+
distribution: "temurin"
67+
java-version: "11"
68+
cache: "maven"
69+
- uses: actions/setup-python@v4
70+
with:
71+
python-version: "3.8"
72+
cache: "pip"
73+
74+
#Run tests
75+
- name: Maven Set version
76+
run: |
77+
mvn versions:set -DnewVersion=${{ needs.meta.outputs.new_release_version }}
78+
- name: Maven Package
79+
run: |
80+
mvn package -DskipTests
81+
- name: Maven Test
82+
run: |
83+
mvn test
84+
- name: Other Test
85+
run: |
86+
./src/test/resources/run-s3-tests.sh
87+
88+
#Store the target
89+
- uses: actions/upload-artifact@v2
90+
with:
91+
name: s3proxy
92+
path: target/s3proxy
93+
- uses: actions/upload-artifact@v2
94+
with:
95+
name: pom
96+
path: pom.xml
97+
98+
Containerize:
99+
runs-on: ubuntu-latest
100+
needs: [runTests, meta]
101+
steps:
102+
#Yes we need code
103+
- uses: actions/checkout@v2
104+
- uses: actions/download-artifact@v2
105+
with:
106+
name: s3proxy
107+
path: target
108+
- uses: actions/download-artifact@v2
109+
with:
110+
name: pom
111+
path: .
112+
# These steps are quick and will work or if fail only because of external issues
113+
- name: Set up QEMU
114+
uses: docker/setup-qemu-action@v2
115+
- name: Set up Docker Buildx
116+
uses: docker/setup-buildx-action@v2
117+
118+
- name: Login to DockerHub
119+
uses: docker/login-action@v2.0.0
120+
if: github.event_name != 'pull_request' && needs.meta.outputs.dockerhub-publish == 'true'
121+
with:
122+
username: ${{ secrets.DOCKER_USER }}
123+
password: ${{ secrets.DOCKER_PASS }}
124+
125+
- name: Login to DockerHub
126+
uses: docker/login-action@v2.0.0
127+
if: github.event_name != 'pull_request'
128+
with:
129+
registry: ghcr.io
130+
username: ${{ github.actor }}
131+
password: ${{ secrets.GITHUB_TOKEN }}
132+
133+
#Generate Meta
134+
- name: Build and push
135+
uses: docker/build-push-action@v3
136+
with:
137+
context: .
138+
platforms: linux/amd64,linux/arm64
139+
push: ${{ github.event_name != 'pull_request' }}
140+
tags: ${{ needs.meta.outputs.container_base }}
141+
labels: ${{ needs.meta.outputs.container_labels }}
142+
build-args: |
143+
BUILDTIME=${{ needs.meta.outputs.container_buildtime }}
144+
VERSION=${{ needs.meta.outputs.container_version }}
145+
REVISION=${{ needs.meta.outputs.container_revision }}
146+
cache-from: type=registry,ref=${{ needs.meta.outputs.container_base }}
147+
cache-to: type=inline
148+
- uses: actions/setup-node@v2
149+
with:
150+
node-version: "14"
151+
- name: Setup regctl
152+
if: github.event_name != 'pull_request' && needs.meta.outputs.dockerhub-publish == 'true'
153+
run: |
154+
curl -L https://github.com/regclient/regclient/releases/download/v0.3.5/regctl-linux-amd64 >/tmp/regctl
155+
chmod 755 /tmp/regctl
156+
157+
- name: Docker meta
158+
if: github.event_name != 'pull_request' && needs.meta.outputs.dockerhub-publish == 'true'
159+
id: docker_action_meta
160+
uses: docker/metadata-action@v4.0.1
161+
with:
162+
images: andrewgaul/s3proxy
163+
flavor: |
164+
latest=false
165+
tags: |
166+
type=sha
167+
type=semver,pattern={{version}},value=${{ needs.meta.outputs.new_release_version }}
168+
type=semver,pattern={{major}},value=${{ needs.meta.outputs.new_release_version }}
169+
type=semver,pattern={{major}}.{{minor}},value=${{ needs.meta.outputs.new_release_version }}
170+
type=ref,event=branch
171+
type=ref,event=pr
172+
type=ref,event=tag
173+
labels: |
174+
org.opencontainers.image.licenses=Apache-2.0
175+
- name: Publish to Docker
176+
if: github.event_name != 'pull_request' && needs.meta.outputs.dockerhub-publish == 'true'
177+
run: |
178+
for line in $CONTAINER_DEST_TAGS; do echo working on "$line"; /tmp/regctl image copy $SOURCE_CONTAINER $line; done
179+
env:
180+
SOURCE_CONTAINER: ${{ needs.meta.outputs.new_release_version }}
181+
CONTAINER_DEST_TAGS: ${{ steps.docker_action_meta.outputs.tags }}

‎.github/workflows/run-tests.yml‎

Lines changed: 0 additions & 54 deletions
This file was deleted.

‎.releaserc‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
{
2+
"tagFormat": 's3proxy-${version}',
3+
"branches": [
4+
{
5+
"name": 'master',
6+
prerelease: false
7+
},
8+
{
9+
"name": 'releases\/+([0-9])?(\.\d+)(\.\d+|z|$)',
10+
prerelease: false
11+
},
12+
{
13+
"name": 'next',
14+
prerelease: false
15+
},
16+
{
17+
name: 'next-major',
18+
prerelease: true
19+
},
20+
{
21+
name: 'develop',
22+
prerelease: true
23+
},
24+
{
25+
name: 'develop\/.*',
26+
prerelease: true
27+
}
28+
]
29+
}

‎Dockerfile‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,8 @@ LABEL maintainer="Andrew Gaul <andrew@gaul.org>"
1313
WORKDIR /opt/s3proxy
1414

1515
COPY \
16-
--from=s3proxy-builder \
17-
/opt/s3proxy/target/s3proxy \
18-
/opt/s3proxy/src/main/resources/run-docker-container.sh \
16+
target/s3proxy \
17+
src/main/resources/run-docker-container.sh \
1918
/opt/s3proxy/
2019

2120
ENV \
@@ -33,6 +32,9 @@ ENV \
3332
S3PROXY_OVERLAY_BLOBSTORE="false" \
3433
S3PROXY_OVERLAY_BLOBSTORE_MASK_SUFFIX="__deleted" \
3534
S3PROXY_OVERLAY_BLOBSTORE_PATH="/tmp" \
35+
S3PROXY_ENCRYPTED_BLOBSTORE="" \
36+
S3PROXY_ENCRYPTED_BLOBSTORE_PASSWORD="" \
37+
S3PROXY_ENCRYPTED_BLOBSTORE_SALT="" \
3638
JCLOUDS_PROVIDER="filesystem" \
3739
JCLOUDS_ENDPOINT="" \
3840
JCLOUDS_REGION="" \

‎README.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,8 @@ and has instructions on how to run it.
2323

2424
Users can [download releases](https://github.com/gaul/s3proxy/releases)
2525
from GitHub. Developers can build the project by running `mvn package` which
26-
produces a binary at `target/s3proxy`. S3Proxy requires Java 8 or newer to run.
26+
produces a binary at `target/s3proxy`. S3Proxy requires Java 11 or newer to
27+
run.
2728

2829
Configure S3Proxy via a properties file. An example using the local
2930
file system as the storage backend with anonymous access:
@@ -155,12 +156,12 @@ for specific storage backends.
155156

156157
* [Apache jclouds](https://jclouds.apache.org/) provides storage backend support for S3Proxy
157158
* [Ceph s3-tests](https://github.com/ceph/s3-tests) help maintain and improve compatibility with the S3 API
158-
* [fake-s3](https://github.com/jubos/fake-s3), [gofakes3](https://github.com/johannesboyne/gofakes3), [S3 ninja](https://github.com/scireum/s3ninja), and [s3rver](https://github.com/jamhall/s3rver) provide functionality similar to S3Proxy when using the filesystem backend
159+
* [fake-s3](https://github.com/jubos/fake-s3), [gofakes3](https://github.com/johannesboyne/gofakes3), [minio](https://github.com/minio/minio), [S3 ninja](https://github.com/scireum/s3ninja), and [s3rver](https://github.com/jamhall/s3rver) provide functionality similar to S3Proxy when using the filesystem backend
159160
* [GlacierProxy](https://github.com/bouncestorage/glacier-proxy) and [SwiftProxy](https://github.com/bouncestorage/swiftproxy) provide similar functionality for the Amazon Glacier and OpenStack Swift APIs
160-
* [minio](https://github.com/minio/minio) and [Zenko](https://www.zenko.io/) provide similar multi-cloud functionality
161161
* [s3mock](https://github.com/findify/s3mock) mocks the S3 API for Java/Scala projects
162162
* [sbt-s3](https://github.com/localytics/sbt-s3) runs S3Proxy via the Scala Build Tool
163163
* [swift3](https://github.com/openstack/swift3) provides an S3 middleware for OpenStack Swift
164+
* [Zenko](https://www.zenko.io/) provide similar multi-cloud functionality
164165

165166
## License
166167

0 commit comments

Comments
 (0)