Skip to content

Commit e4b57e6

Browse files
committed
add an integration and e2e test for the remote infrastructure provider
Signed-off-by: Zachary Nixon <nixozach@amazon.com>
1 parent c4958d1 commit e4b57e6

30 files changed

Lines changed: 2486 additions & 7 deletions

File tree

api/v1alpha1/envoygateway_types.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -551,7 +551,7 @@ type EnvoyGatewayKubernetesCustomProvider struct {
551551

552552
// InfrastructureProviderType defines the types of custom infrastructure providers supported by Envoy Gateway.
553553
//
554-
// +kubebuilder:validation:Enum=Host
554+
// +kubebuilder:validation:Enum=Host;Remote
555555
type InfrastructureProviderType string
556556

557557
const (

charts/gateway-crds-helm/templates/generated/gateway.envoyproxy.io_envoyproxies.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11271,8 +11271,9 @@ spec:
1127111271
useListenerPortAsContainerPort:
1127211272
description: |-
1127311273
UseListenerPortAsContainerPort disables the port shifting feature in the Envoy Proxy.
11274-
When set to false (default value), if the service port is a privileged port (1-1023), add a constant to the value converting it into an ephemeral port.
11274+
When set to false, if the service port is a privileged port (1-1023), add a constant to the value converting it into an ephemeral port.
1127511275
This allows the container to bind to the port without needing a CAP_NET_BIND_SERVICE capability.
11276+
The default value is True, which means no port shifting occurs.
1127611277
type: boolean
1127711278
type: object
1127811279
type:

charts/gateway-helm/charts/crds/crds/generated/gateway.envoyproxy.io_envoyproxies.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11270,8 +11270,9 @@ spec:
1127011270
useListenerPortAsContainerPort:
1127111271
description: |-
1127211272
UseListenerPortAsContainerPort disables the port shifting feature in the Envoy Proxy.
11273-
When set to false (default value), if the service port is a privileged port (1-1023), add a constant to the value converting it into an ephemeral port.
11273+
When set to false, if the service port is a privileged port (1-1023), add a constant to the value converting it into an ephemeral port.
1127411274
This allows the container to bind to the port without needing a CAP_NET_BIND_SERVICE capability.
11275+
The default value is True, which means no port shifting occurs.
1127511276
type: boolean
1127611277
type: object
1127711278
type:

examples/remote-infra/Dockerfile

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
FROM golang:1.26.3 AS builder
2+
3+
ARG GO_LDFLAGS=""
4+
5+
WORKDIR /workspace
6+
COPY go.mod go.sum ./
7+
RUN --mount=type=cache,target=/root/.cache/go-build --mount=type=cache,target=/go/pkg/mod \
8+
go mod download
9+
10+
COPY . ./
11+
RUN --mount=type=cache,target=/root/.cache/go-build --mount=type=cache,target=/go/pkg/mod \
12+
CGO_ENABLED=0 \
13+
GOOS=${TARGETOS} \
14+
GOARCH=${TARGETARCH} \
15+
go build -o /bin/remote-infra -ldflags "${GO_LDFLAGS}" .
16+
17+
# Alpine rather than distroless because the sidecar's exec probes need a
18+
# `test` binary on PATH (busybox provides it).
19+
FROM alpine:3.20
20+
COPY --from=builder /bin/remote-infra /
21+
22+
ENTRYPOINT ["/remote-infra"]

examples/remote-infra/Makefile

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
2+
IMAGE_PREFIX ?= envoyproxy/gateway-
3+
APP_NAME ?= remote-infra
4+
TAG ?= latest
5+
GO_LDFLAGS ?=
6+
7+
.PHONY: docker-buildx
8+
docker-buildx:
9+
docker buildx build . -t $(IMAGE_PREFIX)$(APP_NAME):$(TAG) --build-arg GO_LDFLAGS="$(GO_LDFLAGS)" --load

examples/remote-infra/README.md

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Remote Infrastructure Provider Example
2+
3+
This example is a reference implementation of the Envoy Gateway [Remote
4+
Infrastructure Provider][docs] gRPC service. It receives the Envoy Gateway
5+
Infrastructure IR over gRPC and reconciles a `Deployment` and a `Service` per
6+
proxy fleet against the Kubernetes API.
7+
8+
It is the server used by the `e2e-remote-infra` test target and is intended as
9+
a starting point for writing your own provider — not a production-ready
10+
implementation.
11+
12+
For the full conceptual model and a quickstart that wires this image into an
13+
Envoy Gateway install, see [the task documentation][docs].
14+
15+
## What it does
16+
17+
The provider implements the four RPCs defined in
18+
[`proto/remoteinfra/service.proto`](../../proto/remoteinfra/service.proto):
19+
20+
| RPC | Behavior |
21+
| ------------------------------ | ---------------------------------------------------------------- |
22+
| `CreateOrUpdateProxyInfra` | Reconciles a `Deployment` and `Service` for the IR's proxy fleet |
23+
| `DeleteProxyInfra` | Deletes the `Deployment` and `Service` for the IR's proxy fleet |
24+
| `CreateOrUpdateRateLimitInfra` | No-op. Rate limiting must be provisioned out of band, see below |
25+
| `DeleteRateLimitInfra` | No-op |
26+
27+
The IR is delivered as JSON in the request's `ir_bytes` field. The example
28+
ignores fields it doesn't understand, which is the same forward-compatibility
29+
posture you'll want in your own provider.
30+
31+
## Layout
32+
33+
| Path | Purpose |
34+
| ------------------- | ------------------------------------------------------------------------ |
35+
| `main.go` | Binds a Unix domain socket and starts the gRPC server |
36+
| `synthesizer/` | Translates the IR into desired `Deployment` / `Service` objects |
37+
| `synthesizer/*.tpl` | Embedded Envoy bootstrap template |
38+
| `pb/` | Generated gRPC stubs (regenerated from the proto in this repo) |
39+
| `Dockerfile` | Builds the server image. Alpine base — see note below |
40+
| `Makefile` | `docker-buildx` target produces `envoyproxy/gateway-remote-infra:latest` |
41+
42+
## Build
43+
44+
```shell
45+
make docker-buildx
46+
```
47+
48+
If you're running on `kind`, load the image into the cluster:
49+
50+
```shell
51+
kind load docker-image --name envoy-gateway envoyproxy/gateway-remote-infra:latest
52+
```
53+
54+
## Run
55+
56+
The expected deployment shape is as a native sidecar of the `envoy-gateway`
57+
Pod, sharing an `emptyDir` volume that holds the UDS. The full patch is in
58+
`test/e2e/remote_infra/sidecar-patch.yaml` and is documented in [the task
59+
guide][docs]. By default the server listens on
60+
`/var/run/remote-infra/server.sock` with mode `0660`; both are tunable via
61+
flags:
62+
63+
```text
64+
--socket-path=/var/run/remote-infra/server.sock
65+
--socket-mode=0660
66+
```
67+
68+
The `NAMESPACE` environment variable selects which Kubernetes namespace the
69+
provider writes resources into. The sidecar patch wires this to the Pod's own
70+
namespace via the downward API.
71+
72+
## Limitations
73+
74+
This is a teaching example, not a production provider. In particular:
75+
76+
- It only renders one Deployment and one Service per IR. Real providers will
77+
often want a HorizontalPodAutoscaler, PodDisruptionBudget, and so on.
78+
- It does not provision rate limit infrastructure.
79+
- It does not surface reconcile errors back to Envoy Gateway through the gRPC
80+
response in a structured way; errors are returned as plain strings.

examples/remote-infra/go.mod

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
module github.com/envoyproxy/gateway-remote-infra
2+
3+
go 1.26.3
4+
5+
require (
6+
google.golang.org/grpc v1.81.1
7+
google.golang.org/protobuf v1.36.12-0.20260120151049-f2248ac996af
8+
k8s.io/api v0.36.0
9+
k8s.io/apimachinery v0.36.0
10+
k8s.io/utils v0.0.0-20260210185600-b8788abfbbc2
11+
sigs.k8s.io/controller-runtime v0.24.1
12+
)
13+
14+
require (
15+
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
16+
github.com/emicklei/go-restful/v3 v3.13.0 // indirect
17+
github.com/evanphx/json-patch/v5 v5.9.11 // indirect
18+
github.com/fxamacker/cbor/v2 v2.9.0 // indirect
19+
github.com/go-logr/logr v1.4.3 // indirect
20+
github.com/go-openapi/jsonpointer v0.21.0 // indirect
21+
github.com/go-openapi/jsonreference v0.20.2 // indirect
22+
github.com/go-openapi/swag v0.23.0 // indirect
23+
github.com/google/gnostic-models v0.7.0 // indirect
24+
github.com/google/uuid v1.6.0 // indirect
25+
github.com/josharian/intern v1.0.0 // indirect
26+
github.com/json-iterator/go v1.1.12 // indirect
27+
github.com/mailru/easyjson v0.7.7 // indirect
28+
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
29+
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect
30+
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
31+
github.com/spf13/pflag v1.0.9 // indirect
32+
github.com/x448/float16 v0.8.4 // indirect
33+
go.yaml.in/yaml/v2 v2.4.3 // indirect
34+
go.yaml.in/yaml/v3 v3.0.4 // indirect
35+
golang.org/x/net v0.55.0 // indirect
36+
golang.org/x/oauth2 v0.36.0 // indirect
37+
golang.org/x/sys v0.45.0 // indirect
38+
golang.org/x/term v0.43.0 // indirect
39+
golang.org/x/text v0.37.0 // indirect
40+
golang.org/x/time v0.14.0 // indirect
41+
google.golang.org/genproto/googleapis/rpc v0.0.0-20260226221140-a57be14db171 // indirect
42+
gopkg.in/inf.v0 v0.9.1 // indirect
43+
gopkg.in/yaml.v3 v3.0.1 // indirect
44+
k8s.io/client-go v0.36.0 // indirect
45+
k8s.io/klog/v2 v2.140.0 // indirect
46+
k8s.io/kube-openapi v0.0.0-20260317180543-43fb72c5454a // indirect
47+
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 // indirect
48+
sigs.k8s.io/randfill v1.0.0 // indirect
49+
sigs.k8s.io/structured-merge-diff/v6 v6.3.2 // indirect
50+
sigs.k8s.io/yaml v1.6.0 // indirect
51+
)

0 commit comments

Comments
 (0)