Add `enable-gateways` config option to support mesh-only mode by prashanthjos · Pull Request #1520 · knative-extensions/net-istio

prashanthjos · 2026-03-07T03:43:42Z

Add a new enable-gateways key to the config-istio ConfigMap that allows disabling Istio Gateway creation and reconciliation. When set to "false", the ingress reconciler takes a simplified mesh-only path:

Only mesh VirtualServices are created (no ingress VS, no Gateways)
TLS secret copying and gateway server management are skipped
Readiness probing targets destination service pods directly
LoadBalancer status reports MeshOnly for both public and private
FinalizeKind skips gateway server cleanup

This enables using net-istio purely for service-to-service mesh communication without requiring Istio ingress gateway infrastructure.

The option defaults to "true" for full backward compatibility.

Changes

🎁 Add EnableGateways bool field to Istio config struct, parsed from enable-gateways ConfigMap key (defaults to true)
🎁 Add reconcileMeshOnlyIngress path that creates only mesh VirtualServices when gateways are disabled
🎁 Add mesh-only probing in ListProbeTargets that targets destination service pods directly when gateways are disabled
🧹 Guard FinalizeKind gateway server cleanup behind EnableGateways check

/kind enhancement

Fixes #

Release Note

Add `enable-gateways` key to `config-istio` ConfigMap. When set to "false", net-istio operates in mesh-only mode: no Istio Gateways are created, only mesh VirtualServices are reconciled, and readiness probing targets destination service pods directly. This enables using Knative with net-istio for service-to-service mesh communication without requiring Istio ingress gateway infrastructure. Defaults to "true" for backward compatibility.

TBD - will open a docs PR to document the new enable-gateways config option.

Add a new `enable-gateways` key to the `config-istio` ConfigMap that allows disabling Istio Gateway creation and reconciliation. When set to "false", the ingress reconciler takes a simplified mesh-only path: - Only mesh VirtualServices are created (no ingress VS, no Gateways) - TLS secret copying and gateway server management are skipped - Readiness probing targets destination service pods directly - LoadBalancer status reports MeshOnly for both public and private - FinalizeKind skips gateway server cleanup This enables using net-istio purely for service-to-service mesh communication without requiring Istio ingress gateway infrastructure. The option defaults to "true" for full backward compatibility.

knative-prow · 2026-03-07T03:43:51Z

Welcome @prashanthjos! It looks like this is your first PR to knative-extensions/net-istio 🎉

knative-prow · 2026-03-07T03:43:52Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: prashanthjos
Once this PR has been reviewed and has the lgtm label, please assign dprotaso for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

knative-prow · 2026-03-07T03:43:52Z

Hi @prashanthjos. Thanks for your PR.

I'm waiting for a knative-extensions member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

dprotaso · 2026-03-08T23:15:38Z

Rather than a new option we setup the configuration changes in net-istio to allow for empty gateways. Though we didn't implement it

eg.

net-istio/config/400-config-istio.yaml

Line 49 in 2bcb827

# external-gateways: |

external-gateways: "[]"

Explicitly stating an empty list of gateways (via an yaml string of an empty array) should indicate that we do not want any external gateways.

We can do the same with the local gateways.

dprotaso · 2026-03-08T23:16:03Z

Empty string should remain the default value

prashanthjos · 2026-03-09T18:52:39Z

@dprotaso I tried to address your comments, can you please take a look, if possible can you add ok-to-test command, I tried to run things locally as much as possible, I am nervous about the tests and want to make sure I am not breaking anything.

dprotaso

did a quick pass - here's an initial review

dprotaso · 2026-03-10T15:39:05Z

+    # ingress gateways), set both gateway lists to empty:
+    #   external-gateways: "[]"
+    #   local-gateways: "[]"
+    # This disables gateway creation and probes destination service pods directly.


I don't think we need this in the testdata config

It would make sense to add these examples to the main config

net-istio/config/400-config-istio.yaml

Line 45 in ee32f9d

# external-gateways defines a gateway and Istio service to serve external traffic.

and the local gateway

net-istio/config/400-config-istio.yaml

Line 92 in ee32f9d

# local-gateways: |

It's also probably important that we say that we don't support adding/removing gateways after Knative Services have been created

dprotaso · 2026-03-10T15:39:31Z


 	// IstioNamespace is the namespace containing Istio
 	IstioNamespace = "istio-system"
+


remove these extra new lines in various places. Just makes the more 'busy' than it needs to be

dprotaso · 2026-03-10T17:05:07Z

+// listMeshProbeTargets builds probe targets from the Ingress spec's destination
+// services when no gateways are available (mesh-only mode). It resolves each
+// backend service to its Endpoints and probes the pods directly.
+func (l *gatewayPodTargetLister) listMeshProbeTargets(ing *v1alpha1.Ingress) ([]status.ProbeTarget, error) {


It resolves each backend service to its Endpoints and probes the pods directly.

The intent of probing is to make sure that the routing rules have been propagated. I'm not sure we can do this in mesh because we don't want to probe a pod because it's IP won't be accessible in a sidecar mode. It will probably work in ambient since pod IPs are addressable.

hmm I think you are right, I've reworked the implementation with the assumption that the controller has a istio sidecar( we have enabled the isio side cars for net-istio controllers). listMeshProbeTargets now uses the rule host from the Ingress spec as the probe target directly. The controller's sidecar intercepts the outbound traffic, matches the authority against the mesh VirtualService, and routes to the correct revision service. This verifies the actual mesh routing path end-to-end.

The function is now much simpler, no service lookups and endpoint resolution. I am using the hostname and port from the KIngress spec.

I also used the health check to see if istio-proxy side car exists and then only proceed with probing.

I've reworked the implementation with the assumption that the controller has a istio sidecar( we have enabled the isio side cars for net-istio controllers).

For testing we should assume we the net-istio controller doesn't have a sidecar.

We specifically opt out of it so it can probe the public and internal gateway

net-istio/config/500-controller.yaml

Lines 32 to 35 in baa9bb9

# This must be outside of the mesh to probe the gateways.

# NOTE: this is allowed here and not elsewhere because

# this is the Istio controller, and so it may be Istio-aware.

sidecar.istio.io/inject: "false"

I don't think we can reliably probe the gateways while in the mesh

ok may be I dint phrase it correctly, In Gateway mode (default) ,sidecar.istio.io/inject: "false" stays as the default in 500-controller.yaml. Gateway probing works exactly as before, no mesh involvement and in Mesh-only mode (external-gateways: "[]", local-gateways: "[]"), the controller checks for a sidecar at runtime by hitting localhost:15021/healthz/ready. If no sidecar is detected (default deployment), Mesh probing is skipped entirely. The ingress is marked ready immediately after VirtualService reconciliation. A warning log is emitted telling operators how to enable mesh probing. However if a sidecar is detected (operator explicitly set sidecar.istio.io/inject: "true" while installing net-istio), probes route through the mesh VirtualService for a more thorough readiness check.

So we never assume a sidecar exists. The detection is a one-time check (sync.Once), and the fallback (skip probing) is safe, the mesh VS is the only networking config needed in mesh-only mode, and its creation is verified by the reconciler before marking ready. The two modes are mutually exclusive in their sidecar requirements, we can't probe gateway pods AND route through the mesh simultaneously.

Also I added a clear comment on 202-gateway.yaml and 203-local-gateway.yaml saying that these are not used when we enable mesh mode on the controller although they are statically created. I also added a better comment on 500-controller.yaml about what the sidecar annotation would do.

One important point that I want to make clear is whoever is trying to use net-istio in mesh mode should clone the repo and update the sidecar.istio.io/inject annotation to true and then proceed with kubectl apply.

Also, the intent here is, some teams already have existing gateways and may not want to use them for Knative services or they don't want to create the new ones at all, in such cases we don't want to force the traffic through knative gateway for probing scenarios for various reasons like gateways might already be experiencing high traffic and don't want the probe traffic to go through them. My intent here is net-istio already has so many things we already need and I am trying to fit in the usecases without breaking the existing functionality.

the controller checks for a sidecar at runtime by hitting localhost:15021/healthz/ready

But this won't work in ambient mode I'm assuming?

dprotaso · 2026-03-10T17:12:23Z

/ok-to-test

codecov · 2026-03-10T17:15:23Z

Codecov Report

❌ Patch coverage is 73.39450% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.47%. Comparing base (2bcb827) to head (5a56a46).
⚠️ Report is 12 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/reconciler/ingress/lister.go	66.07%	17 Missing and 2 partials ⚠️
pkg/reconciler/ingress/ingress.go	80.48%	4 Missing and 4 partials ⚠️
pkg/reconciler/ingress/config/istio.go	83.33%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1520      +/-   ##
==========================================
- Coverage   81.96%   81.47%   -0.49%     
==========================================
  Files          23       23              
  Lines        1519     1614      +95     
==========================================
+ Hits         1245     1315      +70     
- Misses        180      201      +21     
- Partials       94       98       +4

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

prashanthjos · 2026-03-11T02:37:27Z

/retest

dprotaso · 2026-03-14T21:40:49Z

Part of this was pulled into #1524

/hold until that merges and this PR gets revisited

knative-prow-robot · 2026-04-02T15:20:13Z

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

knative-prow Bot added the kind/enhancement label Mar 7, 2026

knative-prow Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 7, 2026

knative-prow Bot requested review from dprotaso and skonto March 7, 2026 03:43

prashanthjos added 2 commits March 9, 2026 10:28

Correcting the config-istio to use external-gateways and local-gateways

80e8e20

Correcting the config-istio to use external-gateways and local-gateways

015c687

dprotaso reviewed Mar 10, 2026

View reviewed changes

knative-prow Bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 10, 2026

Addressing the review comments

b8fc727

Addressing comments

5a56a46

knative-prow Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 14, 2026

knative-prow-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 2, 2026


		// IstioNamespace is the namespace containing Istio
		IstioNamespace = "istio-system"

	# This must be outside of the mesh to probe the gateways.
	# NOTE: this is allowed here and not elsewhere because
	# this is the Istio controller, and so it may be Istio-aware.
	sidecar.istio.io/inject: "false"

Conversation

prashanthjos commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Uh oh!

knative-prow Bot commented Mar 7, 2026

Uh oh!

knative-prow Bot commented Mar 7, 2026

Uh oh!

knative-prow Bot commented Mar 7, 2026

Uh oh!

dprotaso commented Mar 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dprotaso commented Mar 8, 2026

Uh oh!

prashanthjos commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dprotaso left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prashanthjos Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prashanthjos Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prashanthjos Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dprotaso commented Mar 10, 2026

Uh oh!

codecov Bot commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

prashanthjos commented Mar 11, 2026

Uh oh!

dprotaso commented Mar 14, 2026

Uh oh!

knative-prow-robot commented Apr 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

prashanthjos commented Mar 7, 2026 •

edited

Loading

dprotaso commented Mar 8, 2026 •

edited

Loading

prashanthjos commented Mar 9, 2026 •

edited

Loading

prashanthjos Mar 11, 2026 •

edited

Loading

prashanthjos Mar 11, 2026 •

edited

Loading

prashanthjos Mar 12, 2026 •

edited

Loading

codecov Bot commented Mar 10, 2026 •

edited

Loading