Skip to content

Unexpected listener draining upon gateway deletion on mergeGateways mode #6534

New issue
New issue
Closed
#6544
Closed
Unexpected listener draining upon gateway deletion on mergeGateways mode#6534
#6544
Assignees
zhaohuabing
Labels
kind/bugSomething isn't working
Milestone
v1.5.0 Release
@shahar-h

Description

@shahar-h
shahar-h
opened on Jul 16, 2025

Description:
When working with merge gateways mode, creating some gateways with the same listener port and then removing the oldest gateway, listener is drained, which causes all connections to be terminated.
The expected behavior is to only remove relevant filter chain.
While observing this issue I noticed that listener name is constructed from the oldest gateway resource, so when deleting it the listener is renamed to the new oldest gateway.

Note: The same issue can also happen in case there are multiple gateways with the same creationTimestamp.

Proposal: listener name can be in format

Repro steps:

  1. Install latest helm chart:
helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.4.2 -n envoy-gateway-system --create-namespace --wait
  1. Create gatewayClass and envoyProxy with mergedGateways enabled:
kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
  parametersRef:
    group: gateway.envoyproxy.io
    kind: EnvoyProxy
    name: config
    namespace: envoy-gateway-system
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: config
  namespace: envoy-gateway-system
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        type: NodePort
        patch:
          type: StrategicMerge
          value:
            spec:
              ports:
              - nodePort: 32500
                port: 80
  mergeGateways: true
EOF
  1. Create backend, gateway, and httproute in consumer1 namespace:
kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: consumer1
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: backend
  namespace: consumer1
---
apiVersion: v1
kind: Service
metadata:
  name: backend
  labels:
    app: backend
    service: backend
spec:
  ports:
    - name: http
      port: 3000
      targetPort: 3000
  selector:
    app: backend
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend
  namespace: consumer1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: backend
      version: v1
  template:
    metadata:
      labels:
        app: backend
        version: v1
    spec:
      serviceAccountName: backend
      containers:
        - image: gcr.io/k8s-staging-gateway-api/echo-basic:v20231214-v1.0.0-140-gf544a46e
          imagePullPolicy: IfNotPresent
          name: backend
          ports:
            - containerPort: 3000
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
---                  
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: eg
  namespace: consumer1
spec:
  gatewayClassName: eg
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      hostname: example.com
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: backend
  namespace: consumer1
spec:
  parentRefs:
    - name: eg
  hostnames:
    - "example.com"
  rules:
    - backendRefs:
        - group: ""
          kind: Service
          name: backend
          port: 3000
          weight: 1
      matches:
        - path:
            type: PathPrefix
            value: /
EOF
  1. Create backend, gateway, and httproute in consumer2 namespace:
kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: consumer2
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: backend
  namespace: consumer2
---
apiVersion: v1
kind: Service
metadata:
  name: backend
  labels:
    app: backend
    service: backend
spec:
  ports:
    - name: http
      port: 3000
      targetPort: 3000
  selector:
    app: backend
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend
  namespace: consumer2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: backend
      version: v1
  template:
    metadata:
      labels:
        app: backend
        version: v1
    spec:
      serviceAccountName: backend
      containers:
        - image: gcr.io/k8s-staging-gateway-api/echo-basic:v20231214-v1.0.0-140-gf544a46e
          imagePullPolicy: IfNotPresent
          name: backend
          ports:
            - containerPort: 3000
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
---                  
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: eg
  namespace: consumer2
spec:
  gatewayClassName: eg
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      hostname: example2.com
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: backend
  namespace: consumer2
spec:
  parentRefs:
    - name: eg
  hostnames:
    - "example2.com"
  rules:
    - backendRefs:
        - group: ""
          kind: Service
          name: backend
          port: 3000
          weight: 1
      matches:
        - path:
            type: PathPrefix
            value: /
EOF
  1. Listener name in configdump is consumer1/eg/http since consumer1/eg is the oldest gateway.
    listener_manager.total_filter_chains_draining metric value is 0
    listener_manager.total_listeners_draining metric value is 0
  2. Delete consumer1 namespace:
kubectl delete ns consumer1
  1. Listener name in configdump is now consumer2/eg/http.
    listener_manager.total_filter_chains_draining metric value is 0
    listener_manager.total_listeners_draining metric value is 1

Environment:
EG 1.4.2.

Metadata

Metadata

Assignees

Labels

kind/bugSomething isn't working

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions