Notice:
This is the "latest" release of Envoy Gateway, which contains the most recent commits from the main branch.
This release might not be stable.
Please refer to the /docs documentation for the most current information.

Proxy Health Check Logs

Envoy Gateway can log health check events for upstream clusters using the healthCheckLog field in the EnvoyProxy CRD’s telemetry section. Events are written as JSON to a configured file sink using Envoy’s event_logger mechanism and the HealthCheckEventFileSink extension.

Note: Health check event logging only applies to xRoutes that have active health checks configured via a BackendTrafficPolicy.

Prerequisites

Follow the steps below to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

Expand for instructions
  1. Install the Gateway API CRDs and Envoy Gateway using Helm:

    helm install eg oci://docker.io/envoyproxy/gateway-helm --version v0.0.0-latest -n envoy-gateway-system --create-namespace
    
  2. Install the GatewayClass, Gateway, HTTPRoute and example app:

    kubectl apply -f https://github.com/envoyproxy/gateway/releases/download/latest/quickstart.yaml -n default
    
  3. Verify Connectivity:

    Get the External IP of the Gateway:

    export GATEWAY_HOST=$(kubectl get gateway/eg -o jsonpath='{.status.addresses[0].value}')
       

    Curl the example app through Envoy proxy:

    curl --verbose --header "Host: www.example.com" http://$GATEWAY_HOST/get
       

    The above command should succeed with status code 200.

    Get the name of the Envoy service created the by the example Gateway:

    export ENVOY_SERVICE=$(kubectl get svc -n envoy-gateway-system --selector=gateway.envoyproxy.io/owning-gateway-namespace=default,gateway.envoyproxy.io/owning-gateway-name=eg -o jsonpath='{.items[0].metadata.name}')
       

    Get the deployment of the Envoy service created the by the example Gateway:

    export ENVOY_DEPLOYMENT=$(kubectl get deploy -n envoy-gateway-system --selector=gateway.envoyproxy.io/owning-gateway-namespace=default,gateway.envoyproxy.io/owning-gateway-name=eg -o jsonpath='{.items[0].metadata.name}')
       

    Port forward to the Envoy service:

    kubectl -n envoy-gateway-system port-forward service/${ENVOY_SERVICE} 8888:80 &
       

    Curl the example app through Envoy proxy:

    curl --verbose --header "Host: www.example.com" http://localhost:8888/get
       

    The above command should succeed with status code 200.

Configure Active Health Checks

Health check event logs require active health checks to be running. Configure a BackendTrafficPolicy targeting your HTTPRoute with an active health check. The example below polls /healthz every three seconds:

cat <<EOF | kubectl apply -f -
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
  name: backend-health-check
  namespace: default
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: backend-route
  healthCheck:
    active:
      type: HTTP
      http:
        path: /healthz
      interval: 3s
      timeout: 1s
      unhealthyThreshold: 3
      healthyThreshold: 1
EOF

Save and apply the following resource to your cluster:

---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
  name: backend-health-check
  namespace: default
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: backend-route
  healthCheck:
    active:
      type: HTTP
      http:
        path: /healthz
      interval: 3s
      timeout: 1s
      unhealthyThreshold: 3
      healthyThreshold: 1

Enable Health Check Event Logging

Configure health check event logging in the EnvoyProxy CRD. When no sinks are specified, events are written to /dev/stdout by default.

cat <<EOF | kubectl apply -f -
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
  parametersRef:
    group: gateway.envoyproxy.io
    kind: EnvoyProxy
    name: hc-event-logging
    namespace: envoy-gateway-system
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: hc-event-logging
  namespace: envoy-gateway-system
spec:
  telemetry:
    healthCheckLog: {}
EOF

Save and apply the following resources to your cluster:

---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
  parametersRef:
    group: gateway.envoyproxy.io
    kind: EnvoyProxy
    name: hc-event-logging
    namespace: envoy-gateway-system
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: hc-event-logging
  namespace: envoy-gateway-system
spec:
  telemetry:
    healthCheckLog: {}

To write events to a specific file instead, configure an explicit sink:

cat <<EOF | kubectl apply -f -
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: hc-event-logging-file
  namespace: envoy-gateway-system
spec:
  telemetry:
    healthCheckLog:
      sinks:
        - type: File
          file:
            path: /var/log/envoy/health-check-events.log
EOF

Save and apply the following resource to your cluster:

---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: hc-event-logging-file
  namespace: envoy-gateway-system
spec:
  telemetry:
    healthCheckLog:
      sinks:
        - type: File
          file:
            path: /var/log/envoy/health-check-events.log

Health check events will now appear in the Envoy proxy container’s standard output in JSON format, for example:

{
  "health_checker_type": "HTTP",
  "host": {
    "socket_address": { "protocol": "TCP", "address": "1.2.3.4", "port_value": 8080 }
  },
  "cluster_name": "default/backend-route/rule/0/match/0/backend-route",
  "timestamp": "2024-01-15T10:23:00.123Z",
  "health_check_failure_event": {
    "failure_type": "ACTIVE",
    "first_check": false
  }
}

Log All Events

When matches is omitted (the default), all health check probe outcomes are logged. To log only specific outcomes, set matches to one or more values; they are ORed together. At least one failure variant and one success variant must be specified together.

ValueLogged when
FailureEvery failed probe, regardless of current health state
FailureTransitionOnly when a host transitions from healthy → unhealthy
SuccessEvery successful probe, regardless of current health state
SuccessTransitionOnly when a host transitions from unhealthy → healthy

To log only on state transitions (the most conservative setting):

cat <<EOF | kubectl apply -f -
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: hc-event-logging-transitions
  namespace: envoy-gateway-system
spec:
  telemetry:
    healthCheckLog:
      matches:
        - FailureTransition
        - SuccessTransition
EOF

Save and apply the following resource to your cluster:

---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: hc-event-logging-transitions
  namespace: envoy-gateway-system
spec:
  telemetry:
    healthCheckLog:
      matches:
        - FailureTransition
        - SuccessTransition

To log every probe result regardless of outcome:

cat <<EOF | kubectl apply -f -
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: hc-event-logging-verbose
  namespace: envoy-gateway-system
spec:
  telemetry:
    healthCheckLog:
      matches:
        - Failure
        - Success
EOF

Save and apply the following resource to your cluster:

---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: hc-event-logging-verbose
  namespace: envoy-gateway-system
spec:
  telemetry:
    healthCheckLog:
      matches:
        - Failure
        - Success

Verify

Trigger a health check failure (e.g. by scaling the backend deployment to zero replicas) and confirm events appear in the proxy logs:

kubectl logs -l gateway.envoyproxy.io/owning-gateway-name=eg -n envoy-gateway-system -c envoy | grep health_checker_type