This is the "latest" release of Envoy Gateway, which contains the most recent commits from the main branch.
This release might not be stable.
Please refer to the /docs documentation for the most current information.
Proxy Health Check Logs
5 minute read
Envoy Gateway can log health check events for upstream clusters using the healthCheckLog
field in the EnvoyProxy CRD’s telemetry section. Events are written as JSON to a
configured file sink using Envoy’s
event_logger
mechanism and the
HealthCheckEventFileSink
extension.
Note: Health check event logging only applies to xRoutes that have active health checks configured via a BackendTrafficPolicy.
Prerequisites
Follow the steps below to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.
Expand for instructions
Install the Gateway API CRDs and Envoy Gateway using Helm:
Gateway API CRD compatibilityThis command installs Gateway API CRDs. If your Kubernetes provider already manages compatible Gateway API CRDs for the cluster, use the provider-managed Gateway API CRD install steps instead.
helm install eg oci://docker.io/envoyproxy/gateway-helm --version v0.0.0-latest -n envoy-gateway-system --create-namespaceInstall the GatewayClass, Gateway, HTTPRoute and example app:
kubectl apply -f https://github.com/envoyproxy/gateway/releases/download/latest/quickstart.yaml -n defaultVerify Connectivity:
Get the External IP of the Gateway:
export GATEWAY_HOST=$(kubectl get gateway/eg -o jsonpath='{.status.addresses[0].value}')Curl the example app through Envoy proxy:
curl --verbose --header "Host: www.example.com" http://$GATEWAY_HOST/getThe above command should succeed with status code 200.
Get the name of the Envoy service created the by the example Gateway:
export ENVOY_SERVICE=$(kubectl get svc -n envoy-gateway-system --selector=gateway.envoyproxy.io/owning-gateway-namespace=default,gateway.envoyproxy.io/owning-gateway-name=eg -o jsonpath='{.items[0].metadata.name}')Get the deployment of the Envoy service created the by the example Gateway:
export ENVOY_DEPLOYMENT=$(kubectl get deploy -n envoy-gateway-system --selector=gateway.envoyproxy.io/owning-gateway-namespace=default,gateway.envoyproxy.io/owning-gateway-name=eg -o jsonpath='{.items[0].metadata.name}')Port forward to the Envoy service:
kubectl -n envoy-gateway-system port-forward service/${ENVOY_SERVICE} 8888:80 &Curl the example app through Envoy proxy:
curl --verbose --header "Host: www.example.com" http://localhost:8888/getThe above command should succeed with status code 200.
Configure Active Health Checks
Health check event logs require active health checks to be running.
Configure a BackendTrafficPolicy targeting your HTTPRoute with an active health
check. The example below polls /healthz every three seconds:
cat <<EOF | kubectl apply -f -
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
name: backend-health-check
namespace: default
spec:
targetRefs:
- group: gateway.networking.k8s.io
kind: HTTPRoute
name: backend-route
healthCheck:
active:
type: HTTP
http:
path: /healthz
interval: 3s
timeout: 1s
unhealthyThreshold: 3
healthyThreshold: 1
EOF
Save and apply the following resource to your cluster:
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
name: backend-health-check
namespace: default
spec:
targetRefs:
- group: gateway.networking.k8s.io
kind: HTTPRoute
name: backend-route
healthCheck:
active:
type: HTTP
http:
path: /healthz
interval: 3s
timeout: 1s
unhealthyThreshold: 3
healthyThreshold: 1
Enable Health Check Event Logging
Configure health check event logging in the EnvoyProxy CRD.
When no sinks are specified, events are written to /dev/stdout by default.
cat <<EOF | kubectl apply -f -
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: eg
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
parametersRef:
group: gateway.envoyproxy.io
kind: EnvoyProxy
name: hc-event-logging
namespace: envoy-gateway-system
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
name: hc-event-logging
namespace: envoy-gateway-system
spec:
telemetry:
healthCheckLog: {}
EOF
Save and apply the following resources to your cluster:
---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: eg
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
parametersRef:
group: gateway.envoyproxy.io
kind: EnvoyProxy
name: hc-event-logging
namespace: envoy-gateway-system
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
name: hc-event-logging
namespace: envoy-gateway-system
spec:
telemetry:
healthCheckLog: {}
To write events to a specific file instead, configure an explicit sink:
cat <<EOF | kubectl apply -f -
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
name: hc-event-logging-file
namespace: envoy-gateway-system
spec:
telemetry:
healthCheckLog:
sinks:
- type: File
file:
path: /var/log/envoy/health-check-events.log
EOF
Save and apply the following resource to your cluster:
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
name: hc-event-logging-file
namespace: envoy-gateway-system
spec:
telemetry:
healthCheckLog:
sinks:
- type: File
file:
path: /var/log/envoy/health-check-events.log
Health check events will now appear in the Envoy proxy container’s standard output in JSON format, for example:
{
"health_checker_type": "HTTP",
"host": {
"socket_address": { "protocol": "TCP", "address": "1.2.3.4", "port_value": 8080 }
},
"cluster_name": "default/backend-route/rule/0/match/0/backend-route",
"timestamp": "2024-01-15T10:23:00.123Z",
"health_check_failure_event": {
"failure_type": "ACTIVE",
"first_check": false
}
}
Log All Events
When matches is omitted (the default), all health check probe outcomes are
logged. To log only specific outcomes, set matches to one or more values;
they are ORed together. At least one failure variant and one success variant must
be specified together.
| Value | Logged when |
|---|---|
Failure | Every failed probe, regardless of current health state |
FailureTransition | Only when a host transitions from healthy → unhealthy |
Success | Every successful probe, regardless of current health state |
SuccessTransition | Only when a host transitions from unhealthy → healthy |
To log only on state transitions (the most conservative setting):
cat <<EOF | kubectl apply -f -
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
name: hc-event-logging-transitions
namespace: envoy-gateway-system
spec:
telemetry:
healthCheckLog:
matches:
- FailureTransition
- SuccessTransition
EOF
Save and apply the following resource to your cluster:
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
name: hc-event-logging-transitions
namespace: envoy-gateway-system
spec:
telemetry:
healthCheckLog:
matches:
- FailureTransition
- SuccessTransition
To log every probe result regardless of outcome:
cat <<EOF | kubectl apply -f -
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
name: hc-event-logging-verbose
namespace: envoy-gateway-system
spec:
telemetry:
healthCheckLog:
matches:
- Failure
- Success
EOF
Save and apply the following resource to your cluster:
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
name: hc-event-logging-verbose
namespace: envoy-gateway-system
spec:
telemetry:
healthCheckLog:
matches:
- Failure
- Success
Verify
Trigger a health check failure (e.g. by scaling the backend deployment to zero replicas) and confirm events appear in the proxy logs:
kubectl logs -l gateway.envoyproxy.io/owning-gateway-name=eg -n envoy-gateway-system -c envoy | grep health_checker_type
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.