For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Monitor and troubleshoot proxy configuration rejections with metrics and events.

The agentgateway proxy validates XDS updates from the control plane before applying them. A rejected resource produces a NACK (negative acknowledgement).

The proxy retains its last known-good configuration for that resource. Traffic continues, but the rejected change is not applied.

The control plane counts rejections in agentgateway_xds_rejects_total on port 9092. The control plane also creates a Kubernetes Warning event on the affected Gateway.

Rejections can occur when the proxy cannot parse a value accepted by the Kubernetes API server and control plane. One example is an invalid agentRuntimeArn in an AgentgatewayBackend.

If only part of a resource is unusable, the proxy can apply the rest and report a configuration warning. Warnings use a separate counter and event reason. See View configuration warnings.

View the rejection metric

1. Port-forward the control plane deployment.

   kubectl port-forward -n agentgateway-system deployment/agentgateway 9092

2. Query the metrics endpoint and search for the rejection counter.

   curl http://localhost:9092/metrics | grep xds_rejects

   Example output when rejections have occurred:

   # HELP agentgateway_xds_rejects_total Total number of xDS responses rejected by agentgateway proxy
   # TYPE agentgateway_xds_rejects_total counter
   agentgateway_xds_rejects_total 3
   

You can use this metric to configure alerts in Prometheus that fire when rejections occur. For guidance on setting up the observability stack, see the OTel stack guide.

View rejection events

A rejection creates a Kubernetes Warning event on the Gateway with reason AgentGatewayNackError. The message is a JSON list with these fields:

• key: The rejected resource.
• error: The error from the proxy.

The resource status does not show the rejection. For example, an AgentgatewayBackend with an invalid agentRuntimeArn still reports Accepted: True.

1. List rejection events in the namespace where your Gateway is deployed.

   kubectl get events -n agentgateway-system --field-selector=reason=AgentGatewayNackError

   Example output:

   LAST SEEN   TYPE      REASON                  OBJECT                       MESSAGE
   8s          Warning   AgentGatewayNackError   gateway/agentgateway-proxy   [{"key":"backend/agentgateway-system/agentcore","error":"error: invalid AgentCore ARN: not-an-arn"}]
   

2. View events on the Gateway resource directly.

   kubectl describe gateway agentgateway-proxy -n agentgateway-system

   Example output:

   Events:
     Type     Reason                 Age   From                           Message
     ----     ------                 ----  ----                           -------
     Warning  AgentGatewayNackError  13s   agentgateway.dev/agentgateway  [{"key":"backend/agentgateway-system/agentcore","error":"error: invalid AgentCore ARN: not-an-arn"}]
   

View configuration warnings

When the proxy can apply a resource after ignoring or replacing an invalid part, it reports a configuration warning. The control plane records warnings separately:

• Counter: agentgateway_xds_warnings_total.
• Gateway event: A Kubernetes Warning event with reason AgentGatewayConfigWarning. The message is a JSON list with these fields:
  • key: The resource with the warning.
  • warn: The warning from the proxy.

Warnings do not increase agentgateway_xds_rejects_total.

Common causes of warnings:

• CEL expressions that pass the Go-based control plane validator but fail the Rust-based proxy parser, such as calling has() with an invalid argument. The proxy replaces the expression with one that always fails.
• Invalid regular expressions in route paths or prompt guards. Invalid path patterns never match. The proxy skips invalid prompt guard patterns.
• Invalid TLS certificates that the proxy cannot load.

Treat each warning as a configuration error. Traffic might behave differently from your configuration.

1. Port-forward the control plane deployment.

   kubectl port-forward -n agentgateway-system deployment/agentgateway 9092

2. Query the metrics endpoint and search for the warning counter. Like the rejection counter, agentgateway_xds_warnings_total appears only after the first warning.

   curl -s http://localhost:9092/metrics | grep xds_warnings

   Example output:

   # HELP agentgateway_xds_warnings_total Total number of xDS responses with configuration warnings from agentgateway proxy
   # TYPE agentgateway_xds_warnings_total counter
   agentgateway_xds_warnings_total 2
   

3. List warning events in the Gateway namespace.

   kubectl get events -n agentgateway-system --field-selector=reason=AgentGatewayConfigWarning

   Example output:

   LAST SEEN   TYPE      REASON                      OBJECT                       MESSAGE
   8s          Warning   AgentGatewayConfigWarning   gateway/agentgateway-proxy   [{"key":"policy/traffic/agentgateway-system/priority-header:transformation:agentgateway-system/httpbin","warn":"invalid CEL expression for transformation: parse: ERROR: <input>:1:20: invalid argument to has() macro\n| has(request.headers['x-priority-level']) ? 'level_' + request.headers['x-priority-level'] : 'level_unknown'\n| ...................^; replacing \"has(request.headers['x-priority-level']) ? 'level_' + request.headers['x-priority-level'] : 'level_unknown'\" with an expression that always fails"}]