.. _version_history_1.38.0: 1.38.0 (Pending) ================= Incompatible behavior changes ----------------------------- *Changes that are expected to cause an incompatibility if applicable; deployment changes are likely required* * **boringssl/fips**: The previous flag for specifying FIPS builds (ie ``--define=boringssl=fips``) will no longer work and has been replaced by ``--config=boringssl-fips``. This change will allow us to better support custom SSL libraries, and will allow FIPS-compliant Envoy to be built with the imminent switch to Bazel bzlmod. * **on_demand**: The on_demand filter, when doing On Demand CDS, will no longer do internal redirects after CDS is successful. Filters appearing in the filter chain before the On Demand Filter will no longer be invoked twice. This behavior can be temporarily reverted by setting the runtime guard ``envoy.reloadable_features.on_demand_cluster_no_recreate_stream`` to ``false``. * **tcp_proxy**: The TCP proxy filter now requires :ref:`max_early_data_bytes ` to be explicitly set when using :ref:`upstream_connect_mode ` modes other than ``IMMEDIATE`` (i.e., ``ON_DOWNSTREAM_DATA`` or ``ON_DOWNSTREAM_TLS_HANDSHAKE``). The field can be set to ``0`` to disable early data buffering while still using delayed connection modes. Configurations using these modes without ``max_early_data_bytes`` will now fail validation at startup. Minor behavior changes ---------------------- *Changes that may cause incompatibilities for some users, but should not for most* * **ext_authz**: Changed the behavior of ``timeout: 0s`` in the HTTP ext_authz filter to mean "no timeout" (infinite) instead of immediate timeout. Previously, ``timeout: 0s`` would cause requests to fail immediately. This aligns with other Envoy timeout configurations where ``0`` means disabled or infinite. * **histograms**: Updated libcircllhist to 0.3.2, which changes how bucket bounds are interpreted. This should not impact production monitoring if the number of samples in the histograms is high. Affected tests were adjusted to account for histogram changes. * **mcp**: Changed the default metadata namespace for the MCP filter from ``mcp_proxy`` to ``envoy.filters.http.mcp``. This change can be reverted by setting the runtime guard ``envoy.reloadable_features.mcp_filter_use_new_metadata_namespace`` to ``false``. * **ratelimit**: Changed the behavior of ``timeout: 0s`` in the HTTP rate limit filter to mean "no timeout" (infinite) instead of immediate timeout. Previously, ``timeout: 0s`` would cause requests to fail immediately. This aligns with other Envoy timeout configurations where ``0`` means disabled or infinite. * **stat_sinks**: OpenTelemetry :ref:`SinkConfig ` stopped reporting empty delta counters and histograms. Bug fixes --------- *Changes expected to improve the state of the world and are unlikely to have negative effects* * **drop_overload**: Fixed a bug where ``drop_overload`` failed to use cached EDS resources. * **ext_authz**: Fixed a bug where headers from a denied authorization response (non-200) were not properly propagated to the client. * **ext_authz**: Fixed the HTTP ext_authz client to respect ``status_on_error`` configuration when the authorization server returns a 5xx error or when HTTP call failures occur. Previously, these error scenarios always returned 403 Forbidden regardless of the configured error status. * **formatter**: Fixed the log formatter in HTTP router upstream logs by correctly setting the downstream connection's ``ConnectionInfoProvider`` in the ``StreamInfo``. * **http**: Fixed a potential file descriptor leak where HTTP/1.1 connections with zombie streams (waiting for codec completion) would not be properly closed when in draining state. This could occur when a response was sent before the request was fully received, causing connections to remain open indefinitely. This change can be reverted by setting the runtime guard ``envoy.reloadable_features.http1_close_connection_on_zombie_stream_complete`` to ``false``. * **http**: Fixed upstream client to not close connection when idle timeout fires before the connection is established. This change can be reverted by setting the runtime guard ``envoy.reloadable_features.codec_client_enable_idle_timer_only_when_connected`` to ``false``. * **load_report**: Fixed an issue upon load-report shutdown race with ADS stream. Introduced proper cleanup of the gRPC stream. * **oauth2**: Fixed OAuth2 refresh requests so host rewriting no longer overrides the original ``Host`` header value. * **odcds**: Fixed a crash (SIGABRT) when destroying OdCDS handles on worker threads. The handle no longer holds a direct reference to the subscription, preventing thread-safety issues during destruction. The subscription now persists in ClusterManagerImpl and is looked up by a config source key. * **overload_manager**: Fixed a resource leak in global connection limit tracking that caused permanent connection rejections when using load shedding (e.g., ``envoy.load_shed_points.tcp_listener_accept``). When connections were rejected due to load shedding after passing the global connection limit check, the allocated connection limit resource was not released, causing the connection counter to become incorrect and leading to ``failed_updates`` in the resource monitor. This resulted in permanent connection rejections even after load subsided. The fix ensures that connection limit resources are properly released when connections are rejected due to load shedding. Also added defensive resource cleanup for edge cases where address processing fails (e.g., ``localAddress()`` or ``peerAddress()`` errors). * **release**: Published contrib binaries now include the ``-contrib`` suffix in their version string. * **spiffe**: Reduce the number of the file watches needed by :ref:`trust_bundles ` when the validator is used in multiple places. Add support for :ref:`watched_directory ` to support Kubernetes environments that rely on atomic symbolic file updates. Add content hashing during file watching to handle excessive watch notifications. * **tls**: Fix on-demand TLS selector to enforce session resumption settings. New features ------------ * **admin**: Added ``filter`` query parameter support to the ``/clusters`` endpoint. The parameter accepts a RE2 regular expression to filter clusters by name. Compatible with the ``format`` parameter for both text and JSON output (e.g., ``/clusters?filter=service&format=json``). * **config**: Added support for :ref:`set_node_on_first_message_only ` to Delta-xDS. Guarded by runtime flag ``envoy.reloadable_features.xds_legacy_delta_skip_subsequent_node``. * **dynamic modules**: Added support for dynamic modules authors to register any combination of http, network, listener, udp listener, and bootstrap filters in the rust SDK. * **dynamic modules**: Introduced the extended ABI forward compatibility mechanism for dynamic modules where modules built with a SDK version can be loaded by Envoy binaries of the next Envoy version. For example, A module built with the v1.38 SDK can now be loaded by an Envoy binary of v1.39. * **dynamic_modules**: Added :ref:`load balancing policies ` support for dynamic modules, enabling custom load balancing algorithms to be implemented in dynamic modules. * **dynamic_modules**: Added a process-wide function registry to the dynamic modules ABI. Modules can register functions by name via ``envoy_dynamic_module_callback_register_function`` and other modules can resolve them via ``envoy_dynamic_module_callback_get_function``, enabling zero-copy cross-module interactions analogous to ``dlsym``. * **dynamic_modules**: Added configurable :ref:`metrics_namespace ` field to ``DynamicModuleConfig``. This allows users to customize the prefix used for all metrics created by dynamic modules. Metrics now appear with the standard ``envoy_`` prefix followed by the namespace in prometheus output (e.g. ``envoy_myapp_requests_total``). The legacy behavior (stripping the namespace prefix from prometheus output) can be restored by setting the runtime guard ``envoy.reloadable_features.dynamic_modules_strip_custom_stat_prefix`` to ``true``. * **ext_authz**: Added support for the ``append_action`` enum in gRPC ext_authz ``OkHttpResponse.headers`` for upstream request header mutations. Previously, only the deprecated ``append`` boolean was checked. Now ``APPEND_IF_EXISTS_OR_ADD``, ``ADD_IF_ABSENT``, ``OVERWRITE_IF_EXISTS``, and ``OVERWRITE_IF_EXISTS_OR_ADD`` actions are fully supported, providing parity with ``response_headers_to_add`` handling. * **formatter**: Added ``%UPSTREAM_DETECTED_CLOSE_TYPE%`` and ``%DOWNSTREAM_DETECTED_CLOSE_TYPE%`` to expose the detected close type of downstream and upstream connections. The possible values are ``Normal``, ``LocalReset``, and ``RemoteReset``. * **formatter**: Added ``QUERY_PARAMS`` support for substitution formatter to log all query params. They can either be logged in their original form or decoded. * **formatter**: Added the new access log formatter ``DOWNSTREAM_LOCAL_CLOSE_REASON``. * **formatter**: Extended ``*_WITHOUT_PORT`` address formatters to accept an optional ``MASK_PREFIX_LEN`` parameter that masks IP addresses and returns them in CIDR notation (e.g., ``%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT(16)%`` returns ``10.1.0.0/16`` for client IP ``10.1.10.23``). * **mcp_router**: Added SSE (Server-Sent Events) streaming support for MCP backend responses. The router now handles SSE responses from backends for ``tools/call`` with direct pass-through streaming, and supports SSE aggregation for fanout operations (``tools/list``, ``initialize``) with incremental event parsing. * **mcp_router**: Added SSE response support for MCP ``resources/list`` fanout aggregation. * **mcp_router**: Added support for MCP client-to-server notification methods ``notifications/cancelled`` and ``notifications/roots/list_changed``. * **mcp_router**: Added support for MCP completion method ``completion/complete`` with routing based on ``ref/prompt`` or ``ref/resource``. * **mcp_router**: Added support for MCP logging method ``logging/setLevel``. * **mcp_router**: Added support for MCP prompt methods ``prompts/list`` and ``prompts/get``. * **mcp_router**: Added support for MCP resource methods ``resources/list``, ``resources/read``, ``resources/subscribe``, and ``resources/unsubscribe``. * **network**: Added support access logging in network filters like http filters, by allowing network filters to register as access logger instances. * **ratelimit**: Added ``RemoteAddressMatch`` action to the rate limit filter. This action will generate a descriptor based on the remote address of the downstream connection by matching it against specified CIDR ranges with support for inversion and formatter substitution. * **ratelimit**: Added per-descriptor ``x-ratelimit-*`` headers support. See the :ref:`x_ratelimit_option ` field documentation for more details. * **ratelimit**: Added support for shadow mode in the local rate limit filter. * **resource_monitors**: Added cgroup v2 support to the CPU utilization resource monitor. The monitor now automatically detects and selects between cgroup v1 and v2 at runtime by checking available cgroup files on the system. This enables the resource monitor to work correctly in both cgroup v1 and v2 environments without configuration changes. * **sse_parser**: Extended the SSE parser utility to support all standard SSE fields: ``id``, ``event`` (as ``event_type``), and ``retry``, in addition to the existing ``data`` field. The ``retry`` field is parsed as a ``uint32_t`` and only accepts values consisting of ASCII digits per the SSE specification. * **stat_sinks**: Added support for exporting OpenTelemetry metrics via HTTP. The new ``http_service`` field in :ref:`SinkConfig ` enables direct OTLP metrics export to backends that only accept HTTP (Dynatrace, Datadog, Elastic), without requiring an intermediate collector sidecar. * **stats**: Added support to limit the number of stats stored in each stats scope in the stats library. * **stats**: The admin prometheus stats endpoint now supports the protobuf exposition format, and will automatically use it if the request contains the correct Accept header, or if query parameter ``prom_protobuf=1`` is set. In a prometheus scrape configuration, add ``PrometheusProto`` to ``scrape_protocols`` to use the protobuf format. Additionally, when using the protobuf exposition format, the admin prometheus stats endpoint now supports `native histograms `_ when using the prometheus protobuf exposition format, using query ``/stats/prometheus?histogram_buckets=prometheusnative``. * **tcp_proxy**: Added an option to emit a log entry when the connection is accepted. * **tls**: Added support for fetching certificates on-demand via SDS in the upstream TLS transport socket using the extension :ref:`on-demand certificate selector `. * **tracers**: Add log events to spans created by OpenTelemetry tracer.