Dynamic A/B Testing with Envoy Wasm Filters in Istio: Zero‑Deploy Traffic Shaping

Modern microservice architectures demand rapid experimentation without incurring the cost of full redeploys or restarting pods. Traditional feature‑flag solutions often sit inside the application layer, requiring code changes, version bumps, and careful rollout coordination. By pushing the decision‑making layer into the data plane, we can alter traffic behavior instantly, safely, and uniformly across all services that share the same mesh. Istio’s EnvoyFilter resource enables exactly this: we can inject a WebAssembly (Wasm) module into Envoy’s filter chain, letting us inspect, mutate, or route HTTP requests based on arbitrary logic—without touching the application code.

Why Wasm in Envoy?

Envoy’s plugin system originally relied on C++ extensions, which required rebuilding the proxy and posed security concerns. The introduction of the Wasm runtime changed the game: plugins are sandboxed, language‑agnostic, and can be hot‑reloaded. Istio leverages this by allowing an EnvoyFilter to reference a Wasm image stored in an OCI registry or a ConfigMap. The Wasm module receives callbacks for each HTTP event (request headers, body, response headers, etc.) and can make decisions based on dynamic data—such as values fetched from a remote configuration service, Kubernetes labels, or even machine‑learning model scores.

For A/B testing, we typically need to:

1. Identify the experiment – via a header, cookie, or query parameter.
2. Assign a variant – using a consistent hashing algorithm (e.g., based on user ID) to guarantee stickiness.
3. Apply the variant – by injecting a custom header, rewriting the URL path, or mirroring traffic to a duplicate service.

All of these steps can be performed inside the Wasm filter, keeping the application oblivious to the experiment.

Building a Simple Wasm Filter for Header‑Based Variant Injection

Below is a minimal Rust‑based Wasm plugin that reads an experiment name from the x-experiment header, hashes the value of the user-id header (or falls back to a cookie), and injects a x-variant header with either A or B. The plugin is intentionally lightweight to illustrate the workflow; production versions would include proper error handling, metrics emission, and external configuration pulls.

// src/lib.rs
use proxy_wasm::traits::*;
use proxy_wasm::types::*;

proxy_wasm::main! {{
    proxy_wasm::set_http_context(|_, _| -> Box<dyn HttpContext> { Box::new(HttpHeaderInjector) });
}}

struct HttpHeaderInjector;

impl HttpContext for HttpHeaderInjector {
    fn on_http_request_headers(&mut self, _: usize, _: bool) -> Action {
        // Extract experiment name (optional) and user identifier
        let experiment = self
            .get_http_request_header("x-experiment")
            .unwrap_or_else(|| "default_experiment".to_string());
        let user_id = self
            .get_http_request_header("user-id")
            .or_else(|| self.get_http_request_header("cookie"))
            .unwrap_or_else(|| "anonymous".to_string());

        // Simple deterministic hash: sum of bytes modulo 2
        let hash: u8 = user_id.bytes().fold(0u8, |acc, b| acc.wrapping_add(b));
        let variant = if hash % 2 == 0 { "A" } else { "B" };

        // Inject the variant header
        self.set_http_request_header("x-variant", Some(variant));

        // Continue filter chain
        Action::Continue
    }
}

To compile this to Wasm:

# Assuming you have the Rust toolchain and the proxy-wasm crate installed
cargo build --target wasm32-unknown-unknown --release
# The output will be in target/wasm32-unknown-unknown/release/libhttp_header_injector.wasm

You can then push the resulting .wasm file to an OCI registry (e.g., GitHub Packages, Docker Hub) or store it in a ConfigMap for cluster‑local testing.

Deploying the Wasm Filter via Istio EnvoyFilter

Once the Wasm module is available, we create an EnvoyFilter that attaches it to the inbound listener of every workload in a specific namespace (or globally). The filter runs in the HTTP connection manager’s request phase, ensuring it sees the original request before any routing decisions.

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: ab-test-wasm-filter
  namespace: istio-system # or the namespace where your meshctl is installed
spec:
  workloadSelector:
    labels:
      app: productcatalog # apply only to a specific service; remove to affect all
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
        listener:
          filterChain:
            filter:
              name: envoy.filters.network.http_connection_manager
              subFilter:
                name: envoy.filters.http.router
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.wasm
          typed_config:
            '@type': type.googleapis.com/udpa.type.v1.TypedStruct
            type_url: type.googleapis.com/proxy_wasm.config.v1.PluginConfig
            value:
              vm_config:
                vm_id: 'ab_test_vm'
                runtime: 'envoy.wasm.runtime.v8'
                code:
                  local:
                    filename: '/etc/wasm/libhttp_header_injector.wasm'
              configuration:
                '@type': type.googleapis.com/google.protobuf.StringValue
                value: '' # optional JSON config passed to the plugin

Explanation of the filter:

• workloadSelector limits the injection to pods with app: productcatalog. Adjust or remove to target other services.
• applyTo: HTTP_FILTER tells Istio we are inserting an HTTP-level filter.
• match ensures we place the filter in the inbound sidecar, right before the router (envoy.filters.http.router). This ordering guarantees our header mutation occurs prior to load‑balancing decisions.
• value defines the Wasm plugin: we reference the compiled module via a local file path. In a production setup you would mount the Wasm file into the sidecar via a sidecar volume or use the oidc/oci image reference mechanism (image: field) to pull it from a registry.

After applying the EnvoyFilter (kubectl apply -f envoyfilter.yaml), Istio automatically reloads the affected sidecars. No pod restart is required; the new Wasm VM is spun up inside Envoy and begins processing traffic immediately.

Validating the Behavior

You can quickly verify that the header injection works with a simple curl test:

curl -v -H "x-experiment: checkout-flow" -H "user-id: alice123" http://productcatalog.default.svc.cluster.local/api/items

In the Istio proxy logs (accessible via istio-proxy container or using istioctl proxy-log), you should see a line similar to:

[2025-03-05T10:12:34.567Z] "GET /api/items HTTP/1.1" 200 - "-" "-" 0 0 123 123 "10.0.0.5" "x-experiment: checkout-flow, user-id: alice123, x-variant: A"

Notice the newly added x-variant: A header. Repeating the request with a different user-id that yields an odd hash will produce x-variant: B.

Extending the Pattern

The basic header‑injection example is a foundation for more sophisticated scenarios:

• Dynamic experiment configuration – store experiment rules (traffic percentages, targeting rules) in a ConfigMap or external service; the Wasm module can fetch them via HTTP calls during initialization.
• Response manipulation – inject alternative payloads, swap JSON fields, or inject feature‑flag scripts directly into HTML responses.
• Traffic mirroring – based on the variant, emit a duplicate request to a shadow service using x-envoy-immediate-host or the Istio mirror filter.
• Observability – emit custom metrics via proxy_wasm::metrics::counter to track variant distribution, enabling real‑time dashboards in Prometheus/Grafana.

Because the Wasm module runs inside the Envoy sandbox, any crash or excessive CPU usage is isolated and will not bring down the sidecar. Istio also provides built‑in metrics (istio_proxy_wasm_vm_*) to monitor plugin health.

Conclusion

By leveraging Istio’s EnvoyFilter and the WebAssembly runtime, we can shift experiment logic from the application layer to the network layer, achieving true zero‑deploy A/B testing. This approach reduces release friction, ensures consistent experiment enforcement across polyglot services, and opens the door to advanced traffic shaping—all while maintaining the safety guarantees of a sandboxed Wasm VM. For teams already invested in a service mesh, adopting Wasm‑based filters is a natural evolution toward more dynamic, observable, and resilient microservice architectures.

Give it a try: compile a simple Wasm plugin, inject it via an EnvoyFilter, and watch your experiments flip flags without touching a single line of service code. Happy traffic shaping!