Record & Replay Real Traffic

The workflow at a glance

Mode is persisted in data/mode.txt and watched at runtime, so a mode change propagates to the proxy without a restart. There are four modes total: LEARN, MOCK, PASSTHROUGH (forward without recording), and TRANSITIONING (the brief interstitial during a LEARN→MOCK transition).

Step 1 — point the proxy at an upstream

The proxy starts in LEARN mode by default. Set BACKEND_URL to the upstream you want to learn from — a local dev server, a staging environment, or a third-party API. Then point your application at the proxy instead of the upstream:

# Plain HTTP — point your app at the proxy on :8080
export API_BASE_URL=http://localhost:8080

# Your app's requests are forwarded to BACKEND_URL and recorded

If your app speaks HTTPS, enable the built-in TLS-MITM listener on :8443 by setting ENABLE_TLS_INTERCEPTION=true, point your client at it with HTTPS_PROXY=http://<agent>:8443, and trust the MITM CA once:

# Fetch and trust the MITM CA (served on :8080; 503 while interception is off)
curl http://localhost:8080/ca.crt > gostly-ca.crt
# then add gostly-ca.crt to your client / OS trust store

See Proxy Setup for the per-OS trust-install steps and the Caddy-fronted TLS alternative.

Step 2 — record in LEARN mode

In LEARN mode the proxy is a transparent pass-through. Every inbound request is forwarded to the upstream; the response is returned to the caller and recorded. Generate traffic the way you normally would — run your test suite, click through the app, or replay a capture. There is no minimum; even a handful of interactions is enough to start replacing upstream calls.

Recording fans out to three independent sinks, each with its own redaction posture:

You can check what's been captured from the dashboard at localhost:3000 or via the control-plane API on port 8000.

Step 3 — transition to build the library

When you trigger a transition, the API reads the recorded traffic, runs it through the scrub pipeline, and persists the result. During the write the proxy enters TRANSITIONING and returns 503 Service Unavailable with a Retry-After header — this is intentional, and prevents partial-library matches mid-write. Once the API signals completion, the proxy reloads the per-service mock library and is ready to serve.

Step 4 — flip to MOCK and replay

Switch to MOCK mode from the dashboard or via the API. The shipped compose runs the control plane fail-closed, so every call on port 8000 carries the X-API-Key header (the value of GHOST_API_KEY from your .env):

# Flip the global mode to MOCK
curl -X POST http://localhost:8000/v1/mode \
  -H "X-API-Key: $GHOST_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"mode": "MOCK"}'

Your app keeps talking to http://localhost:8080. Each inbound request now runs through the match cascade below. The order is fixed: cheaper, more-deterministic tiers run first, and AI only runs when nothing earlier matched.

¹ The statechart engine fires on every tier from bundled fixtures (charge, customer, invoice, order, subscription). The graph editor and per-tenant overrides are Pro+.

Per-service modes

A single proxy instance handles many upstreams at once, and each service carries its own mock library, mode, chaos config, and redaction rules. That means one service can stay in LEARN while another already replays in MOCK — useful when you're recording a new dependency without disturbing the ones you've already locked down:

# Switch one service to MOCK while the others keep recording
curl -X PUT http://localhost:8000/services/{service_id} \
  -H "X-API-Key: $GHOST_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"mode": "MOCK"}'

# Or switch every service at once with the global mode endpoint
curl -X POST http://localhost:8000/v1/mode \
  -H "X-API-Key: $GHOST_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"mode": "MOCK"}'

See Multiple upstream services for host-based and path-based routing. Each deployment is single-tenant (the tenant_id defaults to default); per-tenant row-level-security policies ship as defense-in-depth, but isolation in the default configuration comes from single-tenancy itself.

Seeding without live traffic

If you can't (or don't want to) record live traffic first, seed the library from artifacts you already have. Drag a HAR export, Postman collection, or OpenAPI spec onto the dashboard, which posts it to the corresponding importer:

# Cold-start import — one endpoint per format
curl -X POST 'http://localhost:8000/v1/seed/har?service_id=svc-payments' \
  -H "X-API-Key: $GHOST_API_KEY" \
  -H 'Content-Type: application/json' \
  --data-binary @recording.har

# Also: /v1/seed/postman and /v1/seed/openapi

A seeded service replays through the same cascade as a recorded one — you can start in MOCK with a seeded library and add live recordings later.

Watching for drift

A replayed library is only as good as it is current. When recorded responses diverge from what the live upstream now returns, Gostly emits drift events and surfaces a freshness score (0–100) with a sparkline trend per service, so you can see at a glance which mocks are going stale.

On Pro+ tiers, the AI mock-repair surface can turn a drift event into a proposed fix that an operator approves or rejects. The auto-proposer loop is off by default (enable with ENABLE_AI_MOCK_REPAIR=true); manual approve / reject works either way — Gostly never rewrites a mock without an operator in the loop.

Observing what the proxy did

Every served request increments a Prometheus counter labelled by how it was answered, so you can verify the cascade is doing what you expect. The agent exposes these on /metrics:

ghost_requests_total{match_type="..."}   # one of: session_verbatim, exact,
                                          #   resource_store, resource_transition,
                                          #   smart_swap, generated_cached, miss,
                                          #   learn, transitioning, passthrough, chaos
ghost_mock_library_size                   # mocks currently loaded
ghost_io_errors_total{operation="..."}    # disk-sink failures
axum_http_requests_total                  # HTTP rate
axum_http_requests_duration_seconds       # HTTP latency
# gostly_tls_* family covers the TLS-MITM subsystem

Per response, the X-Ghost-Mock and X-Ghost-Transition headers tell you which tier answered an individual request — handy when debugging why a particular call matched the way it did.

Licensed product vs. the OSS proxy

The workflow above describes the licensed Gostly product, which ships as docker-compose plus container images — there is no host CLI to install; you drive everything through the dashboard and the control-plane API. The OSS proxy is a separate product distributed via Homebrew and a container registry, and it does ship a host CLI. The two share the record-and-replay concept; the tiered AI cascade, drift, statechart overrides, and the team features described here belong to the licensed product.