ADR 023 — Config Storage: Env Vars, Airflow Variables, Airflow Connections¶

Date: 2026-05-07 Status: Accepted Deciders: Data Engineering

Context¶

Today, every piece of configuration in our Airflow deployment — credentials, API base URLs, S3 bucket names, environment flags, debug toggles — is stored as an Airflow Variable. This conflates four distinct kinds of config into a single bucket and creates concrete problems:

1. Security: Airflow Variables are stored as plain text in the metadata Postgres database. The UI masks values whose names match certain patterns (*_password, *_secret, *_api_key), but the underlying storage is unencrypted. Astronomer environment Secrets are encrypted at rest with a separate key.
2. Performance: Variable.get() issues a SELECT query against the metadata DB on every call. For values that never change at runtime (S3 bucket, environment name, base URLs), this is unnecessary overhead.
3. Signal-to-noise: When everything lives in Airflow Variables, the Variables UI becomes a junk drawer. After cleanup, anything in Variables is — by definition — meant to be flipped at runtime, which is operationally useful.
4. Auditability: Variable changes via UI leave a thin audit trail. Astronomer environment variables and Connections defined via deploy config go through the same review/deploy gate as code.

We need a clear rule for where each kind of config belongs.

Decision¶

Use the following rule to choose where a config value lives:

If it's toggled at runtime via the UI → Airflow Variable. If it's a connection with host + auth + port shape → Airflow Connection. Otherwise → Environment variable (Astronomer Secret if it's a credential, regular env var if it's static infrastructure config).

Storage matrix¶

Why each tier exists¶

• Env Secrets are encrypted at rest, never visible in plain text in the UI, and never written to logs. They are the correct home for any credential.
• Env vars (regular) are deploy-time-static infrastructure config. No DB roundtrip per task, and the value is locked to the deployment.
• Airflow Connections exist specifically for the host+auth+port pattern. Hooks and operators expect them. Snowflake, Postgres, S3, MongoDB — these belong here.
• Airflow Variables are the only mechanism that lets ops flip a value via UI without a redeploy. Reserve them for that purpose only.

Classification of Current Variables¶

This is the canonical mapping for the migration tracked in #1014. Values are migrated from Variable.get() to the destination listed below.

→ Astronomer env var (Secret) — credentials & tokens¶

WARBYPARKER_GLADLY_API_TOKEN, WARBYPARKER_GLADLY_EMAIL
CONDENAST_GLADLY_API_TOKEN, CONDENAST_GLADLY_EMAIL
CONDENAST_MEDALLIA_API_KEY, CONDENAST_MEDALLIA_API_TOKEN
CONDENAST_SPROUT_API_KEY, CONDENAST_SPROUT_CLIENT_ID
DATA_ENGINEERING_SERVICE_ACCOUNT
FLARE_VONAGE_CLIENT_ID, FLARE_VONAGE_SECRET
XOOS_API_TOKEN, XOOS_WBP_CLIENT_ID
bamboohr_api_key

→ Astronomer env var (regular) — static infrastructure¶

XO_ENVIRONMENT
S3_BASE_BUCKET
SNOWFLAKE_STAGE_NAME
WARBYPARKER_GLADLY_BASE_URL
CONDENAST_GLADLY_BASE_URL
XOOS_API_BASE_URL
XOOS_WBP_API_BASE_URL

→ Airflow Connection — host+auth shape¶

sf_account / sf_user / sf_pass / sf_database     →  legacy OPERATIONS Snowflake connection
mongo_host / mongo_user / mongo_pass / mongo_db  →  mongo_default connection

The dbt pipelines already use the snowflake_dbt Connection. The legacy sf_* Variables predate that convention and should consolidate to a Connection.

Stays as Airflow Variable — actual runtime tunables¶

XO_DEBUG_LOGGING

This is currently the only value in the deployment that is legitimately meant to be flipped at runtime without a redeploy.

Nuance: HTTP API credentials (Gladly, Medallia, Sprout)¶

These have a host + auth shape and could be modeled as Airflow Connections. We chose env vars instead because:

• Our extractor code does not use Airflow hooks — it builds requests with httpx/requests directly, so a Connection object provides no ergonomics gain
• Env var access is faster (no DB call) and simpler
• The day we wrap an HTTP client in a Hook, we revisit and migrate

Treating these as env Secrets is the correct default until that day.

Consequences¶

• Security improves: Credentials move from plain-text Postgres rows to encrypted Astronomer Secrets. The GCP service account JSON is the highest-priority migration target.
• Performance improves: Tasks that previously made N metadata DB calls per run for static values now do N dict lookups in process memory.
• The Airflow Variables page becomes meaningful: anything you see there is a runtime knob.
• Local dev gets a clear contract: .env.example documents required env vars. Runtime tunables stay in airflow_settings.yaml.
• A dependency on Astronomer's Secrets feature: We accept this — Astronomer is our deployment target and any equivalent platform offers the same primitive.

Options Considered¶

Related¶

• Issue #1014 — migration tracking
• ADR 005 — three-tier stage configuration priority