Safe Sequencing for Control‑Plane and Data‑Plane Changes with Argo CD and Argo Rollouts

The core rule: expand, then contract

The recommended lifecycle is an expand‑and‑contract sequence. First, add new capabilities to the control plane without removing behavior required by the existing data plane. This creates a compatibility window where both old and new workload versions can run. Only after workloads have migrated and coexistence is proven should you remove legacy platform behaviors.

How Argo CD fits: establish prerequisites deterministically

• Waves are sequencing gates, not transactions. They prevent advancement until an earlier wave is synchronized and healthy, but they do not make incompatible schema changes atomic.
• For custom controllers that expose readiness through status, Argo CD supports custom Lua health checks so readiness can reflect real reconciliation instead of mere object creation.
• Hooks: PreSync hooks block the sync if they fail and PostSync hooks can run smoke tests after resources reach healthy sync state. Avoid selective sync for safety‑critical flows because hooks don't run during selective sync and selective sync isn't recorded in history.
• Pruning: Treat automatic deletion conservatively during control‑plane evolution. Argo CD supports disabling pruning (Prune=false) or requiring confirmation (Prune=confirm) to protect critical objects.
• When a CRD and its custom resources are introduced in the same sync, Argo CD skips dry‑run for those custom types so the sync doesn't fail before the API exists.

How Argo Rollouts fits: limit production exposure

Once the control plane is compatible and healthy, Argo Rollouts takes over for progressive delivery of workloads. Rollouts limits production exposure while a new workload version moves toward stable, providing traffic shifting and canary-style promotion that preserves rollback viability when both versions can coexist.

Important constraints and sequencing

• Compatibility must exist before promotion starts. New platform policies must not reject objects produced by the stable workload, and routing must preserve the stable path during the transition.
• The lifecycle of Kubernetes APIs (CRD versions, conversion webhooks, storage version) maps naturally to staged GitOps delivery: add the new API and controller, deploy consumers, migrate stored state if needed, then remove legacy versions after migration.
• Argo Rollouts documentation discourages using Rollouts to deploy infrastructure components such as cert‑manager, CoreDNS, and NGINX. Treat Rollouts as a workload progressive‑delivery tool, not a universal mechanism for platform upgrades.

1. 1Use Argo CD waves to apply CRD and controller (negative waves) and wait for healthy status.
2. 2Deploy new workload manifests managed by Rollouts in a later wave once the control plane is confirmed healthy.
3. 3Use Rollouts' traffic shifting to move traffic progressively to the new workload while keeping the old version available for rollback.
4. 4After workloads and stored states are migrated, remove legacy platform APIs and cleanup in a final contract phase.