The GitOps Operating Model

GitOps is a specific operating model, not a synonym for “we keep YAML in Git”. The defining element is a continuously running reconciler that compares the declared state in a repository against the actual state of the system and converges the two, forever.

The four principles link

1. Declarative. The entire system is described declaratively — desired state, not procedures.
2. Versioned and immutable. The desired state is stored in Git, giving a complete, immutable, auditable history.
3. Pulled automatically. Approved changes are pulled from the repository by software agents, not pushed by an external pipeline.
4. Continuously reconciled. Agents observe actual state and correct any divergence, without being asked.

Principle four is what distinguishes GitOps from CI/CD that happens to deploy Kubernetes manifests. A pipeline applies a change once and stops caring. A reconciler keeps caring.

Push versus pull link

flowchart LR
    subgraph "Push (traditional CD)"
    P1[Git] --> P2[CI pipeline]
    P2 -->|holds cluster credentials| P3[(Cluster)]
    P2 -.->|"applies once,<br/>then forgets"| P3
    end
    subgraph "Pull (GitOps)"
    G1[Git] <-->|"polls / webhook"| G2[Agent inside the cluster]
    G2 -->|"local credentials only"| G3[(Cluster)]
    G3 -->|observed state| G2
    G2 -.->|"reconciles continuously"| G3
    end

The pull model has three concrete advantages that matter beyond architectural neatness:

• No external system holds cluster credentials. The agent uses in-cluster permissions. Your CI system, which executes code from every contributor, never has production write access.
• Drift is corrected, not just detected. A manual change is reverted on the next reconciliation cycle, typically within minutes.
• The cluster can recover itself. Rebuild an empty cluster, point an agent at the repository, and the workloads return without a pipeline run.

The trade-off is feedback latency: the developer’s pipeline reports “merged”, not “deployed and healthy”. Good tooling closes this by writing reconciliation status back to the commit or pull request.

Repository structure link

The most consequential structural decision is separating application source from deployment configuration.

flowchart TD
    subgraph "app-service repo"
    A1[source code] --> A2[CI: build, test]
    A2 --> A3[(Container image<br/>tagged by digest)]
    end
    subgraph "deploy-config repo"
    B1[base/ — shared manifests]
    B2[overlays/dev]
    B3[overlays/staging]
    B4[overlays/production]
    end
    A3 -->|"automated PR: bump image digest"| B2
    B2 -->|promotion PR| B3
    B3 -->|promotion PR| B4
    B2 --> C1[Agent → dev cluster]
    B3 --> C2[Agent → staging cluster]
    B4 --> C3[Agent → prod cluster]

Why separate repositories:

• A deployment (rollback, config change, scaling) does not require an application code change or a rebuild.
• Access control differs: many people change application code; few should change production deployment config.
• The deployment repository’s commit history becomes a precise, auditable record of what ran in production and when — which is exactly what an auditor asks for.

Within the deployment repository, the base-plus-overlays pattern keeps environments genuinely similar: they share the same base and differ only in declared, reviewable patches. If an environment’s overlay grows large, that is a signal the environments have diverged in ways that will bite during a release.

Promotion as a pull request link

Environment promotion becomes a change to the deployment repository, which means it inherits code review, policy checks, and history for free.

sequenceDiagram
    participant CI as App CI
    participant DR as Deploy repo
    participant Pol as Policy checks
    participant Ag as Agent (prod)
    participant K as Production
    CI->>DR: automated PR — bump prod image to sha256:def…
    DR->>Pol: run policy + diff render on the PR
    Pol-->>DR: pass, with rendered manifest diff in the comment
    DR->>DR: human approves (timing decision)
    DR->>Ag: merged commit
    Ag->>K: reconcile to new state
    K-->>Ag: health status
    Ag-->>DR: write status back to the commit

Two practices make this genuinely useful rather than ceremonial: render the diff so reviewers see the resulting manifests rather than a template change, and pin images by digest, not by mutable tag. A tag like v1.4 can point at different content over time, which destroys the reproducibility the whole model exists to provide.

Secrets in a GitOps world link

The obvious problem: if everything is in Git, what about secrets? Three approaches, in ascending order of preference:

Prefer workload identity where the target supports it, external references otherwise, and encrypted-in-repo only for bootstrap material. Never commit plaintext, and note that git history is forever — a leaked secret must be rotated, not just removed in a follow-up commit.

Where GitOps stops working well link

Being honest about the boundaries prevents expensive misapplication.

• Genuinely imperative operations. Database migrations, data backfills, and one-off maintenance are sequences, not states. They need a job runner triggered by the reconciler, not a declarative representation.
• Very high-frequency state changes. Anything changing every few seconds (autoscaling decisions, per-request routing) does not belong in a repository. Declare the policy; let the controller own the instantaneous value.
• Emergency response. During a severe incident, the reconciliation loop can actively fight you by reverting a manual mitigation. Every GitOps installation needs a documented, audited break-glass procedure that suspends reconciliation for a specific scope — and an alert when it is used.
• Cross-cluster orchestration with ordering constraints. “Migrate region A, verify, then region B” is a workflow. Reconcilers are not workflow engines; drive them with one.

Adoption checklist link

• A reconciler continuously converges actual state to declared state.
• Deployment configuration lives in its own repository with its own access control.
• Images are referenced by digest, never by mutable tag.
• Pull requests show the rendered manifest diff, not just template changes.
• No plaintext secrets in Git; workload identity or external references preferred.
• Reconciliation status is written back to the commit or pull request.
• A documented, alerting break-glass procedure exists for incidents.