Skip to content
Hogin Hogin
Go back

PreDelete hooks in Argo CD 3.3: why deleting resources in GitOps was quietly dangerous

8 мин чтения

Before Argo CD 3.3, deleting a resource in GitOps was embarrassingly simple: drop a line from the manifest, sync detects “this is no longer in Git,” and prunes the object right away — whether it’s an ordinary ConfigMap or a StatefulSet holding state that really should have been backed up first. 3.3 adds PreDelete hooks — a new lifecycle stage that finally gives GitOps the “are you sure?” a human normally performs by hand before kubectl delete. Here’s how it works and what to do so stateful workloads stop being one prune away from disaster.

Table of contents

Open Table of contents

How prune used to behave

Argo CD sync has always followed the same logic: compare the desired state (what’s in Git) against the cluster’s actual state, and reconcile the latter to the former. If a resource exists in the cluster but is missing from Git, and prune is enabled (via automated.prune: true or the --prune flag), Argo CD deletes it. There was no checkpoint between “the resource is gone from the manifest” and “the resource is deleted from the cluster” — only PreSync/Sync/PostSync/SyncFail, none of which apply to deletion.

In practice this caused very concrete incidents:

What all three have in common: sync had no notion of “confirm before you delete” — only a binary “in Git / not in Git.” The 3.3 release shipped in the February–March 2026 cycle, and deletion safety was called out as the release’s most notable change — not a new integration or UI feature, but closing exactly this gap.

Pruning a resource before and after PreDelete: deletion used to be immediate, now it only happens after a successful gate

PreDelete: a new lifecycle stage

Argo CD already had sync hooks — PreSync, Sync, PostSync, SyncFail — ordinary Kubernetes manifests (usually a Job) tagged with the argocd.argoproj.io/hook annotation, which the controller applies and waits on at the matching point in the sync. PreDelete works the same way, but it’s tied to deletion instead of applying a manifest: the hook runs after a resource is marked a prune candidate and before Argo CD actually deletes it.

apiVersion: batch/v1
kind: Job
metadata:
  name: predelete-check
  annotations:
    argocd.argoproj.io/hook: PreDelete
    argocd.argoproj.io/hook-delete-policy: HookSucceeded

hook-delete-policy semantics for PreDelete match the other hooks, but with a different consequence:

The key difference from PreSync/PostSync: there, a failed hook blocks applying manifests; here, it blocks deletion specifically. The rest of the Application keeps syncing normally — only the prune of the resource owned by the failing PreDelete hook is held back.

PreDelete hook lifecycle: a resource is marked a prune candidate, a PreDelete Job validates state, and only HookSucceeded clears deletion

Combining it with finalizers and Prune=false

A PreDelete hook doesn’t replace Kubernetes finalizers or the Prune=false sync-option — these are three mechanisms with different scopes, and for stateful resources you should combine them, not pick one over the other.

A finalizer is a Kubernetes-level guarantee: it fires regardless of who triggered the deletion — Argo CD prune, a kubectl delete outside GitOps, or someone’s script. A PreDelete hook only sees deletions that go through Argo CD’s own sync; a manual kubectl delete bypasses it entirely. So for resources where “this must never be deleted without manual confirmation” really matters, don’t rely on the hook alone — keep the Prune=false sync-option directly on the resource, so Argo CD never touches it automatically and deletion stays a fully manual operation:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pgdata-postgres-0
  annotations:
    argocd.argoproj.io/sync-options: Prune=false

PreDelete in this scheme isn’t a replacement for a hard stop — it’s a gate for resources you do want to remain pruneable, but with a mandatory check first: a typical candidate is a Crossplane Claim that’s meant to be deleted along with its environment, but should get a final snapshot or a fresh-backup confirmation beforehand.

What happens to existing Applications without migration

The change is fully backward-compatible: PreDelete is opt-in. If no resource in an Application’s manifests carries the argocd.argoproj.io/hook: PreDelete annotation, prune behavior doesn’t change at all — sync still deletes resources missing from Git immediately, exactly as before 3.3. There’s no retroactive gate turning itself on: teams already relying on the current prune behavior (say, CI that recreates ephemeral environments) won’t see any difference until they add a hook themselves.

The practical takeaway: migration isn’t “upgrade Argo CD and wait” — it’s a concrete task of going through Applications with stateful resources (PVCs, CRDs with finalizers, Crossplane claims) and adding either a PreDelete hook or Prune=false, wherever a safety check matters more than full automation.

Briefly: hydration got faster

The same release also improved hydration performance — generating final manifests from the Git source before sync — through more aggressive caching of intermediate render results for large monorepos with dozens of Applications on a single source. It has nothing to do with deletion safety, but for teams running large GitOps monorepos it’s a noticeable cut in time-to-next-sync — a nice bonus from the same release cycle.

What you need to protect a StatefulSet+PVC

Take a typical case: a StatefulSet running Postgres with an explicitly declared PersistentVolumeClaim that shouldn’t silently vanish on an accidental prune. You need a CSI driver with VolumeSnapshot support (a VolumeSnapshotClass in the cluster) and Argo CD 3.3+.

The PreDelete hook takes a snapshot of the PVC and waits until it’s readyToUse before letting prune proceed:

apiVersion: batch/v1
kind: Job
metadata:
  name: predelete-snapshot-pgdata
  annotations:
    argocd.argoproj.io/hook: PreDelete
    argocd.argoproj.io/hook-delete-policy: HookSucceeded
spec:
  template:
    spec:
      restartPolicy: Never
      serviceAccountName: predelete-snapshot
      containers:
        - name: snapshot
          image: bitnami/kubectl:1.31
          command:
            - /bin/sh
            - -c
            - |
              set -e
              SNAP_NAME="predelete-pgdata-postgres-0-$(date +%s)"
              kubectl apply -f - <<EOF
              apiVersion: snapshot.storage.k8s.io/v1
              kind: VolumeSnapshot
              metadata:
                name: ${SNAP_NAME}
                namespace: ${NAMESPACE}
              spec:
                volumeSnapshotClassName: csi-snapclass
                source:
                  persistentVolumeClaimName: pgdata-postgres-0
              EOF
              kubectl wait --for=jsonpath='{.status.readyToUse}'=true \
                volumesnapshot/${SNAP_NAME} --timeout=120s

Keep the PVC itself on a hard stop too — a snapshot doesn’t remove the risk of the PVC being deleted by the same prune cycle if the hook happens to misfire:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pgdata-postgres-0
  annotations:
    argocd.argoproj.io/sync-options: Prune=false

serviceAccountName: predelete-snapshot needs RBAC to create/get volumesnapshots.snapshot.storage.k8s.io in the namespace — without it the Job fails with Forbidden, and HookFailed blocks the prune, which is exactly the right default here: a blocked sync beats lost data.

How to verify it works

Remove the StatefulSet from the manifests (or rename it temporarily) and sync with prune:

argocd app sync my-app --prune

While the hook runs, the Application shows OutOfSync with progress on the PreDelete hook; check the snapshot logs directly:

kubectl logs job/predelete-snapshot-pgdata -n <namespace>

It’s worth deliberately breaking the hook once (say, pointing at a nonexistent volumeSnapshotClassName) and confirming that argocd app get my-app reports Degraded while the PVC stays put — that’s the proof the gate actually blocks deletion, not just something sitting in YAML.

Bottom line

Declarative deletion in GitOps has long been missing the same “are you sure?” a human performs by hand before kubectl delete — the only way to get it before was building wrappers around sync or leaning on finalizers, which Argo CD’s own prune decision bypasses anyway. PreDelete hooks close exactly that gap: an explicit, UI-visible gate that can fail and actually stop the deletion, instead of just logging it after the fact. It isn’t free — every hook needs to be written, given RBAC, and tested against its own failure path — but for teams running stateful workloads on Argo CD, it’s worth rolling out before the next prune, not after an incident with a lost PVC.


Share this post:

Next Post
In-Place Pod Resize reaches GA in Kubernetes 1.35: resizing CPU/memory without restarting pods