Build deletion and rectification pipelines

Build Deletion and Rectification Pipelines

Honor data subject rights by turning requests into reliable, auditable workflows. Treat deletion and rectification as first class product features with clear ownership, idempotent jobs, vendor propagation, and proof of completion.

Scope and workflow

• Define what you delete vs anonymize vs retain under legal obligation. Document exceptions and how you communicate them.

• Use a stable data subject identifier (DSID) to locate records across services. Derive DSID as a salted hash of the primary key to avoid logging raw IDs.

• Orchestrate with a state machine: received → verified → in_progress → vendor_propagation → completedor denied_with_reason.

Intake and verification

• Provide an authenticated self service portal and a back office tool.

• Verify identity proportionate to risk. Re verify email or require MFA for account level requests.

• Accept request types: delete, rectify, and optional restrict to pause processing while investigating.

POST /privacy/requests
{
  "type": "delete" | "rectify",
  "dsid": "hash...",
  "fields": { "email": "new@example.com" },   // for rectification
  "idempotency_key": "uuid"
}

Inventory and selectors

• Maintain a data map of systems, tables, object stores, search indexes, caches, analytics, and vendors.

• For each system, store a selector for lookup by DSID and the action to perform: hard delete, anonymize, or update.

CREATE TABLE privacy_systems (
  id serial PRIMARY KEY,
  name text UNIQUE,
  selector text,          -- e.g., "user_id = :uid" or "metadata->>'dsid' = :dsid"
  action text             -- delete | anonymize | rectify
);

Deletion pipeline

1. Prepare

   • Revoke sessions and tokens. Disable login to prevent re creation.

   • Queue work items per system with an idempotency key.

2. Execute

   • Relational data: cascade delete where safe. Where constraints block delete, anonymize direct identifiers and sever foreign key links.

   • Object storage: delete all versions. Remove thumbnails and derivatives.

   • Search and caches: delete documents and evict keys. Rebuild affected aggregates.

   • Analytics and logs: remove or aggregate to non identifiable values if feasible. Do not keep raw personal data in logs.

3. Backups and restores

   • Prefer crypto shred for high risk datasets by destroying the per subject DEK.

   • Keep a restore time hook that replays pending deletion markers after a recovery so the subject does not re appear.

4. Vendors

   • Send deletion requests through adapters and record receipts. Retry with backoff and alert on failures.

5. Edge cases

   • Duplicate or merged accounts. Apply to every linked profile.

   • Legal hold. Pause delete and communicate scope and reason.

-- Deletion request and tasks
CREATE TABLE privacy_requests (
  id uuid PRIMARY KEY,
  type text CHECK (type IN ('delete','rectify')),
  dsid text NOT NULL,
  status text NOT NULL DEFAULT 'received',
  created_at timestamptz DEFAULT now(),
  idempotency_key uuid UNIQUE
);

CREATE TABLE privacy_tasks (
  id bigserial PRIMARY KEY,
  request_id uuid REFERENCES privacy_requests(id),
  system text,
  op text,             -- delete | anonymize | rectify | vendor
  payload jsonb,
  status text DEFAULT 'queued',
  attempts int DEFAULT 0,
  last_error text
);

// Idempotent worker sketch
async function runTask(task) {
  const lock = await takeLock(`task:${task.id}`); // prevents double run
  if (!lock) return;
  try {
    await handlers[task.system][task.op](task.payload);
    await markDone(task.id);
  } catch (e) {
    await markRetry(task.id, e.message);
  } finally {
    await releaseLock(lock);
  }
}

Rectification pipeline

1. Validate new values

   • Format check and business rules. Trigger re verification flows where needed, for example new email requires verify link.

2. Apply change

   • Upsert in the source of truth first, then fan out to downstream stores with outbox events.

   • Recompute derived fields and denormalized projections.

3. Propagate

   • Update search indexes, caches, analytics dimensions, and data warehouse slowly changing dimensions when applicable.

   • Notify vendors that store the field. Record confirmations.

4. History

   • Keep a minimal, hashed change audit to prove rectification without storing previous plaintext values.

CREATE TABLE rectification_audit (
  at timestamptz DEFAULT now(),
  dsid text,
  field text,
  old_hash text,
  new_hash text,
  actor text,          -- user or admin id
  decision text        -- allowed or denied
);

Vendors and processors

• Maintain a registry of vendor endpoints, required fields, and SLAs.

• Implement per vendor adapters with signed requests and retries. Capture response body and timestamp for audit.

{
  "vendor": "email_service_x",
  "action": "delete",
  "reference": "req_123",
  "status": "confirmed",
  "confirmed_at": "2025-09-03T20:11:00Z"
}

Testing and dry runs

• Provide a dry run flag that lists what would be removed or corrected per system.

• Create seed data and golden tests for end to end deletion and rectification, including backup restore replay.

• Fuzz test selectors to ensure no over deletion.

User communication and status

• Expose request status to the user and send completion summaries that list systems updated and any lawful exceptions.

• Keep templates for confirmations and denials with clear reasons and escalation paths.

Monitoring and alerts

• Alert on tasks stuck in queued or retry states, vendor adapter failures, or restore replay errors.

• Track lead time from verification to completion and percent completed on first pass.

Quick deletion and rectification checklist

• Stable DSID and complete system inventory with selectors

• Idempotent, state machine driven jobs with locks and retries

• Session revocation and pre delete safeguards

• Hard delete, anonymize, and crypto shred where appropriate

• Search, cache, and analytics invalidation and rebuild

• Vendor propagation with receipts and SLA tracking

• Rectification with validation, re verification, and downstream reindex

• Append only audits for requests, tasks, and vendor confirmations

• Restore time replay of deletion markers

• Dry run mode and e2e tests

Conclusion

Rights workflows only work when they are boringly reliable. With a DSID based inventory, idempotent task runners, vendor adapters, cache and index invalidation, crypto shred for backups, and append only audits, you can fulfill deletion and rectification requests consistently, prove it later, and meet GDPR and CPRA expectations for accuracy and erasure.