Log Sink Points to Non-Existent Bucket: Fixing GCP Dangling Sink Destinations

Log sinks are the plumbing that moves your GCP logs out of the default _Default and _Required buckets and into somewhere useful: Cloud Storage for long-term retention, BigQuery for analysis, or Pub/Sub for streaming. The trouble is that a sink and its destination are loosely coupled. GCP will happily let a sink keep a reference to a Cloud Storage bucket that has been deleted, renamed, or moved to another project. When that happens, the logs you think you are archiving are going nowhere.

The logging_danglingbucket check looks for exactly this situation: a sink whose destination points at a storage bucket that cannot be found.

---

What this check detects

A GCP log sink has a destination field that follows a fixed format. For a Cloud Storage destination it looks like this:

storage.googleapis.com/BUCKET_NAME

This check inspects each sink in scope, extracts the target bucket name, and verifies the bucket actually exists. If the bucket has been deleted, was never created, or lives in a project the sink can no longer reach, the check fires.

Sinks exist at several levels: project, folder, organization, and billing account. A dangling destination can show up at any of them, and org-level sinks are the easiest to forget about because nobody owns them day to day.

---

Why it matters

Logging failures are quiet by design. There is no error thrown when a sink writes to a bucket that does not exist. The export simply fails and the log entries are dropped. You only discover the gap when you go looking for the logs and they are not there, which is usually during an incident or an audit, the two worst moments to find out.

You lose the audit trail you are relying on

Many teams route Cloud Audit Logs to a dedicated Cloud Storage bucket for compliance retention (PCI DSS, HIPAA, SOC 2, and similar frameworks all expect this). If the sink points at a missing bucket, you are out of compliance and you have no record of who did what in the affected window. Reconstructing that after the fact is rarely possible.

Security investigations hit a dead end

When responders pull logs to trace an attacker's movement, gaps line up suspiciously with the time the bucket went missing. Worse, an attacker with sufficient permissions could delete the destination bucket deliberately to cover their tracks, and a sink that keeps pointing at the deleted name produces no obvious alarm.

Cost and capacity assumptions break

If you budgeted retention storage around a certain log volume and the export has silently failed for months, your retention costs look artificially low while your actual coverage is zero. The numbers lie in both directions.

---

How to fix it

There are two paths depending on what you actually want: keep the original destination by recreating the bucket, or repoint the sink at a destination that already exists. Start by confirming the current state.

1. Inspect the sink and its destination

List the sinks at the level you care about and read the destination:

# Project-level sinks
gcloud logging sinks list --project=PROJECT_ID

# Inspect one sink in detail
gcloud logging sinks describe SINK_NAME --project=PROJECT_ID

Look at the destination field. If it reads storage.googleapis.com/my-audit-archive, check whether that bucket exists:

gcloud storage buckets describe gs://my-audit-archive

If you get a 404 or NotFound error, the destination is dangling.

2a. Option A — recreate the bucket

If the bucket should still exist and the name is available, recreate it with the same settings you intended. Pick a region close to your workloads and apply retention and uniform access from the start:

gcloud storage buckets create gs://my-audit-archive \
  --project=PROJECT_ID \
  --location=us-central1 \
  --uniform-bucket-level-access \
  --default-storage-class=NEARLINE

2b. Option B — repoint the sink

Update the sink to send logs to an existing bucket you control:

gcloud logging sinks update SINK_NAME \
  storage.googleapis.com/my-new-audit-archive \
  --project=PROJECT_ID

3. Grant the sink's writer identity write access

This is the step people miss. Every sink writes as a service account (the writer identity), and that identity needs roles/storage.objectCreator on the destination bucket. A freshly created or repointed bucket will not have it.

Get the writer identity:

gcloud logging sinks describe SINK_NAME \
  --project=PROJECT_ID \
  --format="value(writerIdentity)"

That returns something like serviceAccount:[email protected]. Grant it write access to the bucket:

gcloud storage buckets add-iam-policy-binding gs://my-new-audit-archive \
  --member="serviceAccount:[email protected]" \
  --role="roles/storage.objectCreator"

4. Confirm logs are flowing

Wait a few minutes and check that objects are landing in the bucket. Sinks write hourly-partitioned files, so allow time for the first batch:

gcloud storage ls gs://my-new-audit-archive/**

---

How to prevent it from happening again

The root cause is almost always a bucket deleted without checking who depends on it. Close that gap with infrastructure as code and a delete guard.

Manage sinks and buckets together in Terraform

When the sink and its destination live in the same Terraform module, the dependency is explicit and the bucket cannot be removed without Terraform also updating the sink. The writer identity binding is wired in automatically:

resource "google_storage_bucket" "audit_archive" {
  name                        = "my-audit-archive"
  project                     = var.project_id
  location                    = "US-CENTRAL1"
  storage_class               = "NEARLINE"
  uniform_bucket_level_access = true

  retention_policy {
    retention_period = 31536000 # 365 days in seconds
  }
}

resource "google_logging_project_sink" "audit_sink" {
  name        = "audit-archive-sink"
  project     = var.project_id
  destination = "storage.googleapis.com/${google_storage_bucket.audit_archive.name}"
  filter      = "logName:\"cloudaudit.googleapis.com\""

  unique_writer_identity = true
}

resource "google_storage_bucket_iam_member" "sink_writer" {
  bucket = google_storage_bucket.audit_archive.name
  role   = "roles/storage.objectCreator"
  member = google_logging_project_sink.audit_sink.writer_identity
}

Protect the bucket from deletion

Add a bucket-level retention policy or a lifecycle guard so the destination cannot be casually removed. In Terraform, you can also block destroy on the resource:

resource "google_storage_bucket" "audit_archive" {
  # ... other config ...
  lifecycle {
    prevent_destroy = true
  }
}

Gate it in CI/CD with policy as code

Use a policy engine to reject any change that removes a bucket still referenced by a sink, or that leaves a sink without a matching destination resource in the plan. A Terraform plan scanned with OPA or Conftest can catch this before merge:

package gcp.logging

# Deny if a sink destination bucket is not also created in the same plan
deny[msg] {
  sink := input.resource_changes[_]
  sink.type == "google_logging_project_sink"
  dest := sink.change.after.destination
  startswith(dest, "storage.googleapis.com/")
  bucket_name := trim_prefix(dest, "storage.googleapis.com/")

  not bucket_defined(bucket_name)
  msg := sprintf("sink '%s' points at bucket '%s' which is not managed in this plan", [sink.name, bucket_name])
}

bucket_defined(name) {
  b := input.resource_changes[_]
  b.type == "google_storage_bucket"
  b.change.after.name == name
}

---

Best practices

• Use dedicated writer identities. Always create sinks with unique_writer_identity = true so each sink's access is auditable and scoped.
• Keep audit log buckets separate from application data. Isolate compliance and audit exports in their own buckets, ideally in a dedicated logging project with tightly controlled IAM.
• Apply retention locks on audit buckets. A locked retention policy prevents both accidental deletion of objects and tampering, which matters when the logs themselves are evidence.
• Set up an aggregated org-level sink. A single org sink exporting all audit logs to a central bucket gives you one place to monitor and one destination to protect, instead of dozens of per-project sinks each capable of going stale.
• Alert on export errors. Cloud Logging surfaces sink export error metrics. Build an alert on logging.googleapis.com/exports/error_count so a failing sink pages you instead of staying silent.
• Review sinks during decommissioning. When you tear down a project or bucket, list every sink that references it first. Make this a checklist item in your offboarding runbook.

A log sink is only as reliable as the destination behind it. Tie the two together in code, protect the bucket from deletion, and let continuous scanning catch the drift that slips past everything else.