Kinesis Stream Has No Recent Activity: Find and Fix Abandoned Streams

A Kinesis Data Stream that sits empty for a week is rarely a deliberate choice. Most of the time it means a producer stopped writing, a deployment broke an integration, or someone spun up a stream for a project that never shipped. Either way, you are paying for shard-hours on infrastructure that does nothing, and you may be staring at a data pipeline that has silently failed.

This check, kinesis_abandoned, looks at the IncomingRecords metric across your streams and raises a finding when it reads zero for the trailing 7 days.

---

What this check detects

Lensix queries the CloudWatch metric IncomingRecords (namespace AWS/Kinesis) for each Kinesis Data Stream in the account. If the sum of incoming records over the past 7 days is zero, the stream is considered abandoned and flagged.

An empty stream falls into one of a few buckets:

• A leftover from a decommissioned workload that nobody cleaned up.
• A broken producer whose IAM role, endpoint, or application logic changed and stopped writing.
• A genuinely seasonal or low-frequency stream that legitimately receives no traffic for stretches at a time.

The check cannot tell these apart on its own. That distinction is your job, and it changes the remediation completely.

---

Why it matters

Silent pipeline failure

The more serious problem is not the cost. It is what an empty stream often represents. If a stream was actively ingesting clickstream events, application logs, or IoT telemetry last month and now reads zero, something upstream broke. When that data feeds analytics, fraud detection, or billing, the gap can go unnoticed until a report comes back wrong or an alert never fires.

Consider a fraud detection pipeline that consumes transaction events from Kinesis. If the producer service loses its IAM permission after a policy change, records stop flowing. The consumer sees no new records, so it raises no fraud alerts. Everything looks calm precisely because the data stopped, which is the worst kind of failure.

Wasted spend

Provisioned streams charge for every shard whether or not it carries traffic. Abandoned streams accumulate across teams and accounts, and the bill grows quietly. They are easy to create and easy to forget.

Audit and compliance noise

Unused resources clutter your inventory and complicate audits. Every stream that exists is something a reviewer has to account for, and an idle one with broad IAM access attached is an asset worth removing rather than explaining.

---

How to fix it

Start by figuring out which bucket the stream falls into. Do not delete anything until you know.

Step 1: Confirm the stream is actually idle

Pull the IncomingRecords metric directly so you can see the shape of the traffic, not just a single 7-day total.

aws cloudwatch get-metric-statistics \
  --namespace AWS/Kinesis \
  --metric-name IncomingRecords \
  --dimensions Name=StreamName,Value=my-stream \
  --start-time "$(date -u -d '14 days ago' +%Y-%m-%dT%H:%M:%SZ)" \
  --end-time "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  --period 86400 \
  --statistics Sum

If you see records flowing two weeks ago and then a hard stop, you are looking at a broken producer, not an abandoned stream. If it has been zero for the entire window and there is no business reason for periodic traffic, it is a candidate for deletion.

Step 2: Identify the producer

Find what is supposed to be writing to the stream. Check CloudTrail for recent PutRecord and PutRecords calls and the identity behind them.

aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=PutRecords \
  --start-time "$(date -u -d '14 days ago' +%Y-%m-%dT%H:%M:%SZ)" \
  --max-results 20

If the calls stopped at a specific timestamp, line that up against your deployment history. A producer often breaks because of a revoked IAM permission, a changed stream name, a regional misconfiguration, or an exception that silently swallows the failure.

Step 3a: If the producer should be writing, fix it

The most common cause is an IAM permission gap. Confirm the producer's role can write to the stream:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "kinesis:PutRecord",
        "kinesis:PutRecords",
        "kinesis:DescribeStream"
      ],
      "Resource": "arn:aws:kinesis:us-east-1:111122223333:stream/my-stream"
    }
  ]
}

Then verify connectivity end to end by putting a test record:

aws kinesis put-record \
  --stream-name my-stream \
  --partition-key test \
  --data "$(echo -n 'healthcheck' | base64)"

If that succeeds, the issue is in the application, not the platform. Look for swallowed exceptions, a stale endpoint, or a feature flag that disabled the producer.

Step 3b: If the stream is genuinely abandoned, delete it

Check for registered consumers first:

aws kinesis list-stream-consumers \
  --stream-arn arn:aws:kinesis:us-east-1:111122223333:stream/my-stream

If nothing depends on it, delete the stream:

aws kinesis delete-stream \
  --stream-name my-stream \
  --enforce-consumer-deletion

Step 4: Consider on-demand mode for spiky workloads

If the stream is real but its traffic is irregular, switch it from provisioned to on-demand capacity. On-demand bills per gigabyte of data rather than per shard-hour, so an idle period costs almost nothing.

aws kinesis update-stream-mode \
  --stream-arn arn:aws:kinesis:us-east-1:111122223333:stream/my-stream \
  --stream-mode-details StreamMode=ON_DEMAND

---

How to prevent it from happening again

Abandoned streams come from two gaps: no alerting on traffic loss, and no lifecycle ownership. Close both.

Alarm on dropped traffic for active streams

For any stream that should always carry data, set a CloudWatch alarm that fires when IncomingRecords drops to zero. This catches a broken producer in minutes instead of days.

aws cloudwatch put-metric-alarm \
  --alarm-name "kinesis-my-stream-no-records" \
  --namespace AWS/Kinesis \
  --metric-name IncomingRecords \
  --dimensions Name=StreamName,Value=my-stream \
  --statistic Sum \
  --period 3600 \
  --evaluation-periods 3 \
  --threshold 1 \
  --comparison-operator LessThanThreshold \
  --treat-missing-data breaching \
  --alarm-actions arn:aws:sns:us-east-1:111122223333:ops-alerts

The --treat-missing-data breaching flag matters here. Kinesis stops emitting the metric entirely when no records arrive, so without it the alarm would sit in an insufficient-data state instead of alerting.

Tag streams with an owner and purpose

Make ownership explicit at creation time. A stream you can trace back to a team and a project is a stream someone will clean up.

aws kinesis add-tags-to-stream \
  --stream-name my-stream \
  --tags Owner=data-platform,Project=clickstream,Environment=prod

Manage streams as code

Define streams in Terraform or CloudFormation so they are reviewed, version-controlled, and easy to remove when a project ends. A stream that exists only because someone clicked through the console is the one that gets forgotten.

resource "aws_kinesis_stream" "clickstream" {
  name             = "clickstream"
  retention_period = 24

  stream_mode_details {
    stream_mode = "ON_DEMAND"
  }

  tags = {
    Owner       = "data-platform"
    Project     = "clickstream"
    Environment = "prod"
  }
}

Gate it in CI/CD with policy-as-code

Use a tool like OPA or Checkov to require an owner tag on every Kinesis stream before it merges. This is a simple Rego rule that rejects untagged streams:

package terraform.kinesis

deny[msg] {
  resource := input.resource_changes[_]
  resource.type == "aws_kinesis_stream"
  not resource.change.after.tags.Owner
  msg := sprintf("Kinesis stream '%s' must have an Owner tag", [resource.address])
}

---

Best practices

• Default to on-demand mode unless you have steady, predictable throughput that justifies provisioned shards. It eliminates most idle-cost concerns.
• Alarm on traffic loss, not just throttling. Teams instrument WriteProvisionedThroughputExceeded but forget that zero records is also a failure mode.
• Set a sane retention period. The default 24 hours is fine for most pipelines, but extending it has a cost. Match it to how long your consumers might realistically be down.
• Tag everything at creation. Owner, project, and environment tags turn an anonymous resource into an accountable one.
• Review idle resources on a schedule. Treat a flagged abandoned stream as a question, not an automatic delete. Confirm it is dead before you remove it.
• Trace producers when traffic stops. An empty stream is often the symptom, and the root cause is usually a broken or de-permissioned producer worth fixing.

An idle Kinesis stream is cheap to ignore and expensive to misread. Catch it early, confirm what it means, and either revive the pipeline or retire the resource. Both outcomes are better than leaving it running.