Real-AWS deploy composition for recently-interacted-topk

[pequalsnp]

Summary

Lands the prep work needed to start the v1 real-AWS soak documented in STABILITY.md. One terraform apply stands up the multi-source TopK example end to end and exercises the experimental code paths that need operational evidence before promotion (Lambda Kinesis runtime, processor core, Kafka per-partition concurrency).

What this adds

pipeline-counter additions:

• New dedup_enabled flag — provisions <name>_dedup (pk:S hash, native TTL on ttl), grants worker + bootstrap RW, injects DDB_DEDUP_TABLE env var into all three tasks. Exposed as dedup_table_arn / dedup_table_name outputs so sibling modules share the same table.

New pipeline-lambda-kinesis module:

• Kinesis data stream (or BYO via kinesis_stream_arn).
• Lambda function — provided.al2, arm64 by default, deployment-zip driven, content-hashed via filebase64sha256.
• Event-source mapping with BatchItemFailures + bisect_batch_on_function_error; tunable batch_size / parallelization_factor / starting_position (supports AT_TIMESTAMP for bootstrap-to-live handoff).
• IAM: AWSLambdaBasicExecutionRole + Kinesis consumer perms + DDB state RW + optional dedup RW + optional SQS/SNS on-failure destination.
• CloudWatch log group with caller-controlled retention.

examples/recently-interacted-topk/terraform/:

• Composes both modules.
• CloudWatch alarms: Lambda Errors / IteratorAge / Throttles, DDB WriteThrottles on state + dedup, Kinesis WriteProvisionedThroughputExceeded. All route to alarm_sns_topic_arn when set.
• terraform.tfvars.example + runbook (prereqs → image build → Lambda zip → apply → smoke test → observability → teardown). Includes rough cost notes (~$100/mo for a low-volume soak).

Test plan

• terraform validate on both modules and the example composition.
• terraform fmt -check -recursive deploy/terraform examples/recently-interacted-topk/terraform.
• Manual: at least one apply against a sandbox AWS account before merging — this is infra code; validate is necessary but not sufficient.
• Manual: smoke test from the runbook (Kafka + Kinesis events both land in the same DDB row; GetWindow returns the merged Top-N).

Follow-ups (separate PRs)

• Begin the real-AWS soak using this composition. STABILITY.md's "exercised against real (non-local) AWS for at least one full quarter" clock starts on apply.
• Promote pkg/exec/processor + pkg/exec/lambda/{kinesis,dynamodbstreams,sqs} + pkg/source/kafka to mostly stable once the soak is uneventful.

🤖 Generated with Claude Code