Heap

Build me a focused web and product analytics workflow for the personal core of Heap. Requirements:

- Use Next.js 15, TypeScript, ClickHouse, PostgreSQL, and Docker Compose. Model data sources, dated observations, derived metrics, and report snapshots; keep stable source IDs and timestamps.
- Paid product context: Instrument explicit first-party events and avoid automatic capture of unrelated user behavior. Build only this DIY scope: Instrument explicit privacy-conscious first-party events, avoid automatic capture of unrelated user behavior, answer a small set of product questions, and retain raw data under the owner's control.
- For web and product analytics, define the event schema and a permitted first-party source.
- Ingest events with stable IDs, aggregate the requested metric, and retain raw rows for audit.
- Show a dated report with filters and CSV export.
- Use a local web page with input, progress, review, and export views. Required input or access: Docker; ClickHouse.
- Recovery: Ingest idempotently, filter obvious bots, truncate IP data, and avoid fingerprinting.
- Acceptance: with one labelled sample, show the input, saved intermediate state, and exported result; verify this invariant: A replayed batch cannot double-count events; missing or late data must be labelled in the report.
- Out of scope: automatic capture engine, identity graph, data science, governance, and enterprise operations; identity stitching. Keep this a personal, inspectable workflow.
- Include a README with setup, a sample input, required keys or permissions, data location, and the supported scope.

# AGENTS.md — Build guide for Heap

## Project scope
Instrument explicit privacy-conscious first-party events, avoid automatic capture of unrelated user behavior, answer a small set of product questions, and retain raw data under the owner's control.

Catalogue verdict: no. A consolation build is possible, but the paid product's decisive value sits outside a solo rebuild. For Heap, instrument explicit first-party events and avoid automatic capture of unrelated user behavior. The hard boundary is automatic capture engine, identity graph, data science, governance, and enterprise operations, plus data pipeline reliability and analytical depth.
Use the implementation prompt below to define the deliverable. Complete each phase's acceptance checks before extending the scope.

## Working agreement
- Inspect the repository and its existing instructions before choosing paths, dependencies or commands. Keep one coherent stack and explain changes to the proposed architecture.
- Plan a vertical slice that accepts a real input and produces the useful output described below. Persist only the state the prompt calls for; respect memory-only and upstream-managed workflows. Use fixtures only when they are clearly labelled.
- After scaffolding, document the actual install, development, check and build commands in README and keep them synchronized with the package or project manifest. Do not report commands as successful unless they ran.
- Work in small steps. At handoff, list implemented flows, checks actually performed, remaining blockers, and any credentials or provider setup the owner must supply.
- Do not publish, spend money, contact customers, delete source data or run irreversible migrations without the project owner's authorization.

## Prerequisites
- Docker
- ClickHouse
- PostgreSQL
- public HTTPS collector endpoint
- site script access

## Stack and architecture
- Use Next.js 15, TypeScript, ClickHouse, PostgreSQL, and Docker Compose.
- Model data sources, dated observations, derived metrics, and report snapshots; keep stable source IDs and timestamps.
- Interface for this web and product analytics workflow: a local web page with input, progress, review, and export views.

## Security and data integrity
- Ingest idempotently, filter obvious bots, truncate IP data, and avoid fingerprinting.
- Validate imported records and file paths; keep user data within the storage location named in the prompt.
- State the practical limit: automatic capture engine, identity graph, data science, governance, and enterprise operations.

## Agent implementation rules
- Project rule, data: Model data sources, dated observations, derived metrics, and report snapshots; keep stable source IDs and timestamps.
- Project rule, behavior: Create projects, API keys, environments, event names, and a documented event schema.
- Project rule, recovery: Ingest idempotently, filter obvious bots, truncate IP data, and avoid fingerprinting.

## Optional agent skills and references
No installable skill is listed for this guide; use the project rules and the stack documentation.

Read the linked SKILL.md and its dependencies before adding a skill. Select only the skills matching this project's runtime and task; their documentation does not supply API access, credentials or approval to perform external actions. Pin the reviewed revision where the tool supports it. Follow the chosen agent's documented project-level installation mechanism.

## Distribution ideas
These are optional planning notes. Obtain the owner's approval before publishing or contacting anyone.
- Demonstrate this web and product analytics operation with sample or user-owned data: Create projects, API keys, environments, event names, and a documented event schema.
- Show a concrete result or export: Add retention controls, raw-event export, deletion, health checks, and backup instructions.
- State the practical limit: automatic capture engine, identity graph, data science, governance, and enterprise operations.

## Engineering roadmap
1. Phase 1, architecture and data: Use Next.js 15, TypeScript, ClickHouse, PostgreSQL, and Docker Compose. Model data sources, dated observations, derived metrics, and report snapshots; keep stable source IDs and timestamps.
2. Phase 2, implement: Create projects, API keys, environments, event names, and a documented event schema.
3. Phase 3, implement: Add retention controls, raw-event export, deletion, health checks, and backup instructions.
4. Phase 4, review and output: Instrument explicit privacy-conscious first-party events, avoid automatic capture of unrelated user behavior, answer a small set of product questions, and retain raw data under the owner's control. Add retention controls, raw-event export, deletion, health checks, and backup instructions.
5. Phase 5, recovery and acceptance: Ingest idempotently, filter obvious bots, truncate IP data, and avoid fingerprinting. Verify this invariant with a saved fixture: A replayed batch cannot double-count events; missing or late data must be labelled in the report. State the practical limit: automatic capture engine, identity graph, data science, governance, and enterprise operations.

## Paid-product capabilities outside this build
- automatic capture engine, identity graph, data science, governance, and enterprise operations
- identity stitching
- session replay
- warehouse connectors
- high-volume global ingestion and support

## Implementation prompt
Build me a focused web and product analytics workflow for the personal core of Heap. Requirements:

- Use Next.js 15, TypeScript, ClickHouse, PostgreSQL, and Docker Compose. Model data sources, dated observations, derived metrics, and report snapshots; keep stable source IDs and timestamps.
- Paid product context: Instrument explicit first-party events and avoid automatic capture of unrelated user behavior. Build only this DIY scope: Instrument explicit privacy-conscious first-party events, avoid automatic capture of unrelated user behavior, answer a small set of product questions, and retain raw data under the owner's control.
- For web and product analytics, define the event schema and a permitted first-party source.
- Ingest events with stable IDs, aggregate the requested metric, and retain raw rows for audit.
- Show a dated report with filters and CSV export.
- Use a local web page with input, progress, review, and export views. Required input or access: Docker; ClickHouse.
- Recovery: Ingest idempotently, filter obvious bots, truncate IP data, and avoid fingerprinting.
- Acceptance: with one labelled sample, show the input, saved intermediate state, and exported result; verify this invariant: A replayed batch cannot double-count events; missing or late data must be labelled in the report.
- Out of scope: automatic capture engine, identity graph, data science, governance, and enterprise operations; identity stitching. Keep this a personal, inspectable workflow.
- Include a README with setup, a sample input, required keys or permissions, data location, and the supported scope.

## Completion evidence
Demonstrate the prompt's acceptance scenarios against the scoped workflow. Include setup from a clean checkout and failure recovery. Check persistence across restart and export/restore only for the state the prompt says to store; for memory-only tools, confirm that temporary content is discarded as specified. Record actual results and remaining limitations. A detailed plan alone does not establish a working replacement.