Ahrefs

Build me a daily rank tracker and site auditor for my own domain, the one slice
of Ahrefs worth building solo. Requirements:

- Python + httpx. Keywords live in keywords.txt, up to 50 terms; fetch positions
  from a SERP API (Serper.dev or SerpAPI, key in .env, free tiers cover a small
  list). Do not scrape Google directly, that just gets blocked.
- Store keyword, position, ranking URL, and date in SQLite; run daily from cron.
- A localhost dashboard (Flask + Chart.js): a sparkline per keyword and a
  biggest-movers-this-week list.
- A technical audit crawler for my domain: follow internal links, flag broken
  links, missing or duplicate titles and descriptions, redirect chains, and
  pages under 200 words; write findings to audit.md.
- Pull clicks and impressions per query from the Google Search Console API and
  show them beside the rank data. Warn me in the README: budget an hour for the
  Google Cloud console OAuth setup, it is the painful step.
- Local only, no accounts, no telemetry.
- Out of scope: backlink data, keyword volume and difficulty, and competitor
  research. Do not try to fake these by scraping; that dataset is the
  subscription.
- README: API key setup, the Search Console connection steps, and a plain
  statement that Ahrefs is a proprietary web index and this replaces the
  dashboard, not the data.

# AGENTS.md — Build guide for Ahrefs

## Project scope
For one site, crawl pages, check technical SEO, track a small keyword list, and call search APIs where available.

Catalogue verdict: no. You can build a keyword tracker or site audit script, but Ahrefs' value is its proprietary web crawl, backlink index, keyword data, SERP data, and historical datasets.
Use the implementation prompt below to define the deliverable. Complete each phase's acceptance checks before extending the scope.

## Working agreement
- Inspect the repository and its existing instructions before choosing paths, dependencies or commands. Keep one coherent stack and explain changes to the proposed architecture.
- Plan a vertical slice that accepts a real input and produces the useful output described below. Persist only the state the prompt calls for; respect memory-only and upstream-managed workflows. Use fixtures only when they are clearly labelled.
- After scaffolding, document the actual install, development, check and build commands in README and keep them synchronized with the package or project manifest. Do not report commands as successful unless they ran.
- Work in small steps. At handoff, list implemented flows, checks actually performed, remaining blockers, and any credentials or provider setup the owner must supply.
- Do not publish, spend money, contact customers, delete source data or run irreversible migrations without the project owner's authorization.

## Prerequisites
- crawler
- search/SEO APIs
- storage
- rank tracking jobs
- dashboard
- substantial data budget

## Stack and architecture
- Python + httpx. Keywords live in keywords.txt, up to 50 terms; fetch positions from a SERP API (Serper.dev or SerpAPI, key in .env, free tiers cover a small list). Do not scrape Google directly, that just gets blocked.
- Store keyword, position, ranking URL, and date in SQLite; run daily from cron.
- Interface for this SEO/backlink data workflow: a focused SEO/backlink data input, review, and export interface.

## Security and data integrity
- When a crawl or search API call fails, retain the last dated snapshot, show its age, and retry within the provider limit.
- Validate imported records and file paths; keep user data within the storage location named in the prompt.
- State the practical limit: backlink index.

## Agent implementation rules
- Project rule, data: Store keyword, position, ranking URL, and date in SQLite; run daily from cron.
- Project rule, behavior: A localhost dashboard (Flask + Chart.js): a sparkline per keyword and a biggest-movers-this-week list.
- Project rule, recovery: When a crawl or search API call fails, retain the last dated snapshot, show its age, and retry within the provider limit.

## Optional agent skills and references
No installable skill is listed for this guide; use the project rules and the stack documentation.

Read the linked SKILL.md and its dependencies before adding a skill. Select only the skills matching this project's runtime and task; their documentation does not supply API access, credentials or approval to perform external actions. Pin the reviewed revision where the tool supports it. Follow the chosen agent's documented project-level installation mechanism.

## Distribution ideas
These are optional planning notes. Obtain the owner's approval before publishing or contacting anyone.
- Demonstrate this SEO/backlink data operation with sample or user-owned data: A localhost dashboard (Flask + Chart.js): a sparkline per keyword and a biggest-movers-this-week list.
- Show a concrete result or export: A technical audit crawler for my domain: follow internal links, flag broken links, missing or duplicate titles and descriptions, redirect chains, and pages under 200 words; write findings to audit.md.
- State the practical limit: backlink index.

## Engineering roadmap
1. Phase 1, architecture and data: Python + httpx. Keywords live in keywords.txt, up to 50 terms; fetch positions from a SERP API (Serper.dev or SerpAPI, key in .env, free tiers cover a small list). Do not scrape Google directly, that just gets blocked. Store keyword, position, ranking URL, and date in SQLite; run daily from cron.
2. Phase 2, implement: A localhost dashboard (Flask + Chart.js): a sparkline per keyword and a biggest-movers-this-week list.
3. Phase 3, implement: A technical audit crawler for my domain: follow internal links, flag broken links, missing or duplicate titles and descriptions, redirect chains, and pages under 200 words; write findings to audit.md.
4. Phase 4, review and output: For one site, crawl pages, check technical SEO, track a small keyword list, and call search APIs where available. A technical audit crawler for my domain: follow internal links, flag broken links, missing or duplicate titles and descriptions, redirect chains, and pages under 200 words; write findings to audit.md.
5. Phase 5, recovery and acceptance: When a crawl or search API call fails, retain the last dated snapshot, show its age, and retry within the provider limit. Verify this invariant with a saved fixture: A failed crawl retains its last dated snapshot; a report must not imply backlink or keyword coverage the input lacks. State the practical limit: backlink index.

## Paid-product capabilities outside this build
- backlink index
- keyword/SERP databases
- historical data
- scale
- data freshness
- competitive research
- AI-search data

## Implementation prompt
Build me a daily rank tracker and site auditor for my own domain, the one slice
of Ahrefs worth building solo. Requirements:

- Python + httpx. Keywords live in keywords.txt, up to 50 terms; fetch positions
  from a SERP API (Serper.dev or SerpAPI, key in .env, free tiers cover a small
  list). Do not scrape Google directly, that just gets blocked.
- Store keyword, position, ranking URL, and date in SQLite; run daily from cron.
- A localhost dashboard (Flask + Chart.js): a sparkline per keyword and a
  biggest-movers-this-week list.
- A technical audit crawler for my domain: follow internal links, flag broken
  links, missing or duplicate titles and descriptions, redirect chains, and
  pages under 200 words; write findings to audit.md.
- Pull clicks and impressions per query from the Google Search Console API and
  show them beside the rank data. Warn me in the README: budget an hour for the
  Google Cloud console OAuth setup, it is the painful step.
- Local only, no accounts, no telemetry.
- Out of scope: backlink data, keyword volume and difficulty, and competitor
  research. Do not try to fake these by scraping; that dataset is the
  subscription.
- README: API key setup, the Search Console connection steps, and a plain
  statement that Ahrefs is a proprietary web index and this replaces the
  dashboard, not the data.

## Completion evidence
Demonstrate the prompt's acceptance scenarios against the scoped workflow. Include setup from a clean checkout and failure recovery. Check persistence across restart and export/restore only for the state the prompt says to store; for memory-only tools, confirm that temporary content is discarded as specified. Record actual results and remaining limitations. A detailed plan alone does not establish a working replacement.