Knowledge base

Nitpick’s knowledge base (KB) is a local file-system store at ./knowledge-base/. Every run writes to it. Every next run benefits from it. In v2.0+, the KB is also queryable from Claude Code, Cursor, Cline, and any MCP-aware client via nitpick mcp serve.

What’s in it

knowledge-base/
├── page-graph.json          Every discovered page + navigation edges
├── auth/
│   ├── admin.json           Playwright storage state per role
│   └── user.json
├── pages/
│   └── <page_id>/
│       ├── model.latest.json    Current Derived UI Model
│       ├── model.history/       Archived previous versions
│       ├── bugs.json            Deduplicated bug reports
│       ├── flaky-tests.json     Flaky registry
│       ├── resolved-questions.json  Persistent Phase 3.5 resolutions
│       └── baselines/           Visual regression baselines (future)
└── flows/
    └── <flow_name>/
        ├── narrative.yaml
        └── last-result.json

Why it matters

Without KB	With KB
Re-crawl every run	Page graph cached; crawl once
Re-diagnose flaky tests every run	Known flaky → fast-track retry
Re-file the same bug every run	Dedupe on hash, update timestamps
No sense of “what changed”	Diff new model vs archived previous
Surprise when pages change	Diff alerts you immediately

MCP access (v2.0+)

Run nitpick mcp serve to expose the KB over the Model Context Protocol. Any MCP-aware client (Claude Code, Claude Desktop, Cursor, Cline) can then query it directly — no agent runtime needed.

nitpick mcp serve --kb ./knowledge-base --trust read-write

Tools available to MCP clients:

Tool	Description	Trust required
kb_list_pages	List all discovered pages	read
kb_get_page	Get the Derived UI Model for a page	read
kb_search	Full-text search across the KB	read
kb_list_open_bugs	List unfixed bugs	read
kb_get_flow	Get a multi-role flow definition	read
kb_add_bug	Record a new bug	read-write
kb_note_flaky	Mark a test as flaky	read-write
kb_put_resolved_question	Persist a Phase 3.5 resolution	read-write

Trust tiers are set in nitpick.yaml:

mcp:
  kb:
    trust: read-write   # read-write | read-only | deny

See MCP KB server for setup instructions per client.

The Derived UI Model

The single most important artifact. For each page it records headings, fields (with validations and selectors), actions, conditional logic, API contracts, terminal actions, and user-added business rules plus metadata for versioning.

This model is the single source of truth for test generation. Playwright tests are generated only from it.

Model versioning

Every time a model is saved, the previous version is archived to model.history/<timestamp>-v<N>.json. The diff engine compares the two and surfaces changes:

• Fields added / removed
• Required ↔ optional
• Validation rules changed
• Selector drift
• New conditionals
• API contract changes

These changes feed into the unified report’s “Changes since last run” section.

Flaky registry

A test that passes on retry without code changes is flaky. Nitpick tracks this across runs. Next run: if this test fails, Senior QA applies the last successful classification immediately. If it passes 3 consecutive runs stable, it’s removed from the registry.

Bug dedupe

Bugs are hashed on (page_id, test_name, error_signature). First occurrence creates a new bug entry. Subsequent occurrences update last_seen. A future release will add a bug filer that creates a Linear/GitHub ticket on first occurrence and re-comments on recurrences.

Resetting the KB

To start fresh:

rm -rf knowledge-base/

Your next run will re-crawl and re-test from scratch. Useful after major app refactors or when testing a new staging environment.

Portability

The KB is plain JSON files. You can commit it to git (careful with auth state — auth/ contains session tokens), copy between machines, backup and restore, or inspect with jq.

# Inspect the page graph
jq '.pages | keys' knowledge-base/page-graph.json

# See all open bugs
jq '.' knowledge-base/pages/*/bugs.json | grep -v '"fixed": true'