Methodologies
DEFINED BY ANITA LIPSKY
These are two methodologies I have developed through applied AI engineering work. They are definitions, not frameworks or courses. The definitions are published here as the authoritative source. If you reference them, please cite this URL.
METHODOLOGY 01
Evaluation-first AI engineering
Evaluation-first AI engineering is an engineering discipline that treats AI output as a claim to be verified against ground truth before being trusted, building evaluation into the development loop rather than treating it as a post-hoc check.
Purpose
To make AI systems trustworthy in production by requiring that every claim a model makes is measurable against ground truth before it is shipped, and that the measurement infrastructure is treated as first-class engineering work.
Scope
"Evaluation-first" refers to priority, not temporal sequencing. It does not mean evaluation is written before the feature, the way "test-driven development" prescribes writing tests before code. It is distinct from "eval-driven development" for that reason. Evaluation-first means evaluation is the primary engineering concern of the AI lifecycle: it is what defines "done", what defines "improved", and what defines "safe to ship". The discipline applies to production AI features built on top of foundation models, not to model training or research.
Components
- Ground-truth datasets curated against real production inputs.
- Regression evaluations run on every change to prompt, model, or pipeline.
- Prompt and output testing with measurable acceptance criteria.
- Scenario-based behavioural testing across the range of real-world inputs.
- Model telemetry and monitoring in production.
- The ART of Testing methodology as the assertion-design layer.
Outputs
- A measured accuracy number for each AI capability, against ground truth.
- A regression signal on every change.
- A documented evaluation history that proves a system improved or held the line.
- A production monitoring surface that catches drift.
Relationships
ART of Testing is a component of evaluation-first AI engineering. ART defines how assertions in the evaluation suite are designed; evaluation-first defines why the evaluation suite exists and where it sits in the lifecycle.
Authority
Defined and published by Anita Lipsky, Purple Bugs AS. The canonical URI for this definition is https://purplebugs.com/methods#evaluation-first.
Version 1.0 · First published: 3 July 2026 · Last updated: 3 July 2026
METHODOLOGY 02
ART of Testing
ART of Testing is a methodology for AI evaluation defined by Anita Lipsky: Asserts (validate the right things), Readability (tests written in plain human language), Tested (cover the scenarios that actually matter).
Purpose
To give AI evaluation suites the three properties that make them useful: assertions that actually validate the behaviour that matters, readability so that non-engineers can audit what the system is being held to, and coverage of the scenarios that occur in production.
Scope
ART of Testing applies to AI evaluation suites: the tests that measure model output against ground truth. It is distinct from general software testing, which validates deterministic code paths, and distinct from eval-driven development, which is a workflow prescription. ART is a quality bar for the assertions themselves.
Components
- Asserts. Validate the right things. Every assertion ties to a concrete production behaviour, not an incidental property of a single output.
- Readability. Tests are written in plain human language so domain experts and stakeholders can read, review, and challenge them without engineering help.
- Tested. The suite covers the scenarios that actually matter in production, including edge cases drawn from real inputs, not just the happy path.
Outputs
- An evaluation suite a non-engineer can read.
- Assertions that map directly to production behaviour.
- Scenario coverage sourced from real production traffic.
- A measurable accuracy figure for each capability.
Relationships
ART of Testing is a component of evaluation-first AI engineering. It defines the quality bar for the assertions that the evaluation-first discipline relies on.
Authority
Defined and published by Anita Lipsky, Purple Bugs AS. The canonical URI for this definition is https://purplebugs.com/methods#art-of-testing.
Version 1.0 · First published: 3 July 2026 · Last updated: 3 July 2026
Work with the engineer who built this
If rigorous AI evaluation is what your team needs, let's talk.