content-factory/tasks/011-evidence-matrix-and-claim-gate.md

Task 011: Evidence Matrix And Claim Gate

Development description: Build claim extraction, claim-to-evidence mapping, unsupported-claim visibility, insufficient-evidence plan revision, and high-risk unsupported-claim blocking.

Implementation Details

• Backend endpoints:
  • GET /api/articles/{article_id}/evidence
  • PATCH /api/articles/{article_id}/evidence/{evidence_id}
  • Claim review operations under article evidence routes.
• Evidence matrix maps:
  • Plan sections.
  • Claims.
  • Evidence items.
  • Source quality score.
  • Support status.
  • Risk level.
• If research cannot find enough acceptable evidence for the approved plan:
  • Move workflow to PLAN_REVISION_REQUIRED.
  • Do not allow drafting.
  • Show missing evidence reasons to Editor.
• High-risk unsupported claims:
  • Are visible in review screens.
  • Block final approval until resolved.
• Editor can approve/reject evidence and add/remove evidence manually.

Public Interface

• Editor views evidence by section and claim.
• Editor can filter unsupported claims.
• Workflow refuses to draft if evidence is structurally insufficient.

Acceptance Criteria

• TDD pre-requirement: before implementation, write one failing behavior test showing insufficient evidence returns the article to PLAN_REVISION_REQUIRED; proceed one claim/evidence behavior at a time and record evidence in Result.
• Every factual claim is mapped to at least one evidence item or marked unsupported.
• Unsupported claims are visible by section.
• High-risk unsupported claims block final approval.
• Insufficient acceptable evidence blocks draft production and triggers plan revision.
• Editor can approve, reject, add, and remove evidence.
• Evidence UI supports section and unsupported-claim filters.
• Evidence item retains URL, title, source type, summary, quality score, retrieval timestamp, and manifest reference.

Verification

• Run evidence matrix API tests.
• Run frontend evidence UI tests.
• Run a smoke flow with both sufficient and insufficient fake research fixtures.

Result

• Status: Accepted.
• TDD plan: Added apps/backend/tests/integration/test_evidence_matrix_public_api.py first, with the primary failing behavior around insufficient evidence forcing PLAN_REVISION_REQUIRED and blocking draft/final approval; then extended coverage for sufficient evidence flow and manual evidence add/remove operations.
• Red evidence: Before fixes, the evidence matrix behavior test failed with empty evidence/claims because _build_matrix was not fully applied in runtime flow (test_sufficient_evidence_maps_claims_and_allows_draft and insufficient scenario both failed).
• Green evidence: Implemented evidence matrix generation from research manifests, unsupported-claim gating, draft/final approval blockers, manual evidence CRUD operations (POST/PATCH/DELETE under article evidence routes), claim link updates after evidence removal, and FSD evidence UI with unsupported-only + section filters.
• Refactor notes: Added EvidenceCreateRequest contract and shared type generation; consolidated evidence update flows in apps/backend/src/application/evidence.py; ensured article detail response surfaces evidence and claims; updated frontend API client and evidence panel to support add/remove alongside approve/reject.
• Verification output: PYTHONPATH=/private/tmp/pupline-backend-deps python3 -m unittest apps/backend/tests/integration/test_evidence_matrix_public_api.py apps/backend/tests/contracts/test_generated_contract_artifacts.py passed. node apps/frontend/tests/evidence_matrix.model.test.mjs passed. pnpm --filter @pipeline/frontend typecheck passed. bash tests/smoke/evidence-flow.sh passed with Docker Compose after Docker socket escalation.