58 lines
3.7 KiB
Markdown
58 lines
3.7 KiB
Markdown
# Task 011: Evidence Matrix And Claim Gate
|
|
|
|
Development description: Build claim extraction, claim-to-evidence mapping, unsupported-claim visibility, insufficient-evidence plan revision, and high-risk unsupported-claim blocking.
|
|
|
|
## Implementation Details
|
|
|
|
- Backend endpoints:
|
|
- `GET /api/articles/{article_id}/evidence`
|
|
- `PATCH /api/articles/{article_id}/evidence/{evidence_id}`
|
|
- Claim review operations under article evidence routes.
|
|
- Evidence matrix maps:
|
|
- Plan sections.
|
|
- Claims.
|
|
- Evidence items.
|
|
- Source quality score.
|
|
- Support status.
|
|
- Risk level.
|
|
- If research cannot find enough acceptable evidence for the approved plan:
|
|
- Move workflow to `PLAN_REVISION_REQUIRED`.
|
|
- Do not allow drafting.
|
|
- Show missing evidence reasons to Editor.
|
|
- High-risk unsupported claims:
|
|
- Are visible in review screens.
|
|
- Block final approval until resolved.
|
|
- Editor can approve/reject evidence and add/remove evidence manually.
|
|
|
|
## Public Interface
|
|
|
|
- Editor views evidence by section and claim.
|
|
- Editor can filter unsupported claims.
|
|
- Workflow refuses to draft if evidence is structurally insufficient.
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [x] TDD pre-requirement: before implementation, write one failing behavior test showing insufficient evidence returns the article to `PLAN_REVISION_REQUIRED`; proceed one claim/evidence behavior at a time and record evidence in `Result`.
|
|
- [x] Every factual claim is mapped to at least one evidence item or marked unsupported.
|
|
- [x] Unsupported claims are visible by section.
|
|
- [x] High-risk unsupported claims block final approval.
|
|
- [x] Insufficient acceptable evidence blocks draft production and triggers plan revision.
|
|
- [x] Editor can approve, reject, add, and remove evidence.
|
|
- [x] Evidence UI supports section and unsupported-claim filters.
|
|
- [x] Evidence item retains URL, title, source type, summary, quality score, retrieval timestamp, and manifest reference.
|
|
|
|
## Verification
|
|
|
|
- Run evidence matrix API tests.
|
|
- Run frontend evidence UI tests.
|
|
- Run a smoke flow with both sufficient and insufficient fake research fixtures.
|
|
|
|
## Result
|
|
|
|
- Status: Accepted.
|
|
- TDD plan: Added `apps/backend/tests/integration/test_evidence_matrix_public_api.py` first, with the primary failing behavior around insufficient evidence forcing `PLAN_REVISION_REQUIRED` and blocking draft/final approval; then extended coverage for sufficient evidence flow and manual evidence add/remove operations.
|
|
- Red evidence: Before fixes, the evidence matrix behavior test failed with empty `evidence`/`claims` because `_build_matrix` was not fully applied in runtime flow (`test_sufficient_evidence_maps_claims_and_allows_draft` and insufficient scenario both failed).
|
|
- Green evidence: Implemented evidence matrix generation from research manifests, unsupported-claim gating, draft/final approval blockers, manual evidence CRUD operations (`POST/PATCH/DELETE` under article evidence routes), claim link updates after evidence removal, and FSD evidence UI with unsupported-only + section filters.
|
|
- Refactor notes: Added `EvidenceCreateRequest` contract and shared type generation; consolidated evidence update flows in `apps/backend/src/application/evidence.py`; ensured article detail response surfaces evidence and claims; updated frontend API client and evidence panel to support add/remove alongside approve/reject.
|
|
- Verification output: `PYTHONPATH=/private/tmp/pupline-backend-deps python3 -m unittest apps/backend/tests/integration/test_evidence_matrix_public_api.py apps/backend/tests/contracts/test_generated_contract_artifacts.py` passed. `node apps/frontend/tests/evidence_matrix.model.test.mjs` passed. `pnpm --filter @pipeline/frontend typecheck` passed. `bash tests/smoke/evidence-flow.sh` passed with Docker Compose after Docker socket escalation.
|