Task 001: dockerized monorepo foundation

2026-05-21 16:57:58 +03:00
commit d42f0c4392
83 changed files with 4236 additions and 0 deletions
@@ -0,0 +1,61 @@
+# Task 010: Research Fetch And S3 Manifest
+
+Development description: Implement explicit auditable web research with source retrieval metadata, source-section snapshots, object-storage uploads, and Postgres manifest records.
+
+## Implementation Details
+
+- Backend endpoint:
+  - `POST /api/articles/{article_id}/research/start`
+  - `GET /api/articles/{article_id}/research`
+- Research starts only after approved plan.
+- Research uses an explicit search/fetch script, not only free-form agent browsing.
+- For discovered sources, store in object storage:
+  - Relevant content sections.
+  - Page metadata.
+  - Search path to the article.
+  - Agent/source-selection criteria.
+  - Page `<head>` metadata.
+  - Server IP address.
+  - Domain WHOIS owner when available.
+- In Postgres store only:
+  - Research run manifest.
+  - S3/object keys.
+  - Source URLs.
+  - Content hashes.
+  - Artifact types.
+  - Summary metadata required for UI and validation.
+- Retention is indefinite for v1.
+- Provide deterministic fake search/fetch fixtures for tests and demo.
+
+## Public Interface
+
+- Editor starts research for an approved article plan.
+- UI shows research run status and discovered source summary.
+- Backend exposes manifest-linked evidence summaries.
+
+## Acceptance Criteria
+
+- [ ] TDD pre-requirement: before implementation, write one failing behavior test that starts research from an approved plan and verifies a manifest with object keys is produced; proceed one research behavior at a time and record evidence in `Result`.
+- [ ] Research cannot start before plan approval.
+- [ ] Research run creates an agent/job record and article status `RESEARCH_RUNNING`.
+- [ ] Source-section artifacts are uploaded to object storage.
+- [ ] Manifest records include object keys, hashes, URLs, artifact types, and metadata references.
+- [ ] Re-running research creates a new manifest instead of overwriting old artifacts.
+- [ ] UI can show source URL, title, domain, type, summary, retrieval timestamp, and artifact link.
+- [ ] Demo stack can run with deterministic fake research results.
+
+## Verification
+
+- Run research integration tests with fake search/fetch.
+- Verify object storage contains source-section artifact files.
+- Verify Postgres manifest records point to those files.
+- Run Docker Compose research smoke flow.
+
+## Result
+
+- Status: Pending execution.
+- TDD plan: To be filled during execution.
+- Red evidence: To be filled during execution.
+- Green evidence: To be filled during execution.
+- Refactor notes: To be filled during execution.
+- Verification output: To be filled during execution.