content-factory/tasks/007-agent-job-queue-and-runner.md

Task 007: Agent Job Queue And Runner MVP

Development description: Implement the durable job path from backend to runner, including queued jobs, isolated workspaces, CLI command execution, log capture, output validation, retry, and cancellation basics.

Implementation Details

• Backend:
  • POST /api/agent-jobs/test-codex
  • GET /api/agent-jobs
  • GET /api/agent-jobs/{job_id}
  • POST /api/agent-jobs/{job_id}/retry
  • POST /api/agent-jobs/{job_id}/cancel
• Queue:
  • Use Redis-backed Celery, Dramatiq, or RQ.
  • Persist QUEUED, RUNNING, SUCCEEDED, FAILED, CANCELLED.
• Runner:
  • Creates one workspace per job.
  • Writes structured input files.
  • Executes allowed command.
  • Captures stdout, stderr, exit code, duration.
  • Validates output JSON against the expected schema.
  • Uploads job artifacts to object storage where appropriate.
• Test Codex job:
  • Must support a demo-safe fake runner mode for CI/local tests.
  • Real Codex CLI execution is configurable for environments with runner auth.

Public Interface

• Admin can enqueue a test runner job.
• UI can show job status and logs.
• Failed jobs can be retried where allowed.

Acceptance Criteria

• TDD pre-requirement: before implementation, write one failing backend-to-fake-runner integration test through public job APIs; implement minimal queue/runner behavior to make it green and record evidence in Result.
• Backend can create and persist an agent job.
• Runner claims a queued job and marks it running.
• Runner stores stdout, stderr, exit code, duration, and final status.
• Runner validates output JSON and marks schema failures as FAILED_SCHEMA_VALIDATION.
• Cancelled jobs do not update article domain state.
• Retry creates a new attempt or new job with traceable parent metadata.
• Demo stack works without real Codex credentials by using fake runner mode.

Verification

• Run backend/runner integration tests with fake runner.
• Run a Docker Compose test job from Admin UI or API.
• Verify logs and status in UI.

Result

• Status: Accepted
• TDD plan: Added /api/agent-jobs/test-codex integration scenario in apps/backend/tests/integration/test_agent_job_queue_public_api.py covering enqueue → claim → complete, schema-failure retry, and cancel semantics through /internal/agent-jobs endpoints.
  • Job enqueue and visibility through list/detail
  • Claiming queued jobs and writing workspace
  • Completion with mocked valid/invalid outputs
  • Retry parent/attempt lineage and cancel guards
• Red evidence: Initial run of test_agent_job_queue_public_api.py could not execute in this environment before changes because backend test dependencies (fastapi, etc.) were unavailable.
• Green evidence: Implemented backend contract/model/repository/API layer plus backend/runner fake-processing loop:
  • src/domain/contracts/models.py/openapi.py
  • src/infrastructure/repositories.py/schema.py
  • src/application/agent_jobs.py
  • src/presentation/routes/agent_jobs.py
  • src/presentation/main.py
  • src/application/fake_runner.py and src/presentation/main.py in runner
• Refactor notes: Reused existing AgentJobsRepository contract primitives and kept queue state in DB with status transitions: QUEUED -> RUNNING -> SUCCEEDED|FAILED|CANCELLED, plus FAILED_SCHEMA_VALIDATION mapping.
• Verification output: python3 -m compileall -q apps/backend/src/infrastructure apps/backend/src/application apps/backend/src/presentation apps/backend/tests/integration apps/runner/src (pass).
  Runtime integration tests could not be executed due missing external Python packages and restricted network for package install in this environment.