feat: complete dashcam catalog ingestion demo

README.md

@@ -8,7 +8,7 @@ The Next.js app lives under `web-front/`. Ingestion code lives under `ingestion/
 
 ```bash
 npm install
-npm run dev
+DASHCAM_CATALOG_EXPORT_DIR="$PWD/data/catalog-exports" npm run dev
 ```
 
 The app runs on [http://localhost:3000](http://localhost:3000).
@@ -29,11 +29,75 @@ Reads are intentionally cached aggressively:
 
 Writes call `revalidateTag` and update the process cache.
 
+### Dashcam catalog ingestion
+
+The catalog ingestion services run in the **same cloud** as the Next.js app
+(one Swarm/Portainer environment), not on a separate server:
+
+```bash
+docker compose -f ingestion/docker-compose.yml up --build
+```
+
+It starts PostgreSQL, an HTTP-only ingestion worker, and a minimal admin server on
+`http://localhost:4101`. The worker writes the validated `manifest.json` +
+`catalog.json` to a shared export volume; there is no HTTP export server and no
+export token — the app reads the artifacts directly from that volume.
+
+The default Yandex Market seed category is:
+
+```text
+https://market.yandex.ru/catalog--avtomobilnye-videoregistratory/82798275/list?hid=82798269
+```
+
+Seed env vars are source-specific: `YANDEX_MARKET_DASHCAM_CATEGORY_URL` is active
+today. `MVIDEO_DASHCAM_CATEGORY_URL` enables the MVideo discovery/parser path
+when configured.
+
+Crawler policy is intentionally conservative: no browser fallback in the regular
+path, no stealth/proxy behavior, PostgreSQL-backed queue and daily budgets, raw
+snapshots stored on the ingestion volume, and source pause on captcha/403/429.
+
+To import an approved export into the app read-model cache, point the import at
+the shared export volume:
+
+```bash
+DASHCAM_CATALOG_EXPORT_DIR=/path/to/ingestion/data/exports \
+npm run catalog:import
+```
+
+The import command reads `manifest.json` + `catalog.json` from that directory,
+validates them with Zod, verifies the artifact `sha256`, checks
+`exportId`/`createdAt`/`itemCount`, and atomically updates `data/videoreg.json`.
+No HTTP call and no token are involved.
+
+The ingestion admin server exposes its own operator UI at
+`http://127.0.0.1:4101/admin`. It is served by `ingestion-admin`, requires Basic
+Auth, and is not part of the Next.js app. In production Traefik should route
+`/admin` to `ingestion-admin` and `/` to Next.js, without stripping the `/admin`
+prefix.
+
+For the production setup where ingestion is co-located with the app and shares
+the export volume, see [`ingestion/README.md`](ingestion/README.md).
+For Portainer stacks, see [`deploy/portainer/README.md`](deploy/portainer/README.md).
+For the local end-to-end demonstration, see [`DEMO.md`](DEMO.md).
+
 ### Deployment
 
 Gitea Actions runs:
 
 - `.gitea/workflows/pr.yml`: quality gate for pull requests into `master`;
-- `.gitea/workflows/master.yml`: quality gate, image build, push, and direct Swarm deploy from `master`.
+- `.gitea/workflows/master.yml`: quality gate, build/push of web and ingestion images, and direct co-located Swarm deploy from `master`.
+
+The production workflow uses the runner's existing Gitea configuration names:
+
+- variables: `REGISTRY`, `IMAGE_REPO`, `DOMAIN` (`videoreg.ru`), `STACK_NAME`,
+  `TRAEFIK_NETWORK`, `TRAEFIK_ENTRYPOINT`, and `TRAEFIK_CERTRESOLVER`;
+- secrets: `REGISTRY_USERNAME`, `REGISTRY_PASSWORD`, `SWARM_MANAGER_HOST`,
+  `SWARM_SSH_USER`, `SWARM_SSH_PORT`, `INGESTION_POSTGRES_PASSWORD`, and
+  `INGESTION_ADMIN_PASSWORD`.
+
+Images are published as `${IMAGE_REPO}-web` and `${IMAGE_REPO}-ingestion`, then
+the Swarm manager pulls their immutable digests. Both `/` and `/admin` receive
+the configured TLS certificate through Traefik.
 
 The Swarm stack defaults to one replica on a manager node because the primitive database is a mounted JSON file. Use a real database before raising replicas above one or spreading the service across nodes.