## videoreg.ru

Small video registry using TypeScript, a Next.js web frontend, App Router, FSD-oriented folders, and a primitive file-backed JSON database.

The Next.js app lives under `web-front/`. Ingestion code lives under `ingestion/`.

### Local development

```bash
npm install
DASHCAM_CATALOG_EXPORT_DIR="$PWD/data/catalog-exports" npm run dev
```

The app runs on [http://localhost:3000](http://localhost:3000).

### Data storage

By default records are stored in `data/videoreg.json`. Override the path with:

```bash
VIDEOREG_DB_FILE=/absolute/path/videoreg.json npm run dev
```

Reads are intentionally cached aggressively:

- process memory cache for the primitive JSON database;
- Next.js cache tag for dashboard/API reads;
- long HTTP cache headers on the read API.

Writes call `revalidateTag` and update the process cache.

### Dashcam catalog ingestion

The catalog ingestion services run in the **same cloud** as the Next.js app
(one Swarm/Portainer environment), not on a separate server:

```bash
docker compose -f ingestion/docker-compose.yml up --build
```

It starts PostgreSQL, an HTTP-only ingestion worker, and a minimal admin server on
`http://localhost:4101`. The worker writes the validated `manifest.json` +
`catalog.json` to a shared export volume; there is no HTTP export server and no
export token — the app reads the artifacts directly from that volume.

The default Yandex Market seed category is:

```text
https://market.yandex.ru/catalog--avtomobilnye-videoregistratory/82798275/list?hid=82798269
```

Seed env vars are source-specific: `YANDEX_MARKET_DASHCAM_CATEGORY_URL` is active
today. `MVIDEO_DASHCAM_CATEGORY_URL` enables the MVideo discovery/parser path
when configured.

Crawler policy is intentionally conservative: no browser fallback in the regular
path, no stealth/proxy behavior, PostgreSQL-backed queue and daily budgets, raw
snapshots stored on the ingestion volume, and source pause on captcha/403/429.

To import an approved export into the app read-model cache, point the import at
the shared export volume:

```bash
DASHCAM_CATALOG_EXPORT_DIR=/path/to/ingestion/data/exports \
npm run catalog:import
```

The import command reads `manifest.json` + `catalog.json` from that directory,
validates them with Zod, verifies the artifact `sha256`, checks
`exportId`/`createdAt`/`itemCount`, and atomically updates `data/videoreg.json`.
No HTTP call and no token are involved.

The ingestion admin server exposes its own operator UI at
`http://127.0.0.1:4101/admin`. It is served by `ingestion-admin`, requires Basic
Auth, and is not part of the Next.js app. In production Traefik should route
`/admin` to `ingestion-admin` and `/` to Next.js, without stripping the `/admin`
prefix.

For the production setup where ingestion is co-located with the app and shares
the export volume, see [`ingestion/README.md`](ingestion/README.md).
For Portainer stacks, see [`deploy/portainer/README.md`](deploy/portainer/README.md).
For the local end-to-end demonstration, see [`DEMO.md`](DEMO.md).

### Deployment

Gitea Actions runs:

- `.gitea/workflows/pr.yml`: quality gate for pull requests into `master`;
- `.gitea/workflows/master.yml`: quality gate, build/push of web and ingestion images, and direct co-located Swarm deploy from `master`.

The production workflow uses the runner's existing Gitea configuration names:

- variables: `REGISTRY`, `IMAGE_REPO`, `DOMAIN` (`videoreg.ru`), `STACK_NAME`,
  `TRAEFIK_NETWORK`, `TRAEFIK_ENTRYPOINT`, and `TRAEFIK_CERTRESOLVER`;
- secrets: `REGISTRY_USERNAME`, `REGISTRY_PASSWORD`, `SWARM_MANAGER_HOST`,
  `SWARM_SSH_USER`, `SWARM_SSH_PORT`, `INGESTION_POSTGRES_PASSWORD`, and
  `INGESTION_ADMIN_PASSWORD`.

Images are published as `${IMAGE_REPO}-web` and `${IMAGE_REPO}-ingestion`, then
the Swarm manager pulls their immutable digests. Both `/` and `/admin` receive
the configured TLS certificate through Traefik.

The Swarm stack defaults to one replica on a manager node because the primitive database is a mounted JSON file. Use a real database before raising replicas above one or spreading the service across nodes.