Files
huncode 2ce58fcbc0
Master Production Deploy / production (push) Failing after 49s
feat: complete dashcam catalog ingestion demo
2026-07-31 02:01:58 +03:00

2.2 KiB

Demo runbook

The demo uses local PostgreSQL plus the same shared export contract as production. It never writes source prices, sellers, availability, or reviews to the public catalog.

1. Start local ingestion

cp ingestion/.env.example ingestion/.env

Set real local passwords in ingestion/.env, then use five product requests per source for the demonstration:

INGESTION_PRODUCT_DAILY_BUDGET=5 \
INGESTION_FETCH_DELAY_MIN_MS=5000 \
INGESTION_FETCH_DELAY_MAX_MS=5000 \
docker compose --env-file ingestion/.env -f ingestion/docker-compose.yml up -d --build

Follow the worker until the category and product jobs settle:

docker compose --env-file ingestion/.env -f ingestion/docker-compose.yml logs -f ingestion-worker

For a five-card demo, the worker fetches five product cards and marks the rest of the discovered category candidates as rate_limited for that budget day. If MVideo is configured and replies with a CAPTCHA/403/429, it remains paused and visible in the queue; do not retry it through a browser or proxy.

2. Moderate and export

Open http://127.0.0.1:4101/admin, authenticate with the values from ingestion/.env, choose Products, inspect the source URLs, raw technical characteristics, normalized values, and any conflicts. Edit the canonical model or canonical name where needed, approve the cards intended for the demo, then publish the approved export:

docker compose --env-file ingestion/.env -f ingestion/docker-compose.yml run --rm ingestion-worker \
  npm run ingestion:export-catalog

The artifacts appear under data/catalog-exports/:

  • manifest.json
  • artifacts/<exportId>/catalog.json

3. Start the web catalog

DASHCAM_CATALOG_EXPORT_DIR="$PWD/data/catalog-exports" npm run dev

Open http://localhost:3000. The root page reads and validates the shared catalog at request time, displays approved cards, normalized specs, and links back to every source card.

4. Final checks

npm test
npm run typecheck
npm run build
curl -fsS http://localhost:3000/health/ready
curl -u "$INGESTION_ADMIN_USERNAME:$INGESTION_ADMIN_PASSWORD" \
  http://127.0.0.1:4101/admin/api/queue