Skip to content

Operational architecture

wine_platform runs in two operational modes:

  • Mode A — Calibration (manual, Mac): rare, interactive recalibration when camera perspective changes.
  • Mode B — Production (headless, Pi): motion-triggered + daily scheduled vision pipeline.

Mode A — Calibration (manual, Mac)

Goal

Re-establish accurate shelf geometry and cell segmentation when camera perspective changes (e.g. after camera restart, physical movement, or drift).

Calibration updates the reference model used by the production pipeline. Production mode must not modify these references automatically.

Typical steps (operator)

  1. Capture or select a representative shelf snapshot (PNG/JPG).
  2. Manually highlight the shelf structure in blue using the Canva app.
  3. Run ROI adjustment to determine or fine-tune reference corner points.
  4. Generate/update mask(s) for shelf/cells (polc).

Why the manual blue highlighting step exists

Automatic structure detection is intentionally not fully trusted for calibration, because:

  • Camera repositioning may introduce subtle perspective distortion.
  • Lighting conditions may vary.
  • Edge/marker detection can fail under reflections or occlusions.

The manual blue overlay serves as a human-validated structural guide that:

  • Clearly defines shelf boundaries.
  • Reduces ambiguity for ROI detection.
  • Stabilizes downstream mask generation.
  • Ensures deterministic cropping behavior in production mode.

This makes calibration robust and prevents silent geometric drift.

Inputs

  • Shelf snapshot image (PNG/JPG).
  • Manually annotated image with blue structural highlights.
  • Existing configuration (pince_shelf.ini, shared config).

Outputs

  • Updated ROI reference coordinates.
  • Updated shelf/cell mask definitions.
  • Debug images confirming alignment.
  • Persisted reference artifacts used by:
  • crop_with_json.py
  • onnx_batch.py

Architectural boundary

Calibration mode: - May modify reference geometry and masks. - Requires operator supervision. - Is intentionally run on Mac (larger screen, precision).

Production mode: - Must only consume stored references. - Must not auto-adjust geometry. - Must remain deterministic and headless.

Entrypoints

  • workers/wine_inventory/src/pince_shelf/roi/ROI_ADjustment_from_PNG.py
  • workers/wine_inventory/src/pince_shelf/roi/generate_mask_for_polc.py

Failure modes

  • Poor lighting / reflections causing unstable marker/edge detection
  • Wrong snapshot orientation or resolution mismatch
  • Incorrect mask boundaries causing downstream crop errors

Mode B — Production (headless, Raspberry Pi)

Goal

Detect real shelf state (bottle ends) and generate structured results suitable for inventory reconciliation.

Triggers

  • Event trigger: motion detected by Tapo camera (cellar)
  • Scheduled trigger: once per day (health check + drift detection)

Entrypoints (pipeline)

  • Light control:
  • workers/wine_inventory/src/pince_shelf/utils/tapo_manager.py
  • Crop/rectify (using stored references):
  • workers/wine_inventory/src/pince_shelf/vision/crop_with_json.py
  • ONNX inference (bottle-end detection):
  • workers/wine_inventory/src/pince_shelf/vision/onnx_batch.py

Pipeline overview

  1. Switch cellar light ON (Tuya)
  2. Acquire snapshot (from camera capture step or latest saved snapshot)
  3. Rectify shelf perspective and crop shelf/cells using stored ROI references
  4. Run ONNX inference in batch mode (CPUExecutionProvider on Pi)
  5. Write structured outputs + debug artifacts
  6. (Optional, if enabled) call winecellar API to update inventory / movements
  7. Switch cellar light OFF (Tuya)

Data & debug locations (canonical)

  • Snapshots:
  • /data/snapshots/
  • /data/snapshots/tmp/
  • Corner/reference tracking:
  • /data/snapshots/corner_json_history/
  • Runtime debug:
  • /data/cells_runtime/**
  • /data/cells_runtime/**/debug/
  • Proposals/results:
  • /data/proposals/out/
  • /data/proposals/evidence/
  • Models:
  • /data/models/

Failure modes

  • Camera unreachable / RTSP failure / snapshot missing
  • Corner/ROI mismatch causing wrong crop
  • Model/provider mismatch (ONNX runtime errors)
  • Disk full due to debug retention
  • API unreachable (if inventory push is enabled)

Calibration vs Production separation

flowchart LR

  subgraph Calibration["Mode A: Calibration (Manual, Mac)"]
    Snap[Snapshot PNG/JPG]
    Canva[Manual blue highlight <br/> Canva ]
    ROI[ROI Adjustment<br/>ROI_ADjustment_from_PNG.py]
    Mask[Mask Generation<br/>generate_mask_for_polc.py]
    Ref[Reference Geometry<br/>+ Masks]
  end

  subgraph Production["Mode B: Production (Headless, Pi)"]
    Light[Light ON/OFF<br/>tuya_manager.py]
    Crop[Rectify + Crop<br/>crop_with_json.py]
    Infer[ONNX Inference<br/>onnx_batch.py]
    Results[Detection Results<br/>+ Debug Artifacts]
  end

  Snap --> Canva --> ROI --> Mask --> Ref
  Ref --> Crop

  Light --> Crop --> Infer --> Results

  style Calibration fill:#f2f8ff,stroke:#4a90e2
  style Production fill:#f9f9f9,stroke:#555

Responsibilities & boundaries

What production MUST NOT do

  • Recalibrate ROI automatically (only Mode A should change references)
  • Delete reference history silently (must be explicit/retention-controlled)

What production MUST do

  • Be deterministic given stored references + snapshot
  • Emit debug artifacts on failure (so diagnosis is possible headless)