ROI Calibration Interface (Qt Kiosk)¶

This section describes the interactive ROI calibration interface implemented in:

apps/winecellar/frontend/qt_kiosk/pages/pince_roi.py

The page allows the operator to manually define the four perspective corner points of the shelf using the live camera feed.

This defines the reference geometry used later by the vision rectification and cropping pipeline.

Page Layout¶

The ROI page is divided into two panels.

+-----------------------------------------------------+
| LEFT PANEL (controls) | RIGHT PANEL (camera view)   |
|       ~1/3 width      |         ~2/3 width           |
+-----------------------------------------------------+

Left Panel¶

Contains calibration controls:

Set Corner ROI
Delete Last
Delete All
Save ROI-s

Right Panel¶

Displays the live RTSP camera stream and the selected ROI points.

RTSP Camera Feed¶

The page connects to the camera using the RTSP URL defined in:

pince_shelf.env
PINCE_RTSP_URL

The stream is processed in a background thread.

Typical behavior:

~12–15 FPS update rate
automatic reconnection handling
warning if frames temporarily fail

Displayed frames are scaled to the canvas size while preserving aspect ratio.

ROI Selection Workflow¶

The ROI selection is a four-step manual process.

The operator must click the shelf corners in strict order.

Required order¶

1. Top-Left
2. Top-Right
3. Bottom-Right
4. Bottom-Left

The UI explicitly instructs the user:

Tap corners in order:
Top-Left → Top-Right → Bottom-Right → Bottom-Left

Each selected point is displayed as a yellow circle overlay.

Step-by-Step Calibration Procedure¶

Step 1 — Open ROI Page¶

Navigate via the kiosk navigation bar:

ROI

The system displays the live camera stream.

Status message:

ROI page ready.

Step 2 — Freeze the Image¶

Press:

Set Corner ROI

Behavior:

the last live frame is frozen
the camera stream continues internally
point selection becomes enabled

Status message:

ROI mode: tap 4 corners (currently 0/4)

Step 3 — Select Shelf Corners¶

Tap the shelf corners in order.

Each click:

converts widget coordinates → image coordinates
stores the pixel location
displays a yellow circle

Status messages update after every point:

Points: 1/4
Points: 2/4
Points: 3/4
Points: 4/4

When the fourth point is added:

4 points set. You can Save ROI-s.

At this moment:

point selection is automatically disabled
the image remains frozen

Editing ROI Points¶

Two editing buttons are available.

Delete Last¶

Removes the most recently added point.

Example:

Points: 3/4

This is useful when the operator mis-clicks.

Delete All¶

Removes all points.

Result:

Points cleared (0/4)

This allows restarting the selection.

Saving ROI Points¶

Press:

Save ROI-s

Validation rule:

Exactly 4 points must exist

If fewer points exist:

Exactly 4 points are needed to save.

Data Persistence¶

When saving succeeds, the system writes the coordinates to:

shared/config/pince_shelf.ini

Section:

[main_points]

Example:

top_left = 112,64
top_right = 1012,58
bottom_right = 1020,720
bottom_left = 110,735

The points are saved in the same order as the click sequence.

Atomic Configuration Update¶

The configuration file is written using an atomic write process:

temporary file created
configuration written
temporary file replaces original

This guarantees:

no partial file writes
crash safety
valid configuration at all times

Save Confirmation¶

After saving, a confirmation dialog appears:

ROI Saved
ROI corner points were saved successfully.

Button:

Close

When closed:

system navigates automatically to Home page

Session Behavior¶

Important safety behavior:

Leaving the page without saving¶

If the operator navigates away:

ROI data is discarded

Nothing is written to configuration.

Opening the page again¶

The session resets:

points cleared
live stream resumes
selection disabled

Visual Overlay¶

Selected ROI points are displayed as:

yellow circles

Characteristics:

visible on touchscreen
scaled to UI size
drawn over the frozen image

Coordinates are stored in original image pixel space, not screen coordinates.

Coordinate Mapping¶

When the user clicks:

screen position → image pixel coordinate

This mapping accounts for:

aspect ratio scaling
letterboxing margins
canvas resizing

The resulting coordinates always correspond to the original camera frame.

ROI Interaction Flow¶

The ROI calibration interface follows a controlled interaction sequence designed to prevent accidental configuration changes.

flowchart TD

START["Open ROI page"]

LIVE["Live RTSP camera stream"]
SET["Press 'Set Corner ROI'"]

FREEZE["Freeze last frame"]
SELECT["Tap shelf corners"]

P1["Point 1\nTop-Left"]
P2["Point 2\nTop-Right"]
P3["Point 3\nBottom-Right"]
P4["Point 4\nBottom-Left"]

EDIT["Optional edits\nDelete Last / Delete All"]

SAVE["Press 'Save ROI-s'"]

WRITE["Write points to\npince_shelf.ini"]

HOME["Return to Home page"]

START --> LIVE
LIVE --> SET
SET --> FREEZE
FREEZE --> SELECT

SELECT --> P1
P1 --> P2
P2 --> P3
P3 --> P4

P4 --> EDIT
EDIT --> SAVE

SAVE --> WRITE
WRITE --> HOME

Key characteristics:

ROI selection is disabled until "Set Corner ROI" is pressed
exactly four points must be defined
the camera frame remains frozen during selection
saving automatically returns the user to the Home page

ROI Data Flow Into the Vision Pipeline¶

The ROI calibration does not directly perform image processing.
Instead it provides reference geometry that downstream scripts use.

flowchart TD

ROI["ROI Calibration Page"]

INI["pince_shelf.ini\n[main_points]"]

RECTIFY["shelf_rectify.py"]
ROIADJ["ROI_Adjustment_from_PNG.py"]
MASK["generate_mask_for_polc.py"]

RECTIFIED["shelf_rectified.jpg"]
POLYGONS["cells_polygons.json"]
MASKIMG["mask.png"]

CROP["crop_with_json.py"]
DETECT["onnx_batch.py"]

ROI --> INI

INI --> RECTIFY
INI --> ROIADJ

RECTIFY --> RECTIFIED
ROIADJ --> POLYGONS

POLYGONS --> MASK
MASK --> MASKIMG

RECTIFIED --> CROP
POLYGONS --> CROP

CROP --> DETECT

The calibration provides the four perspective corner points, which define:

the shelf perspective transform
the rectified shelf image
ROI polygons used for bin cropping
mask generation for bottle-end detection

Calibration vs Production Separation¶

Calibration and production pipelines are strictly separated.

flowchart LR

CAL["Calibration Mode"]

ROI["ROI Page"]
MASKGEN["Mask Generation"]
CONFIG["pince_shelf.ini"]

PROD["Production Vision"]

SNAP["take_clean_snapshot.py"]
CROP["crop_with_json.py"]
ONNX["onnx_batch.py"]
COMPARE["DB_compare.py"]

ROI --> CONFIG
MASKGEN --> CONFIG

CONFIG --> SNAP
CONFIG --> CROP

SNAP --> CROP
CROP --> ONNX
ONNX --> COMPARE

Important design guarantees:

Calibration updates configuration files only
Production pipeline reads configuration only
Production never recalculates geometry automatically
Calibration is manual and deterministic

Operational Safety¶

The system protects calibration integrity through several safeguards.

Mechanism	Purpose
Manual activation	Prevent accidental calibration
Frozen frame	Prevent moving reference image
Four-point requirement	Enforce valid perspective geometry
Atomic config write	Prevent corrupted configuration
Session reset on page exit	Avoid partial calibration

When Calibration Should Be Performed¶

Calibration is required only in exceptional situations:

camera physically moved
shelf geometry modified
cropping accuracy degraded
large perspective drift observed

Under normal conditions the calibration remains stable for long periods.

Practical Tips for Operators¶

To obtain reliable calibration:

ensure the shelf is fully visible in the camera frame
avoid reflections or strong lighting changes
tap exact shelf corners, not the bottle edges
verify the order of corner selection
if a mistake occurs, use Delete Last instead of restarting

Correct calibration significantly improves:

perspective rectification
bin polygon accuracy
bottle detection reliability

Failure Scenarios¶

No camera frame¶

If RTSP is not available:

No frame yet. Wait for camera stream.

RTSP connection failure¶

Status message:

RTSP open failed (check PINCE_RTSP_URL)

Camera hiccups¶

Temporary read errors produce warnings:

RTSP read hiccup…

Integration with Vision Pipeline¶

The saved corner points are used by:

shelf_rectify.py
ROI_Adjustment_from_PNG.py
generate_mask_for_polc.py

These scripts:

compute the perspective transform
generate rectified shelf images
define ROI polygons for cropping
produce mask images for bottle detection

Safety Design¶

The ROI page is intentionally designed to:

require manual activation
require explicit Save
never overwrite calibration accidentally
isolate calibration from production

Production vision pipelines only read these stored values.

They never recompute them automatically.

Summary¶

The ROI calibration interface allows the operator to:

freeze the live shelf image
manually define four shelf corner points
correct mistakes interactively
persist calibration safely

This calibration step defines the geometric foundation of the entire automated wine detection pipeline.