Skip to content

Viewport

Jump to bottom
Chris Chronopoulos edited this page Aug 12, 2023 · 1 revision

The Viewport is a graphical display widget with extended functionality, used in several features of Parallax. The Main Window has two Viewports by default; here they are showing two camera feeds during an experiment:

view_ports

It is also used in the Checkerboard Tools (mono and stereo) and the Generate Template Tool.

The Viewport can display images and video from a variety of sources, but most commonly it is used to display a video feed from a camera. To select a camera feed to view, do:

Right-click > Parallax > Cameras

And select your camera by name and serial number. If no supported cameras are connected, then you will only see one option: MockCamera, which displays white noise for the purpose of experimenting with the GUI (and also debugging).

Mouse wheel up/down over the Viewport will zoom in and out. There is no limit to the zoom scale, and you can easily zoom in to the scale of individual pixels:

pixels

To reset the zoom to full-frame, middle-click over the Viewport. Clicking and dragging will pan the view.

Left-clicking will place a Pixel Cursor over the image:

cursor

The Pixel Cursor defines the pixel coordinates used in calibrations and triangulations.

Focus Controllers

The Modular Insertion System features a remote focusing system controlled by the open-source mis-focus-controller. This provides a USB interface which can be controlled by Parallax. To select a controller / channel for a particular camera, right-click the Viewport through which that camera is streaming and select:

Right-click > Parallax > Focus Controllers > (Controller Serial # and Channel)

Then, focus/defocus for this camera is controlled by Control + mouse wheel over the Viewport, and Control + Shift + mouse wheel enables fine-tuned control of the focus.

Filters

The video feed in the Viewport can be processed with a video filter. To enable one, navigate to:

Right-click > Parallax > Filters > (select filter)

Filters are image processing functions which convert each frame in the stream into a new frame, typically either a processed version of the frame, or an overlay. Parallax currently supports 4 different filters:

  • Brightness and Contrast: Software processing of image brightness and contrast
  • Difference: Each output frame is the absolute difference between the last two input frames. Useful for basic background subtraction while detecting motion
  • Checkerboard (fast): This uses OpenCV's findChessboardCorners to quickly detect checkerboard corners in each frame. Good for debugging, or moving checkerboards (?), but we recommend using the smoothed version below for accuracy.
  • Checkerboard (smooth): Uses a larger convolution window and a 5-frame boxcar filter to achieve more accurate corner detection with less jitter. Much better for camera calibrations - one drawback is that the 5-frame boxcar introduces a ~1 second latency between pose updates and the settling of the corners.

An example of the Checkerboard detector (smooth version) is shown below:

image

Detectors

Detectors are similar to filters in that they operate in real-time on a series of video frames, but instead of generating a new frame as output, the output is the pixel coordinate of a detected object, namely the probe tip. A detector on a Viewport will automatically update the position of the Pixel Cursor so that it can be used in a calibration procedure, or in triangulation.

To select a probe detector for the Viewport, do:

Right-click > Parallax > Detectors > (select detector)

Currently, 2 different probe detectors are supported in Parallax:

  • Template Match - This implements OpenCV's matchTemplate to find the most likely match with a given template. Selecting the template file from the dialog window that pops up after selecting. To generate a template, use the Generate Template Tool.
  • SLEAP - SLEAP is a machine-learning-based object detector and pose estimator. If you have a SLEAP model trained on your image data, you can load it into this detector and track objects in real-time through the Viewport. See here for more details on using SLEAP within Parallax.

PyQtGraph features

Since the Viewport is based on PyQtGraph's GraphcisView, it inherits many of its great features, and some of these are exposed through the right-click menu of the Viewport:

  • View all - reset the zoom (same as middle-click)
  • X-axis, Y-axis: set the extent of the viewing window manually
  • Export: Save the current frame as a PNG file.
Clone this wiki locally