Table of Contents

Architecture

See also:

This page documents the CapCognition SDK architecture at a conceptual level and explains how the main building blocks connect across General, .NET MAUI, and .NET LTS.

General

Overview

CapCognition follows a pipeline architecture: images/frames enter, one or more processors run recognition, results are delivered to your app, and optional components can draw overlays or stream outputs.

Note

Core concepts (terminology)

  • Source (Capture): produces images/frames (camera, file, RTSP, …)
  • Recognizer: controls the frame detection by the processors and passes back the results
  • Processor: runs recognition on a frame (e.g., barcode scanning, license plate detection)
  • Options: configure processors behavior
  • Overlay: optional drawing layer for previews/debugging

Step-by-step data push frame flow

  1. Your application selects a source (mainly file based sources) and starts capture.
  2. The SDK converts incoming frames into a format usable by processors and passes the image to the application.
  3. The application sends the image to the recognizer
  4. The recognizer passes this image forward to all processors to run recognition on the image.
  5. A result object is produced and delivered back to your app (callbacks/return values depending on the called SDK method).
  6. If enabled, processors also update an overlay (visualization).
flowchart LR
  B[Source] --> A(Recognizer)
  A --> C[[Processor 1]]
  C -.-> F[/Overlay\]
  C --> H[[Processor 2]]
  H --> I{{Result}}
  I --> K(Recognizer)
  H -.-> J[/Overlay\]
  C --> D{{Result}}
  D --> K(Recognizer)
  K --> E[Your App]

Step-by-step data pull frame flow

  1. Your application selects a source (maily camera or stream based sources) and starts capture.
  2. Your application starts continuous recognition (sequential or parallel) on the recognizer and passes 2 callbacks to the recognizer
    • Image grabber callback
    • Result delivery callback
  3. The Recognizer starts pulling the images from your application for forwarding them to the processors
  4. The results are passed back to your application by the result callback
  5. If enabled, processors also update an overlay (visualization).
flowchart LR
  B[Your App]-. Start continuous recognition .-> A(Recognizer)
  A --> C{{Frame Grab CB}}
  C --> |Frame|A
  A --> D[[Processor 1]]
  D --> |Result|A
  A --> E[[Processor 2]]
  E --> |Result|A
  A --> |Results|F{{Your Result Handler}}
Note
  • The Recognizer decides on the number of threads that the processors execute in parallel.
  • The Recognizer ensures that the “Frame Grab CB” is always called sequentially, i.e., one after the other, even if the processors are executed in parallel in multiple threads.
  • The result of “Your Result Handler” is delivered in the context of the processor thread and is not thread-synchronized. It is therefore called in parallel in the respective thread.
  • Never block in “Your Result Handler” otherwise the processor thread will be blocked and will not be able to process any more images.
  • Block the “Frame Grab CB” for as short a time as possible.

Temporal license plate results

The .NET and .NET MAUI LPR packages provide an optional LicensePlateTemporalTracker for applications that need a stable plate result across a vehicle's passage. It sits alongside the application result handling: recognition results are supplied to the tracker, and the application can react to its stable-result events. Recognition, capture, and overlays remain part of the existing pipeline.

flowchart LR
  Source[Frames] --> Recognition[Recognition pipeline]
  Recognition --> Results[Recognition results]
  Results --> App[Application]
  Results -. Optional LPR tracking .-> Tracker[LicensePlateTemporalTracker]
  Tracker --> Stable[Stable plate updates]
  Stable --> App

The tracker modes describe application workflows and the trade-off that matters to each one:

  • ParkingOfficer (parking enforcement): use when an operator handles one vehicle at a time and should keep that vehicle's plate available until explicitly moving on to the next vehicle.
  • Barrier (gates and access control): use when a reliable plate is more valuable than the earliest possible output, for example before deciding whether to open a barrier.
  • CarDetection (slow or free-flow traffic): use when a quick result is useful while vehicles continue moving, such as vehicle counting or traffic observation.

Select the mode that matches how the application encounters vehicles. The tracker reports plate updates for the application to present or act on; applications that do not need stabilized LPR output can continue consuming recognition results directly. See the MAUI license plate sample and the .NET Stream and Recognize sample for integration guidance.

Licensing model

Production use requires a license for each enabled capability. Calling a module's Use() without a license argument selects trial mode for testing and development. Trial mode has no calendar expiry, but runtime restrictions apply.

Integration Initializers Trial behaviour
.NET image recognition BarcodeRecognition.Use, LicensePlateDetection.Use, or YoloModelDetection.Use Recognition operations per application start are limited; restart after the limit is reached.
.NET RTSP capture VideoStreamCapturing.Use, plus each recognition module used Capture/output operations are restricted in trial mode; recognition limits apply separately.
MAUI camera + barcode Capture.Use, Recognition.Use, BarcodeDetection.Use The view displays a Trial watermark. Direct recognizer use has a recognition limit per application start.
MAUI camera + LPR/YOLO Capture.Use, Recognition.Use, plus LicensePlateDetection.Use or YoloModelDetection.Use View watermark and module trial restrictions apply.

Pass the corresponding purchased license string to each initializer for production. Image-only .NET recognition does not require the RTSP capture initializer. Trial operation counts are module/version-specific; do not depend on a fixed count in application logic. See the LPR model license notice before using recognition models.

If you build your own features/processors, you can also integrate licensing for custom capabilities via the CapCognition licensing library.

Examples and reference integrations:

Key takeaways

  • The core architecture is Source → Processor(s) → Result, with optional Overlay and Output.
  • The same mental model applies across .NET MAUI and .NET LTS, but the integration surface differs.

.NET MAUI

Overview

The MAUI SDK supports Android and iOS only. For Windows, Linux, or Raspberry Pi, use the .NET LTS SDK.

The .NET MAUI flavor focuses on in-app camera preview + recognition with a simple XAML/programmatic integration. Two central UI components are:

  • Image capture view (camera preview + frame acquisition)
  • Recognition view (camera preview + recognition processors + result events)
Note

MAUI-specific concepts

  • Views + Handlers: the MAUI view delegates work to platform-specific handlers (Android/iOS).
  • Processor registry: processors are resolved by option type via a registration dictionary.
  • Overlay registry: the platform overlay maintains a list of overlay layers and draws them each frame.

Step-by-step data flow

  1. Your MAUI page hosts a view (e.g., the Recognition view) and requests permissions.
  2. The view opens the camera and starts the preview via the platform handler.
  3. Frames are captured and handed to the recognition pipeline.
  4. The view executes configured processors sequentially or in parallel (when enabled and multiple processors exist).
  5. Results are emitted via events; overlays are updated if overlay drawing options are enabled.
flowchart LR
  UI[MAUI App UI] --> |Start Recognition|RV[Recognition View]
  RV --> Cam[Camera]
  RV -. Start recognition .-> Rec[Recognizer]
  Rec --> |GrabFrameCB|RV
  RV -. Frame .-> Rec
  Rec --> Processors[[Processors]]
  Rec --> |Results|RV
  Processors --> |Results|Overlays[[Overlays]]
  RV -. Draw .-> Overlays
  RV --> |Results|UI

Key takeaways

  • MAUI integration centers around views that manage camera lifecycle and recognition.
  • Overlay drawing is implemented via platform overlays that maintain a layer list and redraw on invalidation.
  • Parallel processing is used in conjunction with the pull-frame method to ensure the best possible data flow with minimal processor power.

.NET LTS

Overview

The .NET LTS flavor targets desktop/server-style applications and provides:

  • High-performance capturing and decoding (including RTSP + H.264 decoding) on supported platforms.
  • A processor-based recognition pipeline similar to MAUI (but without MAUI Views).
  • Optional output sinks, including HLS output support.
Note

.NET LTS building blocks

  • Video sources: webcam / file / RTSP stream sources
  • Processors: recognition processors configured via options
  • Output sinks: optional sinks that publish frames/streams (e.g., HLS)

Step-by-step data flow

  1. Create a capture source for your input (e.g., RTSP URL).
  2. Start capture and feed frames into processing.
  3. Configure and run the processor(s) for recognition.
  4. Emit results back to your application code.
  5. Optionally create an output sink (e.g., HLS) to publish a stream.
  6. If you use HLS output: host the generated files via an HTTP server (the SDK produces the stream artifacts; your app serves them).
flowchart TB
  B[Your App] -. Start streaming .-> Video(Video Source)
  B -. Start continuous recognition .-> A(Recognizer)
  A --> C{{Frame Grab CB}}
  C --> |Get Frame|Video
  Video -. Frame .-> C
  C -. Frame .-> A
  A --> D[[Processor 1]]
  D -..-> |Result|A
  A --> E[[Processor 2]]
  E -..- |Result|A
  A --> |Results|F{{Your Result Handler}}
  F --> |Results|B
  B -. Create .-> HLS(HLS Video Output Sink)
  HLS --> |Produce|Files[(Files)]
  HTTP -. Share .->  Files
  HLS --> |Connect|Video
  Video -. Packets .->  HLS
  B -. Create .-> HTTP(HTTP Host)

Notes on Low-Latency HLS (LL-HLS)

The .NET LTS capturing/recognition stack includes HLS output support (including low-latency HLS). A concrete example can be found in the repository’s command test app:

  • It creates an HLS output sink and starts capture/output.
  • It hosts the generated .m3u8 and segment files via ASP.NET Core static file hosting (including correct MIME types).

Key takeaways

  • .NET LTS is the right fit for Windows/Linux/ARM64 deployments and RTSP-based ingestion.
  • Output sinks enable streaming use-cases; HLS output requires your application to provide the HTTP hosting layer.