Table of Contents

Stream and Recognize (.NET LTS)

This sample shows how to:

  1. Connect to an RTSP stream.
  2. Start sequential detection of QR codes and LPs -> Detect 20 frames
  3. Start parallel detection of QR codes and LPs
    • For demonstration:
      • Throttle Barcode detection
      • Terminate barcode processing after 20 frame requests (not 20 successful recognitions)
  4. Stop the parallel phase after the supplied duration and clean up resources. The sequential phase runs first, with a 200-second timeout.

Prerequisites

Applies to CapCognition SDK 2.2.x, .NET 10, on Windows x64, Linux x64, or Linux ARM64 (Raspberry Pi). Use a reachable RTSP camera and compatible 2.2.x packages. Deploy the codec libraries and the three 640n LPR model files in Assets for offline use. The local-model helpers in this sample require the files to exist; missing models cause an error. Codec initialization can download missing native libraries and therefore require internet access. See licensing for trial and production initialization.

NuGet setup

Install the current modular packages:

dotnet add package CapCognition.Net
dotnet add package CapCognition.Net.BarcodeScanning
dotnet add package CapCognition.Net.LicensePlateDetection

Initialize CapCognition once at application startup before calling StreamAndRecognize:

using CapCognition.Net.BarcodeScanning.Processor;
using CapCognition.Net.CaptureSources;
using CapCognition.Net.Core.Capture;
using CapCognition.Net.LicensePlateDetection.Processor;
using Microsoft.Extensions.Logging;

using var loggerFactory = LoggerFactory.Create(builder => builder.SetMinimumLevel(LogLevel.Information));
CapCognition.Net.Core.CapCognition.Initialize(loggerFactory,
[
    VideoStreamCapturing.Use(/* Add your capture license here */),
    BarcodeRecognition.Use(/* Add your barcode license here */),
    LicensePlateDetection.Use(/* Add your LPR license here */),
]);
CaptureControl.Instance.InitializeAsync().GetAwaiter().GetResult();

Example code

The method below uses the current package APIs. Include these namespaces in the sample project:

using CapCognition.Net;
using CapCognition.Net.BarcodeScanning.Common;
using CapCognition.Net.BarcodeScanning.Processor;
using CapCognition.Net.CaptureSources;
using CapCognition.Net.CaptureSources.VideoStream;
using CapCognition.Net.Core.Capture;
using CapCognition.Net.Core.Processing;
using CapCognition.Net.Core.Processing.Common;
using CapCognition.Net.LicensePlateDetection.CapCognitionLicensePlateTracking;
using CapCognition.Net.LicensePlateDetection.Common;
using CapCognition.Net.LicensePlateDetection.Common.Extensions;
using static CapCognition.Net.LicensePlateDetection.Common.LicensePlateDetectionRecognitionOptions;
using CapCognition.Net.LicensePlateDetection.Processor;
using SkiaSharp;

private static void StreamAndRecognize(Uri url, TimeSpan duration, bool useTemporalTracker = false)
{
    //Optional: enable streaming log for connection debugging
    CapCognition.Net.Core.CapCognition.EnableStreamingLogs = true;

    //assuming url is valid rtsp url with username and password if needed (rtsp://<username>:<password>@<ip>:<port>)
    var captureSourceOptions = new VideoStreamCaptureSourceOptions(url, null, null);

    captureSourceOptions.AdditionalStreamingOptionsToSet = new Dictionary<string, string>();
    captureSourceOptions.AdditionalStreamingOptionsToSet["timeout"] = "4000000";
    captureSourceOptions.AdditionalStreamingOptionsToSet["user_agent"] = "LibVLC/3.0.16 (LIVE555 Streaming Media v2021.04.09)";

    var captureSource = CaptureControl.Instance.CreateVideoCaptureSource(captureSourceOptions);
    if (captureSource == null)
    {
        Console.WriteLine("Failed to create video capture source");
        return;
    }

    try
    {
        var success = captureSource.StartCapture();
        if (!success)
        {
            Console.WriteLine("Failed to start capture");
            return;
        }

        var videoCaptureSource = captureSource.ToVideoStreamCaptureSource();

        var frameSize = videoCaptureSource.SourceFrameSize;
        if (frameSize == null)
        {
            Console.WriteLine("Failed to get source frame size");
            return;
        }

        var options = new RecognitionOptionBuilder()
            .AddBarcodeRecognitionOption()
                .SetBinarizer(BarcodeRecognitionOptions.BinarizerType.AdaptiveHybridBinarizer)
                .SetBarcodeFormats([
                    BarcodeRecognitionOptions.BarcodeFormat.QRCode
                ])
                .UseFastQRFinder()
                .UseStandardQRDecoding()
                .Done()
            .AddLicensePlateDetectionRecognitionOption()
                .UseCroppedImageForRecognition()
                .DoAutomaticDetectionOptimization()
                .UseLocalModelsForRecognitionMode(RecognitionModeType.CountryLicencePlateThenVehicle)
                    .SetLocalModelBasePath("Assets")
                    .UseLocalLicensePlateModelFromType(LPModelType.LPModelType640n)
                    .UseLocalLicensePlateTextModelFromType(LPTextModelType.LPTextModelType640n)
                    .UseLocalVehicleModelFromType(VehicleModelType.VehicleModelType640n)
                    .Done()
                .ConfigureOption(option =>
                {
                    option.ClearOldModelsFromCacheFolder();
                })
                .Done()
            .Build();

        try
        {
            var resultDrawingOptions = new ResultDrawingOptionsBuilder()
                .AddBarcodeDrawingOptions()
                    .DrawBoundingBox()
                    .BoundingBoxColor(SKColors.Blue)
                    .BoundingBoxStyle(SKPaintStyle.Stroke)
                    .BoundingBoxStrokeWidth(3f)
                    .Done()
                .AddLicensePlateDetectionDrawingOptions()
                    .DisplayVehicleSurroundingBox()
                    .VehicleSurroundingRectColor(new SKColor(0, 255, 0, 40))
                    .VehicleSurroundingRectStyle(SKPaintStyle.Fill)
                    .VehicleSurroundingRectStrokeWidth(1f)
                    .LicensePlateSurroundingRectColor(SKColors.Green)
                    .LicensePlateSurroundingRectStyle(SKPaintStyle.Stroke)
                    .LicensePlateSurroundingRectStrokeWidth(2f)
                    .LicensePlateNonValidatedSurroundingRectColor(SKColors.Red)
                    .Done()
                .Build();

            using var procFinishedSem = new SemaphoreSlim(0);
            using var tracker = useTemporalTracker
                ? new LicensePlateTemporalTracker(LicensePlateTrackingMode.CarDetection)
                : null;
            using var recognizer = new Recognizer();
            success = recognizer.PrepareProcessorsAsync(frameSize.Value.Width, frameSize.Value.Height, options).GetAwaiter().GetResult();
            if (!success)
            {
                Console.WriteLine("Failed to prepare streaming recognition processors");
                return;
            }

            //optional: disable streaming log not to get flooded with packet messages
            CapCognition.Net.Core.CapCognition.EnableStreamingLogs = false;
            Console.WriteLine("Start sequential pull-frame based processing");
            Console.WriteLine("--------------------------------------------");

            const int noOfSequentialFramesToProcess = 20;
            if (tracker != null)
            {
                tracker.ResultChanged += OnStablePlateChanged;
            }

            GraphicProcessorControl.ResultDeliveryCallback resultDeliveryCallback;
            if (tracker != null)
            {
                // Temporal tracking needs every frame's result, including frames with no detection.
                resultDeliveryCallback = tracker.CreateCallback();
            }
            else
            {
                resultDeliveryCallback = (bmp, result) =>
                {
                    if (result.Results.Count == 0)
                    {
                        Console.WriteLine("No elements could be recognized");
                        return;
                    }
                    Console.WriteLine($"Got results: {result}");
                };
            }

            var started = recognizer.StartContinuousSequentialImageProcessing(
                grabImageCB: (ref result, cancellationToken) =>
                {
                    var bitmap = videoCaptureSource.GetBitmap();
                    result.Bitmap = bitmap;

                    if (result.FrameNo >= noOfSequentialFramesToProcess)
                    {
                        procFinishedSem.Release();
                    }
                },
                resultDeliveryCB: resultDeliveryCallback,
                alwaysDeliverResults: tracker != null,
                resultDrawingOptions: resultDrawingOptions);

            if (!started)
            {
                Console.WriteLine("Failed to start continuous sequential image processing");
                return;
            }

            var completed = procFinishedSem.Wait(TimeSpan.FromSeconds(200));
            recognizer.StopContinuousImageProcessing();
            if (!completed)
            {
                Console.WriteLine("Sequential processing timed out");
                return;
            }

            Console.WriteLine("Start parallel pull-frame based processing");
            Console.WriteLine("---------------------------------------");

            int noOfActiveProcessors = options.Count();
            var lastFrameNos = new long[noOfActiveProcessors];

            started = recognizer.StartContinuousParallelImageProcessing(
                grabImageCB: (ref result, cancellationToken) =>
                {
                    var bitmap = videoCaptureSource.GetBitmap();
                    result.Bitmap = bitmap;

                    lastFrameNos[result.ProcessorIndex] = result.FrameNo;

                    if (noOfActiveProcessors == 2 && result.ProcessorIndex == 0)
                    {
                        //throttle processor 0 (barcode) to demonstrate parallel processing with different speeds
                        result.DelayBeforeNextRequestInMilliseconds = 200;

                        //Show termination of a single processor without interrupting the other processors -> Barcode detection ends after 20 frame requests, LPR detection goes on
                        if (result.FrameNo >= noOfSequentialFramesToProcess)
                        {
                            Console.WriteLine($"Processor {result.ProcessorIndex} finished processing cause {noOfSequentialFramesToProcess} frames processed");
                            result.ContinueRequest = GraphicProcessorControl.ImageGrabResult.ContinuousRequest.StopProcessing;
                        }
                    }
                },
                resultDeliveryCB: (bmp, result) =>
                {
                    if (result.Results.Count == 0)
                    {
                        Console.WriteLine("No elements could be recognized");
                        return;
                    }
                    Console.WriteLine($"Got results: {result.ToString()}");
                },
                resultDrawingOptions: resultDrawingOptions);

            if (!started)
            {
                Console.WriteLine("Failed to start continuous parallel image processing");
                return;
            }

            Task.Delay(duration).GetAwaiter().GetResult();
            recognizer.StopContinuousImageProcessing();

            //output the number of processed frames per processor
            for (int procIdx = 0; procIdx < noOfActiveProcessors; procIdx++)
            {
                Console.WriteLine($"Last processed frame for processor {procIdx}: {lastFrameNos[procIdx]}");
            }
        }
        finally
        {
            options.Dispose();
        }
    }
    finally
    {
        try
        {
            captureSource.StopCapture();
        }
        finally
        {
            CaptureControl.Instance.RemoveCaptureSource(captureSource);
        }
    }
}

private static void OnStablePlateChanged(LicensePlateTemporalTracker tracker, StableLicensePlateResult result)
{
    using (result)
    {
        Console.WriteLine($"Stable plate update: {result}");
    }
}

Step-by-step explanation

  1. Build capture options

    • VideoStreamCaptureSourceOptions is created for the RTSP url.
    • AdditionalStreamingOptionsToSet passes low-level FFmpeg/stream options (e.g., timeout, user_agent).

  2. Create and start the capture source

    • CaptureControl.Instance.CreateVideoCaptureSource(...) creates the capture source.
    • StartCapture() starts the pipeline.
    • On failure, the code prints an error, stops capture (if needed), and releases the source via RemoveCaptureSource(...).

  3. Get a stream-specific source and frame size

    • captureSource.ToVideoStreamCaptureSource() provides stream-specific APIs, including pulling frames via GetBitmap().
    • SourceFrameSize is required to prepare the recognizer for the correct input resolution.

  4. Configure recognition options (barcode + license plate detection)

    • RecognitionOptionBuilder() composes a multi-processor pipeline.
    • Barcode recognition is restricted to QR codes and uses UseFastQRFinder() + UseStandardQRDecoding().
    • License plate detection is configured to auto-optimize and run in “country plate then vehicle” mode.
    • This sample requires the selected model files in Assets. Local-model helpers do not fall back to downloads. Offline use also requires native codec libraries to be deployed.

  5. Configure drawing/overlay options (optional)

    • ResultDrawingOptionsBuilder() enables drawing bounding boxes for barcodes and license plates.

  6. Prepare processors for the incoming stream size

    • recognizer.PrepareProcessorsAsync(width, height, options) allocates and initializes internal processor resources.

  7. Run sequential continuous processing (pull-frame)

    • StartContinuousSequentialImageProcessing(...) pulls frames through grabImageCB (GetBitmap()) and delivers results via resultDeliveryCB.
    • The sample processes a fixed number of frames (noOfSequentialFramesToProcess = 20) and signals completion via SemaphoreSlim, then calls StopContinuousImageProcessing().

  8. Run parallel continuous processing (pull-frame) with per-processor control

    • StartContinuousParallelImageProcessing(...) runs multiple processors in parallel (one per recognition option).
    • The sample throttles processor 0 (barcode) using result.DelayBeforeNextRequestInMilliseconds = 200.
    • It also stops only the barcode processor after 20 frames via result.ContinueRequest = ...StopProcessing, while the other processor(s) continue.
    • lastFrameNos[] tracks the last processed frame per processor.

  9. Stop after a fixed duration and clean up

    • The parallel phase waits for the supplied duration (Task.Delay(...)), then stops continuous processing. The using declaration disposes the recognizer on every exit path.
    • The finally block stops capture (videoCaptureSource.StopCapture()) and removes the capture source (CaptureControl.Instance.RemoveCaptureSource(...)).

Calling the sample

Put the methods in your application class. After initialization, call:

StreamAndRecognize(new Uri("rtsp://camera-address/stream"), TimeSpan.FromSeconds(15));

Replace the camera URL with your own. The 15 seconds apply to the parallel phase after sequential processing completes. Output includes frame recognition results and the last frame requested by each processor. Cleanup runs on success, early return, and exceptions.

Practical notes

  • RTSP URLs often look like rtsp://<username>:<password>@<ip>:<port>/....
  • Build this sample with .NET 10 / C# 14. Changing collection expressions to arrays alone is not enough for older compilers because the callbacks also use lambda parameter modifiers.
  • Call StreamAndRecognize(url, duration, useTemporalTracker: true) to enable LicensePlateTemporalTracker; false keeps the direct per-frame output. The tracker is applied to the sequential phase.