YOLO model recognition (MAUI) – camera + overlay toggle

Applies to SDK 2.2.x. The MAUI SDK supports Android and iOS only, using .NET MAUI 9 or 10. See requirements and licensing.

This sample is a simplified version of the YoloModelRecognitionPage from CapCognition.Maui/Test/Test.Maui.

Goals:

load a YOLO ONNX model from the app package
build the recognition options in OnOpenYoloDetection
use a RecognitionView in XAML
start/stop YOLO recognition
keep a button to toggle overlays (bounding boxes)

Prerequisites

Package(s): CapCognition.Maui and CapCognition.Maui.YoloModelDetection
The ONNX model must be included in the app package as a MauiAsset (typically via Resources/Raw).

Example (csproj):

<ItemGroup>
  <MauiAsset Include="Resources\Raw\fruits-yolo-model.onnx" LogicalName="fruits-yolo-model.onnx" />
</ItemGroup>

1) Create options (MainPage)

This method matches the setup from OnOpenYoloDetection and loads the model from the app package.

using CapCognition.Maui.Core.Shared.Common;
using CapCognition.Maui.YoloModelDetection;

private void OnOpenYoloDetection(object? sender, EventArgs e)
{
    var modelFileName = "fruits-yolo-model.onnx";

    var modelStream = new YoloModelDetectionRecognitionOptions.ModelStreamInfo(
        FileSystem.Current.OpenAppPackageFileAsync(modelFileName).GetAwaiter().GetResult(),
        modelFileName);

    var options = new RecognitionOptionBuilder()
        .AddYoloModelDetectionOption()
            .UseFittedImageForRecognition()
            .UseModel(modelStream)
            .ConfigureOption(option =>
            {
                option.CreateAndLoadModel();
            })
            .AddYoloModelDetectionOverlayDrawingOption()
                .EnableOverlays()
            .Done()
        .Done()
        .Build();

    Navigation.PushAsync(new YoloModelRecognitionPage(options));
}

2) UI (XAML)

Minimal UI: a RecognitionView, an overlay button, and start/stop for continuous recognition.

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://schemas.microsoft.com/dotnet/2021/maui"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:views="clr-namespace:CapCognition.Maui.Views;assembly=CapCognition.Maui"
             x:Class="YourApp.YoloModelRecognitionPage"
             Title="YOLO Recognition">

    <Grid>
        <views:RecognitionView
            x:Name="RecognitionView"
            RequestedPreviewResolutionWidth="1920"
            RequestedCaptureResolutionWidth="1080"
            CaptureIntervalInMs="0"
            UseFrontCamera="False"
            FillPreviewToView="False"
            DisplayWatermark="False" />

        <!-- Simple overlay controls -->
        <VerticalStackLayout
            Padding="12"
            Spacing="8"
            HorizontalOptions="Start"
            VerticalOptions="Start">

            <Button Text="{Binding OverlayText}" Clicked="OnToggleOverlay" />
            <Button Text="{Binding RecognitionText}" Clicked="OnToggleRecognition" />

        </VerticalStackLayout>
    </Grid>
</ContentPage>

3) Code-behind (simplified)

Key points:

Options are injected into the RecognitionView (RecognitionView.AddOption(...)).
The model must be loaded before opening the camera.
Overlays are controlled via YoloModelDetectionRecognitionOptions.DrawingOptions.EnableOverlays.

using CapCognition.Maui.Core.Shared.Common;
using CapCognition.Maui.YoloModelDetection;
using SkiaSharp;

namespace YourApp;

public partial class YoloModelRecognitionPage : ContentPage
{
    public YoloModelRecognitionPage(List<RecognitionOptions> options)
    {
        _options = options;
        InitializeComponent();
        BindingContext = this;

        foreach (var option in options)
        {
            RecognitionView.AddOption(option);
        }

        _yoloOptions = RecognitionView.GetOption<YoloModelDetectionRecognitionOptions>();

        OverlayText = "O: On";
        RecognitionText = "Start Recognition";

        Loaded += OnLoaded;
    }

    public string OverlayText { get; set; }
    public string RecognitionText { get; set; }

    protected override async void OnAppearing()
    {
        base.OnAppearing();
        OpenCamera();
    }

    private void OnLoaded(object? sender, EventArgs e)
    {
        var result = await RecognitionView.RequestAllPermissionAsync();
        if (!result)
        {
            await Navigation.PopAsync();
        }

        OpenCamera();
    }

    protected override void OnDisappearing()
    {
        RecognitionView.CloseCamera();
        base.OnDisappearing();
    }

    private void OpenCamera()
    {
        if (!RecognitionView.Initialized || RecognitionView.CameraIsOpen)
        {
            return;
        }

        RecognitionView.CameraOpenedEvt -= OnCameraOpened;
        RecognitionView.CameraOpenedEvt += OnCameraOpened;
        var success = RecognitionView.OpenCamera();
    }

    private void OnCameraOpened(bool success)
    {
        RecognitionView.CameraOpenedEvt -= OnCameraOpened;

        if (!success)
        {
            Console.WriteLine("Camera open failed");
            return;
        }

        Console.WriteLine("###Camera opened");
    }

    private void OnToggleOverlay(object? sender, EventArgs e)
    {
        _overlayOn = !_overlayOn;
        OverlayText = _overlayOn ? "O: On" : "O: Off";

        if (_yoloOptions != null)
        {
            _yoloOptions.DrawingOptions.EnableOverlays = _overlayOn;
        }

        // If you use bindings with auto-notify (INotifyPropertyChanged), notify here.
        BindingContext = null;
        BindingContext = this;
    }

    private void OnToggleRecognition(object? sender, EventArgs e)
    {
        if (!RecognitionView.CameraIsOpen)
        {
            Console.WriteLine("Camera not open");
            return;
        }

        if (RecognitionView.IsRecognizing)
        {
            var success = RecognitionView.StopContinuousRecognition();
            if (success)
            {
                RecognitionView.RecognitionResultEvt -= OnImageRecognized;
            }

            RecognitionText = "Start Recognition";
        }
        else
        {
            var success = RecognitionView.StartContinuousRecognition();
            if (success)
            {
                RecognitionView.RecognitionResultEvt += OnImageRecognized;
            }

            RecognitionText = "Stop Recognition";
        }

        BindingContext = null;
        BindingContext = this;
    }

    private void OnImageRecognized(RecognitionResult result, SKBitmap bitmap)
    {
        // Results are drawn as overlay directly in the preview when overlays are enabled.
        Console.WriteLine(result);
    }

    private bool _overlayOn = true;
    private readonly List<RecognitionOptions> _options;
    private readonly YoloModelDetectionRecognitionOptions? _yoloOptions;
}

Step-by-step explanation

Ship the model as an app asset
- Include the ONNX file as a MauiAsset so it is available at runtime from the app package.
Open the model from the app package
- FileSystem.Current.OpenAppPackageFileAsync(modelFileName) returns a stream to the packaged asset.
Configure recognition options
- AddYoloModelDetectionOption() enables the YOLO processor.
- UseModel(modelStream) attaches your ONNX model.
- CreateAndLoadModel() loads the model upfront (important so later steps don’t fail due to an unloaded model).
Enable overlay drawing
- AddYoloModelDetectionOverlayDrawingOption().EnableOverlays() turns overlay output on by default.
Register options in the RecognitionView
- In the page, the RecognitionOptions are registered via RecognitionView.AddOption(...).
Open the camera (after the model is loaded)
- In OnLoaded, the page checks whether _yoloOptions.LoadedYoloModel is available.
- Optionally, the capture resolution is adjusted using LoadedYoloModel.InputSize.
Start/stop recognition
- The button calls StartContinuousRecognition() and StopContinuousRecognition().
- Subscribe to RecognitionResultEvt to receive per-frame results.
Toggle overlays (keep the button)
- The overlay button sets _yoloOptions.DrawingOptions.EnableOverlays.
- When enabled, the RecognitionView draws bounding boxes directly on the preview.

Table of Contents