Taking AI Dev Gallery Code Cross Platform with Uno Platform – iOS

by

Following my previous posts that introduced the AI Dev Gallery, added AI code to an Uno Platform application and adjusting the AI code to work with Android, in this post we’re going to look at the adjustments required to get the AI code to work on iOS. The issues with iOS aren’t apparent until you go to run the application, in my case on the iOS simulator (from Windows). The application builds successfully if you right-click the project and select Build or Rebuild. However, when you click Run, the build will fail with the following error:

1>Tool xcrun execution finished (exit code = 1).
1>C:\Program Files\dotnet\packs\Microsoft.iOS.Sdk.net8.0_18.0\18.0.8314\targets\Xamarin.Shared.Sdk.targets(1648,3): error : clang++ exited with code 1:
1>C:\Program Files\dotnet\packs\Microsoft.iOS.Sdk.net8.0_18.0\18.0.8314\targets\Xamarin.Shared.Sdk.targets(1648,3): error : ld: library 'f:/packages/microsoft.ml.onnxruntime.extensions/0.13.0/buildTransitive/net6.0-ios15.4/../../runtimes/ios/native/onnxruntime_extensions.xcframework.zip' not found
1>C:\Program Files\dotnet\packs\Microsoft.iOS.Sdk.net8.0_18.0\18.0.8314\targets\Xamarin.Shared.Sdk.targets(1648,3): error : clang++: error: linker command failed with exit code 1 (use -v to see invocation)

This appears to be a known issue tracked on the onnx runtime github repository in relation to building an .NET MAUI application. In the issue there is a workaround that suggests manually extracting the zip file that contains the native reference. However, if you do this, you’ll need to manually do it on each computer you want to run the application on.

Let’s walk through how we can apply the workaround suggest in the issue in a way that will work on any computer that wants to run the application, without having to apply a bunch of manual steps. The first thing to do is to extract the two native references from the NuGet packages Microsoft.ML.OnnxRuntime and Microsoft.ML.OnnxRuntime.Extensions:

  • packages\microsoft.ml.onnxruntime.extensions\0.13.0\runtimes\ios\native\onnxruntime_extensions.xcframework.zip
  • packages\microsoft.ml.onnxruntime\1.20.1\runtimes\ios\native\onnxruntime.xcframework.zip

Each of these zip files was extracted into a Library folder at the solution level, resulting in the following folder structure.

Each of the NuGet packages, Microsoft.ML.OnnxRuntime and Microsoft.ML.OnnxRuntime.Extensions, includes a targets file that contains a single NativeReference item, for example.

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <ItemGroup Condition="('$(OutputType)'!='Library' OR '$(IsAppExtension)'=='True')">
    <NativeReference Include="$(MSBuildThisFileDirectory)..\..\runtimes\ios\native\onnxruntime.xcframework.zip">
      <Kind>Static</Kind>
      <IsCxx>True</IsCxx>
      <SmartLink>True</SmartLink>
      <ForceLoad>True</ForceLoad>
      <LinkerFlags>-lc++</LinkerFlags>
      <WeakFrameworks>CoreML</WeakFrameworks>
    </NativeReference>
  </ItemGroup>
</Project>

Unfortunately, it appears that the MSBuild task that is responsible for extracting the zip file and resolving the native reference, doesn’t appear to be working correctly. Since we’ve manually extracted the zip files into a location where they can be checked in, alongside the application, we just need to update the NativeReference by changing the Directory.Build.targets to the following:

<Project>
    <ItemGroup Condition="(('$(OutputType)'!='Library' OR '$(IsAppExtension)'=='True') AND '$(TargetFramework)'=='net8.0-ios')">
        <NativeReference Remove="@(NativeReference)" />

        <NativeReference Include="$(MSBuildThisFileDirectory)..\Library\onnxruntime.xcframework">
            <Kind>Static</Kind>
            <IsCxx>True</IsCxx>
            <SmartLink>True</SmartLink>
            <ForceLoad>True</ForceLoad>
            <LinkerFlags>-lc++</LinkerFlags>
            <WeakFrameworks>CoreML</WeakFrameworks>
        </NativeReference>
        <NativeReference Include="$(MSBuildThisFileDirectory)..\Library\onnxruntime_extensions.xcframework">
            <Kind>Static</Kind>
            <IsCxx>True</IsCxx>
            <SmartLink>True</SmartLink>
            <ForceLoad>True</ForceLoad>
            <LinkerFlags>-lc++</LinkerFlags>
            <WeakFrameworks>CoreML</WeakFrameworks>
        </NativeReference>
    </ItemGroup>
</Project>

With this change, the ObjectDetection application will now run on iOS. Here’s it running in on the iOS simulator via the Remote Simulator.

Leave a comment