Mono AOT decoder workaround for slow jpeg decoding. by JimBobSquarePants · Pull Request #2762 · SixLabors/ImageSharp · GitHub
Skip to content

Mono AOT decoder workaround for slow jpeg decoding. - #2762

Merged
JimBobSquarePants merged 5 commits into
release/3.1.xfrom
js/mono-aot-decoder-workaround
Jul 31, 2024
Merged

JimBobSquarePants merged 5 commits into
release/3.1.xfrom
js/mono-aot-decoder-workaround

Conversation

@JimBobSquarePants

@JimBobSquarePants JimBobSquarePants commented Jul 3, 2024 •

Copy link
Copy Markdown
Member

Prerequisites

  • I have written a descriptive pull-request title
  • I have verified that there are no overlapping pull-requests open
  • I have verified that I am following the existing coding patterns and practice as demonstrated in the repository. These follow strict Stylecop rules 👮.
  • I have provided test coverage for my change (where applicable)

Description

This PR is an attempt to work around issues found with the Mono AOT compiler which causes slow performance on IOS Android

dotnet/runtime#71210

In the linked Issue, Analysis from the Mono team highlighted this indirection as a culprit.

The AOT compiler does appear to have problems figuring out which instances to generate. In this specific case, the caller is
ImageDecoderUtilities:Decode<Rgba32> which calls IImageDecoderInternals::Decode on an argument. So in theory, the aot compiler could figure out that the call could possible go to JpegDecoderCore::Decode<Rgba32> and generate that instance. Currently, this kind of analysis is not done.

This PR removes that indirection completely by introducing an internal base class, ImageDecoderCore for all XXDecoderCore instances. In addition, seeding has been introduced for SpectralConverter<TPixel> and others.

This is currently untested but I'm confident that this should improve matters.

@beeradmoore

beeradmoore commented Jul 10, 2024 •

Copy link
Copy Markdown

@beeradmoore

Copy link
Copy Markdown

Added all the numbers.

My takeaway from this (as someone who uses images, but doesn't really know much about image encoding) is:

  • Jpg is way slower than png
  • Debug mode in almost all platforms is slower than I would have expected
  • Jpgs on an android device in debug mode is wild
  • Shocked to see jpg on android emulator be faster than android on device
  • I don't see any noticeable difference in the results of this PR and the currently published 3.1.4. I don't know if there is meant to be additional changes to using AOT mode for the tests or something.

More than happy to add and run more tests as requested. Attaching the raw results numbers below (if anyone wants to see the results per run).

ImageSharpMAUITest-Results.zip

@JimBobSquarePants

Copy link
Copy Markdown
Member Author

@beeradmoore I've been doing some R&D in order to understand the Android performance. I can't see a configuration value in your sample to enable LLVM which is required for maximum performance. Did I miss something?

https://learn.microsoft.com/en-us/dotnet/android/building-apps/build-properties#enablellvm

@beeradmoore

Copy link
Copy Markdown

I did not.

I added

    <PropertyGroup Condition="$([MSBuild]::GetTargetPlatformIdentifier('$(TargetFramework)')) == 'android' AND '$(Configuration)' == 'Release'">
        <EnableLLVM>true</EnableLLVM>
    </PropertyGroup>

And during executing all the tests it gets to about 15/40 (half way through jpg load and resize run) and it crashes with

07-16 12:36:03.133  1730  1778 W WindowManager: Exception thrown during dispatchAppVisibility Window{647a9a8 u0 com.beeradmoore.imagesharpmauitest/crc64df7e0c4a761c65fa.MainActivity EXITING}
07-16 12:36:03.133  1730  1778 W WindowManager: android.os.DeadObjectException
07-16 12:36:03.133  1730  1778 W WindowManager: 	at android.os.BinderProxy.transactNative(Native Method)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at android.os.BinderProxy.transact(BinderProxy.java:586)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at android.view.IWindow$Stub$Proxy.dispatchAppVisibility(IWindow.java:552)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at com.android.server.wm.WindowState.sendAppVisibilityToClients(WindowState.java:3217)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at com.android.server.wm.WindowContainer.sendAppVisibilityToClients(WindowContainer.java:1293)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at com.android.server.wm.WindowToken.setClientVisible(WindowToken.java:403)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at com.android.server.wm.ActivityRecord.setClientVisible(ActivityRecord.java:7100)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at com.android.server.wm.ActivityRecord.postApplyAnimation(ActivityRecord.java:5820)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at com.android.server.wm.ActivityRecord.commitVisibility(ActivityRecord.java:5762)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at com.android.server.wm.Transition.finishTransition(Transition.java:1257)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at com.android.server.wm.TransitionController.finishTransition(TransitionController.java:925)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at com.android.server.wm.WindowOrganizerController.finishTransition(WindowOrganizerController.java:489)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at android.window.IWindowOrganizerController$Stub.onTransact(IWindowOrganizerController.java:278)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at com.android.server.wm.WindowOrganizerController.onTransact(WindowOrganizerController.java:199)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at android.os.Binder.execTransactInternal(Binder.java:1500)
07-16 12:36:03.133  1730  1778 W WindowManager: 	at android.os.Binder.execTransact(Binder.java:1444)

This is new to me and I have no idea what or why it is doing this. I have re-installed AndroidOS between then and now, this could be related, it could be AOT compile. Tonight I can revert back to not having LLVM and see if to happens again or not. It could also be I did no have developer mode enabled so maybe it was going to sleep in 15sec 🤷‍♂️

I did a single run of with v3.1.4 in release mode running the JpgLoad test and got 1116.8ms. For the local nuget I got 1126.6ms.

I referred to the previous thread and grabbed info from this comment,

and just put

<EnableLLVM>true</EnableLLVM>
<RunAOTCompilation>true</RunAOTCompilation>
<AndroidEnableProfiledAot>false</AndroidEnableProfiledAot>

in my main PropertyGroup to ensure it is enabled.

With that I got 516ms for the local nuget. Huge improvement!

I can test again tonight to see what ones of those above properties are required, if my release Android targeting below was working as intended.

@JimBobSquarePants

Copy link
Copy Markdown
Member Author

@beeradmoore Did you ever get those updated Android benchmarks?

@beeradmoore

Copy link
Copy Markdown

@JimBobSquarePants , aside from that single test I did not. Creating a reminder to do it this weekend.

@JimBobSquarePants

Copy link
Copy Markdown
Member Author

Thanks @beeradmoore very much appreciated!

@beeradmoore

Copy link
Copy Markdown

So some more checking into things. For these results I will call things run #1 - #6 with each run having a different csproj config. These different configs can also impact filesize.

I think we are also focusing on changes to Android, so these tests are only on Release mode on my physical Pixel 2 XL. If there are any specific AOT flags or anything you want me to re-run on other platforms just let me know what ones.

Settings used for each run:

Run 1

Of the 3 options listed in the other runs, this run did not have them in the csproj. I did not lookk what the net8.0-android defaults are for release mode.

Run 2

<EnableLLVM>true</EnableLLVM>

This resulted in a crush mid way through the second test. While the tests could be ran individually there is an issue if you can't run them back to back so I am considering these invalid.

07-27 14:06:30.804 7895 7961 E gesharpmauitest: * Assertion at /__w/1/s/src/mono/mono/mini/aot-runtime.c:5220, condition `plt_entry' not met
07-27 14:06:30.806 7895 7961 F libc : Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 7961 (.NET TP Worker), pid 7895 (gesharpmauitest)

More info on these crashes below.

Run 3

<EnableLLVM>true</EnableLLVM>
<RunAOTCompilation>true</RunAOTCompilation>

This also resulted in a crash.

07-27 14:12:55.543 8325 8359 F libc : Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 8359 (.NET TP Gate), pid 8325 (gesharpmauitest)
07-27 14:12:55.543 8325 8378 E gesharpmauitest: * Assertion at /__w/1/s/src/mono/mono/mini/aot-runtime.c:5220, condition plt_entry' not met 07-27 14:12:55.543 8325 8358 E gesharpmauitest: * Assertion at /__w/1/s/src/mono/mono/mini/aot-runtime.c:5220, condition plt_entry' not met

More info on these crashes below.

Run 4

<EnableLLVM>true</EnableLLVM>
<RunAOTCompilation>true</RunAOTCompilation>
<AndroidEnableProfiledAot>false</AndroidEnableProfiledAot>

Run 5

<EnableLLVM>true</EnableLLVM>
<AndroidEnableProfiledAot>false</AndroidEnableProfiledAot>

Same as Run 4, but without the RunAOTCompilation defined. I did not check the defaults for these values, but size and performance appears teh same so I am going to assume RunAOTCompilation is defaulting to true.

Run 6

<EnableLLVM>false</EnableLLVM>
<AndroidEnableProfiledAot>true</AndroidEnableProfiledAot>

This is the same as run 5 but it swaps EnableLLVM and AndroidEnableProfiledAot around. Size and results appears this is the default when no params are defined.

Results (3.1.4)

Run JpgLoad JpgResize PngLoad PngResize aab size
1 1114.9ms 1353.9ms 34.0ms 41.2ms 17.8 MB
2 Crash Crash Crash Crash 17.7 MB
3 Crash Crash Crash Crash 17.7 MB
4 515.3ms 712.2ms 26.9ms 35.4ms 24.1 MB
5 512.0ms 698.8ms 26.6ms 33.9ms 24.1 MB
6 1116.4ms 1355.3ms 33.9ms 40.9ms 17.8 MB

Results (this pr)

Run JpgLoad JpgResize PngLoad PngResize aab size
1 1124.5ms 1361.0ms 33.2ms 41.0ms 17.8 MB
2 Crash Crash Crash Crash 17.7 MB
3 Crash Crash Crash Crash 17.7 MB
4 510.1ms 709.8ms 27.2ms 34.5ms 24.1 MB
5 512.7ms 713.2ms 26.8ms 34.8ms 24.1 MB
6 1123.1ms 1352.3ms 34.1ms 40.5ms 17.8 MB

Crashes

The crashes looks like they are discussed in this issue in the dotnet runtime repo.

Conclusion

My takeaway from this is the currently release 3.1.4 and this PR perform the same, at least for the limited tests I am doing. Other situations, or possibly other devices, may perform differently. What the new code is doing is above my skillset, but I don't see it degrading performance so I don't think its a bad change.

If people want more performance with ImageSharp for Android then they will get it at an incrased filesize by enabling these options for release mode Android builds.

<PropertyGroup Condition="$([MSBuild]::GetTargetPlatformIdentifier('$(TargetFramework)')) == 'android' AND '$(Configuration)' == 'Release'">
    <EnableLLVM>true</EnableLLVM>
    <AndroidEnableProfiledAot>false</AndroidEnableProfiledAot>
</PropertyGroup>

Using EnableLLVM is likely not enough as you could still have crashes, but those crashes may not appear all the time. EG. I can run the second test individually and it will pass, but if I run all tests back to back it will crash halfway through the second set of tests.

I am unsure if there are more optimisations you can do, or if the next step of improvemnets requires the dotnet team to do things on their end.

I am also unsure if writing C# code using Android SDK for image manipulation gets it going any faster or if I am now approching limits on my 7 year old phone. Doing so would then require more devtime to write platform specific code for every single image manipulation piece I'd do for each platform I'd run my app on. It's also then just a lot more code in general to maintain. Whereas ImageSharp appears to do a great job across the board, except for Android where it just does an ok job OR I just use pngs everywhere, becaause they were always fast everywhere 😅

@JimBobSquarePants

Copy link
Copy Markdown
Member Author

Thanks @beeradmoore for the monumental effort you've put in here. I think this should actually enable us to close off several longstanding issues.

Judging by this comment and this comment the following combination should be the recommended one for Android.

<EnableLLVM>true</EnableLLVM>
<RunAOTCompilation>true</RunAOTCompilation>
<AndroidEnableProfiledAot>false</AndroidEnableProfiledAot>

iOS performance looks fantastic and Android, I imagine will improve with new phones and codegen improvements. I think most complaints are either outdated or the result of miscofiguration.

Adding the Android and iOS specific MAUI configuration from your sample with an explanation to this section of the docs will help eleviate issues going forward.

Despite relative equality of the performance metrics of the two source versions I think merging this PR is still a wise choice. It's much easier to maintain internally as it removes misdirection.

@beeradmoore

Copy link
Copy Markdown

I think one of the things that will catch people off guard is running in debug mode being slower. I would expect things to be slower in debug mode, but not 14sec to load an image. That would have caught me off guard and I would have questioned the library long before trying in release mode.

@JimBobSquarePants

Copy link
Copy Markdown
Member Author

I think one of the things that will catch people off guard is running in debug mode being slower. I would expect things to be slower in debug mode, but not 14sec to load an image. That would have caught me off guard and I would have questioned the library long before trying in release mode.

Yeah... Definitely need to docuement that also and raise issues with the relevant parties.

@JimBobSquarePants JimBobSquarePants mentioned this pull request Jul 31, 2024
4 tasks done
@JimBobSquarePants

Copy link
Copy Markdown
Member Author

@JimBobSquarePants
JimBobSquarePants merged commit aad5cfa into release/3.1.x Jul 31, 2024
@JimBobSquarePants
JimBobSquarePants deleted the js/mono-aot-decoder-workaround branch July 31, 2024 12:25
This was referenced Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants