FaceLandmark uses blendshape, which causes serious frame drops on Sony Xperia8. You need to switch the background to make it work properly.

[silingfei]

Plugin Version or Commit ID

v0.16.1

Unity Version

6000.0.50f1

Your Host OS

Windows 10

Target Platform

Android

Description

I'm using blendshape in my facelandmark example. I've tested it with both synchronous and asynchronous execution, and on a GPU. On a Sony Xperia 8 (and other lower-end devices), I experience significant frame drops. Switching the app to the background and then back again returns it to a normal 60fps. Are there any good solutions?

Code to Reproduce the issue

No response

Additional Context

No response

[homuler]

Can you confirm whether the change mentioned in this comment has any impact on this issue?

[silingfei]

I have verified this solution one by one, but there is still no improvement

[silingfei]

Sorry, using CPU mode will significantly improve xperia8, but the CPU Helio G88 is only about 10 frames, and it will return to 60 frames after switching to the background.

[homuler]

Could you investigate what exactly is happening and provide an explanation of the situation?
Right now, various topics seem to be mixed together (e.g. CPU/GPU, sync/async, xperia8 or others, etc...), so it would also help if you could clarify the differences under each condition.

For example, it would be helpful to have information such as:

• The exact definition of “fps” in this context
  • If you are measuring it by inserting code, please share the code differences and logs
• Performance differences when blendshapes are not being output
• How performance changes when using different devices or delegates
• How performance differs before and after putting the app into the background
• Whether there are any cases where performance changes after applying the change mentioned in the above comment

If this is the same issue as before, the investigation has stalled, so unless you can identify which part of MediaPipe is taking time, there may not be much I can do on my side (since I cannot reproduce the issue on my own device).

[silingfei]

1. Without outputting blend shapes, all devices had normal fps.
2. After enabling blend shapes, testing the CPU, CPU ASYNC, and GPU, the Helio G88 experienced significant lag, with single-frame delays exceeding 400ms.
3. Switching to the background and then back again restored the frame rate to 60 fps, and recognition returned to normal.
4. All the solutions mentioned in the comments above did not improve the performance on the Helio G88.

[homuler]

As noted in #1383, I wasn’t able to reproduce it on my end, and unless we can identify which specific line of MediaPipe’s code is taking time, there’s nothing more I can do at the moment.
If possible, please try building the library yourself and pinpointing the problem.

By the way, could you check whether the official MediaPipe sample app works as expected?