Execute the application on ARM CPU instead of NPU

ML Embedded Evaluation Kit
#1

Hello all

First of all thanks a lot for putting out a detailed set of instructions and also having discussion platform.

I have been starting to use this ML Embedded Evaluation Kit to deploy applications on MPS3 board and was wondering if its possible to execute the whole application (tflite model) on ARM Cortex M55 CPU instead of ARM NPU. (I want to compare the time taken by the CPU and NPU)

If its possible, could you kindly let me know what are the steps needed to be done to execute the target application on ARM CPU on MPS3 board? Thanks in advance for all the replies.

#2

Hi,

I believe all you need to do to run only on Cortex-M55 and not at all on Ethos-U55 is to omit the Vela compiler step. If you take your original, quantised .tflite file and link that with the application code then no operators should run on the Ethos-U55.

Hope this helps, Pete

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

#3

To add to what Pete has mentioned above, if you are looking to run the existing use cases with their default models (as downloaded by set_up_default_resources.py) on the CPU, you can set the parameter -DETHOS_U_NPU_ENABLED=OFF when you configure the CMake project (see all the configuration options here).
And if you’re deploying your own tflite file using the generic inference runner, you can skip the Vela compiler step to make the inference NOT use the NPU.

Hope this helps.

#4

Thanks a lot for the quick responses. :slight_smile:

I’m trying to run the existing use cases and if I set DETHOS_U_NPU_ENABLED=OFF while configuring the project and try to deploy the application on MPS3 board, I’m getting the following erros

ERROR - Ethos-U operator present in the model but this build does not include Ethos-U drivers
ERROR - Failed to initialise model

Am I missing a step here?

Thanks in advance for your response!

#5

Hi Surya,

Error indicates that your application still loads a model optimised for Ethos-U (processed by vela). Build scripts generate source files based on a model file. It might be that you have not cleared your build directory from the previous build results and the old model source file is still picked up.

Please, try using different build dir or cleaning the old one.

By the way, there is another build option you may want to set: -DCPU_PROFLIE_ENABLED=1. This will enable active CPU cycle counts output.

#6

Thanks for the reply. :slight_smile:

Yeah using a different build directory seems to have worked. I’m already using CPU Profiler to check the CPU cycles.