Replacing an existing KWS model with a new one

#1

Hi,

I have been trying to replace an existing KWS model in the following repo, although it is a private repo, but you might have access to that:
https://github.com/alifsemi/ml-kws-ic-example-ensemble-share
Firstly, I was wondering if the documents are up to date.
Because I found that the structure of directories/subdirectories have changed a bit but seemingly it is not updated in the document.
Aside from that, in KWS.md document, it has been said that for replacing micronet_m model with a different model (let’s say a previous one such as dnn-m-quantized), we just need to point to the .tflite file, however, I tracked the commits that your team pushed and found out that there are some files (.cc and header files) in the source directory, have been modified for replacing the DS-CNN with Micronet-m.
That’s why I am a bit confused if either the document is updated or we need to modify different files in order to be able to change the model.

Any help would be greatly appreciated
Thank you in advance,
Sina

#2

Hi Sina,

When changing to a different model for one of the usecases, e.g. KWS, you will need to point to the different TFLite file when building the application. Also, as you have worked out already, will need to make changes to the .hpp and .cc file that abstracts details about the model. For KWS this will be MicroNetKwsModel.hpp and MicroNetKwsModel.cc

This only needs to be done if the operators used by the new TFLite model are different to those used by the existing model. If they are then you will need to add these to the EnlistOperations method (source/application/api/use_case/kws/src/MicroNetKwsModel.cc - ml/ethos-u/ml-embedded-evaluation-kit - Gitiles) and also change this value here: source/application/api/use_case/kws/include/MicroNetKwsModel.hpp - ml/ethos-u/ml-embedded-evaluation-kit - Gitiles if more operators are now in the model.

Please be aware that the repo you have linked to is likely from a very old fork of our ml-embedded-evaluation-kit repo so may be a bit out of date now. I will also check if documentation in our repo can be clearer about this change that needs to be made when swapping model.

Let me know if any of this is unclear or you need some more assistance and we will be happy to help.

Many thanks,
Richard

#3

Hi Richard,

First, I appreciate your time giving to that for replying to my concern.
Based on the document, the mentioned flow is exactly what I understood. I must modify the enlisted operations in accordance with the operations in a model in this case a KWS model and the number of max operations also need to adapt.
However, for my information (since I need to summarize a full replacement flow), there are two possible cases that should be considered:
1- Replacing the model with a different model in the same repository (ML Zoo)
2- Replacing with a completely different model (from other external repository, not ML Zoo)

For the first part, let’s think of replacing the micornet model with dnn medium quantized (although this is an old version of ARM KWS model, for our consideration, it might be helpful). I changed the enlisted operation to:

bool arm::app::MicroNetKwsModel::EnlistOperations()
{
    //this->m_opResolver.AddReshape();
    //this->m_opResolver.AddAveragePool2D();
    //this->m_opResolver.AddConv2D();
    //this->m_opResolver.AddDepthwiseConv2D();
    this->m_opResolver.AddFullyConnected();
    this->m_opResolver.AddRelu();
    this->m_opResolver.AddSoftmax();
    this->m_opResolver.AddQuantize();
    this->m_opResolver.AddDequantize();
    if (kTfLiteOk == this->m_opResolver.AddEthosU()) {
        info("Added %s support to op resolver\n",
            tflite::GetString_ETHOSU());
    } else {
        printf_err("Failed to add Arm NPU support to op resolver.");
        return false;
    }
    return true;
}

Then, I changed the max operation, but the indexes of input and output also has changed to this (since the input shape is (1,250)), please let me know if this modification looks good and these places are the only parts that should be modifed:

image
image1153Ă—847 131 KB

For the second part, if the .tflite model is not implemented by ARM, what are the other files that should be modifed, I am thinking of a different MFCC, and other factors.

Please accept my apologies for providing and asing for too much detail here.
Any comment and feedback would be greatly appreciated.

Best,
Sina

#4

Hi Sina,

Thanks for the questions raised. You’re right in thinking that there might be other changes required especially if the model you’re looking at has custom preprocessing steps (like you say MFCC - if the model has been trained with a different pipeline, this would be different). From a high level perspective you would have to look at:

  • Model ops (covered in Richard’s post and our documentation) enlisted.
  • Row/col indices in the model header file (like the one you pointed to in your response).
  • Preprocessing steps
    • MFCC for example - any custom way of calculating the coefficients that are fed in.
  • Post processing steps
    • Labels (the new model might have different keywords or their indices might be different)

For the models in the Arm model zoo, we’ve had to deal only with ops, indices and labels. You might find this CR link useful. It covers the migration from the old model to the current micronet one.

Useful changes:

Hope this helps.

Kshitij

1 Like
#5

One thing to add to Kshitij’s response:

If switching to the other Arm trained networks such as DNN medium the preprocessing (MFCC feature extraction) is basically the same so you shouldn’t need to change this. However, you will need to adjust the feature FrameLength and FrameStride values set here: usecase.cmake

The DNN variant of KWS models I believe should use values of 640 for both FrameLength and FrameStride, while others used 320 for the stride. Changing to 640 for both should then change the size of the final features to match the input shape of (1, 250) for that model.

1 Like
#6

Kshitij and Richard,

Thank you so much for these information.
I will try to put them into effect to see if I could get the desired output.
Once again appreciate your help and please let me continue this discussion if I come up with any other issue.

My best regards,
Sina

#7

@Burton2000 and @kshitij-sisodia-arm,

Thank you so much for providing the information on how to replace a model.
However, there was some slight modification that was needed to be done and after that the model has been successfully replaced with another ARM zoo model for kws.
However, I am having some issue with replacing the model with an outside model. To be accurate, I am trying to replace an Edge Impulse model that I recently designed for KWS application. The preprocessing parameters are exactly the same (as far as I could track, since their platform for preprocessing steps are visual and we are able to put some numbers for different elements, i.e. MFCC coeff numbers and etc).

image
image784Ă—780 26.7 KB

I could successfully build the project with this replacement, however, the behavior is different which I guess it might be from preprocessing steps that the model is trained based on the corresponding inputs.
My question, is the preprocessing step used in ARM implementation was different than others? I mean whether the MFCC calculation is different than others? or even other parts of preprocessing, such as shifting, normalization, and etc. Do you think if it’s possible to make this replacement?
I would appreciate any feedback.
Sina

#8

Hi Sina,

apologies for the delayed response - we have a few colleagues out on vacation but hope to come back to you soon.

Thanks, Pete

1 Like
#9

Hi @asadisina,

Apologies for the late reply.

However, there was some slight modification that was needed to be done and after that the model has been successfully replaced with another ARM zoo model for kws.

Good to know. The KWS models in Arm Model Zoo have similar training routines in terms of pre-processing, so only minor changes would be expected.

I could successfully build the project with this replacement, however, the behavior is different which I guess it might be from preprocessing steps that the model is trained based on the corresponding inputs.
My question, is the preprocessing step used in ARM implementation was different than others? I mean whether the MFCC calculation is different than others?

You are right, the MFCC could be different - the way the normalisation works or something else (for example, the Logarithm might be taken with base 10 or base e). You would need to have a the full pre-processing pipeline’s Python code to figure out the differences.

You might find that the way MFCC works in our repo also has two specialisations: one works for KWS and one for ASR. You’d find that the ASR implementation overrides more functions from the base MFCC implementation. These differences are based on the preprocessing pipleine used when training the model (see librispeech_mfcc.py) , it is imperative that you have access to something similar for the model you wish to deploy.

Hope this helps.

Thanks,
Kshitij

1 Like