Qualcomm AI Engine Direct - Mimi Enablement Stage 2

[winskuo-quic]

Summary

• Support Mimi Encoder
• Support Mimi Decoder
• Support OP: CDist
• Workaround for 0D tensor scenario since QNN does not support 0D tensor

Commands to Execute the model
10 sec as a batch (1 inference for 10 sec audio):
python examples/qualcomm/oss_scripts/moshi/mimi.py -b build-android -s $DEVICE -m SM8650 --chunks_per_batch 125

80s as a batch (125 inferences for 10 sec audio)
python examples/qualcomm/oss_scripts/moshi/mimi.py -b build-android -s $DEVICE -m SM8650

Test plan

• UT for CDist
• UT for 0D tensor

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10098

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 08759d6 with merge base e29a4b5 ():

NEW FAILURE - The following job has failed:

• Check Labels / Check labels (gh)
  RuntimeError: Error checking labels: PR does not have required labels

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[winskuo-quic]

Hi @cccclai, @iseeyuan, @billmguo,
This PR is to enable mimi without streaming mode.
I have listed to execution commands in the summary section.
Please have a look.
Thanks

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[cccclai]

Does the logic in https://github.com/pytorch/executorch/blob/main/backends/transforms/rank_0_to_rank_1.py can be applied here?

[winskuo-quic]

Hi @cccclai,
Thanks for reviewing the PR.
I believe the pass you are suggesting is to change input from 0d to 1d tensor.
However, for our case, this 0D tensor happened during select op in the middle of graph. We just removed this select op since it does not affect the logic of the graph.
The exact point where 0D tensor occurs in mimi is under moshi/quantization/core_vq.py, where is tries to create a 1D tensor and retrieve index 0.