Besides GPU, can any other edge accellerators run it. EX: (Hailo AI Hat for RPI)
Can the parakeet model run on the Hailo AI Hat for my Raspberry Pi project?
How about Nvidia's Orin Jetson Nano?
Hi @Flyingcrabs
I'm not exactly sure since I don't have these devices, but judging by these examples they support onnxruntime:
https://github.com/hailo-ai/Hailo-Application-Code-Examples/tree/main/runtime/hailo-8/python/hailo_onnxruntime
https://www.elinux.org/Jetson_Zoo#ONNX_Runtime
If this works, you can use my library - onnx-asr to run parakeet models.
Thank you for the response, but my question is really about if the hardware such as the HailoAIHat can acutally run the model in a timely manner. The HAILO HAT is good at convolution and quantization. Will this model be a good fit?
I am running it in HF Space in real-time mode with 2 vCPU only so I bet it can.
https://huggingface.co/spaces/WJ88/NVIDIA-Parakeet-TDT-0.6B-v2-INT8-Real-Time-Mic-Transcription