Show HN: Speakrs Full PyAnnotate pipeline in Rust/ONNX 20-37x times faster macOS https://ift.tt/TVthOw4
Show HN: Speakrs Full PyAnnotate pipeline in Rust/ONNX 20-37x times faster macOS Speakrs implements the full pyannote community-1 style diarization pipeline in Rust: segmentation, powerset decode, overlap-add aggregation, binarization, embedding, PLDA, and VBx clustering. There is no Python runtime in the library path. Inference runs on ONNX Runtime or native CoreML, and the rest of the pipeline stays in Rust. It is 20x-30x faster on macOS, but only 2-3x faster on linux/cuda (depending on CPU). Few reasons its faster: 1. Speakrs is using coreml versions of the models. I exported the models specifically to run on coreml. PyAnnote just runs the same the same PyTorch versions through MPS (Metal) on macOS. 2. PyAnnote is not a single model, its a few different models put together in a pipeline, the readme has some info on the full pipeline. 3. Speakrs optimizes the pipeline so different parts can run on CPU, Neural Engine and GPU. Speakrs has a batch mode, where you can run on multiple fil...