Stream audio in, get JSON transcripts out. We operate the models and the GPUs. From about $0.50 per audio hour.
4 architectures
CTC decoding, a token-and-duration transducer, an encoder-decoder transformer and a language-model decoder are each implemented against the same Rust trait. An abstraction that survives four families this different is a working abstraction, and adding a fifth will not change the API you integrate against.
1 binary
The production runtime is a single Rust binary. No Python interpreter, no PyTorch, no ONNX runtime and no separate inference server sit behind it. Inference runs on Candle, and weights are memory-mapped straight off disk. Python appears only offline, converting checkpoints before an image is built.
0 infrastructure
You never provision a GPU, install a CUDA toolkit, download a checkpoint or keep an ML stack running. You open a WebSocket and send audio. Pick the engine that matches your language and your budget, and pay for the audio hours you actually process.
Four engines are available. They differ in architecture, in the languages they were built for, in whether they can give you timestamps, and in what they cost per audio hour. One of them fits your audio better than the other three.
Russian audio at the lowest rate
A single-language model that does one job. If your audio is Russian, this is both the specialised choice and the cheapest one. It reports "ru" on every response because it was never built to do anything else.
MODEL_TYPE=gigaamai-sage/GigaAM-v3Broad European coverage, including Ukrainian
A transducer rather than an attention decoder, which is a genuinely different way to turn acoustics into tokens. Its declared language set is the widest European one we run, and it is the only engine here that lists Ukrainian alongside Russian.
MODEL_TYPE=parakeetnvidia/parakeet-tdt-0.6b-v3Language-model decoding for messier audio
The decoder is an actual language model, not a CTC head or a transducer. That changes how the transcript is formed: it is generated token by token with full text context, with a guard that aborts degenerate repetition loops.
MODEL_TYPE=qwen3Qwen3-ASRWidest language range, and the only engine returning confidence
The reference multilingual model, and the one to pick when you cannot predict what language will arrive. It is also the only engine here that returns a per-segment confidence value alongside the text.
MODEL_TYPE=whisperopenai/whisper-large-v3-turboProvisioning is handled by hand. That is deliberate at our size: it means the deployment, the engine and the access rules are set up for your case rather than guessed by a wizard.
The language you transcribe, roughly how many audio hours a month, and the domains or IP addresses that will connect.
A dedicated instance running one engine on one device, reachable at its own hostname. Weights are already inside the image, so nothing is downloaded at run time.
Access control sits in front of the endpoint. Restrict by allowed domain or by IP address, whichever suits your integration.
Open the WebSocket, send 16 kHz mono PCM, read JSON back. Partial transcripts arrive while you are still sending, and a final one when you finish.
Billing follows audio hours at the rate of the engine you run. No seats, no tiers, no minimum commitment.
Want to talk it through before committing to anything? Request access and describe your use case.
There is a single route. You open it, push raw samples at it, and read transcripts back as they improve. Nothing to configure per request, and nothing model-specific in your client: switching engines later means pointing at a different endpoint, not rewriting the integration.
const ws = new WebSocket("wss://<your-endpoint>/transcribe");
ws.onopen = () => {
// 16 kHz mono Float32Array, samples in [-1, 1].
// Any framing works: the service concatenates the bytes it receives.
const pcm = getFloat32Audio();
const CHUNK = 8000; // 0.5 s
let i = 0;
const timer = setInterval(() => {
if (i >= pcm.length) {
clearInterval(timer);
ws.send(JSON.stringify({ action: "finish" }));
return;
}
const slice = pcm.subarray(i, i + CHUNK);
ws.send(slice.buffer.slice(slice.byteOffset, slice.byteOffset + slice.byteLength));
i += CHUNK;
}, 500);
};
ws.onmessage = (event) => {
const msg = JSON.parse(event.data);
// Read the TOP-LEVEL is_final. Partials are complete re-transcriptions of
// everything received so far, so replace the transcript, never append to it.
transcript.textContent = msg.text;
if (msg.is_final) {
console.log("rtf", msg.rtf, "audio_s", msg.audio_duration_secs);
ws.close();
}
};import asyncio, json, wave
import numpy as np
import websockets
URI = "wss://<your-endpoint>/transcribe"
async def transcribe(path: str) -> str:
with wave.open(path, "rb") as wf:
assert wf.getframerate() == 16000, "RustSTT expects 16 kHz audio"
assert wf.getnchannels() == 1, "RustSTT expects mono audio"
pcm16 = np.frombuffer(wf.readframes(wf.getnframes()), dtype=np.int16)
audio = pcm16.astype(np.float32) / 32768.0 # -> f32 in [-1.0, 1.0]
final_text = ""
async with websockets.connect(URI) as ws:
async def receive():
nonlocal final_text
async for raw in ws:
msg = json.loads(raw)
if msg.get("is_final"): # top-level field only
final_text = msg.get("text", "")
return
print("partial:", msg.get("text", ""))
receiver = asyncio.create_task(receive())
for i in range(0, len(audio), 8000): # 0.5 s chunks
await ws.send(audio[i:i + 8000].tobytes())
await asyncio.sleep(0.5)
await ws.send('{"action": "finish"}')
await receiver
return final_text
Every message carries rtf: inference time divided by audio duration. Below
1.0 means the audio was transcribed faster than it plays. Read it from the final result,
where it describes the whole utterance.
{
"is_final": true,
"text": "the transcript of everything sent on this connection",
"inference_time_secs": 0.31,
"audio_duration_secs": 6.0,
"rtf": 0.0517,
"model_name": "gigaam-v3-e2e-ctc",
"segments": [
{ "start": 0.0, "end": 6.0, "text": "...", "confidence": null }
],
"language": "ru"
}Segment behaviour differs per engine. Whisper returns 30-second chunk segments with a confidence value, GigaAM returns chunk segments above 30 seconds, and Parakeet and Qwen3-ASR return none. No engine returns word-level timestamps.
Architecture, language coverage, timestamp behaviour and price are properties of the engines, so they are here. Accuracy and latency are not: we publish no benchmark numbers because we have not measured them under conditions worth quoting.
| Property | GigaAM v3 E2E CTC gigaam | Parakeet TDT v3 parakeet | Qwen3-ASR qwen3 | Whisper Large v3 Turbo whisper |
|---|---|---|---|---|
| Architecture | Conformer encoder with a CTC head | FastConformer, LSTM prediction network, token-and-duration transducer | AuT audio encoder feeding a Qwen3 language-model decoder | Encoder-decoder transformer |
| Parameters | 220M | 627,090,606 | 0.6B | 809M |
| Languages | Russian only | 25 European languages | 20 languages listed | Broadest coverage |
| Segments | Chunk-level segments above 30 seconds of audio | No segments or timestamps | No segments or timestamps | Chunk-level segments with a confidence value |
| Word timestamps | No | No | No | No |
| Per audio hour | ~$0.50 | ~$0.75 | ~$0.75 | ~$1.00 |
Scroll the table sideways to see every engine.
The rate follows the engine you run. There are no seats, no tiers, no monthly minimum and no commitment to sign. Ten hours of Russian audio through GigaAM comes to roughly $5.
Prices are approximate and quoted in US dollars. Managed endpoints include access control for
up to three of your domains; each additional domain is $10. localhost cannot be
used as an allowed domain. IP-based allowlisting is available instead if that suits your
integration better.
Organisational deployment for teams that need several endpoints, their own subdomain-based access configuration and provisioning handled as a commercial relationship rather than ticket by ticket. Terms are set per organisation.
An annual licence to run the decoder on hardware you own and pay for. The service runs inside your own infrastructure, and the licence carries no access to ours. Supplied in a restricted form, tied to a specific engine and configuration.
That is not a stylistic preference, it is visible in how the image is assembled. Python does the offline work of turning upstream checkpoints into safetensors. The stage that does it is then left behind, and the final image copies in exactly two things.
chef
FROM CUDA 12.2 devel
Rust toolchain and cargo-chef.
planner
FROM chef
Produces the dependency recipe so cargo layers stay cacheable.
builder
FROM chef
Cooks the dependencies, then builds the release binary.
copied out the release binary
downloader
FROM CUDA 12.2 runtime
Python, torch, huggingface_hub and safetensors. Downloads and converts the checkpoints.
copied out converted weights
Not inherited by the final image
runtime
FROM CUDA 12.2 runtime
ca-certificates, curl, libssl3, cloudflared. Copies in the binary and the weights.
Every tensor operation runs through Candle, the pure-Rust framework. There is no libtorch, no ONNX Runtime, no whisper.cpp, no Triton and no separate inference server to keep alive. Weights are memory-mapped from safetensors or GGUF and read straight into the graph.
Greedy CTC decoding, Whisper autoregressive decoding with temperature fallback, TDT greedy decoding with duration prediction and language-model decoding with a KV cache are each implemented in this codebase. The two largest crates are the from-scratch Conformer and FastConformer implementations.
A running instance makes no outbound calls. The weights are already in the image, so there is no model registry to reach, no telemetry endpoint and no analytics. Audio is transcribed in memory and never written to disk, and the service keeps no transcript database.
A deployment is one process holding one engine on one device, with the weights already baked into its image. Scaling out means more of those, which is why a deployment can be dedicated to a single customer without anything clever happening underneath.
NVIDIA CUDA Production path
CUDA 12. The compute capability an image targets is a build-time argument, so a self-hosted build can be matched to the GPU you actually have. GigaAM and Parakeet run in BF16 here.
CPU Supported
F32 precision, and all four engines run. This is the local and development path rather than the volume one.
Apple Metal Custom build
The Metal path exists and is the most carefully hardened one: a staged device probe runs before heavy weights load, and explicit barriers sit between inference stages. It needs a build configured for it.
The device is fixed when the process starts. There is no automatic fallback: an unavailable device is a startup error rather than a silent downgrade to something slower.
| Concern | Managed | Self-hosted |
|---|---|---|
| GPU hardware and its cost | We provide it | You provide it |
| Model weights and conversion | We handle it | Covered by the licence terms |
| Container operation and updates | We handle it | You handle it |
| TLS and public hostname | We handle it | You handle it |
| Access to our infrastructure | Included | None |
| Billing basis | Per audio hour | Annual licence |
Samples in, a transcription result out. No model type appears anywhere in the server, and no engine crate knows another exists. That is what makes the wire protocol stable while the catalogue underneath it changes.
pub trait AsrModel: Send {
fn name(&self) -> &str;
fn model_type(&self) -> ModelType;
fn sample_rate(&self) -> u32 { 16_000 }
fn supported_languages(&self) -> &[&str];
fn model_info(&self) -> ModelInfo;
fn transcribe(&mut self, samples: &[f32], options: &TranscribeOptions)
-> AsrResult<TranscriptionResult>;
}A CTC head, a transducer, an encoder-decoder transformer and a language model all sit behind those six methods without leaking their differences upward. Adding a fifth engine touches the new crate, one enum, one dispatch arm and one feature flag. It does not touch the protocol, the response schema or your client.
Wire surface
asr-server 192 LoC
axum WebSocket server, env config, connection loop. Builds the service binary, named rustasr in the repository.
Dispatch
asr-engine 236 LoC
Resolves a model type to a decoder behind Cargo features, so a build can ship exactly one engine.
Engines
model-parakeet 2216 LoC
FastConformer, LSTM prediction network, TDT decoder.
model-gigaam 1454 LoC
Conformer encoder, CTC head, greedy CTC decoder.
model-whisper 768 LoC
Chunking, no-speech filtering, fallback decoding.
model-qwen3 257 LoC
Adapter onto the Qwen3 pipeline.
Qwen3 pipeline
asr-pipeline 996 LoC
Mel to encoder to decoder, tokenising and output parsing.
aut-encoder 1057 LoC
AuT audio encoder with rotary positional encoding.
qwen3-decoder 1108 LoC
Qwen3 decoder layers, KV cache, audio to text projector.
Foundations
asr-core 1113 LoC
The AsrModel trait, shared types, typed errors, model registry, weight-file resolution.
audio 684 LoC
WAV loading, mono downmix, resampling, mel-spectrogram extraction.
Tensors
candle
candle-core, candle-nn and candle-transformers. Pure Rust, no external runtime.
Around 10,100 lines of Rust across the workspace, on edition 2024. The two largest crates are the from-scratch Parakeet and GigaAM implementations, which is where the claim about not being a wrapper actually lives.
Raw PCM, 32-bit float, little-endian, 16 kHz, mono, with samples in the range -1.0 to 1.0. You send it as WebSocket binary frames. The service does not decode container formats and does not resample, so converting to 16 kHz mono is your side of the boundary.
No engine returns word-level timestamps. Whisper Large v3 Turbo returns segments at 30-second chunk granularity with a confidence value. GigaAM returns chunk-level segments for audio longer than 30 seconds. Parakeet and Qwen3-ASR return no segments at all. If timestamps matter to your product, choose Whisper.
GigaAM v3 E2E CTC is the Russian specialist and the lowest rate at about $0.50 per audio hour. Parakeet, Qwen3-ASR and Whisper all list Russian too, so if you also need other languages on the same endpoint, one of those is the better fit.
No. Each deployment runs exactly one engine on exactly one device, both fixed when the process starts. Switching engines means a different endpoint. That is a deliberate property: an instance holds one set of weights and nothing else competes for them.
The service transcribes audio in memory and never writes it to disk. There is no database, no object storage and no cache in the service. We do not publish a formal retention policy, a compliance certification or a deletion guarantee, and we will not claim ones we do not have.
Managed endpoints sit behind an access-control layer that we configure for you. You can restrict access by allowed domain or by IP address, whichever suits your integration. The standard configuration includes up to three of your domains. localhost cannot be used as an allowed domain.
Real-time factor: inference time divided by audio duration. A value below 1.0 means the audio was transcribed faster than it plays. Every response carries it, partial and final. Read it from the final result, where it describes the whole utterance unambiguously.
The production path is an NVIDIA GPU with CUDA 12. The compute capability an image targets is a build-time argument, so the build can be matched to the GPU you have. A CPU backend exists and works for all four engines. An Apple Metal backend exists in the code and requires a custom build.
Email us with the language you need to transcribe, roughly how many audio hours per month, and the domains or IP addresses that will connect. We provision an endpoint and send you the integration details. Onboarding is handled by hand, not by a signup form.
The language, roughly how many audio hours a month, and the domains or IP addresses that will connect. That is enough for us to come back with an engine recommendation and an endpoint.