Export: this page prints cleanly (File → Print → Save as PDF; the layout has print styles). A PDF, the standalone HTML, every chart as SVG and PNG, and the underlying tables as CSV are in asr/reports/corpus-datasheet-export.zip.

Rebbe Yiddish ASR Corpus

Everything the speech-recognition models in this project train and test on, updated 2026-09-24 with the new source data, the v4 results, the audit of the delivered transcripts and the plan for v5: where the audio and text come from, how they were paired and cut into clips, what was filtered, how the fixed evaluation sets were chosen, the pseudo-labeled extension, and what each training run did with it. All figures are computed from the manifests in asr/data/manifests/ and the evaluation reports in asr/reports/; nothing here is estimated.

Hours are audio inside clips. Words are counted on the training-target text (punctuation kept, vowel points removed). Hebrew years: 5711 = 1950/51 … 5752 = 1991/92; 5783 = 2022/23 … 5787 = 2026/27.

What one training example is

A clip is a stretch of at most 28 s of the Rebbe's speech, cut from a Daily Sicha recording, paired with the words the published Yiddish hanacha (the written transcript of the talk) gives for that stretch. The model sees the 16 kHz mono audio and learns to emit the text. A typical clip is 20 s long and carries about 42 words at 2.1 words per second.

Where the data comes from

The Daily Sicha (thedailysicha.com) publishes one ~10-minute excerpt of a farbrengen per day with a Yiddish hanacha, and for recent years a sync file with segment timings. The project holds the daily mp3s for five publication years (5783–5787) and the site's text for each; the source talks span 1950 to 1992. Four older archive years (5778–5781, 1,005 files) have audio only and are the transcription target of the project.

New since 2026-09-23: hanachos and audio for 5775–5781

The Daily Sicha delivered the Yiddish hanachos as one PDF per year (5768–5782 and 5784–5787) together with the recordings of 5775–5777. The four years the project had been transcribing without text (5778–5781) and three more years are now paired: 1,758 days, 306 h of recordings, 2.31 M words, more than doubling the corpus once aligned. Nine PDFs have a clean text layer. Ten (5768–5770, 5774–5780) are set in custom-encoded fonts: every glyph code is a fixed permutation of the Hebrew alphabet, different per font. They are decoded exactly rather than OCR'd: pipeline/pdf_fontmap.py hill-climbs each font's code→letter map by the share of decoded words that exist in the training vocabulary (0.91–0.93 for the body font) and pins the bold header font with the words every day's header shares (בס"ד. "השיחה היומית" ליום, the year, התוכן, the month names). Tesseract on the same pages scored 0.39 WER; the exact decode leaves no systematic errors (the drafts-vs-text numbers below equal the clean-layer year).

Splits

Splitting is by day, never by clip, so no test audio shares a recording with training audio. The rule, fixed on 2026-09-16 and never changed:

Dev days: 001 3-Tishrei, 028 11-Cheshvan, 055 13-Kislev, 082 15-Teiveis, 109 17-Shvat, 137 20-Adar, 164 27-Nisan, 191 28-Iyar, 023 2-Cheshvan 5787, 050 3-Kisleiv 5787, 077 5-Teiveis 5787, 104 7-Shvat 5787, 131 9-Adar 1 5787, 158 10-Adar 2 5787, 185 13-Nisan 5787, 213 20-Iyar 5787, 240 24-Sivan 5787, 267 25-Tamuz 5787, 294 28-Av 5787, 321 29-Elul 5787

Test days: 227 13-Tamuz, 237 24-Tamuz, 242 1-Av, 246 6-Av, 251 12-Av, 254 15-Av, 260 22-Av, 263 26-Av, 265 28-Av, 268 1-Elul, 272 6-Elul, 278 13-Elul, 281 17-Elul, 284 20-Elul, 001 3-Tishrei 5787, 006 9-Tishrei 5787, 009 13-Tishrei 5787, 016 24-Tishrei 5787, 018 26-Tishrei 5787, 021 30-Tishrei 5787

Caveats for the scientist: (1) the dev WER printed during training uses a fixed 200-clip subset of dev decoded with the HF generate path; the full-dev and test numbers in the results section use the full sets. (2) One test day, 9 Tishrei 5787, scores 0.28–0.34 for every model and is a suspected reference-quality outlier; it is kept because the split is frozen. (3) Out-of-vocabulary rate against the clean training text: dev 1.1%, test 1.2%.

Clip anatomy

Alignment quality

For the 92% of clips timed by the project's own aligner, the anchor rate is the share of a clip's reference words that were matched exactly or fuzzily to a word the Yiddish Whisper model heard at a known time; the remaining words are placed by interpolation. Clips below 0.30 are excluded at build time, and clip boundaries must fall on anchored words.

Text conventions and normalization

Training target

normalize.training_text: the hanacha paragraph text with vowel points (nikkud) removed, punctuation and the abbreviation marks kept (geresh ' and gershayim "), bracket characters removed but the bracketed words kept (they are spoken asides; dropping them raised WER for every model). 27,644 of 30,212 raw references carried nikkud; 68.8% of clips contain at least one gershayim abbreviation.

Scoring text

normalize.scoring_text: Hebrew letters only. All WER and CER figures anywhere in the project are computed on this form, so punctuation, quotes and spelling of marks never count as errors, but a misspelled abbreviation does.

Reference frames

Site sync timings can refer to the raw excerpt or to the daily file with its dedication; calibrate_site_timings.py decides per day and compute_offsets.py supplies the offset. Days whose offset could not be established are never used with site timings.

Decoder note

Whisper's default token-suppression list blocks the ASCII double quote, which in Hebrew text is the gershayim. Every fine-tuned model learned to write רש"י and הקב"ה but could not emit them through faster-whisper until 2026-09-21; the production decoder (HF fp32 on the GPU, VAD windows, no timestamp tokens) has no such list.

Pseudo-labeled extension (v3 self-training)

A second pseudo-label set (26,797 clips, 169.4 h) was cut on 2026-09-22 from the third transcription pass (model turbo lr 3e-5, whole-file WER 0.081); the two v4 self-training runs use it, the v3 runs below used the first set. Result of v3 on the full test set: 0.082 / 0.084 / 0.086 whole-file for the three recipes — each 0.5–1 point better than its starting model, none better than the supervised lr 3e-5 model (0.081).

The 1,005 recordings of 5778–5781 have no human text. They were transcribed with the best model (turbo full fine-tune, lr 2e-5) through the production decoder; each VAD window of up to 28 s became a clip with the model's text as its label. Windows with mean token log-probability below −0.6, fewer than two words, or repetitive text (decoder loops, 32 windows) were dropped, and labels were normalized exactly like human targets.

Training runs

All runs fine-tune ivrit.ai's Yiddish-adapted Whisper (large-v3 or large-v3-turbo) in fp32 on Apple-silicon GPUs, one run per Mac Studio, from the same clean manifest; batch 8 × 4 accumulation (32 clips per step), 870 steps per epoch, linear decay with 200 warm-up steps, evaluation every 500 steps on the 200-clip dev subset. Lines are dev WER; a dot is an evaluation.

Reading the curves: freezing the encoder (amber) plateaus at 0.19–0.22, LoRA (teal, purple) lands near 0.084, full fine-tuning of the turbo model reaches 0.0765 at lr 2e-5. v2 finished with large-v3 at lr 2e-5 on 0.0723 and turbo at lr 3e-5 on 0.0730 (both better than the v1 winner); continuing the v1 winner at lr 1e-5 only reached 0.0757. The v3 self-training runs ended with the best in-training scores of all (0.0686–0.0693) but scored 0.082–0.086 whole-file on the full test set, behind the production model — the 200-clip dev subset overstates models trained on pseudo-labels, so only the full test set ranks them. Pseudo-labels lifted each starting model by 0.5–1 point without surpassing the best supervised recipe. v4 probes higher learning rates, the turbo recipe on large-v3, a seed repeat of the production run, and self-training from the strongest starting points with fresher labels.

Results on the fixed test set

Two metrics, both against the human hanacha of the 20 test days: clip WER on the 488 test clips (audio cut on known timings) and whole-file WER where each day's full recording is decoded with voice-activity chunking and scored against the whole hanacha — the number that matters for the product. Decoder matters as much as model: the same weights score 0.132, 0.124 and 0.088 whole-file under the three decoders tried. The v2 winner on whole files is the turbo model trained at lr 3e-5 for 4 epochs (0.081, CER 0.045); large-v3 at lr 2e-5 is best on clips (0.083) but decodes three times slower and is a hair worse on whole files (0.082).

Accuracy on the product years: the delivered transcripts against the hanachos

With the PDFs in hand, the 1,005 machine transcripts of 5778–5781 (third pass, production model) can be scored against the printed text for the first time. These are whole-file numbers over all days, including days whose printed text is a different sicha or a heavily edited one, so they are upper bounds; the fixed test set (0.081) sits in the same range.

Where the errors are now

v5: the enlarged dataset and the runs on it (in progress)

Status at the snapshot: the fleet has aligned 5781 and is at 1,170 of the remaining 1,511 days (5775–5780, wave 2; ~6 days per minute). When it finishes, every node and this Mac rebuild the dataset from the same day records, the production model scans the train split, the filter drops clips with CER > 0.6, and a guarded launcher starts the runs below unless the cleaned manifest is small or a year lost more than 25% of its clips. Rules kept: data/splits.json unchanged (the 20 dev and 20 test days stay in 5786–5787); the new years go to train only, except test_b = 20 evenly spaced 5781 days (clean text layer) that are never trained on and are reported next to the main test set from now on. Expected size: about 66,000 clips, ~370 h (30,212 clips / 168 h in v4).

v4 verdict, complete (all six runs scored): on the old data nothing beats the production recipe; the seed repeat (0.080 vs 0.081 whole-file) puts run-to-run noise at ~0.001, so table differences under 0.003 mean nothing. Higher learning rates (4e-5, 5e-5) are worse; self-training never wins; large-v3 at lr 3e-5 (0.086 clips / 0.081 whole-file) equals the turbo on whole files at 2.5× the cost and does not reach large-v3 at lr 2e-5 on clips. The v5 data is the only remaining lever.

Next steps

1. Finish the v5 data (today). Alignment wave 2 → dataset → production-model scan → filter. Read the per-year drop rate: a year above ~10% means a decode or alignment problem in that year's text, not bad audio.
2. Train the three v5 runs (Thu–Sat) and score each on the fixed test set through the GPU path (whole-file WER is the ranking metric) and on test_b. Decide by whole-file test WER; anything under 0.003 apart is a tie.
3. If a v5 model wins: a fourth Phase 7 pass with it and the auto windows over 5775–5781 (the project owner's go-ahead needed; the third pass stands until then), then the per-year drafts-vs-hanacha table again.
4. Attack the remaining errors where they are: half of the remaining word errors are words never seen in training (names, sources, loshon-kodesh); the text-only years (5768–5770, 5774, 5782: ~1.4 M words) are material for a spelling/normalization layer or decoder biasing, and for consistent abbreviation forms.
5. Compute. The large-v3 runs take 30+ h per Mac Studio; the GB10 (CUDA, bf16) would cut that to hours once its GPU memory and disk are freed. Fleet alignment should move from static shards to a shared queue (the tail idled the fleet ~1 h per wave).
6. Product and review. Synced subtitles (an SRT per file exists) are the preferred first deliverable; nothing is published or made searchable until Rabonim/Mashpi'im and the hanachos' author have reviewed; ask The Daily Sicha team for the 5782 recordings (text exists) and confirm the two 5781 days whose printed text does not match the audio (17 Kislev, 9 Teves).
7. Housekeeping. Report every number from the GPU path only (faster-whisper long-form loses ~3 points and suppresses the gershayim by default); keep output_superseded/ and the windowing reports as the audit trail.

Reproducing any number on this page