Conversation
Exports CoreEncoderStatefull, CoreDecoderStatefull, and FrameSyncNet weights to C source files (rade_enc_v2_data, rade_dec_v2_data, rade_sync_data) for use in the radae_nopy C port. Mirrors the rx2.py weight loading pattern: instantiate with w1_dec=w1_dec_stateful=128, filter mismatched checkpoint entries, then use core_encoder/decoder_statefull_load_state_dict() to copy weights from the trained CoreEncoder/Decoder into the stateful variants. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
stateful_encoder_v2.py: reference Python script that runs CoreEncoderStatefull and CoreDecoderStatefull one step at a time, matching C port behaviour. Uses bottleneck=0 and rx2.py weight-loading pattern. CMakeLists.txt: add radae_nopy_v2_enc and radae_nopy_v2_dec ctests. Rename existing V1 nopy ctests to radae_nopy_v1_* for clarity. Both V2 tests pass with loss 0.096 (enc: C enc -> Python dec, dec: Python enc -> C dec vs Python dec, delta 0.000). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Compares C radae_v2_tx IQ output against tx2.py reference. Checks data diff_rms/sig_rms < 3% and EOO max diff < 1e-4. Observed: 1.2% data diff (encoder float differences), 9e-6 EOO diff. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tests the full C tx -> C rx pipeline at high SNR (no noise): lpcnet_demo features -> radae_v2_tx -> radae_v2_rx -> loss check. Loss threshold 0.2 (observed 0.083 noiseless). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Four new ctests (radae_nopy_v2_rx_awgn, radae_nopy_v2_rx_awgn_low, radae_nopy_v2_rx_mpp, radae_nopy_v2_rx_mpp_low) check that the C port of the V2 receiver (radae_v2_rx) achieves acceptable loss vs the reference features across AWGN and MPP channels at high and low SNR. Each test uses inference.sh to generate channel IQ, decodes with the C receiver, and checks the result with loss.py. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds three new ctests for the RADE V2 C receiver: - radae_nopy_v2_rx_eoo: EOO detection on MPP channel at EbNodB=8 - radae_nopy_v2_rx_snr_high: SNR estimator at high SNR (EbNodB=12, tol ±3.0 dB) - radae_nopy_v2_rx_snr_low: SNR estimator at low SNR (EbNodB=2, tol ±2.0 dB) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds guidance for contributors on test types, reproducibility requirements, and how to perform a software-only loss baseline test using tx2.py, rx2.py, and loss.py. Also improves the Stored File Tests section with clearer OTA vs simulated channel paths. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add radev2 mode to asr_test.sh (genie receiver via inference.sh), radev2 function to asr_test_top.sh, plot_wer_v2 to radae_plots.m, soundfile fallback in asr_wer.py for torchaudio >= 2.6, and README updates for new test commands and soundfile dependency. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Allows running specific test modes e.g. `./asr_test_top.sh radev2 -n 100` instead of always running all tests. Removes hardcoded exit 0 that was blocking radev2/ssb/rade sweeps from running. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- plot_wer_v2: larger fonts, 2 dB x-axis ticks, xmin=-6 for even tick labels - asr_test_top.sh: denser low-SNR points for rade/radev2, g_mpp_1200s.f32 for all MPP sweeps, updated datestamp to 260702 - README: updated MPP fading file instructions and plot examples Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- src/ch.c: add --fading_adv option to seek into fading file before processing, matching codec2-dev ch behaviour; radae ch now has PATH priority in asr_test.sh - asr_test.sh: add CPAPR as 4th column in SSB/700D results (after WER to preserve column 3=WER compatibility); extract per-file CPAPR from ch stderr log - asr_test_top.sh: prepend "# n=$n" comment to each results file so sample count is recorded for future reference - radae_plots.m: plot_wer_v2 EPS output uses 5dB x-axis divisions and xmin=-5 for clean grid; PNG retains 2dB steps and xmin=-6 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix font restore bug: PNG block no longer restores fonts before EPS print - set_fonts(30) for EPS gives 9pt labels matching other paper figures - Legend shifted right 10% and up 40% to clear SSB MPP curve - Y axis truncated at 40% WER for EPS; PNG retains 60% Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- asr_test.sh: add CPAPR as last column in SSB/700D results; fix V1 (rade mode) inference cmd to include --tanh_clipper --ssb_bpf; reorder PATH so radae ch takes precedence over codec2-dev ch - asr_test_top.sh: switch to g_mpp_4000s.f32 (was 1200s, too short for n=200); add # n=$n comment to results files; update n=200 default (n=500 requires ~120G RAM); update results_file prefix to 260704_asr - radae_plots.m: tune plot_wer_v2 for EPS output (font size, legend position, x-axis range, y-axis truncation at 40% WER) - README.md: update MPP fading file generation to 4000s Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace all radae_v2_tx/radae_v2_rx references with radae_tx --v2 / radae_rx --v2. Fix EOO test regex to match new "End-of-over detected" message. V1 and V2 ctests now symmetric through the same API programs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Consistent with radae_nopy change. Update fprintf format to %.1f. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Owner
Author
|
Bit of a radical suggestion - but can we depreciate the C API from this (reference) RADE Python repo? Would remove a lot of cruft. Please let me know what you think. |
Exercises the V1 EOO callsign channel and V2 BPSK data symbol channel through the public API using the new standalone C test binaries. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Encodes speech WAV → RADE WAV → decoded WAV and checks for a positive mean SNR in the rade_rx_wav summary, confirming sync was achieved. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
V2 WAV pipeline sanity check: encode speech WAV with rade_tx_wav --v2, decode with rade_rx_wav --v2, confirm positive SNR while in sync. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Decodes a KiwiSDR or similar OTA WAV recording using the V2 Python rx pipeline. Resamples to 8kHz, converts to f32 IQ, runs rx2.sh with AGC, writes all diagnostic artefacts to a subdirectory named after the input file, and generates a plots.png via octave plot_v2_logs. Report is terse by default (key fields only, no bash -x trace); pass --verbose for full output. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…after Scope Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
RADAE_NOPY_BUILD_DIR -> RADE_C_BUILD_DIR radae_nopy_* tests -> rade_c_* tests Update README cmake invocation and ctest pattern to match. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds per-frame loss statistics (mean/median/percentiles/outliers) and histogram output, plus PNG export. Reworks the --plot time axis to use seconds instead of raw frame indices, and aligns the features_hat2 comparison curve using its acq_time offset. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The per-frame loss loop was starting at min_start but using features_hat_seq_length - min_start frames, which skipped the last min_start frames that are included in min_loss. Now uses features_hat_seq_length so --stats mean exactly matches the reported min_loss value. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Four new ctests verifying the AGC port to rade_rx_v2_process() (freedv/rade_c#dr-v2-rx-gain), gated behind RADE_C_BUILD_DIR: - rade_c_v2_rx_agc_clean_high/low: C tx -> C rx, no channel impairment, gain 4/0.25. Isolates gain-staging regressions from channel noise -- the freedv-gui/backend bug this was written for would have been invisible against the existing MPP-only AGC tests' loose tolerance, but stands out clearly on a clean channel. - rade_c_v2_rx_agc_mpp_high/low: mirrors rade_c_v2_rx_mpp/_low with the same gain mismatch, Python tx (inference.sh) since it's the C rx's AGC under test. Thresholds reuse the existing mpp tests' values (0.15/0.40); measured AGC-on loss (0.103/0.285) sits well inside them, matching the Python reference's own AGC ctest results for the same gain values. All 4 pass; full existing rade_c_v2_* suite (13 tests) re-run alongside them with no regressions now that AGC defaults to on. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Two new ctests model the actual verification process used to find and fix today's AGC gap: C tx -> C rx, clean channel, +/-6dB gain mismatch (matching the real freedv-gui/freedv-backend TX level bug, rather than the AGC tests' wider +/-12dB) compared against a gain=1.0 baseline with a tight 0.01 delta -- sensitive enough to clearly fail if AGC regressed (measured delta today: 0.000). Also fixes a real bug in the previous commit's 4 AGC ctests: the two clean-channel ones (rade_c_v2_rx_agc_clean_high/low) never generated their own features_in.f32, silently relying on it being left over from whichever test happened to run first. Run standalone, or after a brian_g8sez.wav-based test, this would either fail outright or -- worse -- silently compare against the wrong wav file's features. All four clean-channel tests now generate features_in.f32 from wav/all.wav explicitly. Verified by deliberately corrupting features_in.f32 with brian_g8sez.wav's data first, then running rade_c_v2_rx_gain_p6db standalone -- passes correctly regardless of prior state. Full rade_c_v2_* suite (19 tests) passes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Uses rade_tx_wav/rade_rx_wav -f feature file output to measure loss end-to-end (no noise, nominal gain). Threshold set at 5% above software-only baseline (rade_c#989770e, AGC on, wav/all.wav): V2: 0.088 (baseline 0.084), V1: 0.119 (baseline 0.113). Tighter than the 10% integrator criterion -- no HW in the loop. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Validates that Re{tx+noise} noise folding is handled correctly by the
WAV rx. Negative-frequency noise aliases to positive frequencies,
giving the real rx ~3 dB less effective SNR than the complex IQ rx.
Path A: complex IQ rx at EbNodB=12 (SNR3k ~5.6 dB).
Path B: real WAV rx at EbNodB=15 (+3 dB to compensate noise penalty).
Both reach the same loss; delta threshold 0.005.
Sensitivity validated: correct 3 dB offset gives delta~0.002 (PASS),
wrong 2.5 dB offset gives delta~0.011 (FAIL).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two new ctests (rade_c_readme_smoke_wav, rade_c_readme_smoke_iq) run the exact quick-start commands from the README using voice.wav. No loss threshold -- just verifies the example pipelines complete without error. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Follow rade_c rename of voice.wav -> input_sample.wav. Also give wav and IQ smoke test outputs distinct /tmp names (rade_c_smoke_wav_* and rade_c_smoke_iq_*) so they don't clobber each other. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
9 tasks
…section verification_procedure.md: formal procedure for integration verification before Phase D OTA testing. Covers software loopback (mandatory), OTAC and OTC (at least one mandatory for hardware integrations). Requires clean signal path (transfer function of 1) in both directions, loss.py from this repo, and wav/all.wav as the test file. template.md: checklist form for testers to complete and submit as evidence of a verified integration. README.md: replace verbose prose with a short intro linking to the procedure, plus the baseline command lines. Baseline framing updated to require re-running with current version rather than using a cached loss value. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…cal OR Explains the rationale for the L2/L3 choice: OTAC covers all application integration paths; OTC is the appropriate alternative for SDRs and hardware radios where OTAC is impractical (e.g. Flex). At least one is mandatory. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ixes - Add 'Worked Example of Loss Tests' section to verification procedure, showing clip value selection with before/after loss plots - Add automated pass/fail subsection using --compare/--delta for CI use - loss.py: print FAIL and exit(1) for --compare failures; replace quit() with sys.exit(1) throughout for correct CI return codes Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…an PNGs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tested pipeline: lpcnet_demo -> tx2.py -> f32toint16.py --real -> sox -> WAV -> rade_rx_wav gives loss 0.083, consistent with baseline. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Revised RADE verification procedure - thanks Peter & Mooneer for review
…lay versus loss issue
…lay versus loss issue
…mate Fixes Issue #8 (C receiver loss varying with --prepend_noise delay). The per-block RMS gain wasn't aligned to the estimated symbol timing, so shifting the true symbol boundary against the fixed block grid produced delay-dependent gain error. A persistent, per-sample IIR-smoothed power estimate removes the block-grid dependency. Also adds the 260830_inf AGC-curve config to compare_models_inf.sh, and an INF_OPTS passthrough + stddev summary to test/v2_c_timing.sh.
--no_eoo (rx2.py/radae_v2.py): lets algorithm-level testing isolate the V2 EOO detector's known non-zero false-positive rate, which was otherwise corrupting long-file AGC/delay curve runs by triggering a spurious resync partway through. compare_models_inf.sh: 260830_inf now runs with --no_eoo; new 260831_inf config sweeps loss-vs-SNR at a few fixed --prepend_noise delays (single AGC gain) to show the V2 rx is delay-robust, not just gain-robust. run_model_rx2 now threads --prepend_noise through to inference.sh (previously only reached rx2.sh via passthrough, silently dropped). radae_plots.m / test/v2_c_timing.sh: new loss_delay_plot() and RESULTS/PLOT env vars, so the existing delay sweep can also produce a LaTeX-includable (epslatex) loss-vs-delay figure for the V2 test report, in addition to its existing stdout summary.
New ctest sweeps 3 delay points spanning the documented worst-case
window and checks C receiver loss stddev stays below 0.002. Empirically
confirmed against both AGC implementations: old block-based RMS gives
stddev ~0.0026 (fails), new IIR-smoothed gives ~0.0000 (passes) --
verified by temporarily building rade_c's pre-fix commit and running
the exact same test, not just assumed from the fix's own before/after
numbers.
Also fixes test/v2_c_timing.sh's C-side delta_hat extraction: it was
grepping for a diagnostic-only debug line specific to the
investigation branch, rather than the pre-existing (now CLI-reachable
via -v 3, see rade_c) per-frame status line. Hardened summary-row
construction to use ${var:-NA} instead of raw substitution, so a
missing field shows up as a visible "NA" rather than silently
shifting all subsequent columns (exactly the failure mode that broke
the original comparison against the pre-fix binary).
New figures/table: loss vs SNR at 3 fixed delays (260831_inf), loss vs delay at fixed high SNR (v2_delay_loss + table), AGC curves regenerated (260830_inf, now includes V1 baseline). Notes the regenerated section's commit hashes since the rest of the report predates the AGC fix. Fixes an underscore-escaping bug in loss_delay_plot's legend text (LaTeX "Missing $" error) and widens its y-axis range for consistent scaling with the other loss plots.
The loss-vs-delay sweep (Figure 4, Table 5) compared Python without AGC (default off) to C with AGC (default on) -- not a matched test. Rerun with PY_OPTS="--agc" so both sides run with AGC enabled; updated stddev/min/max in the table and the cited command line. Still flat and tightly matched (stddev 0.0006 Python, 0.0005 C), now a genuine apples-to-apples result.
…or V2 Uses rade_tx_wav/rade_rx_wav instead of Python inference for RADE V2 in ota_test.sh, so the real-hardware OTC test can exercise the C engine through a known-good reference application. Also fixes the V2 loss reference to use each engine's own DSP-extracted input features (features_in_tx2.f32) rather than a single shared features_in.f32, since the C and Python genie decode paths have different acquisition timing and were being measured against a reference that didn't match either. Adds matching ota_test_v2_c_awgn/mpp ctests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note GitHub workflow tests run here will only test Python code, as that's the default. See https://github.com/drowe67/radae_nopy/tree/dr-radev2 for V2 C Port test results.