Detection with Guarantees: Distribution-Free Control of Non-Monotone Event-Level Miss Rates in Polyphonic Sound Event Detection

Source abstract

Deployed sound event detection (SED) systems ship with thresholds tuned for benchmark scores and no statement about how many events they miss. We study distribution-free, finite-sample control of the event-level miss share for a frozen SED system. Under standard event matching rules the miss rate is not monotone in the threshold: collars un-match predictions that grow as it drops, and permissive thresholds merge detections, so neither end of the axis is a safe anchor; fixed-extent bounding-box decoding restores exact monotonicity under every rule. On RealDESED with a reproduced state-of-the-art baseline, feasibility comes first: across four training runs no thresholding of any studied decoding attains a 10% miss share (tightest floor 0.113±0.005), so the requestable level is bounded below by that floor, which also fixes our operating levels from validation data alone. Comparing seven calibration routes under a proven-certificate criterion and a repeated-splits acceptance gate shows that the gate rewards slack, not validity: passing it is no evidence of a guarantee. The tight certificates control the clip-balanced (per-clip average) miss rate: the certified frontier attains a 20% clip-balanced level at 379 false positives per hour, while the induced pooled share runs 2–5 points higher and admits only a capped surrogate certificate. Finally, we measure where certificates break: subgroups, decoder swaps, corpus transfer.