Crash: avformat decoder cache (size 4) evicts state another thread is still decoding with #1318

Environment: MLT 7.40.0 (Fedora 44 package, x86_64), FFmpeg 8.1.2. The code involved is unchanged on master (0926a75).

Found while building a GTK4 video editor on MLT; the repro below uses only melt/libmlt.

What happens

producer_avformat keeps its decoder state in the service cache "producer_avformat", whose default size is 4 process-wide (mlt_service_cache_put()). When a fifth producer decodes, the least recently used entry is evicted and its state closed, even if another thread is decoding with it at that moment. So more than four producers decoding concurrently on different threads crash.

Repro

Sixteen threads, each with its own avformat producer of the same clip (all opened on the main thread first, so the lazy-init races can't hit), read 150 frames of video and audio (C, 60 lines): https://github.com/iDoMeteor/u-studio-video-editor/tree/70841c3/tools/upstream-repros/mlt/avformat-cache-limit

ffmpeg -f lavfi -i testsrc2=s=640x360:r=25:d=8 -f lavfi -i sine=d=8 \
    -c:v libx264 -g 25 -pix_fmt yuv420p -c:a aac clip.mp4
cc -g cachelimit.c -o cachelimit -pthread $(pkg-config --cflags --libs mlt-framework-7)
for i in $(seq 30); do ./cachelimit clip.mp4 || echo crash; done | grep -c crash        # 30
for i in $(seq 30); do ./cachelimit clip.mp4 raise || echo crash; done | grep -c crash  # 0

Every crash is in __pthread_mutex_lock ← mlt_properties_get ← producer_get_frame (avformat) ← mlt_service_get_frame. raise calls mlt_service_cache_set_size(…, "producer_avformat", 17) first: 0 of 30. With 8 threads the rate varies with load (2 to 13 of 30 here).

Expected: concurrent decoding on separate producers is safe, or at worst slower. Actual: use of freed decoder state.

Notes

Either an entry in use should survive eviction until its user is done with it, or the limit should be documented as having to cover every concurrent decoder. kdenlive raises the size for the same reason; we set it to threads + 2 per track.