Possible new FLAC problem with FL Solar BARs

[atruskie]

https://data.acousticobservatory.org/audio_recordings/802072

flac-decoding-bug.mp4

$ flac -d -f .\20210521T000000+1000_Spyglass-Wet-B_802072.flac .\20210521T000000+1000_Spyglass-Wet-B_802072.flac.wav

flac 1.3.4
Copyright (C) 2000-2009  Josh Coalson, 2011-2016  Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

20210521T000000+1000_Spyglass-Wet-B_802072.flac: WARNING, cannot check MD5 signature since it was unset in the STREAMINFO
done


20210521T000000+1000_Spyglass-Wet-B_802072.flac.wav: ERROR while decoding metadata
                                                     state = FLAC__STREAM_DECODER_END_OF_STREAM

It looks like most of the file is valid until the very last frames.

It's not much of a bug, but it might be enough to cause some problems.

Reported by @ninascarpelli

[atruskie]

OK, this might be affecting a larger range of people. We've had reports of this causing indefinite hangs in python programs

Two more sample files:

https://api.acousticobservatory.org/audio_recordings/590000/original (log file: "Z:\sd_card_rip\20210423_51_harvest_3\202\Card_758_20210423\logfile.txt", no faults logged in log file)

https://api.acousticobservatory.org/audio_recordings/588000/original

[atruskie]

I ran ffmpeg to convert the files, to see if it's decoder could get anywhere.

It looks like it stops at the same point the flac decoder does and emits just the first half the good portion of the file.

So for the 588000 file above:

• File size is 215,828,588 bytes
• Last valid FLAC frame is at 215,825,380
• Difference is 3208 bytes. The average frame size seems to be about 3220 bytes, so last frame is invalid?
• Metaflac reports duration as 7,194.8422675 seconds ( 158,646,272 samples )
• trancoded wave file is 3686.028481 seconds long
• last valid flac frame is at 3,686.1213
• These files have been repaired with FL010
  • A previous version of FL010 halved the erroneous sample count and used that value without counting frames

Theory A:

Could be a compound issue? Last frame invalid plus FL010

Theory B:

Could be an incorrect FL010 application. If we applied the modern FL010 fix algorithm that counts frames does the problem go away?

• Doesn't seem likely; the FLAC decoding errors seem new

---

Looking at a spectrogram, the last bytes of the file don't appear to WAVE samples so that discounts at FL011 (partial flac encoding problem)

---

Python code that hits a snag with the files

import soundfile

def load_audio(path, offset_s, window_size_s):
  with epath.Path(ap).open('rb') as f:
    sf = soundfile.SoundFile(f)
    offset = int(offset_s * sf.samplerate)
    window_size = int(window_size_s * sf.samplerate)
    audio = sf.read(offset + window_size)
    audio = audio[offset:offset + window_size]
  return audio

with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
  print(len(sf) / sf.samplerate / 60, flush=True)
  for i in range(120):
    print(i, flush=True)
    future = executor.submit(load_audio, ap, i * 60, 10)
    audio = future.result(timeout=5)
audio, sr = librosa.load(ap, sr=None)
print(audio.shape)