Apparently this function uses a bitrate heuristic to determine which
track is best. We don't want or need that, so just select the first
track with default disposition (e.g. FlagDefault=1 in Matroska).
Most WebM files don't have their default duration defined, so we need
to parse the Opus frame header to determine the duration. This is
needed for buffered range calculation.
Instead of using a single track entry for all blocks in the file, use a
lookup to get the info needed to calculate the timestamp for the
specific track a block belongs to. No change in behavior for
SampleIterator, since that only returns blocks from the track that was
passed. This will be useful for MSE, since it demuxes all tracks at
once.
Otherwise, the promise handlers may never be invoked, and we won't know
whether we're matching expectations.
Prior to 39d865b, this test did not actually check if the PulseAudio
stream was able to connect before passing the test. After that commit,
it would instead try to post the rejection to the main thread, which
would either lead to a crash preventing a use-after-free on the main
event loop, or in the more common case, the event loop would be freed
and never invoke the rejection callback.
This allows us to avoid returning a PlaybackStream in cases where the
async initialization fails.
This is a step towards more graceful fallbacks when audio fails in
AudioMixingSink.
For web audio, I reckon an occasional misjudged channel layout is
better than more frequent exceptions.
Signed PCM is normalized with unsigned max divided by 2, not
signed max. If you divide by the signed max (32767), you get headroom
that can exceed the threshold below -1.0. It's not audible, this mostly
matters for tests that assume correct normalization. But it turns out
there's no shortage of "golden ears" jackholes out there who swear they
can hear the difference.
The way that other classes interact with IncrementallyPopulatedStream
is now through a virtual interface MediaStream and MediaStreamCursor.
This way, we can have simpler implementations of reading media data
that will not require an RB tree and synchronization.
...and abstract away the stream/cursor blocking/aborting functionality
so that demuxers can implement or ignore those methods as they see fit.
This is a step towards implementing a wrapper demuxer for MSE streams.
We were allowing Matroska blocks with fixed-size lacing to contain
frames with non-divisible sizes. This should not be possible, as it
inherently means that trailing bytes will be discarded.
We now have a valid and invalid testcase for fixed-size lacing to
ensure our handling remains correct.
We don't actually need a Vector stack of bytes read for each element
we're reading out of a Matroska file, we already have the C++ stack
in which we can store the start and end of the master elements we're
reading.
This fixes an issue where seeks while parsing master elements would not
increment m_octets_read, so the master element could continue reading
further than intended.
This could cause a BlockGroup followed by a SimpleBlock to read as if
the BlockGroup contained the SimpleBlock, meaning that SampleIterator
would skip the SimpleBlock.
A test is added to ensure this doesn't regress again.
This saves us from having our own color conversion code, which was
taking up a fair amount of time in VideoDataProvider. With this change,
we should be able to play high resolution videos without interruptions
on machines where the CPU can keep up with decoding.
In order to make this change, ImmutableBitmap is now able to be
constructed with YUV data instead of an RBG bitmap. It holds onto a
YUVData instance that stores the buffers of image data, since Skia
itself doesn't take ownership of them.
In order to support greater than 8 bits of color depth, we normalize
the 10- or 12-bit color values into a 16-bit range.
This fixes a compile issue on FreeBSD where this would not compile as
the `pulse/pulseaudio.h` header is not in the default search path,
instead it is in `/usr/local/include'. This is a problem because this
test manually includes `PulseAudioWrappers.h`.
This ensures that we're using the reader for the particular thread that
the block was read from, avoiding any race conditions between seeks and
reads across threads.
We only need to get the frames from a block when requested by the
demuxer, so factor that out into a function that it can call when it is
outputting frames.
Implement PlaybackStream using WASAPI. The design is similar to
PlaybackStreamAudioUnit in that it uses a task queue. A high priority
thread is used to render the stream. All the stream controls save for
the exit being requested which happens on destruction of the stream are
managed by the render thread.
Due to the design of the windows audio mixer the audio we receive must
be resampled to match the sample rate of the mixer. We use a float based
interleaved PCM stream which matches both our existing code and the
audio mixer which internally usues floats.
Having to use a mutex around a queue for the task queue is suboptimal,
in a future PR a MPSC queue could be added to AK and used instead.
Refactor the FFmpeg and Matroska demuxers to consume data through
`IncrementallyPopulatedStream::Cursor` instead of a pointer to fully
buffered.
This change establishes a new rule: each track must be initialized with
its own cursor. Data providers now explicitly create a per-track context
via `Demuxer::create_context_for_track(track, cursor)`, and own pointer
to that cursor. In the upcoming changes, holding the cursor in the
provider would allow to signal "cancel blocking reads" so an
in-flight seek can fail immediately when a newer seek request arrives.
Windows currently doesn't have a LADYBIRD_AUDIO_BACKEND set, this means
Audio::PlaybackStream::create() always returns an Error. We should not
perform assertions in TestPlaybackStream that assume an implementation
always exists.
Instead of specifying the sample rate, channel count/map, etc. to the
PlaybackStream, we'll now use the output device's sample specification
whenever possible. If necessary, the stream will fall back to sane
default.
This hugely simplifies AudioMixingSink, since it no longer has to take
care of reinitializing the stream with a new sample specification when
it encounters a track with a higher sample rate or more channels. We
wouldn't be likely to benefit from this anyway, since it turns out that
at least Windows's virtual surround doesn't work through WASAPI at all,
and WASAPI likely wouldn't support downmixing.
This commit breaks playback of audio files that don't match the system
default audio device's sample rate and channel count. The next commit
introduces a converter into the pipeline to allow mixing of any sample
specification.
We were already assuming that our streams were using floats, we may as
well hardcode this. If we ever encounter a platform API that doesn't
support or convert from float, we can always bring this back.
Also, since we don't support big-endian systems, remove that check in
PulseAudioWrappers.
Posting callbacks to the main thread is now predicated on whether the
event loop reference is alive, preventing a stack-use-after-return.
The data providers will also check if they've been requested to exit
before calling deferred_invoke, though this is not going to be the case
unless the media element gets GCed while the media is playing.
Demuxer creation and track+duration extraction are moved to a separate
thread so that the media data byte buffer is no longer accessed from the
main thread. This will be important once the buffer is populated
incrementally, as having the main thread both populate and read from the
same buffer could easily lead to deadlocks. Aside from that, moving
demuxer creation off the main thread helps to be more responsive.
`VideoDataProvider` and `AudioDataProvider` now accept the main thread
event loop pointer as they are constructed from the thread responsible
for demuxer creation.
The stream was being kept alive until the moment before we check if the
context is still alive. The stream's control thread holds a reference
to the PulseAudioContext, so that should almost never be destroyed
before the VERIFY in the test. Instead, wait at most 100ms for it to be
destroyed.
We can't control whether the instantiation mutex is held when
~Weakable() is called, so we need to implement this via a static raw
pointer instead to ensure that all operations on it are effectively
atomic.
Otherwise, if the sample iterator resides in a block with multiple
frames before the seek, the demuxer will output all the remaining
frames from that block before moving on to the block at the seeked
position.
