Dmitry Shmidt authored 3 years ago and Mattijs Korpershoek committed 1 year ago

Currently when using the v4l2_codec2 HAL as documented, it will
fail to function and one will see the following error in the
logcat:
03-11 05:25:09.121   568  2059 D CCodecBufferChannel: [c2.v4l2.avc.decoder#295] Created input block pool with allocatorID 16 => poolID 19 - OK (0)
03-11 05:25:09.122   568  2059 D C2PlatformStorePluginLoader: Failed to load library: libc2plugin_store.so (dlopen failed: library "libc2plugin_store.so" not found)
03-11 05:25:09.122   568  2059 D C2PlatformStorePluginLoader: Handle or CreateAllocator symbol is null

It seems this is due to the C2PlatformStorePluginLoader being
system library which tries to load and call out to
libc2plugin_store, which isn't allowed (vendor can call to
system, but system can't call to vendor).

This patch sets libc2plugin_store library to be vendor_available
which sets it in the system image.

Apparently this isn't the right solution, but without it
the libraries will not load.

It seems the libc2plugin_store loader logic may not be valid
post-Treble?
https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/codec2/vndk/C2PlatformStorePluginLoader.cpp;l=38



We need to understand what a proper solution would be, or what
vendors use for this.

Bug: 170199771
Test: none
Signed-off-by: Dmitry Shmidt <dimitrysh@google.com>
[jstultz: Added some detail to the commit message, narrowed
 change to just libc2plugin_store]
Signed-off-by: John Stultz <john.stultz@linaro.org>
Change-Id: I7e4486ad95fb32a75ae9cec075672de0c7de2586
Link: https://android-review.googlesource.com/c/platform/external/v4l2_codec2/+/2007320
Link: https://android-review.googlesource.com/c/platform/external/v4l2_codec2/+/2234008


Signed-off-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>

ag/18183905 changes the constructor arguments of C2BufferQueueBlockPoolData.

Bug: 232711062
Test: m
Change-Id: I63a9f9b1c7d8368af4c80777a9f4dcba4b581df0

Currently, getDmabufId() is placed at C2VdqBqBlockPool.h. However,
the header file already has many dependencies. The caller of
getDmabufId() would need to add many unnecessary dependency libraries.

This CL splits the method to a dedicated file to reduce this issue.

Bug: 205501954
Test: m -j32

Change-Id: Ie7e366ddf77ceec105cbacf6d16e6b6dc3a58198

Importing buffer into DRM/KMS just to obtain unique_id is redundant
and can cause issues if wrong DRM/KMS device selected.

Bug: 205501954

Signed-off-by: Roman Stratiienko <r.stratiienko@gmail.com>
Change-Id: I0f20cf8f230f26be4641a85bf1f7ec4d47690494

Currently, getDmabufId() is placed at C2VdqBqBlockPool.h. However,
the header file already has many dependencies. The caller of
getDmabufId() would need to add many unnecessary dependency libraries.

This CL splits the method to a dedicated file to reduce this issue.

Bug: 205501954
Test: m -j32

Change-Id: Ie7e366ddf77ceec105cbacf6d16e6b6dc3a58198

Currently, getDmabufId() is placed at C2VdqBqBlockPool.h. However,
the header file already has many dependencies. The caller of
getDmabufId() would need to add many unnecessary dependency libraries.

This CL splits the method to a dedicated file to reduce this issue.

Bug: 205501954
Test: m -j32

Change-Id: Ie7e366ddf77ceec105cbacf6d16e6b6dc3a58198

Importing buffer into DRM/KMS just to obtain unique_id is redundant
and can cause issues if wrong DRM/KMS device selected.

Signed-off-by: Roman Stratiienko <r.stratiienko@gmail.com>
Change-Id: I0f20cf8f230f26be4641a85bf1f7ec4d47690494

Bug: None
Test: mmm external/v4l2_codec2
Change-Id: I21d5d30ce32279dfb1cbe2b40d435d2e9707f5aa

Originally, we hard-coded the DRM device path and name. However, the
value might be different between each platform. This CL makes them
customizeable by setting Android property.

Bug: 189059445
Test: android.media.cts.AdaptivePlaybackTest
Change-Id: If2418244b5854bafffd2d038dc85af323db7657f

C2PooledBlockPool is used by VideoFramePool. Currently VideoFramePool
has exponential backoff retry, so we don't need another retry logic in
C2PooledBlockPool. This CL removes the unnecessary retry logic.

Bug: 160110407
Test: tast.arc.VideoDecodeAccel.h264_vm

Change-Id: I73ffd4b27000846eeea1e5bffbedb21e16795593

C2BufferQueueBlockPool implementation changed for supporting waitable
object. Reflects the change.

Bug: 157111613
Change-Id: I4cfc14221bd95e962774df63573eb216f234b462

At ag/13824423 we fixed the description of
WrapNativeCodec2GrallocHandle() that the returned handle should be
closed. This CL closes the handle if the ownership is not transfered
to C2GrallocAllocation via priorGraphicAllocation() call.

Bug: 181541291
Test: android.media.cts.AdaptivePlaybackTest

Change-Id: Ifbe842f715977f4f6f79dfd2b33efbee9c0d1191

Currently we wait the fence of the buffer with timeout. To reduce the
chance of timeout, we move the waiting call to the last point, just
before returning the buffer.

Bug: 178770649
Test: android.media.cts.AdaptivePlaybackTest
Change-Id: Idf2b4a40397a6632120e6d04c1b95b97933de457

C2ArcProtectedGraphicAllocator allocates the protected buffer when
priorGraphicAllocation() is called, and releases the protected buffer
if all the corrresponding C2GraphicAllocations are released.

Originally, we store the the registered graphic buffers with
GraphicBuffer. Each time when fetchGraphicBlock() is called, we
call priorGraphicAllocation() to convert GraphicBuffer to
C2GraphicAllocation. The component releases the C2GraphicAllocation
after decoding a frame and passing it to framework. So at the end we
continue allocating and releasing protected buffers during decoding.

This CL changes to store C2GraphicAllocation directly. We don't call
priorGraphicAllocation() when recycling the buffer, and the registered
C2GraphicAllocations are released when resolution change or component
is destroyed.

Bug: 181541291
Test: android.media.cts.AdaptivePlaybackTest

Change-Id: Ib0cf9fe9d4609f3a566c514429d281c1c54b458d

Bug: 160110407
Test: android.media.cts.AdaptivePlaybackTest
Change-Id: Ie8a18e7f47469ae940fac0ca91fa68826006f13d

|mComponentOwnedUniquedIds| was used to prevent the issue that buffers
might be attached twice to new surface. But ag/13490974 change the way
to switch surface. This issue doesn't exist anymore. This CL removes
the unnecessary mComponentOwnedUniquedIds data.

Also, because the extra field |mUniqueId| of C2VdaBqBlockPoolData is
used to update |mComponentOwnedUniquedIds|, it's not needed as well.
This CL removes C2VdaBqBlockPoolData class and uses its parent class
directly.

Bug: 174188958
Test: android.media.cts.AdaptivePlaybackTest
Test: android.media.cts.MediaCodecPlayerTest#testPlaybackSwitchViews

Change-Id: I588f3896a8ad6cceb90c844981e7ab95defe2092

During decoding, graphic buffers are owned by component (i.e. local
buffers) or CCodec framework (i.e. remote buffers).  When switching
surface, the CCodec framework migrates remote buffers to the new
surfaces, then call C2BufferQueueBlockPool::configureProducer().
C2BufferQueueBlockPool implementation then migrates local buffers.
However, the CCodec and C2BufferQueueBlockPool implementation don't
guaratee to migrate all the existing buffers, and might allocate new
buffers. This violates the V4L2 stateful API's requirement.

Originally, C2VdaBqBlockPool migrates all the buffers again, and tries
to detach the slot of buffers migrated by CCodec framework. we assume
there should be only at most N buffers at BufferQueue, where N is the
number passed via setMaxDequeuedBufferCount(). However, this
assumption is wrong, so we failed to detach all buffers migrated by
CCodec framework.

In this CL, C2VdaBqBlockPool migrates local buffers when switching
surface, similar to C2BufferQueueBlockPool. After we dequeue
NUM_BUFFER_SLOTS buffers, all the migrated buffers should be dequeued
at least once. Then we could detect the lost buffers, and attach them
to the new surface.

Bug: 174188958
Test: android.media.cts.AdaptivePlaybackTest
Test: android.media.cts.MediaCodecPlayerTest#testPlaybackSwitchViews

Change-Id: Ifc30b62e69608be4ebc24ce1ea1a97be11f25a37

Originally, the definition of C2BufferQueueBlockPoolData is hidden at
cpp file. C2VdaBqBlockPoolData has to handle the ownership transition
itself with extra workaround.

At ag/13473071, we expose C2BufferQueueBlockPoolData at header file.
This CL makes C2VdaBqBlockPoolData inherit C2BufferQueueBlockPoolData
directly. With this CL, C2VdaBqBlockPoolData could share the same
logic of the ownership transition.

Bug: 174188958
Test: android.media.cts.AdaptivePlaybackTest

Change-Id: Ib73b2f8bc36bbd9724523624415551a7efadda73

At ag/13473071 we expose C2BufferQueueBlockPoolData at
C2BqBufferPriv.h, which needs C2BlockInternal.h header. This CL adds
the corresponding header_libs at plugin_store because it includes
C2BqBufferPriv.h.

Bug: 174188958
Test: m -j32

Change-Id: I0eec5db85b6d7d05ad97df142cb61a9ea2adbe76

Although H2BGraphicBufferProducer is only used by C2VdaBqBlockPool,
the logic of H2BGraphicBufferProducer is not highly coupled. This CL
splits the class to a dedicated file.

Bug: 160110407
Test: mmm external/v4l2_codec2
Change-Id: Icec2d189fe9df832d71aeb7b0f25ec22df8cc2d0

This CL removes the unnecessary "android" namespace prefix for
status_t constant. Also it replaces "NO_ERROR" to "OK", which is the
preferred constant.

Bug: 160110407
Test: m -j32

Change-Id: Ia93e82d6425812b169b85ee96b5d3db0d1c23214

This CL changes the order of the function call at configuring producer
to avoid unnecessary call when any error occurs or the producer is not
changed.

Bug: 160110407
Test: android.media.cts.MediaCodecPlayerTest#testPlaybackSwitchViews

Change-Id: I194f929ab7dbdaa10ab55dd3a11ca4a1c9a0d40a

Originally, protected buffer allocator has restriction that we can
only call priorGraphicAllocation() method once for each buffer. But we
have to call priorGraphicAllocation() again when switching surface. So
surface switching is not supported at protected playback.

However, the restriction is relaxed at ag/13461483. This CL removes
the protected playback check at surface switching.

Bug: 160110407
Test: m -j32

Change-Id: I21154fa9a9f69db1e2eee89f296daae8932239d1

Originally, we use IGBP's maximum dequeued buffer count to restrict
the total allocated buffer count. However, the maximum dequeued buffer
count only affects the free BufferQueue slot count, not related to
allocated buffer count. Also, we already track the allocated buffer
count by TrackedGraphicBuffers class.

This CL sets a big enough maximum dequeued buffer count when
configuring the producer. It simplifies the logic in general.

Bug: 160110407
Test: android.media.cts.AdaptivePlaybackTest

Change-Id: Ibd8859395732c85521b2e609b46af4b203d969c0

This CL polishes queryGenerationAndUsage method, including:

- change the return type to status_t to reduce type mapping
- rename to queryGenerationAndUsageLocked to indicate the method
  should be called during locked
- remove unnecessary fence waiting

Bug: 160110407
Test: android.media.cts.AdaptivePlaybackTest

Change-Id: Ie85acde8bdfdc5b6e2620e590aed9dd606e1b507

This CL tracks the current status of allowAllocation(). If the target
status is the same as the expected status, then we skip the call.
Also, this CL disables allocation when we dequeue more buffers then
requested.

Bug: 160110407
Test: android.media.cts.AdaptivePlaybackTest

Change-Id: I860db3be92b06f18c7f5a0e7b6a391ccd26bb6f6

This CL adds the type alias for the unique identifier of graphic
buffers, and the BufferQueue slot index for readability.

Bug: 160110407
Test: m -j32

Change-Id: I778956f909b8df3a226beb5b657493f10a7cf7f1

This CL avoids reusing the status variable to prevent the potential
bug that the value is misused.

Bug: 160110407
Test: android.media.cts.MediaSyncTest

Change-Id: I8c0a3a45657603bc800923da31b7fa00d59fa13e

Originally we assumed the buffers at BufferQueue is not moved to
different slot, so we tracked the graphic buffers by storing tuple of
(slot index, unique ID, C2Allocation). However, the slot index of the
buffer might be changed.

In this CL, we change to store two maps instead:
- unique ID to C2Allocation
- slot index to GraphicBuffer

The map from unique ID to C2Allocation is for tracking the fixed
buffer set. The map from slot index to GraphicBuffer is updated when
IGBP::dequeueBuffer() returns BUFFER_NEEDS_REALLOCATION, no matter if
the buffers is tracked or not. With these two maps, we can handle the
case that the consumer move graphic buffers to different BufferQueue
slot.

Bug: 177484971
Test: android.media.cts.AdaptivePlaybackTest
Test: android.media.cts.MediaSyncTest

Change-Id: Iee99e8d0b00dae463cf6483ca8c971cc84f9ea49

Originally we assume the buffers at BufferQueue are not moved to
different slot. Then we could track the buffers by the slot index.
However, MediaSync implementation detach buffers and attach back to
BufferQueue. The slot index might be changed.

This CL changes to use GEM handle as the identifier of graphic
buffers. Because all the graphic buffers are allocated by gralloc, we
could get the GEM handle by DRM. V4L2GraphicAllocator is not used
anymore.

Bug: 177484971
Test: android.media.cts.AdaptivePlaybackTest

Change-Id: I6b7f3f9a6c1d6b97d0e44aa2a19bc71ad3bab880

When running MediaSyncTest, Fence::waitForever() is stuck for few
seconds, and it blocks other methods due to the mutex. This CL changes
waitForever() method to wait() with timeout. Also we enlarge the
timeout to 32 ms.

Bug: 177484971
Test: android.media.cts.AdaptivePlaybackTest
Test: android.media.cts.MediaSyncTest#testPlayVideo

Change-Id: I6d8d2c3befed35e17e16f322edfaa4245797254c

Currenlty C2VdaBqBlockPool is highly coupled with IGBP.
V4L2DecodeComponent cannot work with the buffers not received from the
IGBP. This CL removes the fallback when the IGBP is not configured.

Bug: 160110407
Test: android.media.cts.AdaptivePlaybackTest
Change-Id: I61e3ad01fb85a7b8003c948a5b3cbba341888d3c

queryGenerationAndUsage() could access |mProducer| member, so we don't
need to pass the argument into the method.

Bug: 160110407
Test: android.media.cts.AdaptivePlaybackTest#testH264_*

Change-Id: I40cad58b5ff9cfbff03f172ec119d530b5248b63

BufferReleasedNotifier and C2VdaBqBlockPoolData notify
C2VdaBqBlockPool when some event happens. Originally these two classes
hold shared_ptr of C2VdaBqBlockPool. However they should not hold the
ownership of C2VdaBqBlockPool. This CL changes these two classes to
hold weak_ptr of C2VdaBqBlockPool.

Bug: 160110407
Test: android.media.cts.AdaptivePlaybackTest

Change-Id: Ibc8c960dabc16542f83f5e6ff082dec70edf2481

Added SPDX-license-identifier-BSD to:
  accel/Android.bp
  common/Android.bp
  components/Android.bp
  plugin_store/Android.bp
  service/Android.bp
  store/Android.bp
  tests/c2_comp_intf/Android.bp

Added SPDX-license-identifier-BSD legacy_unencumbered to:
  Android.bp
  tests/c2_e2e_test/jni/Android.mk

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all

Exempt-From-Owner-Approval: janitorial work
Change-Id: I4ef032a3849d650e13761a868b45309275d408a2

Bug: 160110407
Test: m -j32
Change-Id: Icb1e50296610388df945ed00d6eca685c3a27944

This CL changes some non-fatal or non-error log from ALOGE to ALOGD.

Bug: 175489166
Test: m -j32

Change-Id: I0b9a60138cf4f3405fdf958f6a093086272c4c13

After fetching new IGBP slot times out, we should retry the fetching
when buffer released at IGBP. We use a flag
|mBufferReleasedAfterTimedOut| to catch the case that the buffer is
released between fetching timed out and setting the retry callback.
If the flag is set, then we should execute the callback immediately,
instead of waiting for the next buffer released event.

However, we forgot to reset the flag when executing the callback. It
caused we continuously retry the buffer fetching until the next buffer
released event. This CL fixes this issue.

Bug: 175489166
Test: run com.google.android.exoplayer.gts.DashTest#testH264Adaptive
      and check setMaxDequeuedBufferCount() only retry when buffer
      released
Change-Id: I8f7a36af50e5eebfcbecedc0f126c86f9427fa95

VideoFramePool is used to wrap C2BlockPool and request buffers for
V4L2Decoder. When resolution changes, V4L2Decoder creates a new
VideoFramePool instance to request buffers with new resolution.
Also, C2VdaBqBlockPool is designed to provide buffers with the same
resolution. So there only one VideoFramePool could exist at the same
time.

This CL destroys the previous VideoFramePool before getting the new
one, and checks the buffer format passed to
C2VdaBqBlockPool::fetchGraphicBlock() is the same as the format
requested by requestNewBufferSet().

Also, this CL simplifies the signature of the callback of getting
VideoFramePool. Originally we put the returned value at output
argument because Chromium callback with WeakPtr doesn't support
returned value. But we actually could avoid WeakPtr here because
V4L2Decoder is always destroyed before V4L2DecodeComponent. It's safe
to use ::base::Unretained(this).

Bug: 174287571
Test: android.media.cts.AdaptivePlaybackTest#test{H264,Vp8,Vp9}_*

Change-Id: Ib06855e7ccc3db5508a4a3f6403bceedb9b5f59d

configureProducer() doesn't return any status. When this method fails,
the component should stop working immediately. In this CL,
C2VdaBqBlockPool replies error at fetchGraphicBlock(), which is the
main method called by the component, if any error occurs at the
previous configureProducer().

Bug: 162902320
Test: android.media.cts.MediaCodecPlayerTest#testPlaybackSwitchViews
Change-Id: I4a9537b7657fef604ba2e020bcaeb0398ae2a5d0