Support C++20 standard #26590
Close https://github.com/opencv/opencv/issues/26589
Related https://github.com/opencv/opencv_contrib/pull/3842
Related: https://github.com/opencv/opencv/issues/20269
- do not arithmetic enums and ( different enums or floating numeric)
- remove unused variable
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Fix gstreamer backend with manual pipelines #24243
- Fix broken seeking in audio/video playback
- Fix broken audio playback
- Fix unreliable seeking
- Estimate frame count if it is not available directly
- Return -1 for frame count and fps if it is not available.
- Return 0 for fps if the video has variable frame rate
- Enable and fix tests
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [X] I agree to contribute to the project under Apache 2 License.
- [X] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [X] The PR is proposed to the proper branch
- [-] There is a reference to the original bug report and related work => Reproducible test provided
- [-] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [X] The feature is well documented and sample code can be built with the project CMake
1. Download two test videos:
```bash
wget https://github.com/ietf-wg-cellar/matroska-test-files/raw/master/test_files/test1.mkv
wget https://test-videos.co.uk/vids/jellyfish/mkv/360/Jellyfish_360_10s_5MB.mkv
```
2. I modified a OpenCV videoio sample to demonstrate the problem, here it is the patch: http://dpaste.com//C9MAT2K6W
3. Build the sample, on Ubuntu:
```bash
g++ -g videocapture_audio_combination.cpp -I/usr/include/opencv4 `pkg-config --libs --cflags opencv4` -o videocapture_audio_combination
```
4. Play an audio stream with seeking BEFORE the fix:
```bash
$ ./videocapture_audio_combination --audio "filesrc location=test1.mkv ! queue ! matroskademux name=demux demux.audio_0 ! decodebin ! audioconvert ! appsink"[ERROR:0@0.009] global cap.cpp:164 open VIDEOIO(GSTREAMER): raised OpenCV exception:
OpenCV(4.8.0-dev) ./modules/videoio/src/cap_gstreamer.cpp:153: error: (-215:Assertion failed) ptr in function 'get'
[ WARN:0@0.009] global cap.cpp:204 open VIDEOIO(GSTREAMER): backend is generally available but can't be used to capture by name
ERROR! Can't to open file: filesrc location=test1.mkv ! queue ! matroskademux name=demux demux.audio_0 ! decodebin ! audioconvert ! appsink
```
5. Play a video stream with seeking BEFORE the fix:
```bash
$ ./videocapture_audio_combination --audio "filesrc location=Jellyfish_360_10s_5MB.mkv ! queue ! matroskademux name=demux demux.video_0 ! decodebin ! videoconvert ! video/x-raw, format=BGR ! appsink drop=1"
[ WARN:0@0.034] global cap_gstreamer.cpp:1728 open OpenCV | GStreamer warning: Cannot query video position: status=1, value=22, duration=300
CAP_PROP_AUDIO_DATA_DEPTH: CV_16S
CAP_PROP_AUDIO_SAMPLES_PER_SECOND: 44100
CAP_PROP_AUDIO_TOTAL_CHANNELS: 0
CAP_PROP_AUDIO_TOTAL_STREAMS: [ WARN:0@0.034] global cap_gstreamer.cpp:1898 getProperty OpenCV | GStreamer: CAP_PROP_AUDIO_TOTAL_STREAMS property is not supported
0
[ WARN:0@0.034] global cap_gstreamer.cpp:1817 getProperty OpenCV | GStreamer: CAP_PROP_POS_MSEC property result may be unrealiable: https://github.com/opencv/opencv/issues/19025
Timestamp: 0.6218
Timestamp: 33.1085
Timestamp: 67.1274
Timestamp: 100.1182
Timestamp: 133.1204
Timestamp: 167.1195
Timestamp: 200.1161
Timestamp: 233.1147
Timestamp: 267.1194
Timestamp: 300.1202
[ WARN:0@0.338] global cap_gstreamer.cpp:1949 setProperty OpenCV | GStreamer warning: GStreamer: unable to seek
0:00:00.338215907 3892572 0x5592899c7580 WARN basesrc gstbasesrc.c:3127:gst_base_src_loop:<filesrc0> error: Internal data stream error.
0:00:00.338235884 3892572 0x5592899c7580 WARN basesrc gstbasesrc.c:3127:gst_base_src_loop:<filesrc0> error: streaming stopped, reason not-linked (-1)
0:00:00.338264287 3892572 0x5592899c7580 WARN queue gstqueue.c:992:gst_queue_handle_sink_event:<queue0> error: Internal data stream error.
0:00:00.338270329 3892572 0x5592899c7580 WARN queue gstqueue.c:992:gst_queue_handle_sink_event:<queue0> error: streaming stopped, reason not-linked (-1)
[ WARN:0@0.339] global cap_gstreamer.cpp:2784 handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module filesrc0 reported: Internal data stream error.
[ WARN:0@0.339] global cap_gstreamer.cpp:1199 startPipeline OpenCV | GStreamer warning: unable to start pipeline
Number of audio samples: 0
Number of video frames: 10
[ WARN:0@0.339] global cap_gstreamer.cpp:1164 isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
```
6. Play an audio stream with seeking AFTER the fix:
```bash
$ ./videocapture_audio_combination --audio "filesrc location=test1.mkv ! queue ! matroskademux name=demux demux.audio_0 ! decodebin ! audioconvert ! appsink"CAP_PROP_AUDIO_DATA_DEPTH: CV_16S
CAP_PROP_AUDIO_SAMPLES_PER_SECOND: 48000
CAP_PROP_AUDIO_TOTAL_CHANNELS: 2
CAP_PROP_AUDIO_TOTAL_STREAMS: [ WARN:0@0.025] global cap_gstreamer.cpp:1903 getProperty OpenCV | GStreamer: CAP_PROP_AUDIO_TOTAL_STREAMS property is not supported
0
Timestamp: 0.0000
Timestamp: 24.0000
Timestamp: 48.0000
Timestamp: 72.0000
Timestamp: 96.0000
Timestamp: 120.0000
Timestamp: 144.0000
Timestamp: 168.0000
Timestamp: 192.0000
Timestamp: 216.0000
Timestamp: 3500.0000
Timestamp: 3504.0000
Timestamp: 3528.0000
Timestamp: 3552.0000
Timestamp: 3576.0000
Timestamp: 3600.0000
Timestamp: 3624.0000
Timestamp: 3648.0000
Timestamp: 3672.0000
Timestamp: 3696.0000
Timestamp: 3720.0000
Timestamp: 3744.0000
Timestamp: 3768.0000
Timestamp: 3792.0000
Timestamp: 3816.0000
Timestamp: 3840.0000
Timestamp: 3864.0000
Timestamp: 3888.0000
Timestamp: 3912.0000
Timestamp: 3936.0000
```
7. Play a video stream with seeking AFTER the fix:
```bash
$ ./videocapture_audio_combination --audio "filesrc location=Jellyfish_360_10s_5MB.mkv ! queue ! matroskademux name=demux demux.video_0 ! decodebin ! videoconvert ! video/x-raw, format=BGR ! appsink drop=1"
[ WARN:0@0.033] global cap_gstreamer.cpp:1746 open OpenCV | GStreamer warning: Cannot query video position: status=1, value=22, duration=300
CAP_PROP_AUDIO_DATA_DEPTH: CV_16S
CAP_PROP_AUDIO_SAMPLES_PER_SECOND: 44100
CAP_PROP_AUDIO_TOTAL_CHANNELS: 0
CAP_PROP_AUDIO_TOTAL_STREAMS: [ WARN:0@0.034] global cap_gstreamer.cpp:1903 getProperty OpenCV | GStreamer: CAP_PROP_AUDIO_TOTAL_STREAMS property is not supported
0
Timestamp: 0.0000
Timestamp: 33.0000
Timestamp: 67.0000
Timestamp: 100.0000
Timestamp: 133.0000
Timestamp: 167.0000
Timestamp: 200.0000
Timestamp: 233.0000
Timestamp: 267.0000
Timestamp: 300.0000
0:00:00.335931693 3893501 0x55bbe76ad920 WARN matroskareadcommon matroska-read-common.c:759:gst_matroska_read_common_parse_skip:<demux:sink> Unknown CueTrackPositions subelement 0xf0 - ignoring
0:00:00.335952823 3893501 0x55bbe76ad920 WARN matroskareadcommon matroska-read-common.c:759:gst_matroska_read_common_parse_skip:<demux:sink> Unknown CueTrackPositions subelement 0xf0 - ignoring
0:00:00.335988029 3893501 0x55bbe76ad920 WARN basesrc gstbasesrc.c:1742:gst_base_src_perform_seek:<filesrc0> duplicate event found 184
Timestamp: 3467.0000
Timestamp: 3500.0000
Timestamp: 3533.0000
Timestamp: 3567.0000
Timestamp: 3600.0000
Timestamp: 3633.0000
Timestamp: 3667.0000
Timestamp: 3700.0000
Timestamp: 3733.0000
Timestamp: 3767.0000
Timestamp: 3800.0000
Timestamp: 3833.0000
Timestamp: 3867.0000
Timestamp: 3900.0000
Timestamp: 3933.0000
Timestamp: 3967.0000
Timestamp: 4000.0000
Timestamp: 4033.0000
Timestamp: 4067.0000
Timestamp: 4100.0000
```
Address https://github.com/opencv/opencv/issues/22868
Used the same defaults as it's done for FFmpeg
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
```
force_builders=Custom
build_image:Custom=gstreamer:16.04
buildworker:Custom=linux-1
```
Some GStreamer elements may produce buffers with very non
standard strides, offsets and/or even transport each plane
in different, non-contiguous pointers. This non-standard
layout is communicated via GstVideoMeta structures attached
to the buffers. Given this, when a GstVideoMeta is available,
one should parse the layout from it instead of generating
a generic one from the caps.
The GstVideoFrame utility does precisely this: if the buffer
contains a video meta, it uses that to fill the format and
memory layout. If there is no meta available, the layout is
inferred from the caps.
* Fix gst error handling
* Use the return value instead of the error, which gives no guarantee of being NULL in case of error
* Test err pointer before accessing it
* Remove unreachable code
* videoio(gstreamer): restore check in writer code
Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
* videoio/gstreamer: Add support for GRAY16_LE.
* videoio/gstreamer: added BGRA/BGRx support
Co-authored-by: Maksim Shabunin <maksim.shabunin@gmail.com>
videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF
* HW acceleration in FFMPEG backend
* fixes on Windows, remove D3D9
* HW acceleration in FFMPEG backend
* fixes on Windows, remove D3D9
* improve va test
* Copyright
* check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+
* CAP_MSMF test on .mp4
* .mp4 in test
* improve va test
* Copyright
* check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+
* CAP_MSMF test on .mp4
* .mp4 in test
* .avi for GStreamer test
* revert changes around seek()
* cv_writer_open_with_params
* params.warnUnusedParameters
* VideoCaptureParameters in GStreamer
* open_with_params
* params->getUnused
* Reduce PSNR threshold 33->32 (other tests use 30)
* require FFMPEG 4.0+; PSNR 30 as in other tests
* GStreamer AVI-demux plugin not installed in Ubuntu test environment?
* fix build on very old ffmpeg
* fix build on very old ffmpeg
* fix build issues
* fix build issues (static_cast)
* FFMPEG built on Windows without H264 encoder?
* fix for write_nothing test on VAAPI
* fix warnings
* fix cv_writer_get_prop in plugins
* use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs
* internal function hw_check_device() for device check/logging
* two separate tests for HW read and write
* image size 640x480 in encode test
* WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency)
* exception on VP9 SW encoder?
* rebase master; refine info message
* videoio: fix FFmpeg standalone plugin build
* videoio(ffmpeg): eliminate MSVC build warnings
* address review comments
* videoio(hw): update videocapture_acceleration.read test
- remove parallel decoding by SW code path
- check PSNR against the original generated image
* videoio: minor fixes
* videoio(test): disable unsupported MSMF cases (SW and HW)
* videoio(test): update PSNR thresholds for HW acceleration read
* videoio(test): update debug messages
* "hw_acceleration" whitelisting parameter
* little optimization in test
* D3D11VA supports decoders, doesn't support encoders
* videoio(test): adjust PSNR threshold in write_read_position tests
* videoio(ffmpeg): fix rejecting on acceleration device name mismatch
* videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging
* videoio: rework VideoAccelerationType behavior
- enum is not a bitset
- default value is backend specific
- only '_NONE' and '_ANY' may fallback on software processing
- specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration.
* videoio(test): fix for current FFmpeg wrapper
Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
- migrate GStreamer backed
- migrate FFmpeg backend (with switch on legacy API)
- cv_videoio_capture_retrieve_cb_t uses Mat type instead of number of channels
Add CV_16UC1/GRAY16_LE support to GStreamer backend for VideoWriter
* videoio(backend): add Writer_open_with_params to plugin API
This will allow arbitrary parameters to be passed to plugin backends
* videoio(gstreamer): add GRAY16_LE/CV_16UC1 writing support to GStreamer
This introduces a new property VIDEOWRITER_PROP_DEPTH, which defaults to
CV_8U, but for GStreamer can be set to CV_16U.
Also, fix another test to not fail if plugin isn't found, copying logic
from the read_write test.
* videoio(plugin): fix handling plugins with previous API level
* videoio: coding style
* fix warning
A running GMainLoop processes many events on the GLib/GStreamer
world. While some things may work without it, many others wont.
Examples of these are signals, timers and many other source
events. The problem becomes more concerning by the fact that
some GStreamer elements rely on signals to work.
This commit allows the user to specify an OpenCV option to
start a main loop, if needed. Since the loop blocks, this is
done in a separate thread.
* feature: Extend VideoWriter to accept vector of parameters
- Add additional constructor and `open` method for `VideoWriter`
those accept a vector of parameters
- Move actual implementation of the `VideoWriter::open` to general method
which accepts vector of parameters
- Propagate parsed parameters map up to actual video backend construction
* fix: Change VideoWriter constructor description to suppress doc warning
* refactor: Rollback newlines changes
* feature: Changed VideoWriter parameters workflow
* feature: Log unused parameters in VideoWriter open
* doc: Fix VideoWriter `isColor` parameter description
* fix: int to bool VC++ conversion warning
* doc: Remove information about `isColor` flag usage.
According to the gstreamer docs [1], the GstMessage pointer returned by
gst_bus_pop() is nullable, meaning NULL is a valid return value.
Previously, gst_is_missing_plugin_message would throw an assert when its
message object parameter would fail the GST_IS_MESSAGE macro check,
crashing the entire process (unless running in a try-catch block of course).
Instead of relying on valid messages, check if the message object itself is
valid before passing it to other gstreamer functions.
[1] https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstBus.html#gst-bus-pop
Signed-off-by: Christopher N. Hesse <raymanfx@gmail.com>
Kumataro
Maksim Shabunin
Alexander Smorkalov
Ahmed Essam
Csaba Kertész
Maxim Milashchenko
Alexander Smorkalov
Michael Gruner
OpenCV Developers
Yulv-git
Alexander Alekhin
Jonas VautherinandAlexander Alekhin
zypandMaksim Shabunin
Mikhail NikolskiiandAlexander Alekhin
joshdoe
Vadim Levin
Brian Wignall
Alexander Alekhin
Christopher N. Hesse