Skip to content

Commit

Permalink
πŸŽ‰ Released VidGear Stable v0.3.0 [#352]
Browse files Browse the repository at this point in the history
πŸ““ Complete Release Notes: https://abhitronix.github.io/vidgear/latest/changelog/#v030-2023-01-26

#### New Features ✨
- **WriteGear:** 
    * Added support for user-defined and higher than 8-bit depth input frames pixel-format.
      + Added support for higher than 8-bit depth frames with datatypes of unsigned integer(`uint`) kind and element size `2`.
      + Added `dtype` parameter to internal `Preprocess` method for passing input frames datatype.
      + Implemented auto-calculation of input pixel-format based on number of channels in higher than 8-bit depth frames.
      + Added various known working pixel-formats(based on number of channels), supported by all prominent computer vision libraries.
      + Added support for up to 1-channel(`gray16-le/be`) to all the way up to 4-channels(`bgra64-le/be`) in input frames.
      + Added endianness little(`le`) or big(`be`) at the suffix of pixel-format based on byte-order of input frames datatypes.
      + Extended support for higher RGB 8-bit depth frames through RGB mode.
    * Added support for user-defined custom input pixel-format.
      + Added new `-input_pixfmt` attribute to `output_params` dictionary parameter for easily specifying custom input pixel-format.
      + Added newly implemented `get_supported_pixfmts` method import for verifying user-defined input pixel-format against Installed FFmpeg supported pixel-formats. Unsupported values will be discarded. 
      + Implemented runtime datatype validation check, such that all input frames must have same datatype.
    * Added support for Context Managers for proper handling of resources via `with` statement for allocating and releasing resources precisely. (Suggested by @sueskind)
      + Implement the `__enter__()` and `__exit__()` methods.
      + Added `__enter__` method that returns reference to the WriteGear Class.
      + Added `__exit__` method that automatically executes `close()` for performing the cleanup operations and handling exception gracefully. 
- **StreamGear:** 
    * Added support for Context Managers for proper handling of resources via `with` statement for allocating and releasing resources precisely. (Suggested by @sueskind)
      + Implement the `__enter__()` and `__exit__()` methods.
      + Added `__enter__` method that returns reference to the StreamGear Class.
      + Added `__exit__` method that automatically executes `close()` for performing the cleanup operations and handling exception gracefully. 
- **WebGear:**
    * Added way to completely disable Data-Files Auto-Generation WorkFlow.
      + Added new `skip_generate_webdata` boolean optional attribute(`False` by default) to completely disable Data-Files Auto-Generation WorkFlow.
      + This flag enables only `/video` route for disabled Data-Files Auto-Generation WorkFlow.
      + Implemented JSONResponse as placeholder response instead of Index, `404` and `500` HTML pages, when workflow is disabled. _(Note: Index HTML page will throw `404` status code.)_
      + Added necessary imports.
- **Helper:**
    * Added more robust implementation of validate_audio method.
      + Added new more robust regex pattern for extracting audio-samplerate.
      + Added new `validate_audio` method for calculating accurate bitrate(in kbps) from audio samplerate, channels, bit-depth values.
      + Implemented new patterns and logic for accurately extracting audio channels and bit-depth from given metadata.
    * Added support for Linux video device path _(such as `/dev/video0`)_.
- **Maintenance:** 
    * Logging current vidgear version when vidgear APIs are called, not at import.
      + Added `logcurr_vidgear_ver` helper function to facilitate logging current vidgear version, when called within a API.
      + Implemented `ver_is_logged` global variable in helper to log version only once, which can modifiable with `logcurr_vidgear_ver` method only. Followed recommendation given in official python docs: https://docs.python.org/3/faq/programming.html#how-do-i-share-global-variables-across-modules
      + Current version can only be logged by VidGear APIs with the logging turned on _(i.e. `logging=True`)_.
- **Docs:**
    * Added new WriteGear Bonus Example:
      + Added "Using WriteGear's Compression Mode with `v4l2loopback` Virtual Cameras bonus python example.
      + Added related prerequisites and dependencies for creating `v4l2loopback` Virtual Cameras on Linux machines.
      + Added both With/Without-Audio cases for "Using WriteGear's Compression Mode for YouTube-Live Streaming".
    * Added `content.code.copy` and `content.tabs.link` features.
    * Added docs related to `skip_generate_webdata` optional attribute.
    * Added feedback features to mkdocs.yml.
    * Added `404.html` static template to `mkdocs.yml`.
- **CI:**
    * Added v4l2loopback support for testing `/dev/video0` device on Linux machines.
    * Added test cases for newer implementation of `validate_audio` method.
    * Added `test_skip_generate_webdata` to test `skip_generate_webdata` optional attribute.
    * Added tests for user-defined and higher than 8-bit depth input frames pixel-format.


#### Updates/Improvements ⚑️
- WriteGear: 
    * Completely revamped code structure and comments.
      + Updated comments, description, and logging messages to more sensible and developer friendly.
      + Implemented operator short-circuiting to cleanup code as much as possible.
      + Renamed `startFFmpeg_Process` internal class method to `start_FFProcess`.
      + Renamed `Preprocess` internal class method to `PreprocessFFParams`.
      + Renamed `startCV_Process` internal class method to `start_CVProcess`.
      + Renamed `initiate` internal class parameter to `initiate_process`.
      + Renamed `force_termination` internal class parameter to `forced_termination`.
      + Enabled `output_params` parameters logging in both modes.
      + Improved `compression` and `logging` parameters boolean value handling.
      + Impelemented `stdout` closing to cleanup pipeline before terminating.
- Helper:
    *  Updated `validate_audio` method with improved and more robust regex patterns for identifying audio bitrate in ay audio file.
- Setup.py:
    * Bumped version to `0.3.0`.
    * Replaced `>=` comparsion operator with more flexible `~=`.
    * Replaced `distutils.version.LooseVersion` with `pkg_resources.parse_version`.
- Maintenance: 
    * Replaced depreciated `LooseVersion` with `parse_version`.
    * Updated `Retry` package to be imported from `requests.adapters`.
    * Moved terminal and python code text area to Question GitHub Form Schema.
    * Removed unnecessary imports.
    * Removed redundant code.
    * Improved logging messages.
    * Updated code comments.
    * Updated method descriptions.
    * Refactored code.
    * Increased coverage.
- Bash Script:
    * Updated FFmpeg Static Binaries links to latest date/version tag to `12-07-2022`.
    * Removed depreciated binaries download links and code.
- Docs:
    * Replaced all `raw.githubusercontent.com` GIF URLs with `user-images.githubusercontent.com`.
    * Reformatted `custom.css` and added missing comments.
    * Updated sponsor block.
    * Enabled Code Highlights.
    * Updated announcement bar.
    * Reduced `webgear_rtc.gif` size.
    * Updated Zenodo badge and the BibTeX entry.
- CI:
    * Added more flexible formats to `return_testvideo_path` function.
    * Updated `test_write` test for higher than 8-bit depth input frames pixel-format in WriteGear's Compression Mode.
    * Updated `actions/checkout` to `v3`.
    * Updated `actions/setup-python` to `v4`.
    * Updated `codecov/codecov-action` to `v3`.
    * Moved `test_colorspaces` test to CamGear tests.
    * Added deffcode library import.
    * Re-stuctured yaml code.

#### Breaking Updates/Changes πŸ’₯
- WriteGear: 
    * Renamed `output_filename` string parameter to `output`.
      + Since WriteGear API accepts all sorts of streams _(such as valid filename/path/URL)_ for encoding, thereby changing parameter name to `output` will be more true to its purpose.
      + Renaming `output_filename` to `output` in WriteGear API will also help user to not accidentally assume WriteGear supports only encoding of video files.
      * It matches the `output` parameter in StreamGear which basically does the same thing.
    * Renamed `cmd` parameter in `execute_ffmpeg_cmd()` class method to more sensible `command`.
    * `ValueError` will be raised if datatype of input frames mismatches Writegear API

#### Bug-fixes πŸ›
- Camgear: 
    * Fixed `CamGear.read()` blocked unnecessarily.
      + πŸ’¬ When `THREADED_QUEUE_MODE` is enabled `CamGear.read()` blocks for an excessive duration when attempting to read past the end of a stream.
      + Added `None` frame to the queue at the end to signal we're done.
      + Added `terminate` Event check before continuing.
    * Fixed deadlock on exit.
      + πŸ’¬ The deadlock is due to `self.__queue.get(timeout=self.__thread_timeout)` line in `read()` method, which still waits for timeout(thread_timeout) to happen when main `update()` thread was already terminated on exit and queue was empty. Since there was no way to signal queue that stream is already ended, the blocking `queue.get()` keeps on waiting until timeout occurs.
      + The solution was to signal `queue.get()` that stream is already ended by putting `None` in queue on exiting the main `update()` thread.
- ScreenGear: 
    * Fixed `ScreenGear.read()` blocked during cold startup.
      + πŸ’¬ During startup, `ScreenGear.read()` doesn't checks if queue is empty before continuing.
- WriteGear: 
    * Fixed gstpipeline_mode not activating when wrongly assuming `output` value as valid path.
    * Fixed name 'compression' is not defined bug.
    * Fixed `AttributeError`.
- Helper:
    * Fixed `fltp` keyword in regex pattern causing non-ftlp streams to be not recognized.
    * Fixed response.headers returning `content-length` as Nonetype since it may not necessarily have the Content-Legth header set.
      + Reason: The response from gitlab.com  contains a Transfer-Encoding field as `'Transfer-Encoding': 'chunked'`, which means data is sent in a series of chunks, so the Content-Length header is emitted. More info: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding#Directives
    * Fixed Linux video device paths still not working. 
      + Moved `helper.py` logic to WriteGear and StreamGear APIs resp.
    * Fixed KeyError for empty metadata.
- Setup:
    * Pinned `pyzmq==24.0.1` working version.
    * Removed redundant patch for the issue.
- Maintaince:
    * Fixed missing pkg name `import_dependency_safe` functions calls.
- Bash Script: 
    * Fixed gstreamer installation.
- CI:
    * Fixed missing v4l2loopback apt dependency on Linux envs.
    * Added fix for RTCPeerConnection fails to create RTCDtlsTransport (Related issue: aiortc/aiortc#804)
      + Pinned `cryptography==38.0.4` in dependencies.
    * Pinned Linux image to `ubuntu-20.04` in github actions.
    * Fixed No module named 'httpx' bug.
      + Added `httpx` library import.
    * Fixed F821 undefined name bug.
    * Fixed Gstreamer bug.
- Docs:
    * Fixed hyperlinks to new GitHub's form schemas. 
    * Fixed non-rendering images in README.md 
      + Replaced all relative image/gifs paths with absolute URLs in README.md.
    * Fixed badges/shields#8671 badge issue in README.md
    * Fixed GitLab CDN links throwing blocked by CORS policy bug.
      + Replaced gitlab GitHack CDN links with with bitbucket.
    * Fixed DASH playback failing by setting the `maxAttempts` to Infinity.
    * Removed `x-sign` glow-text effect CSS.
    * Fixed several typos (suggested by @timgates42)
    * Fixed coverage badge.
  • Loading branch information
abhiTronix authored Jan 26, 2023
2 parents 6e4eaf6 + 8cbfbff commit 3572ef3
Show file tree
Hide file tree
Showing 72 changed files with 1,976 additions and 1,189 deletions.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ body:
render: shell
- type: textarea
attributes:
label: Python Code
label: Python Code(Optional)
description: Share python code you're using(only if available) here or else leave this block empty. You can directly paste your python code here, no need for any backticks or formatting.
render: python
- type: input
Expand Down
10 changes: 10 additions & 0 deletions .github/ISSUE_TEMPLATE/question.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,16 @@ body:
description: Please describe your question thoroughly here. What are you trying to accomplish?
validations:
required: true
- type: textarea
attributes:
label: Terminal log output(Optional)
description: Copy and paste any relevant terminal log output(only if applicable) here or else leave this block empty. You can enable logging with `logging=True` in any Vidgear API for getting verbose terminal logs.
render: shell
- type: textarea
attributes:
label: Python Code(Optional)
description: Share python code you're using(only if available) here or else leave this block empty. You can directly paste your python code here, no need for any backticks or formatting.
render: python
- type: input
attributes:
label: VidGear Version
Expand Down
35 changes: 21 additions & 14 deletions .github/workflows/ci_linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,37 +19,38 @@ on:
branches:
- testing
paths-ignore:
- '**/*.md'
- '**/*.html'
- '**/*.js'
- '**/*.css'
- 'docs/**/*'
- "**/*.md"
- "**/*.html"
- "**/*.js"
- "**/*.css"
- "docs/**/*"
- README.md
- mkdocs.yml
pull_request:
branches:
- testing
paths-ignore:
- '**/*.md'
- 'docs/**/*'
- "**/*.md"
- "docs/**/*"

jobs:
test:
name: CI Test vidgear
runs-on: ubuntu-latest
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ['3.7', '3.8', '3.9', '3.10']
python-version: ["3.7", "3.8", "3.9", "3.10"]
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: install apt_dependencies
run: |
sudo apt-get update -qq
sudo apt-get install -qq unzip wget -y
sudo apt-get install -qq dos2unix -y
sudo apt-get install ffmpeg v4l2loopback-dkms v4l2loopback-utils linux-modules-extra-$(uname -r) -y
- name: prepare bash_scripts
run: |
dos2unix scripts/bash/prepare_dataset.sh
Expand All @@ -61,7 +62,9 @@ jobs:
pip install -U pip wheel numpy
pip install -U .[asyncio]
pip uninstall opencv-python -y
pip install -U flake8 six codecov pytest pytest-asyncio pytest-cov yt_dlp mpegdash paramiko m3u8 async-asgi-testclient
pip install -U flake8 six codecov httpx pytest pytest-asyncio pytest-cov yt_dlp mpegdash paramiko m3u8 async-asgi-testclient
pip install -U deffcode
pip install cryptography==38.0.4
if: success()
- name: run prepare_dataset_script
run: bash scripts/bash/prepare_dataset.sh
Expand All @@ -80,13 +83,17 @@ jobs:
if: success() && matrix.python-version == 3.7
- name: pytest without_ENV
run: |
sudo modprobe v4l2loopback devices=1 video_nr=0 exclusive_caps=1 card_label='VCamera'
timeout 1200 pytest --verbose --cov=vidgear --cov-report=xml --cov-report term-missing vidgear/tests/ || code=$?; if [[ $code -ne 124 && $code -ne 0 ]]; then exit $code; else echo "EXIT_CODE=$code" >>$GITHUB_ENV; fi
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
if: success() && matrix.python-version != 3.7
- name: Display exit code
run: echo "Exit code was = $EXIT_CODE"
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
uses: codecov/codecov-action@v3
with:
name: ${{ matrix.python-version }}
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: true
verbose: true
if: ${{ env.EXIT_CODE != '124' }}
if: ${{ env.EXIT_CODE != '124' }}
12 changes: 6 additions & 6 deletions .github/workflows/deploy_docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,10 @@ jobs:
if: github.event_name == 'release' && github.event.action == 'published'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: actions/setup-python@v2
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: install_vidgear
Expand Down Expand Up @@ -75,10 +75,10 @@ jobs:
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: actions/setup-python@v2
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: install_vidgear
Expand Down Expand Up @@ -118,11 +118,11 @@ jobs:
if: github.event_name == 'push' && github.ref == 'refs/heads/testing'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
with:
fetch-depth: 0
- run: git checkout testing
- uses: actions/setup-python@v2
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: install_vidgear
Expand Down
Loading

0 comments on commit 3572ef3

Please sign in to comment.