Skip to content

Commit

Permalink
⚡️Enhancements, Refactoring, and Maintenance Updates for StreamGear A…
Browse files Browse the repository at this point in the history 
…PI [#401]

💬 This commit includes significant refactoring, documentation updates, and enhancements across StreamGear, WriteGear, and NetGear APIs. It introduces safer process termination methods, improved parameter handling, and enhanced documentation for better user guidance. Additionally, it addresses several bugs and improves the overall maintainability and clarity of the codebase.

- Refactored stream copy handling to address errors with unsupported parameters (Fixes #396).
  - Moved settings for "-vf" and "-aspect" inside conditional blocks.
  - Added warnings and discarded these parameters in stream copy mode.
  - Fixed incompatibility of stream copy with Real-time Frames Mode.
    - Added warnings and discarded `-vcodec copy` when using this mode.
- Deprecated `terminate()` method, introducing `close()` for safer process termination.
  - Updated documentation and tests to reflect these changes.
- Introduced new `-enable_force_termination` attribute for immediate FFmpeg process termination.
- Restricted `-livestream` parameter to Real-time Frames Mode only.
  - Disabled live streaming for video files and updated relevant logging.
- Removed non-essential aspect ratio parameter to prevent unwanted distortions (Fixes #385).
  - Enhanced warning messages and clarified documentation.
- Updated `close()` methods for handling gracefully signal interruptions on different systems with device audio streams.
  - Adjusted signals for graceful termination based on the operating system.

#### WriteGear API:
- Simplified the logic for formatting output parameters.


#### NetGear API:
- Fixed Secure Mode failing to work on conflicting ZMQ Contexts:
  - Handled "Address in use" error more gracefully and disabled secure mode if errors occur.
  - Improved handling of ZMQ Authenticator and Certificates.
- Enhanced logging and error handling for secure mode.
  - Logged Authenticator start/stop events.
- Handled socket session expiration more gracefully in `recv_handler`.
- Fixed `msg_json` undefined when terminating context in the `recv_handler` method.
- Ensured proper termination of the ZMQ context and socket when closing the NetGear instance.
- Added Admonition for warning users about the Client's end must run before the Server's end to establish a secure connection in Secure Mode.

#### Helper Class:
- Added a custom deprecated decorator to mark deprecated functions and parameters.
  - It displays a warning message when a deprecated function or parameter is used.
- Updated `extract_time` helper function regex to handle milliseconds.

#### Setup
- Updated setup.py to use the latest `pyzmq` version to address installation issues (Fixes #399).

#### Documentation:
- StreamGear Updates:
  - Improved the overview section's description and wording.
  - Updated usage examples for both Single-Source Mode and Real-time Frames Mode.
    - Updated StreamGear usage examples for device audio input.
    - Refactored sections for Live Streaming usage.
      - Added warning for unsupported `-livestream` parameter in Single-Source Mode.
    - Updated usage example for device video source.
  - Addressed deprecation of the `terminate()` method in favor of the new `close()` method.
    - Updated respective notices for the deprecated `terminate()` method and `rgb_mode` parameter.
  - Added the use of new `-enable_force_termination` parameter.
  - Added a new FAQ entry about the deprecated `rgb_mode` parameter.
  - Added a deprecation warning admonition for the `rgb_mode` parameter in the `stream()` method.
    - Removed the obsolete usage example for deprecation RGB mode with StreamGear.
  - Added documentation and usage of the new `-enable_force_termination` parameter.
    - Modified the warning message to mention that forced termination can cause corrupted output in certain scenarios.
  - Updated the docstring for the `stream()` method and `transcode_source()` method.
  - Refactored the StreamGear API Parameters documentation to enhance clarity and readability.
    - Refined the description of the `-streams` attribute of the StreamGear API.
- Updated the documentation for the `-disable_force_termination` parameter in WriteGear API.
- Improved overall documentation quality, including enhancing clarity, readability, consistency, and precision.
- Added detailed explanations, practical examples, best practices, and clearer usage patterns.
- Fixed various issues like typos, formatting errors, code highlighting issues, and grammar inconsistencies.
- Updated sections, code examples, admonitions, and comments for better clarity and accuracy.
- Fixed minor typo in `js_hook.py`.

#### CI
- Improved parameterized test cases to support floating point values.
- Fixed various typos and code issues in tests.
- Updated StreamGear tests to use the new `close()` method instead of the deprecated terminate() method.
- Updated tests of various APIs for better coverage and reliability.
- Enabled `kill=True` in `close()` in NetGear Tests.
- Fixed expected duration value in parameterized test case from `8` to `8.44` since `test_extract_time` function now supports floating point values.
- Fixed `test_secure_mode` NetGear test:
  - Added `"127.0.0.1"` address to allow common endpoint for connection.
  - Added `"jpeg_compression":False` to disable frame compression, allowing frame to be the same while assertion.


#### Maintenance:
- Applied short-circuiting to simplify code across various APIs and tests.
- Improved logging, parameter validation, and added descriptive dialogs across various APIs.
  • Loading branch information
@abhiTronix
abhiTronix authored Jun 7, 2024
2 parents 20dec0a + c101402 commit d4243ab
Show file tree
Hide file tree
Showing 24 changed files with 1,237 additions and 1,136 deletions.
14 changes: 7 additions & 7 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -427,21 +427,21 @@ In addition to this, WriteGear also provides flexible access to [**OpenCV's Vide
<img src="https://abhitronix.github.io/vidgear/latest/assets/images/streamgear_flow.webp" alt="NetGear API" width=80%/>
</p>

> _StreamGear automates transcoding workflow for generating Ultra-Low Latency, High-Quality, Dynamic & Adaptive Streaming Formats (such as MPEG-DASH and Apple HLS) in just few lines of python code._
> _StreamGear streamlines and simplifies the transcoding workflow to generate Ultra-Low Latency, High-Quality, Dynamic & Adaptive Streaming Formats like MPEG-DASH and Apple HLS with just a few lines of Python code, allowing developers to focus on their application logic rather than dealing with the complexities of transcoding and chunking media files._
StreamGear provides a standalone, highly extensible, and flexible wrapper around [**FFmpeg**][ffmpeg] multimedia framework for generating chunked-encoded media segments of the content.
StreamGear API provides a standalone, highly extensible, and flexible wrapper around the [**FFmpeg**](https://ffmpeg.org/) multimedia framework for generating chunk-encoded media segments from your multimedia content effortlessly.

SteamGear is an out-of-the-box solution for transcoding source videos/audio files & real-time video frames and breaking them into a sequence of multiple smaller chunks/segments of suitable lengths. These segments make it possible to stream videos at different quality levels _(different bitrates or spatial resolutions)_ and can be switched in the middle of a video from one quality level to another – if bandwidth permits – on a per-segment basis. A user can serve these segments on a web server that makes it easier to download them through HTTP standard-compliant GET requests.
With StreamGear, you can transcode source video/audio files and real-time video frames into a sequence of multiple smaller chunks/segments of suitable lengths. These segments facilitate streaming at different quality levels _(bitrates or spatial resolutions)_ and allow for seamless switching between quality levels during playback based on available bandwidth. You can serve these segments on a web server, making them easily accessible via standard **HTTP GET** requests.

SteamGear currently supports [**MPEG-DASH**](https://www.encoding.com/mpeg-dash/) _(Dynamic Adaptive Streaming over HTTP, ISO/IEC 23009-1)_ and [**Apple HLS**](https://developer.apple.com/documentation/http_live_streaming) _(HTTP Live Streaming)_. But, Multiple DRM support is yet to be implemented.
SteamGear currently supports both [**MPEG-DASH**](https://www.encoding.com/mpeg-dash/) _(Dynamic Adaptive Streaming over HTTP, ISO/IEC 23009-1)_ and [**Apple HLS**](https://developer.apple.com/documentation/http_live_streaming) _(HTTP Live Streaming)_.

SteamGear also creates a Manifest file _(such as MPD in-case of DASH)_ or a Master Playlist _(such as M3U8 in-case of Apple HLS)_ besides segments that describe these segment information _(timing, URL, media characteristics like video resolution and bit rates)_ and is provided to the client before the streaming session.
Additionally, StreamGear generates a manifest file _(such as MPD for DASH)_ or a master playlist _(such as M3U8 for Apple HLS)_ alongside the segments. These files contain essential segment information, _including timing, URLs, and media characteristics like video resolution and adaptive bitrates_. They are provided to the client before the streaming session begins.

**StreamGear primarily works in two Independent Modes for transcoding which serves different purposes:**

- **Single-Source Mode:** In this mode, StreamGear **transcodes entire video file** _(as opposed to frame-by-frame)_ into a sequence of multiple smaller chunks/segments for streaming. This mode works exceptionally well when you're transcoding long-duration lossless videos(with audio) for streaming that required no interruptions. But on the downside, the provided source cannot be flexibly manipulated or transformed before sending onto FFmpeg Pipeline for processing. **_Learn more about this mode [here ➶][ss-mode-doc]_**
- **Single-Source Mode :cd: :** In this mode, StreamGear **transcodes entire video file** _(as opposed to frame-by-frame)_ into a sequence of multiple smaller chunks/segments for streaming. This mode works exceptionally well when you're transcoding long-duration lossless videos(with audio) for streaming that required no interruptions. But on the downside, the provided source cannot be flexibly manipulated or transformed before sending onto FFmpeg Pipeline for processing. **_Learn more about this mode [here ➶][ss-mode-doc]_**

- **Real-time Frames Mode:** In this mode, StreamGear directly **transcodes frame-by-frame** _(as opposed to a entire video file)_, into a sequence of multiple smaller chunks/segments for streaming. This mode works exceptionally well when you desire to flexibility manipulate or transform [`numpy.ndarray`](https://numpy.org/doc/1.18/reference/generated/numpy.ndarray.html#numpy-ndarray) frames in real-time before sending them onto FFmpeg Pipeline for processing. But on the downside, audio has to added manually _(as separate source)_ for streams. **_Learn more about this mode [here ➶][rtf-mode-doc]_**
- **Real-time Frames Mode :film_frames: :** In this mode, StreamGear directly **transcodes frame-by-frame** _(as opposed to a entire video file)_, into a sequence of multiple smaller chunks/segments for streaming. This mode works exceptionally well when you desire to flexibility manipulate or transform [`numpy.ndarray`](https://numpy.org/doc/1.18/reference/generated/numpy.ndarray.html#numpy-ndarray) frames in real-time before sending them onto FFmpeg Pipeline for processing. But on the downside, audio has to added manually _(as separate source)_ for streams. **_Learn more about this mode [here ➶][rtf-mode-doc]_**

### StreamGear API Guide:

Expand Down
100 changes: 54 additions & 46 deletions docs/gears/netgear/advanced/secure_mode.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ Secure Mode uses a new wire protocol, [**ZMTP 3.0**](http://zmtp.org/) that adds

Secure Mode can be easily activated in NetGear API through `secure_mode` attribute of its [`options`](../../params/#options) dictionary parameter, during initialization. Furthermore, for managing this mode, NetGear API provides additional `custom_cert_location` & `overwrite_cert` like attribute too.


&nbsp;

## Supported ZMQ Security Layers
Expand All @@ -47,11 +46,12 @@ Secure mode supports the two most powerful ZMQ security layers:

&nbsp;


!!! danger "Important Information regarding Secure Mode"

* The `secure_mode` attribute value at the Client's end **MUST** match exactly the Server's end _(i.e. **IronHouse** security layer is only compatible with **IronHouse**, and **NOT** with **StoneHouse**)_.

* In Secure Mode, The Client's end **MUST** run before the Server's end to establish a secure connection.

* The Public+Secret Keypairs generated at the Server end **MUST** be made available at the Client's end too for successful authentication. If mismatched, connection failure will occur.

* By Default, the Public+Secret Keypairs will be generated/stored at the `$HOME/.vidgear/keys` directory of your machine _(e.g. `/home/foo/.vidgear/keys` on Linux)_. But you can also use [`custom_cert_location`](../../params/#options) attribute to set your own Custom-Path for a directory to generate/store these Keypairs.
Expand All @@ -60,8 +60,11 @@ Secure mode supports the two most powerful ZMQ security layers:

* **IronHouse** is the strongest Security Layer available, but it involves certain security checks that lead to **ADDITIONAL LATENCY**.


* Secure Mode only supports `libzmq` library version `>= 4.0`.




&nbsp;

Expand Down Expand Up @@ -125,10 +128,56 @@ For implementing Secure Mode, NetGear API currently provide following exclusive

Following is the bare-minimum code you need to get started with Secure Mode in NetGear API:

#### Server's End
!!! alert "In Secure Mode, Client's end MUST run before the Server's end to establish a secure connection!"

#### Client's End

Open your favorite terminal and execute the following python code:

!!! tip "You can terminate client anytime by pressing ++ctrl+"C"++ on your keyboard!"

```python linenums="1" hl_lines="6"
# import required libraries
from vidgear.gears import NetGear
import cv2

# activate StoneHouse security mechanism
options = {"secure_mode": 1}

# define NetGear Client with `receive_mode = True` and defined parameter
client = NetGear(pattern=1, receive_mode=True, logging=True, **options)

# loop over
while True:

# receive frames from network
frame = client.recv()

# check for received frame if Nonetype
if frame is None:
break

# {do something with the frame here}

# Show output window
cv2.imshow("Output Frame", frame)

# check for 'q' key if pressed
key = cv2.waitKey(1) & 0xFF
if key == ord("q"):
break

# close output window
cv2.destroyAllWindows()

# safely close client
client.close()
```

#### Server's End

Then open another terminal on the same system and execute the following python code to send the frames to our client:

!!! tip "You can terminate both sides anytime by pressing ++ctrl+"C"++ on your keyboard!"

```python linenums="1" hl_lines="9"
Expand Down Expand Up @@ -171,49 +220,7 @@ stream.stop()
server.close()
```

#### Client's End

Then open another terminal on the same system and execute the following python code and see the output:

!!! tip "You can terminate client anytime by pressing ++ctrl+"C"++ on your keyboard!"

```python linenums="1" hl_lines="6"
# import required libraries
from vidgear.gears import NetGear
import cv2

# activate StoneHouse security mechanism
options = {"secure_mode": 1}

# define NetGear Client with `receive_mode = True` and defined parameter
client = NetGear(pattern=1, receive_mode=True, logging=True, **options)

# loop over
while True:

# receive frames from network
frame = client.recv()

# check for received frame if Nonetype
if frame is None:
break

# {do something with the frame here}

# Show output window
cv2.imshow("Output Frame", frame)

# check for 'q' key if pressed
key = cv2.waitKey(1) & 0xFF
if key == ord("q"):
break

# close output window
cv2.destroyAllWindows()

# safely close client
client.close()
```

&nbsp;

Expand All @@ -222,11 +229,12 @@ client.close()

### Using Secure Mode with Variable Parameters


#### Client's End

Open a terminal on Client System _(where you want to display the input frames received from the Server)_ and execute the following python code:

!!! alert "In Secure Mode, Client's end MUST run before the Server's end to establish a secure connection!"

!!! info "Note down the local IP-address of this system(required at Server's end) and also replace it in the following code. You can follow [this FAQ](../../../../help/netgear_faqs/#how-to-find-local-ip-address-on-different-os-platforms) for this purpose."

!!! danger "You need to paste the Public+Secret Keypairs _(generated at the Server End)_ at the `$HOME/.vidgear/keys` directory of your Client machine for a successful authentication!"
Expand Down
26 changes: 12 additions & 14 deletions docs/gears/streamgear/introduction.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,15 @@ limitations under the License.

## Overview

> StreamGear automates transcoding workflow for generating _Ultra-Low Latency, High-Quality, Dynamic & Adaptive Streaming Formats (such as MPEG-DASH and Apple HLS)_ in just few lines of python code.
> StreamGear streamlines and simplifies the transcoding workflow to generate _Ultra-Low Latency, High-Quality, Dynamic & Adaptive Streaming Formats like MPEG-DASH and Apple HLS_ with just a few lines of Python code, allowing developers to focus on their application logic rather than dealing with the complexities of transcoding and chunking media files.
StreamGear provides a standalone, highly extensible, and flexible wrapper around [**FFmpeg**](https://ffmpeg.org/) multimedia framework for generating chunked-encoded media segments of the content.
StreamGear API provides a standalone, highly extensible, and flexible wrapper around the [**FFmpeg**](https://ffmpeg.org/) multimedia framework for generating chunk-encoded media segments from your multimedia content effortlessly.

SteamGear is an out-of-the-box solution for transcoding source videos/audio files & real-time video frames and breaking them into a sequence of multiple smaller chunks/segments of suitable lengths. These segments make it possible to stream videos at different quality levels _(different bitrates or spatial resolutions)_ and can be switched in the middle of a video from one quality level to another – if bandwidth permits – on a per-segment basis. A user can serve these segments on a web server that makes it easier to download them through HTTP standard-compliant GET requests.
With StreamGear, you can transcode source video/audio files and real-time video frames into a sequence of multiple smaller chunks/segments of suitable lengths. These segments facilitate streaming at different quality levels _(bitrates or spatial resolutions)_ and allow for seamless switching between quality levels during playback based on available bandwidth. You can serve these segments on a web server, making them easily accessible via standard **HTTP GET** requests.

SteamGear currently supports [**MPEG-DASH**](https://www.encoding.com/mpeg-dash/) _(Dynamic Adaptive Streaming over HTTP, ISO/IEC 23009-1)_ and [**Apple HLS**](https://developer.apple.com/documentation/http_live_streaming) _(HTTP Live Streaming)_.
SteamGear currently supports both [**MPEG-DASH**](https://www.encoding.com/mpeg-dash/) _(Dynamic Adaptive Streaming over HTTP, ISO/IEC 23009-1)_ and [**Apple HLS**](https://developer.apple.com/documentation/http_live_streaming) _(HTTP Live Streaming)_.

SteamGear also creates a Manifest file _(such as MPD in-case of DASH)_ or a Master Playlist _(such as M3U8 in-case of Apple HLS)_ besides segments that describe these segment information _(timing, URL, media characteristics like video resolution and adaptive bit rates)_ and is provided to the client before the streaming session.
Additionally, StreamGear generates a manifest file _(such as MPD for DASH)_ or a master playlist _(such as M3U8 for Apple HLS)_ alongside the segments. These files contain essential segment information, _including timing, URLs, and media characteristics like video resolution and adaptive bitrate_. They are provided to the client before the streaming session begins.

!!! alert "For streaming with older traditional protocols such as RTMP, RTSP/RTP you could use [WriteGear](../../writegear/introduction/) API instead."

Expand All @@ -59,9 +59,9 @@ SteamGear also creates a Manifest file _(such as MPD in-case of DASH)_ or a Mast

!!! tip "Useful Links"

- Checkout [this detailed blogpost](https://ottverse.com/mpeg-dash-video-streaming-the-complete-guide/) on how MPEG-DASH works.
- Checkout [this detailed blogpost](https://ottverse.com/hls-http-live-streaming-how-does-it-work/) on how HLS works.
- Checkout [this detailed blogpost](https://ottverse.com/hls-http-live-streaming-how-does-it-work/) for HLS vs. MPEG-DASH comparison.
- Checkout [this detailed blogpost ➶](https://ottverse.com/mpeg-dash-video-streaming-the-complete-guide/) on how MPEG-DASH works.
- Checkout [this detailed blogpost ➶](https://ottverse.com/hls-http-live-streaming-how-does-it-work/) on how HLS works.
- Checkout [this detailed blogpost ➶](https://imagekit.io/blog/hls-vs-dash/) for HLS vs. MPEG-DASH comparison.


&thinsp;
Expand All @@ -71,14 +71,12 @@ SteamGear also creates a Manifest file _(such as MPD in-case of DASH)_ or a Mast
StreamGear primarily operates in following independent modes for transcoding:


??? warning "Real-time Frames Mode is NOT Live-Streaming."
???+ alert "Real-time Frames Mode itself is NOT Live-Streaming :material-video-wireless-outline:"
To enable live-streaming in Real-time Frames Mode, use the exclusive [`-livestream`](../params/#a-exclusive-parameters) attribute of the `stream_params` dictionary parameter in the StreamGear API. Checkout [this usage example ➶](../rtfm/usage/#bare-minimum-usage-with-live-streaming) for more information.

Rather, you can enable live-streaming in Real-time Frames Mode by using the exclusive [`-livestream`](../params/#a-exclusive-parameters) attribute of `stream_params` dictionary parameter in StreamGear API. Checkout [this usage example](../rtfm/usage/#bare-minimum-usage-with-live-streaming) for more information.
- [**Single-Source Mode :material-file-video-outline:**](../ssm/overview) : In this mode, StreamGear **transcodes entire video file** _(as opposed to frame-by-frame)_ into a sequence of multiple smaller chunks/segments for streaming. This mode works exceptionally well when you're transcoding long-duration lossless videos(with audio) for streaming that required no interruptions. But on the downside, the provided source cannot be flexibly manipulated or transformed before sending onto FFmpeg Pipeline for processing.


- [**Single-Source Mode**](../ssm/overview): In this mode, StreamGear **transcodes entire video file** _(as opposed to frame-by-frame)_ into a sequence of multiple smaller chunks/segments for streaming. This mode works exceptionally well when you're transcoding long-duration lossless videos(with audio) for streaming that required no interruptions. But on the downside, the provided source cannot be flexibly manipulated or transformed before sending onto FFmpeg Pipeline for processing.

- [**Real-time Frames Mode**](../rtfm/overview): In this mode, StreamGear directly **transcodes frame-by-frame** _(as opposed to a entire video file)_, into a sequence of multiple smaller chunks/segments for streaming. This mode works exceptionally well when you desire to flexibility manipulate or transform [`numpy.ndarray`](https://numpy.org/doc/1.18/reference/generated/numpy.ndarray.html#numpy-ndarray) frames in real-time before sending them onto FFmpeg Pipeline for processing. But on the downside, audio has to added manually _(as separate source)_ for streams.
- [**Real-time Frames Mode :material-camera-burst:**](../rtfm/overview) : In this mode, StreamGear directly **transcodes frame-by-frame** _(as opposed to a entire video file)_, into a sequence of multiple smaller chunks/segments for streaming. This mode works exceptionally well when you desire to flexibility manipulate or transform [`numpy.ndarray`](https://numpy.org/doc/1.18/reference/generated/numpy.ndarray.html#numpy-ndarray) frames in real-time before sending them onto FFmpeg Pipeline for processing. But on the downside, audio has to added manually _(as separate source)_ for streams.

&thinsp;

Expand Down
Loading

0 comments on commit d4243ab

Please sign in to comment.