Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

added reframer_ts_vtt filter. #7

Open
wants to merge 7 commits into
base: integration
Choose a base branch
from
Open

added reframer_ts_vtt filter. #7

wants to merge 7 commits into from

Conversation

jadarve
Copy link

@jadarve jadarve commented Dec 23, 2024 •
edited
Loading

Given a Transport Stream file containing Metadata as PES packets with metadata format as "WVTT", the new rftsvtt filter reads the content of each PES packet and creates a WebVTT cue output that can be used by the dasher to create a subtitles track for both DASH and HLS.

image

Running GPAC as:

/home/juan/workspace/git/gpac/bin/gcc/gpac \
  -i /home/juan/workspace/notebooks/GStreamer/out_subtitle.ts \
  rftsvtt \
  -o "/home/juan/workspace/notebooks/GStreamer/gpac/hls/playlist.m3u8:gpac:buf=-150:seg_sync=no:segdur=1.0:cdur=1.0:hlsc=true:cmaf=cmf2:dmode=static:template=segment:rawsub=true" \
  -graph

creates a filter graph as follows:

Filters connected:
fin (src=/home/juan/workspace/notebooks/GStreamer/out_subtitle.ts) (idx=1)
-(PID out_subtitle.ts) m2tsdmx (dyn_idx=4)
--(PID P1T2) rftsvtt (idx=2)
---(PID P1T2) dasher (dyn_idx=7)
----(PID manifest_m3u8) fout (dst=/home/juan/workspace/notebooks/GStreamer/gpac/hls/playlist.m3u8:gpac:buf=-150:seg_sync=no:segdur=1.0:cdur=1.0:hlsc=true:cmaf=cmf2:dmode=static:template=segment:rawsub=true) (idx=3)
----(PID P1T2)          ufvtt (dyn_idx=11)
-----(PID P1T2) writegen (dyn_idx=12)
------(PID P1T2) fout (dst=/home/juan/workspace/notebooks/GStreamer/gpac/hls/segment_track65_init.vtt:gfopt:buf=-150:seg_sync=no:segdur=1.0:cdur=1.0:hlsc=true:cmaf=cmf2:dmode=static:template=segment:rawsub=true:noinitraw:frag:xps_inband=no:psshs=moov) (idx=8)
----(PID P1V3)          mp4mx (dyn_idx=13)
-----(PID P1V3) fout (dst=/home/juan/workspace/notebooks/GStreamer/gpac/hls/segment_track66_init.mp4:gfopt:buf=-150:seg_sync=no:segdur=1.0:cdur=1.0:hlsc=true:cmaf=cmf2:dmode=static:template=segment:rawsub=true:frag:xps_inband=no:psshs=moov:mime=video/mp4) (idx=9)
----(PID P1A4)          mp4mx (dyn_idx=14)
-----(PID P1A4) fout (dst=/home/juan/workspace/notebooks/GStreamer/gpac/hls/segment_track67_init.mp4:gfopt:buf=-150:seg_sync=no:segdur=1.0:cdur=1.0:hlsc=true:cmaf=cmf2:dmode=static:template=segment:rawsub=true:frag:xps_inband=no:psshs=moov:mime=audio/mp4) (idx=10)
--(PID P1V3) rfnalu (dyn_idx=5)
---(PID P1V3) dasher (dyn_idx=7)
--(PID P1A4) rfadts (dyn_idx=6)
---(PID P1A4) dasher (dyn_idx=7)

Notice that, as this is HLS, the rawsub=true flag is used. The output playlist defines the subtitles track:

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-INDEPENDENT-SEGMENTS

#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="1",AUTOSELECT=YES,URI="playlist_1.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=2117639,CODECS="avc1.42C01E,mp4a.40.2",RESOLUTION=640x480,FRAME-RATE=25,AUDIO="audio",SUBTITLES="subs",CLOSED-CAPTIONS=NONE
playlist_2.m3u8

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio",NAME="3",AUTOSELECT=YES,URI="playlist_3.m3u8",CHANNELS="2"

and playlist_1.m3u8

#EXTM3U
#EXT-X-TARGETDURATION:1
#EXT-X-VERSION:6
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-INDEPENDENT-SEGMENTS
#EXTINF:1,
segment_track65_1.vtt
#EXTINF:1,
segment_track65_2.vtt
#EXTINF:1,
segment_track65_3.vtt
#EXTINF:1,
segment_track65_4.vtt
#EXTINF:1,
segment_track65_5.vtt
#EXTINF:1,
segment_track65_6.vtt
#EXTINF:1,
segment_track65_7.vtt
#EXTINF:0.8,
segment_track65_8.vtt

#EXT-X-ENDLIST

where each .vtt file contains 1 second of data.

estigma88
estigma88 reviewed Jan 7, 2025
View reviewed changes
src/filter_core/filter_register.c Show resolved Hide resolved
src/filters/dmx_m2ts.c Outdated
@@ -479,6 +486,7 @@ static void m2tsdmx_declare_pid(GF_M2TSDmxCtx *ctx, GF_M2TS_PES *stream, GF_ESD
if (esd->decoderConfig && (esd->decoderConfig->streamType==GF_STREAM_OD))
stream->flags |= GF_M2TS_ES_IS_MPEG4_OD;
} else {
// TODO: Adarve this is the path for WVTT
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need it?

src/filters/dmx_m2ts.c Outdated
@@ -745,7 +752,13 @@ static void m2tsdmx_send_packet(GF_M2TSDmxCtx *ctx, GF_M2TS_PES_PCK *pck)

/*pcr not initialized, don't send any data*/
// if (! pck->stream->program->first_dts) return;
if (!pck->stream->user) return;

// if (pck->stream->pid == 65) GF_LOG(GF_LOG_ERROR, GF_LOG_CONDITION, ("m2tsdmx_send_packet: PID 65"));
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need it?

src/filters/dmx_m2ts.c
@@ -795,6 +808,8 @@ static void m2tsdmx_send_packet(GF_M2TSDmxCtx *ctx, GF_M2TS_PES_PCK *pck)
}


// if (pck->stream->pid == 65) GF_LOG(GF_LOG_ERROR, GF_LOG_CONDITION, ("m2tsdmx_send_packet: creating packet %llu\n", len));
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need it?

src/filters/reframe_ts_wvtt.c Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@estigma88 estigma88 estigma88 left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jadarve @estigma88