FFmpeg SCTE-35 Stream Type 0x6 bin_data is well supported by threefive

[bbgdzxng1]

Latest ffmpeg and threefive and SCTE-35 Info

futzu wrote.. "I don't know how to stop ffmpeg from changing the SCTE35 stream type to 0x6."

FYI - I did some experiments with this and it seems to be a limitation of FFmpeg's TS muxer.

If I use the NUT container (FFmpeg's proprietary container format), the -tag:d can be used to set the type. In theory, this tag should also be capable of setting the type via a fourcc value, but that does not seem to be supported for data streams. Hex or decimal work with a NUT container though.

In this case, the stream description was changed from scte_35 to none.

$ ffmpeg -hide_banner -i "https://futzu.com/xaa.ts" -map 0:v:0 -codec:v copy -streamid 0:0x101 -map 0:a:0 -codec:a copy -streamid 1:0x102 -map 0:d:0 -codec:d:0 copy -streamid 2:0x103 -tag:d:0 "0x86" -f nut OUTFILE.nut -y

# or

$ ffmpeg -hide_banner -i "https://futzu.com/xaa.ts" -map 0:v:0 -codec:v copy -streamid 0:0x101 -map 0:a:0 -codec:a copy -streamid 1:0x102 -map 0:d:0 -codec:d:0 copy -streamid 2:0x103 -tag:d:0 "134" -f nut OUTFILE.nut -y

$ ffprobe OUTFILE.nut
  Stream #0:0: Video: h264 (Main) (H264 / 0x34363248), yuv420p(tv, bt470bg), 544x576 [SAR 64:33 DAR 544:297], 50 fps, 50 tbr, 90k tbn, 50 tbc
  Stream #0:1(eng): Audio: mp2 (P[0][0][0] / 0x0050), 48000 Hz, stereo, s16p, 128 kb/s
  Stream #0:2: Data: none ([134][0][0][0] / 0x0086)

If the same command is used with an MPEG-TS container, the FFmpeg muxer overwrites the type to be 0x6. At runtime / console output, it claims it is setting the 0x86, but when probed, the file output is 0x6.

In this case, the stream type was changed from scte_35 to bin_data.

$ ffmpeg -hide_banner -i "https://futzu.com/xaa.ts" -map 0:v:0 -codec:v copy -streamid 0:0x101 -map 0:a:0 -codec:a copy -streamid 1:0x102 -map 0:d:0 -codec:d:0 copy -streamid 2:0x103 -tag:d:0 "0x86" -f mpegts OUTFILE.ts -y

$ ffprobe OUTFILE.ts
  Stream #0:0[0x101]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg, top first), 544x576 [SAR 64:33 DAR 544:297], 25 fps, 50 tbr, 90k tbn, 50 tbc
  Stream #0:1[0x102](eng): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 128 kb/s
  Stream #0:2[0x103]: Data: bin_data ([6][0][0][0] / 0x0006)

This is not an issue with threefive, it is purely informational and can be closed. I recognize that threefive successfully works around FFmpeg's limitation. Raising the ticket was just a means of sharing knowledge and narrowing down where FFmpeg was struggling. Neither does the ticket solve the issue because the ultimate goal is setting it in a TS container, not a NUT container. Feel free to close this issue.

[futzu]

-tag:d:0 "0x86"

So that's how it works...
I tried using -tag, but I didn't know how to specify the stream.
I've never used the NUT container, I'm going to have to check it out.

I've been contemplating adding some code to change
a stream type in the PMT, that's where I get the stream type.

I don't know where the fourcc tag is stored in mpegts, that probably should set too though.

I'm going to make this a discussion, because I just noticed there was a option to do that,
and in hopes that you'll help me figure it out.

Have you ever played with mplex?

mplex13818-1.1.8.tar.gz

The information is appreciated man, post anytime you like.

Adrian

[bbgdzxng1]

NUT container is FFmpeg's native container. It is just a very simplistic container that "tries not to mess with stuff". It has no practical use outside of FFmpeg, but it is a great container for just playing with stuff within FFmpeg.

I've been contemplating adding some code to change a stream type in the PMT, that's where I get the stream type.

or just use TS Duck? That's a great tool for PMT mapping and stuff. I use TS Duck and pipe it into threefive... They are very complementary tools. You can do all sorts of TS manipulation.

$ tsp --verbose --input "hls" INPUTURL --processor "rmsplice" --dry-run --continue --processor "pcrextract" --pid "500" --scte35 | python3 -c 'import threefive; threefive.decode()'

Your code would make an an amazing base for a plugin to replace TS Duck's "rmsplice". Your code is way more sophisticated on the SCTE-35 side. TS Duck can do any of the TS manipulation rather well, and the dude who writes it seems like a good sort. If you don't know TS Duck, check it out.

Your code is also more sophisticated than TS Duck, because you have the basis for manifest-based ad markers. I don't think there are any other open-source projects that do ad marker decoding from HLS manifest.

I don't know where the fourcc tag is stored in mpegts, that probably should set too though.

Yeah, I don't know where that is set either.

mplex

I have looked at it in the past. I got scared. I ran away.

Having been a big user of threefive for a couple of years, I was amused to see that you are also the author of scc2vtt in a different repo. SCTE-35 & EIA-608/scc in HLS are two areas where I get very involved. ccextractor is my go-to, but I have played with scc2vtt. Thanks for open-sourcing your code.

Finally, I'm intrigued by where you got https://futzu.com/xaa.ts from - and don't feel like you have to answer. That particular TS stream was created about 2007-2009 in a proof of concept experiment with Channel 4 in the UK to do realtime replacement of ads for a student-focused IPTV service. I recognize the metadata in the stream. I never thought I would see that TS stream over a decade later.

[futzu]

I am so sorry, I was not aware of your additional comments, I am just now reading them.
scc2vtt was part on a captioning and subtitle project i was on for about a year and a half. I still have captioning turn on, I like it .
xaa.ts came from that scte35 ffmpeg feature request. I ran the split command on the video, that's why the xaa name

[bbgdzxng1]

All good, Futzu. There was nothing of meaning that warranted a reply.

I keep an eye on your work in threefive and regularly update to latest - it is so reassuring to see the level of maintenance and features that you put in (like HLS support and your new SDT tables). On behalf of your anonymous user base who who can't even read a line of Python - thanks for making the tool so usable in a plain command line environment.

"Great tool AAAA+++++. Would recommend."

[futzu]

@bbgdzxng1 It seems many of my users aren't pythonic, so I'm going to make a command line tool out of the threefive/go version.
What options do you want in a command line tool?

[bbgdzxng1]

Ooops, just seen this. Sorry dude. For me, the simplest would be piping a TS into a threefive.decode() installable binary - although that's the one that is pretty easy anyway once you have your head around python.

What separates threefive from the rest is actually the HLS handling. The industry is separated into two groups...
(1) the broadcasters who deal with SCTE-35 as solely a data track. They operate in a TS domain. They are typically responsible for putting the markers in the stream with expensive kit and want an independent tool to verify that they conform to SCTE-35 best practice. They are typically originators and engineers. I expect many of those users rely on TSDuck's rmsplice, or TSDuck's newer splicemonitor plugin.
(2) The digital-OTT folks who are doing dynamic ad insertion using something like Uplynk, YoSpace, Brightcove Unicorn, GAM/DFP or MediaTailor. They operate in the HLS & manifest domain, generally using one of the non-SCTE-35 ad marker formats CUE-OUT/IN, but also need to understand what is happening in those mystical base-64 SCTE-35 messages. They are consumers of SCTE-35.

It is that second group where threefive really excels. Your toolset is the first single open source tool that acknowledges that SCTE is used in HLS. A use case for a standalone binary would be:

• Feed a master playlist (full URL, not locally hosted) into threefive
• Print a decoded, JSON output of base-64 SCTE markers in the data track
• Print a decoded, JSON output of base-64 SCTE markers in some of the more common vendor specific SCTE markers in the manifest (eg EXT-OATCLS-SCTE35)
• Print any interpreted non-SCTE ad markers in the various non-standard, vendor specific formats
• Install via the 'nice' CLI package managers, ie homebrew / apt / chocolatey

All of the above can be done today with scripting, piping and a combination of threefive / FFmpeg / TS Duck. It is just not very accessible to the non-pythonic user, but it works! And I am a very grateful user.

The trouble is more to do with python, not with your tool. An example is that the user sees threefive on github... Wow, that looks amazing.

• Ok, now I wanna play with it.
• Oh crap, I gotta install python on my mac.
• Hang on, python comes with my mac. Cool, Python 2.7.
• But I should be using Python3. Ok, learn how to install that with homebrew. Ok, now I have 2 pythons and I gotta play with my environment variables in my bash profiles
• Ok, now pip. This looks neat. What's the difference between pip and pip3? A few more hours go by. (pip is super neat, btw)
• Where does pip install stuff? I'm not running as admin, oh crap, where has pip put all the stuff I wanted. Some weird error messages keep appearing about upgrading pip, even through I've just upgraded pip. Oh well.
• Ok, I just got me head around pip and python... But I'm told I should be using pypy
• Oh no, I'm on an M1 mac and pypy is not there just yet. Back to python3.
• Right, now how do I invoke a python command?

I exaggerate for the sake of it, because your tl;dr of "Up and Running in Less Than 7 Seconds" was actually the most useful snippet I have ever seen in a python project because it provides (a) how to install (b) a hosted sample and (c) a reference output. Big kudos! Most of the above moans are not threefive, but are rather the daunting nature of the python-pip-pypy ecosystem.

I pressed on through, because I really like your stuff, but I suspect there is a similar demographic of non-pythonic users who kinda just give up. But there is the Uplynk, YoSpace, Brightcove Unicorn, GAM/DFP or MediaTailor userbase who just want to validate or monitor ad insertion and a command-line tool may suit them.

As a developer of what is fundamentally a library, the user's scripting / python learning curve is not your problem, and you are doing all this stuff for free - so please don't interpret this as anything more than a very satisfied and grateful user explaining their journey.

threefive is a library of functions that enables an enlightened developer to do pretty much anything in the SCTE-35 domain. But there is another less enlightened user group who are more bash/tool/binary focused. Some projects separate this into the library (for devs), eg https://github.com/x42/libltc and the tool (for users), eg https://github.com/x42/ltc-tools. Or libav (the library, not the confusing fork) vs FFmpeg the tool.

Does that help?

[futzu]

Yes it does. I'm going to do a stand alone binary with threefive/go. It is stupid fast.

[futzu]

Ive been thinking about the monitoring thing, expect something soon.

[futzu]

Finally, I'm intrigued by where you got https://futzu.com/xaa.ts from - and don't feel like you have to answer. That particular TS stream was created about 2007-2009 in a proof of concept experiment with Channel 4 in the UK to do realtime replacement of ads for a student-focused IPTV service. I recognize the metadata in the stream. I never thought I would see that TS stream over a decade later.

I never gave you an answer, I think I found that video on a ffmpeg feature request or bug report or something.

cli tools

You got me thinking about cli tools, I am about to bundle some cli tools together.

• a threefive cli
• a keyframe detector cli
• x9k3 hls segmenter cli
• a hls m3u8 and segment parser cli
• gumd , a multicast sender daemon cli
• re_cc, a continuity counter restamping cli

All of them use the reader function from threefive. Meaning they all support files, http(s), multicast, and udp URIs,
they also support reading from stdin, so you can pipe stuff in.

[bbgdzxng1]

As a separate comment to the above, which is what I checked back here to share...another kind developer has proposed a patch to FFmpeg for SCTE-35 passthrough. I thought it was relevant to the original topic about 0x86 stream type in MPEG-TS muxer and wanted to share with you http://ffmpeg.org/pipermail/ffmpeg-devel/2021-November/thread.html#287613. Who knows whether the patch will be accepted, but I can be pretty sure that threefive would be the test case for validating that FFmpeg patch. And if it isn't used as the test case, it should be!

On inspection, the proposed patch of the FFmpeg maining list may only work with one of -copyts, -mpegts_copyts 1 or -vsync passthrough is used, and may not not work if one of the timestamp manipulation techniques such as -vsync cfr|vfr|drop, -avoid_negative_ts -start_at_zero, -use_wallclock_as_timestamps, -output_ts_offset, setpts= filter or -bsf:v setts= filter is used. It would also be limited to MPEG-TS output. So the patch may get some pushback due to interaction of other FFmpeg internal functions. But that should all come out of testing, ideally using your awesome threefive.

There have been many SCTE-35 patches proposed to FFmpeg over the years, but none-of them have made it to stable, but maybe this is the start of something.

As ever, thank-you!

[futzu]

I get the feeling the ffmpeg guys don't care much for SCTE35... I don't really know.
I need to check that out. Thank you.

You might like this example.
https://github.com/futzu/scte35-threefive/blob/master/examples/stream/cue2vtt.py

[bbgdzxng1]

Thanks futzu - I saw that cue2vtt.py a few weeks ago - and it looks very useful for debugging. I have not played with it fully - but I really like the way you are thinking. I don't trust whether any of the webVTT players are frame accurate or not, but even semi-accurate visualization is very helpful for quick and dirty. So thanks!

Most of my world is ad markers in a segmented HLS stream... and the one feature that FFmpeg, MVP and libav are lacking on is the playback of segmented webVTT within HLS. But I remux the HLS chunks into a single TS with TSDuck, run your cue2vtt and play the webVTT alongside the remuxed TS file, much like you do with mplayer, which pick ups the vtt subs because it has a common filename to the media.

I'm surprised (and relieved) that noone ever published a formal standard for SCTE-35 into a generic webVTT metadata track. We have webVTT captions, webVTT chapters, webVTT generic metadata but fortunately no standards-body has defined a method for communicating SCTE-35 via webVTT between systems. I say relieved because there more than enough ways of passing ad-insertion cues between systems already. Using webVTT for system>human visualization is very nice.

I was trying to work out a method of writing the (decoded) SCTE-35 from either threefive or TS Duck's splice_monitor to a named pipe/mkfifo, feed the named pipe to FFmpeg's metadata filter and then use FFmpeg's drawtext filter to overlay the frame metadata containing the decoded SCTE message over the video, but it all relied on pipes and writes (because ultimately, I'm just a monkey with a typewriter rather than a programmer).

At present, I use FFmpeg's drawtext to overlay the PTS on the video, and compare the PTS in the threefive decode. It is a manual process and non-realtime, but good enough for a bear of little brain.

Your vtt method is a useful non-realtime visualizer and a lot more elegant than my attempt. Thanks.

[futzu]

I am surprised too. I was trying to do the same thing with ffmpeg,and driving myself nuts. Updated the cue2vtt.py last night. Mplayer doesnt recognize the webVTT formating, so everything is aligned center. I am thinking about combining it with mpegts.js project so I can play the video in a webpage with formatted webvtt subtitles.
I may do one in ASS subtitles, partly forthe formatting, but mostly just for the jokes. :)

[bbgdzxng1]

Lots of great stuff coming from the SoftWorks crew on subtitle rendering in FFmpeg's libav. I have been lurking in ffmpeg-devel and there was a huge amount of activity in Nov around subtitle frame handling and conversion. As far as I read it, it amounts to subtitle frame handling and a -filter:s and I think a subtitle API. http://ffmpeg.org/pipermail/ffmpeg-devel/2021-November/thread.html#288221. It looks to be a pretty substantial overhaul, subject to code review. One to watch.

With current stable and a -strict experimental flag you can unlock some of the HLS+webVTT functionality at least to a point that you can see the demuxer acknowledge webVTT in debug, but it is still experimental and non-functional.

$ ffmpeg -strict experimental -protocol_whitelist "http,https,tls,tcp" -format_whitelist "hls,mpegts,webvtt" -allowed_extensions ALL -i https://example.com/master-containing-webvtt.m3u8 -map 0 -f null -

If and when anything hits mainstream FFmpeg, I'll let you know.

[futzu]

Speaking of HLS, check out x9k3.

[futzu]

kabuki can change the stream type back to 0x86 (scte35)

[bbgdzxng1]

Sorry, I did not see your comment. I have been watching kabuki for a month. Very cool, Adrian. Very cool.

And x93k has been progressing nicely. So nice to see that there's now an open-source solution for inserting SCTE messages in streams. I have been watching that too.

Loving cuei, btw.

I hope you are well, sir.

[midicase]

I there a way to hack a fix into ffmpeg for this? I'm digging through the code and having trouble finding where this occurs.
Got this far but lose track:
https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/mpegts.c#L2478

I've read through kabuki but my python is so rusty. I don't really need to make a durable fix but need to run a live remux to test downstream scte35 haling.

Thanks

[futzu]

Ive never really looked at ffmpeg's code but what if I update kabuki so you can pipe a stream through it?
I did that last week for SuperKabuki and it works well.
Then you could do

{ ffmpeg command goes here} -  | kabuki -o outfile.ts

Would that solve your problem?

[futzu]

I updated kabuki, git pull you some.
version is 0.0.3

a@debian:~$ pypy3 kabuki.py -v
0.0.3
a@debian:~$ 

Here's the input

a@debian:~$ ffprobe pv.ts -hide_banner
[mpegts @ 0x55ddbc4045c0] start time for stream 5 is not set in estimate_timings_from_pts
Input #0, mpegts, from 'pv.ts':
  Duration: 00:00:20.33, start: 1.400000, bitrate: 17696 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 59.94 fps, 59.94 tbr, 90k tbn
  Stream #0:1[0x101]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:2[0x102]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:3[0x103]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:4[0x104]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:5[0x105]: Data: bin_data ([6][0][0][0] / 0x0006)
Unsupported codec with id 98314 for input stream 5
a@debian:~$ 

• then I ran this command with the updated kabuki.py

a@debian:~$ cat pv.ts | pypy3 kabuki.py -p 0x105 | ffplay -hide_banner  -

• output

261
pid match
Input #0, mpegts, from 'pipe:':    0KB vq=    0KB sq=    0B f=0/0   
  Duration: N/A, start: 1.400000, bitrate: N/A
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 59.94 fps, 59.94 tbr, 90k tbn
  Stream #0:1[0x101]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:2[0x102]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:3[0x103]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:4[0x104]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:5[0x105]: Data: scte_35
  17.30 A-V: -0.019 fd=   0 aq=   32KB vq= 2498KB sq=    0B f=0/0   

• to write to a file

a@debian:~$ cat pv.ts | pypy3 kabuki2.py -p 0x105 -o kout.ts
261
pid match

ffprobe -hide_banner kout.ts
Input #0, mpegts, from 'kout.ts':
  Duration: 00:00:20.24, start: 1.400000, bitrate: 17650 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 59.94 fps, 59.94 tbr, 90k tbn
  Stream #0:1[0x101]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:2[0x102]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:3[0x103]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:4[0x104]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:5[0x105]: Data: scte_35
Unsupported codec with id 98305 for input stream 5

so you can do something like

 ffmpeg -i  mpegts/pcrvid.ts -map 0 -c copy -f mpegts - | pypy3 kabuki.py -p 0x105 | ffplay  -

[midicase]

Fighting the install of new_reader on arch.

python ~/bin/kabuki.py -v
0.0.3
pypy3 ~/bin/kabuki.py -v
Traceback (most recent call last):
  File "/home/mikeg/bin/kabuki.py", line 7, in <module>
    from new_reader import reader
ModuleNotFoundError: No module named 'new_reader'

Dunno if that has an impact. I have to do a lot of piping (hls source):

ffmpeg  -re -i  http://x.x.x.x:xx/720/funroads720SCTE35.m3u8  -map 0 -c copy -f mpegts - |  python3 ~/bin/kabuki.py -p 0x103 | ffmpeg -i pipe:  -map 0 -c copy -f mpegts udp://127.0.0.1:2000

ffprobe udp://127.0.0.1:2000
Input #0, mpegts, from 'udp://127.0.0.1:2000':
  Duration: N/A, start: 4.683844, bitrate: N/A
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 59.94 fps, 59.94 tbr, 90k tbn
  Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 131 kb/s
  Stream #0:2[0x102]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 7 kb/s
  Stream #0:3[0x103]: Data: bin_data ([6][0][0][0] / 0x0006)

I will revisit try to get this running on Centos7 and/or Debian. Looks promising though.

[midicase]

I need to restream the fixed stream, so I'll try to add unicast output.

[futzu]

Fighting the install of new_reader on arch.

python ~/bin/kabuki.py -v
0.0.3
pypy3 ~/bin/kabuki.py -v
Traceback (most recent call last):
  File "/home/mikeg/bin/kabuki.py", line 7, in <module>
    from new_reader import reader
ModuleNotFoundError: No module named 'new_reader'

Dunno if that has an impact. I have to do a lot of piping (hls source):

ffmpeg  -re -i  http://x.x.x.x:xx/720/funroads720SCTE35.m3u8  -map 0 -c copy -f mpegts - |  python3 ~/bin/kabuki.py -p 0x103 | ffmpeg -i pipe:  -map 0 -c copy -f mpegts udp://127.0.0.1:2000

ffprobe udp://127.0.0.1:2000
Input #0, mpegts, from 'udp://127.0.0.1:2000':
  Duration: N/A, start: 4.683844, bitrate: N/A
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 59.94 fps, 59.94 tbr, 90k tbn
  Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 131 kb/s
  Stream #0:2[0x102]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 7 kb/s
  Stream #0:3[0x103]: Data: bin_data ([6][0][0][0] / 0x0006)

I will revisit try to get this running on Centos7 and/or Debian. Looks promising though.

pip3 install new_reader

Tell me if that doesnt work for you.

[futzu]

https://github.com/futzu/gumd , gumd does multicast, and it's real easy to use.
I just tried it and I was able to read the stream as UDP

cat pv.ts | pypy3 ./kabuki.py -p 0x105 | gumd -a 127.0.0.1:2000

in another terminal I ran

ffplay udp://127.0.0.1:2000

and it worked just fine.

a@debian:~/build/clean/x9k3$ ffplay  udp://127.0.0.1:2000

Input #0, mpegts, from 'udp://127.0.0.1:2000':0KB sq=    0B f=0/0   
  Duration: N/A, start: 1.400000, bitrate: N/A
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 59.94 fps, 59.94 tbr, 90k tbn
  Stream #0:1[0x101]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:2[0x102]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:3[0x103]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:4[0x104]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
  Stream #0:5[0x105]: Data: scte_35

[futzu]

I had to rename kabuki six2scte35 so I could make it a pkg.
https://github.com/futzu/six2scte35

pip3 install six2scte35 gumd

or 
pypy3 -mpip install six2scte35 gumd

then you can do something like

ffmpeg  -re -i  http://x.x.x.x:xx/720/funroads720SCTE35.m3u8  -map 0 -c copy -f mpegts -  | six2scte35 -p 0x105 | gumd -a 127.0.0.1:2000

That's about as concise I can make it.

[futzu]

Good God Man, don't leave me in suspense :)

[midicase]

Worked like charm.

ffmpeg -re -i http://x.x.x.x:xx/720/funroads720SCTE35.m3u8 -map 0 -c copy -f mpegts -pkt_size 1316 -flush_packets 0 - | six2scte35 -p 0x103 | gumd -a x.x.x.x:3001

Probe gave me:

Input #0, mpegts, from 'udp://x.x.x.x:3001':
  Duration: N/A, start: 4.676511, bitrate: N/A
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 59.94 fps, 59.94 tbr, 90k tbn
  Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 131 kb/s
  Stream #0:2[0x102]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 7 kb/s
  Stream #0:3[0x103]: Data: scte_35

[futzu]

Cool. I didn't know gumd could do that until I tried it yesterday. gumd also does 1316 packet size by default.

I am glad you got it working,
Adrian

[futzu]

Hey I got a message about binding to a specific interface, I was thinking that maybe an issue and I should make it so you can specify the interface, I will try update it tonight, but it will probably be tomorrow.

[futzu]

@midicase

Some changes I think you'll dig.

pypy3  gums.py -i ~/mpegts/udp.livetv.ts  -b 192.168.12.160  -u -a 192.168.12.160:4444

	Stream: udp://192.168.12.160:4444
	Source: 192.168.12.160:43594

• I dropped the port for bind address
• The Source ip and port are values directly from the socket self.sock.getsockname()
• UDP sockets are now non-blocking
• I changed the name to gums.
• bin/gumd and bin/gums scripts are both available and both use gums.

usage: gums [-h] [-i INPUT] [-a ADDR] [-u] [-b BIND_ADDR] [-t TTL] [-v]

optional arguments:

  -h, --help            show this help message and exit
  
  -i INPUT, --input INPUT
                        like "/home/a/vid.ts" or "udp://@235.35.3.5:3535" or "https://futzu.com/xaa.ts"
                        
  -a ADDR, --addr ADDR  Destination IP:Port like "227.1.3.10:4310"
    
  -b BIND_ADDR, --bind_addr BIND_ADDR
                        Local IP to bind to like "192.168.1.34". Default is 0.0.0.0
                        
  -t TTL, --ttl TTL     Multicast TTL 1 - 255
 
  -u, --unicast         Use Unicast instead of Multicast 

  -v, --version         Show version

[midicase]

In C we just test if multicast address, no need for extra arg:
if ((inet_addr(address) & 0x000000F0) == 0x000000E0)
Python might have to range compare. Less stuff for users to figure out, the easier.

If you want more optional features to add:
Source-specific multicast (SSM) receive. https://en.wikipedia.org/wiki/Source-specific_multicast
RTP handling (sending is easy). https://en.wikipedia.org/wiki/Real-time_Transport_Protocol

SSM we use for backup systems. Some broadcast plants send redundant streams from multiple sources and the client can switch between them as needed (usually for redundancy purposes).

RTP is just a re-ordering protocol. It's most edffective in SMPTE 2022-7. Dual identical RTP streams, client reads both and if any ordered packet is missing, is grabbed from the other RTP stream. RTP requires buffering which can introduce unwanted jitter.

[big-tuna]

What I have been trying to sort out is adding gums to six2scte35, it kind of make sense to be able to send  the output over a network. It works, but I have noticed an occasional video artifact with high bit rate streams. Testing for multicast does make sense, I didn't think of that. I had to drop the nonblocking, My sockets kept getting stuck open. SSM I don't know. I have been wanting to add srtp to new_reader though.

…

On Mar 15, 2023, 3:48 PM, at 3:48 PM, midicase ***@***.***> wrote: In C we just test if multicast address, no need for extra arg: `if ((inet_addr(address) & 0x000000F0) == 0x000000E0) ` Python might have to range compare. Less stuff for users to figure out, the easier. If you want more optional features to add: Source-specific multicast (SSM) receive. https://en.wikipedia.org/wiki/Source-specific_multicast RTP handling (sending is easy). https://en.wikipedia.org/wiki/Real-time_Transport_Protocol SSM we use for backup systems. Some broadcast plants send redundant streams from multiple sources and the client can switch between them as needed (usually for redundancy purposes). RTP is just a re-ordering protocol. It's most edffective in SMPTE 2022-7. Dual identical RTP streams, client reads both and if any ordered packet is missing, is grabbed from the other RTP stream. RTP requires buffering which can introduce unwanted jitter. -- Reply to this email directly or view it on GitHub: #33 (comment) You are receiving this because you are subscribed to this thread. Message ID: ***@***.***>

[umaygank]

Hi futzu, i had use this ..
ffmpeg -re -i http://x.x.x.x:xx/720/funroads720SCTE35.m3u8 -map 0 -c copy -f mpegts - | six2scte35 -p 0x105 | gumd -a 127.0.0.1:2000
its running well, but no PCR detected ..
Can you help me for this issue..

[futzu]

Sorry, I'm just seeing this. Which program is saying no PCR detected?
Cut and paste the errors for me.