Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract MP4 metadata-based chapters #1851

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

MiSikora
Copy link
Contributor

@MiSikora MiSikora commented Nov 1, 2024

This PR adds extraction of metadata-based MP4 chapters. It is a follow-up to the issue in the old ExoPlayer repository: google/ExoPlayer#2316. I used the suggestion from this comment.

I think you're correct that MP4 files don't use ID3 tags, however the approach we've taken for MP4 metadata has been to convert it into the equivalent ID3 tags as far as is possible. It usually is possible since the metadata atoms in MP4 are in most cases equivalent to ID3 tags. See extractor.mp4.MetadataUtil. For chapter metadata, my guess is that the chpl atom is used, as defined in this spec. It looks like such an atom could probably be converted into a ChapterTOCFrame and N ChapterFrames. Each ChapterFrame would have a single TIT2 sub-frame holding the title.

In the current solution, I only create ChapterFrames and do not use ChapterTOCFrame, but I'm open to adding it if necessary.

Specification

It is quite difficult to find reliable documentation around MP4 chapters. The best document describing them I found is this one. I attached the relevant part of the documentation at the bottom of this PR.

In summary, there are two types of chapters: QuickTime chapters, which are multiplexed into the data stream itself, and Nero chapters, which use a classic metadata approach. This PR handles only these chapters. To the best of my knowledge, most of M4A editors embed both types of chapters in the media files.

Test Data

I created the test sample in the following way:

  1. I copied the pre-existing sample_empty_track.mp4.
  2. I executed ffmpeg -i sample_empty_track.mp4 -f ffmetadata sample-metadata.txt.
  3. I appended the sample-metadata.txt file with these chapters:
[CHAPTER]
TIMEBASE=1/1000
START=0
END=200
title=Chapter 1

[CHAPTER]
TIMEBASE=1/1000
START=201
END= 400
title=Chapter 2

[CHAPTER]
TIMEBASE=1/1000
START=800
END=850
title=Chapter 3
  1. I executed ffmpeg -i sample_empty_track.mp4 -i sample-metadata.txt -map_metadata 1 -codec copy sample_with_chapters.mp4 and used the sample_with_chapters.mp4 file in tests.

MP4 metadata-based chapter specification

"Nero" chapters are an alternative to Quicktime chapters implemented by Nero software suite. It aims at providing a simpler, metadata-based chapter description akin to Vorbis chapters.

They are implemented as a specific atom located at moov.udta.chpl

The contents of the atom is as follows

Type (1) Data Notes
int32 Atom Size As part of standard MP4 atom header
char[4] Atom Name As part of standard MP4 atom header; value is "chpl"
byte Version Atom version
int24 Flags Atom flags (none known so far)
byte Reserved Unknown reserved byte
int32 Chapter count Number of chapters
--- --- --- Following lines are repeated for each chapter
int64 Chapter start time Uses 100-nanosecond base; divide by 10 000 to get milliseconds
byte String data size Size of following string data
string Chapter name Uses UTF-8 encoding; size of binary data is declared on previous field

(1) : Big-Endian convention

To my knowledge, understanding of Nero chapters comes from retro-engineering, as there are no official specifications.

NB1 : Quicktime player, iTunes and the built-in iOS audiobook player support Quicktime chapters only, and ignore Nero chapters entirely.

NB2 : Some players such as VLC seem to fail reading Nero chapters properly when there are more than 255 of them, for instance on (very) long audiobooks. As the Nero structure actually allows for any number of chapters to be written, I'm unsure if this is a bug or a part of the Nero standard I'm unaware of...

String chapterName = chpl.readString(titleLength);
ChapterFrame chapterFrame =
new ChapterFrame(
/* chapterId= */ Integer.toString(i),
Copy link
Contributor Author

@MiSikora MiSikora Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure if something else or perhaps C.INDEX_UNSET should be used as the ID. MP4 chapters do not have this concept.

@MiSikora MiSikora changed the title Extract mp4 metadata-based chapters Extract MP4 metadata-based chapters Nov 1, 2024
colorTransfer = 3
lumaBitdepth = 8
chromaBitdepth = 8
metadata = entries=[TSSE: description=null: values=[Lavf61.1.100], CHAP, CHAP, CHAP, Mp4Timestamp: creation time=0, modification time=0, timescale=1000]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line asserts that the chapters are detected in the file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants