[iOS] Active Sessions Break on Video Playback - 10.10

[JPKribs]

Describe the bug

It looks like something changed in 10.10 that causes the /sessions endpoint not to work in the manner we're currently expecting.

Application version

1.3(2) & Main

Where did you install the app from?

TestFlight

Device information

iPhone 11, Emulator

OS version

iOS 18

Jellyfin server version

10.10.0

[JPKribs]

@LePips it might make sense to disable active sessions for 1.3? I think this is an SDK update needed but I could be wrong. If so, I think we can disable this view until we're ready to update the SDK? Or, wrap the view to hide it if the server is 10.10?

[LePips]

We will just update the SDK for 1.3. 1.3 will be a bigger update with my video player changes and I will want a tvOS TestFlight alongside it as well.

[JPKribs]

We will just update the SDK for 1.3

Just to confirm, all the way to 10.10? Since, for this specific issue, it looks like it's 10.10 API related. I only ask because I know for now we're on 10.8 so I don't want to add any pressure to accelerate this quicker than needed. IMO, active sessions are nice to have but far from core functionality.

Should we also add a message/warning to a 1.2.X build that the minimum server version will be changing in 1.3?

[sjorge]

Weird, I had this work yesterday on 10.10.0?

Let me try again.

[sjorge]

That is with the latest testflight build, maybe something changed in the recent changes that have not made it into testflight yet?

[JPKribs]

Hm. I'm getting the error on both TestFlight and Main on both existing server and brand new server with consistency

[sjorge]

Interesting, I think I figured it out.
While playing audio it's fine, but if i play a video it breaks with the same message! And it remains broken until I stop all video playback and play an audio file and relaunch the app.

[sjorge]

Edit 3, iOS keeps uploading the wrong picture

More info! It seems when transcoding is involved it breaks!

Anything direct play is fine, i vaguely remember there being some changes wrt the transcode reasons the server reports.

I wonder if it's a deserialization issue.

[LePips]

You should be able to check the decoding error in the logs in Settings for the endpoint.

[sjorge]

Interesting, when it's working the call to the sessions endpoint returns valid JSON, when it fails, the json isn't parsable by jq either. It might be a 10.10.0 bug?

jq: parse error: Invalid numeric literal at line 118, column 24

Which is this line:

"Name": ""The Deep Breath Before the Plunge"",

Edit:

This makes no sense, the same endpoint + params is called by the webclient, and firefox does get valid json back 🤷‍♂️

[sjorge@ulysses ~]$ jq . < /tmp/x | grep Before
          "Name": "\"The Deep Breath Before the Plunge\"",
            "Name": "\"The Deep Breath Before the Plunge\"",

Is there some escaping done before the json is parsed in swiftfin, that might be doing something wrong with escaped quotes? That still doesn't explain why it sometimes works and sometimes doesn't though.

[LePips]

This is probably a server bug, there shouldn't be quotes in there.

[JPKribs]

This appears to be resolved on 10.10.1. Can you validate @sjorge its working the same for you?

[sjorge]

This appears to be resolved on 10.10.1. Can you validate @sjorge its working the same for you?

On 10.10.0 I had about a 75% ish success rate of it just working, only watched 1 thing on 10.10.1 and that was working too.

[sjorge]

Unfortunately still broken on 10.10.1, it's just very tempermental.

This is the response copied from the log when I just tripped the error earlier today, but playing different episode on the same client after it was fine again 🤷

[
{
"AdditionalUsers": [],
"ApplicationVersion": "0.17.9",
"Capabilities": {
"PlayableMediaTypes": ["Video", ",", "Audio"],
"SupportedCommands": ["DisplayContent", ",", "SetSubtitleStreamIndex", ",", "SetAudioStreamIndex", ",", "DisplayMessage", ",", "SendString", ",", "VolumeUp", ",", "VolumeDown", ",", "SetVolume", ",", "Mute", ",", "Unmute", ",", "ToggleMute"],
"SupportsMediaControl": true,
"SupportsPersistentIdentifier": true
},
"Client": "Android TV",
"DeviceId": "<snip>",
"DeviceName": "Zircon",
"HasCustomDeviceName": false,
"Id": "<snip>",
"IsActive": true,
"LastActivityDate": "2024-11-04T20:57:32.6383889Z",
"LastPlaybackCheckIn": "2024-11-04T20:57:29.6403074Z",
"NowPlayingItem": {
"BackdropImageTags": [],
"ChannelId": null,
"Chapters": [],
"Container": "mkv",
"DateCreated": "2023-10-01T08:09:58.5403239Z",
"EnableMediaSourceDisplay": true,
"ExternalUrls": [
{
"Name": "AniDB",
"Url": ["https://anidb.net/episode/269769"](https://anidb.net/episode/269769)
}
],
"GenreItems": [],
"Genres": [],
"HasSubtitles": true,
"Height": 720,
"Id": "<snip>",
"ImageBlurHashes": {
"Backdrop": {
"<snip>": "HmF=jqIVNHM{f+V@R+ozWB~WNGRjWCWBV@kCogRj"
},
"Primary": {
"<snip>": "dICP9b=|a0$*~C-BnOxaVtrsjFozv~rsspxuZ%aKxa%2",
"<snip>": "WRHVCz0JRRnnD%Im%M?bIBD*xuIn9G%Mt6WTfkxut8IU%LxuIVs="
}
},
"ImageTags": {
"Primary": "<snip>"
},
"IndexNumber": 5,
"IsFolder": false,
"IsHD": true,
"LocalTrailerCount": 0,
"LocationType": "FileSystem",
"MediaStreams": [
{
"AspectRatio": "16:9",
"AudioSpatialFormat": "None",
"AverageFrameRate": 23.976025,
"BitDepth": 8,
"BitRate": 970251,
"Codec": "h264",
"DisplayTitle": "720p H264 SDR",
"Height": 720,
"Index": 0,
"IsAVC": true,
"IsAnamorphic": false,
"IsDefault": true,
"IsExternal": false,
"IsForced": false,
"IsHearingImpaired": false,
"IsInterlaced": false,
"IsTextSubtitleStream": false,
"Language": "jpn",
"Level": 41,
"NalLengthSize": "4",
"PixelFormat": "yuv420p",
"Profile": "High",
"RealFrameRate": 23.976025,
"RefFrames": 1,
"ReferenceFrameRate": 23.976025,
"SupportsExternalStream": false,
"TimeBase": "1/1000",
"Type": "Video",
"VideoRange": "SDR",
"VideoRangeType": "SDR",
"Width": 1280
},
{
"AudioSpatialFormat": "None",
"BitRate": 157384,
"ChannelLayout": "stereo",
"Channels": 2,
"Codec": "aac",
"DisplayTitle": "Japanese - AAC - Stereo - Default",
"Index": 1,
"IsAVC": false,
"IsDefault": true,
"IsExternal": false,
"IsForced": false,
"IsHearingImpaired": false,
"IsInterlaced": false,
"IsTextSubtitleStream": false,
"Language": "jpn",
"Level": 0,
"LocalizedDefault": "Default",
"LocalizedExternal": "External",
"Profile": "LC",
"SampleRate": 48000,
"SupportsExternalStream": false,
"TimeBase": "1/1000",
"Type": "Audio",
"VideoRange": "Unknown",
"VideoRangeType": "Unknown"
},
{
"AudioSpatialFormat": "None",
"Codec": "ass",
"DisplayTitle": "English - Default - ASS",
"Height": 0,
"Index": 2,
"IsAVC": false,
"IsDefault": true,
"IsExternal": false,
"IsForced": false,
"IsHearingImpaired": false,
"IsInterlaced": false,
"IsTextSubtitleStream": true,
"Language": "eng",
"Level": 0,
"LocalizedDefault": "Default",
"LocalizedExternal": "External",
"LocalizedForced": "Forced",
"LocalizedHearingImpaired": "Hearing Impaired",
"LocalizedUndefined": "Undefined",
"SupportsExternalStream": true,
"TimeBase": "1/1000",
"Title": "English",
"Type": "Subtitle",
"VideoRange": "Unknown",
"VideoRangeType": "Unknown",
"Width": 0
}
],
"MediaType": "Video",
"Name": "File: Glint",
"OriginalTitle": "File《氷刃》のモニカ",
"ParentBackdropImageTags": ["<snip>"],
"ParentBackdropItemId": "<snip>",
"ParentId": "<snip>",
"ParentIndexNumber": 1,
"Path": "/mm/anime/S/Spy Kyoushitsu (2023)/Spy Kyoushitsu (2023) - 05 - File Glint (C85BA78E).mkv",
"PremiereDate": "2023-08-10T00:00:00.0000000Z",
"PrimaryImageAspectRatio": 1.777777777777778,
"ProductionYear": 2023,
"ProviderIds": {
"AniBD": "269769",
"AniDB": "269769"
},
"RunTimeTicks": 14235010000,
"SeasonId": "<snip>",
"SeasonName": "Season 1",
"SeriesId": "<snip>",
"SeriesName": "Spy Kyoushitsu (2023)",
"SeriesPrimaryImageTag": "<snip>",
"SeriesStudio": "KADOKAWA",
"ServerId": "<snip>",
"SpecialFeatureCount": 0,
"Studios": [],
"Taglines": [],
"Trickplay": {
},
"Type": "Episode",
"VideoType": "VideoFile",
"Width": 1280
},
"NowPlayingQueue": [],
"NowPlayingQueueFullItems": [],
"PlayState": {
"AudioStreamIndex": 1,
"CanSeek": true,
"IsMuted": false,
"IsPaused": false,
"MediaSourceId": "<snip>",
"PlayMethod": "Transcode",
"PlaybackOrder": "Default",
"PositionTicks": 299610000,
"RepeatMode": "RepeatNone",
"SubtitleStreamIndex": 2
},
"PlayableMediaTypes": ["Video", "Audio"],
"RemoteEndPoint": "<snip>",
"ServerId": "<snip>",
"SupportedCommands": ["DisplayContent", "SetSubtitleStreamIndex", "SetAudioStreamIndex", "DisplayMessage", "SendString", "VolumeUp", "VolumeDown", "SetVolume", "Mute", "Unmute", "ToggleMute"],
"SupportsMediaControl": true,
"SupportsRemoteControl": true,
"TranscodingInfo": {
"AudioChannels": 2,
"AudioCodec": "aac",
"Bitrate": 5107256,
"CompletionPercentage": 50.73196295612016,
"Container": "ts",
"Framerate": 564,
"HardwareAccelerationType": "qsv",
"Height": 720,
"IsAudioDirect": true,
"IsVideoDirect": false,
"TranscodeReasons": ["SubtitleCodecNotSupported"],
"VideoCodec": "hevc",
"Width": 1280
},
"UserId": "<snip>",
"UserName": "sjorge",
"UserPrimaryImageTag": "<snip>"
}
]

I redacted the uuids just in cae but otherwise I made no changes, it is again 'invalid' json but I wonder if part of it is because of the log -> response body seems to try to do some 'encoding' of it's own. Like the link looked 'OK' but it was clickable, after hitting the share button it seems to have become markdown? I don't think this is a reliable way of getting the response JSON. Aside from maybe MITM'ing on the reverse proxy but give it sometimes breaks/unbreaks on the same playback session it would be hard to capture.

[sjorge]

After a lot more looking at broken and non broken requests, I think I've figured out the actualy problem. The weird quotes in the log were because the log view on iOS renders them weird... the actualt issues seems to be this;

    "Capabilities": {
      "PlayableMediaTypes": [
        "Audio",
        ",",
        "Video"
      ],
      "SupportedCommands": [
        "MoveUp",
        ",",
        "MoveDown",

Parts of the things under caps now have extra "," entries. (I captured this on the reverse proxy side so nothing could have messed with the content.

Probably: jellyfin/jellyfin#12948

[JPKribs]

Probably: jellyfin/jellyfin#12948

That looks very promising! Thank you for all your help troubleshooting this.

My only question is why is this inconsistent? Does this only happen if there are multiple capabilities reported back from the server? That is, if there's only a single capability or no capabilities, there's no comma and therefore no issue?

[sjorge]

Probably: jellyfin/jellyfin#12948

That looks very promising! Thank you for all your help troubleshooting this.

My only question is why is this inconsistent? Does this only happen if there are multiple capabilities reported back from the server? That is, if there's only a single capability or no capabilities, there's no comma and therefore no issue?

I think it depends on what clients are 'active' not all seem to return SupportedCommands for example.
So far a session from the web or ios expo client seems to trigger it, but one from infuse does not.

Edit:
I think it also applies to other structures, like chapter and media segment info. As I have seen it with infuse a few times, but I mostly watch anime which has neither segments or chapters and it's not an issue.

[sjorge]

jellyfin/jellyfin#12949 made it into todays release... 🤞

[JPKribs]

Unfortunately, I am still seeing this post 10.10.2

[LePips]

Is it the same issue with the decoding? If so, I would recommend finding that log in Settings/Logs and filing a bug on the server.

[JPKribs]

Is it the same issue with the decoding? If so, I would recommend finding that log in Settings/Logs and filing a bug on the server.

I'd be happy to file a bug but I'm having trouble understanding what's even failing to decode. I have this narrowed down to when a client starts having a TranscodingInfo is when this begins having decoding errors. So, transcoding, remuxing, or anything Infuse seems to cause this issue. I did a comparison between a valid decoding log and an invalid decoding log and this was the only difference (Aside from timestamps):

    "TranscodingInfo": {
      "AudioChannels": 2,
      "AudioCodec": "aac",
      "Bitrate": 7348577,
      "CompletionPercentage": 0,
      "Container": "mp4",
      "Framerate": 0,
      "HardwareAccelerationType": "qsv",
      "Height": 1080,
      "IsAudioDirect": false,
      "IsVideoDirect": false,
      "TranscodeReasons": ["ContainerNotSupported", "VideoCodecNotSupported", "AudioCodecNotSupported"],
      "VideoCodec": "h264",
      "Width": 1920
    },

All of these should be accounted for in the SDK so I'm not sure what the bug report would even be. I see two PRs that relate to 'TranscodingInfo' that could have been added to Jellyfin Server in 10.10 that could be our culprit:

jellyfin/jellyfin#12587
jellyfin/jellyfin#12561

But I'm not sure if adding a Server-Side DTO creates issues on our end for decoding?

For the new SessionInfoDTO, the difference I can see is that the SDK has fullNowPlayingItem which was removed in 10.10. Server has a LastPausedDate field that isn't in the SDK. Neither or these are used in this feature nor are they used in the TranscodingInfo. I could be wrong that there are other culprits but this was my 100% recreation as to when this fails to decode.

Sorry I'm not more helpful. If you think this is enough info to go from I can make a bug report but I'm just not even sure what I'm looking at that I can point to as a 'fix this' type request.