Rotating devices (Smartphones)

[morbificagent]

Hi together,

at the moment i am trying to get this scenario to work:

Smartphone -> OvenPlayer as WebRTC Input -> OME Server -> LLHLS Output

It works great except of one important scenario:
If the user starts streaming on the smartphone the resolution/aspect ratio (portrait) is used for the conversion in OME.
But now the user thinks "oh rotationg the device is a good idea".
Result:

If transcoding in ome is used, the aspect ratio stayes on portrait even if the input is now landscape, so the picture is distorted.
If transcoding is off and bypass is used the aspect ratio is changing properly... but some playing devices looking the LLHLS stream arent fine with this and the player crashes/stops playing on this change.

So i think the best solution would be that ome is converting the input stream to the target resolution (1280x720 for example) while putting black bars to it when the aspect ratio of the input dont match (like it does at the moment) but keeping this dynamic for acpect ratio changes from the input...

Im not sure if there is any solution at the moment that i can reach with configchanges...?

Would be great if someone has an idea.

[bchah]

I misread the question earlier - are you using OvenLiveKit for WebRTC transmission? And what kind of smartphone / OS? I have not seen this issue on iPhone, for instance - it properly pads the aspect ratio when the device is rotated.

[bchah]

And on your input are you using MediaConstraints to set the input resolution / aspect ratio accordingly? This would be required when you call getUserMedia. Here is some sample code from my implementation (it's part of a class so not syntactically perfect in this context but you get the idea):

// Get constraints

 let supportedConstraints = navigator.mediaDevices.getSupportedConstraints();

            constraints = {
                audio: {
                    deviceId: selectedDevices.audioDeviceId
                },
                video: {
                    deviceId: selectedDevices.videoDeviceId
                }
            }

  // Set 720p, for example
      constraints.video.width = { ideal: 1280, max: 1280, min: 1280, exact: 1280 };
      constraints.video.height = { ideal: 720, max: 720, min: 720, exact: 720 };

            if (supportedConstraints.frameRate) {
                constraints.video.frameRate = '30';
            }

            if (supportedConstraints.autoGainControl) {
                constraints.audio.autoGainControl = false;
            }

            if (supportedConstraints.noiseSuppression) {
                constraints.audio.noiseSuppression = false;
            }

            if (supportedConstraints.sampleRate) {
                constraints.audio.sampleRate = 48000;
                constraints.audio.sampleSize = 4;
            }

            if (supportedConstraints.channelCount) {
                constraints.audio.channelCount = 2;
            }

And then for initiating the MediaStream:

OvenLiveKit.getUserMedia(constraints).then((userMedia) => {

                let videoTrack = userMedia.getVideoTracks()[0];
                let audioTrack = userMedia.getAudioTracks()[0];
                const streamSendUrl = "wss://yourwebrtcinputurl";
                const connectionConfig = {
                   preferredVideoFormat: "H264",
                    iceTransportPolicy: "relay",
                     maxVideoBitrate: 2000
               }
        OvenLiveKit.startStreaming(streamSendUrl, connectionConfig);
});

[morbificagent]

I misread the question earlier - are you using OvenLiveKit for WebRTC transmission? And what kind of smartphone / OS? I have not seen this issue on iPhone, for instance - it properly pads the aspect ratio when the device is rotated.

Hi bchah,
many thanks for you reply.
Yes i am using OvenLiveKit for WebRTC transmission (sorry i wrote Ovenplayer).
And the test-inputdevice was an Android.

I have tried an iPad a few minutes ago and in my config the behavior is the same.
I have tested more, this time with a screen capture where i resize a window (same effect):

Smartphone -> OvenLiveKit as WebRTC Input -> OME Server (bypass)
= Result ok after source-rotation on Android as player (video.js/LLHLS)
= Result problematic after source-rotation on iOS as player (video.js/LLHLS) -> Stream crashes
= Result ok after source-rotation on Windows as player (video.js/LLHLS)
= Result ok after source-rotation on Windows as player (VLC/LLHLS)
= Result ok after source-rotation on all devices (OvenPlayer/WebRTC)

.
.
.
.
Smartphone -> OvenLiveKit as WebRTC Input -> OME Server (transcoding) -> LLHLS Output
= Result distorted after source-rotation on all devices (video.js/LLHLS)
= Result distorted after source-rotation on all devices (OvenPlayer/WebRTC)

So even the result looks a little bit mixed when using bypass in this "Source-Rotation-Scenario", the result is always distorted when using transcoding in OME.

And because of the MediaConstraints...
I am using the Demo-Code from here (https://github.com/AirenSoft/OvenPlayer/blob/master/demo/demo_input.html) at the moment.
Im not sure it it makes any difference for this rotation/dynamic padding problem...