What is better for motion estimation in hybrid codec?

[cheptil]

Hello, Roman

I investigate the possibility of integrate some motion estimation approach into learnable codec and have three ways:

1. Optical flow with confidence interval on flow vectors so one can remove noisy vectors
2. ME received from ME only Nvenc mode, but it is not clear, do them optimal in rate-distortion meaning
3. MV from compressed video file H.264, as I understand, it is real decoded MV

But each of these three ways has its own disadvantages:

1. Optical flow is EPE/MSE optimized, which is not directly related with rate-distortion optimization
2. ME received from ME only Nvenc mode are optimized for unknown criterion
3. MV from compressed video file H.264 honest optimized in rate-distortion meaning, but AVC is too old now

How do you think, which approach is most promising for rate-distortion optimization to integrate into learnable codec?

[RomanArzumanyan]

Hi @cheptil

I must mention that I'm not a real researcher and have almost zero experience in ML applications to video coding.
So I can only share my past experience with you which covers mostly conventional coding standards.

As you know, there are 2 motion estimation approaches:

• Real ME which aims on finding the best match between objects (minimize the SAD difference).
• RD-optimized ME which aims on minimizing rate penalty when compressing motion compensation residue.

Later often leads to non-trivial approaches like motion vector derivation which manages to save on rate by giving away some MV precision.

So IMO you're facing a problem which requires you to do not an apple-to-apple comparison.
One thing I cat tell you for is you can always go from real ME to RD-optimized ME but I'm not sure if it's possible to go backwards.

However recently I came across couple very interesting data compression articles which may be helpful to you (that's more of a gut feeling of mine - maybe it's possible to do the optical flow MV RD optimization in latent space):

• Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, Nick Johnston VARIATIONAL IMAGE COMPRESSION WITH A SCALE HYPERPRIOR, ICLR 2018
• D. Minnen, J. Ballé, G. Toderici. “Joint autoregressive and hierarchical priors for learned image compression. ” Advances in Neural Information Processing Systems, 2018

Or just read any relevant Johannes Ballé papers, they are top quality.

[RomanArzumanyan]

Hi @cheptil

does the classic codec make full pass after motion estimation stage for calculate rate-distortion cost for MV?

Encoder doesn't have to follow any predefined algorithm as long as it produces conformant output (which is regulated by coding standard).

So it depends.

Particular encoder implementation has a liberty to do RD optimization the way it likes. If you'd like to see how it's done "in real life" I recommend you to look at projects like livbpx and libx265 to see the actual behavior of codecs like VP9, AV1 and HEVC.

P. S.
Basically, knowing how to avoid full exhaustive search while maintaining minimal loss to reference (full exhaustive search) implementation is the bread and butter of any real-world encoder implementation. All sorts of heuristics and ML applications are user to achieve that tradeoff.

[cheptil]

Hello, @RomanArzumanyan

I am diving into topic of ME RDO and based on my current understanding there are:

1. ME RDO where R is mean bits cost for coding ME and do not include residual
2. ME RDO where R is mean bits cost for full residual coding pass and include residual

As how vvenc maintainer answered me:
"A full residual coding pass is only done for a single optimal MV candidate"
fraunhoferhhi/vvenc#344

So based on your experience, does it exists at least one codec with ME RDO of second type (full pass) or there are no such examples due to complexity?

[RomanArzumanyan]

Hi @cheptil

does it exists at least one codec with ME RDO of second type (full pass) or there are no such examples due to complexity?

Honest answer to this question is that I don't know.
Real-world implementations don't do that and that's the only material I've worked with.

P. S.
I've seen your profile and it looks like you're Russian.
If that's true - I know couple researchers who do ML video coding at Huawei Moscow office.
I can ask them to share some thoughts on this topic as I'm clearly running out of depth.
Can't promise anything of course.

If you're interested in that - please send me an email, you can find my address in GitHub profile.

[cheptil]

The thing is that I am one of the researchers who do ML video coding at Huawei Moscow office...

[RomanArzumanyan]

Alright, the snake has bitten it's own tail.
The next thing to do is to clone reference encoder (libvpx has much cleaner code base, I recommend that) and lurk through the code base.