Option to Auto-resize Emoji Height to Text Height

[vencabot]

Is your feature request related to a problem? Please describe.
When using emojis alongside text, the text and emoji are misaligned due to emoji being larger or smaller than text (based on the image uploaded).

Describe the solution you'd like
It would be nice if, at In My Library -> Chatrooms -> Channel emojis, there was a checkbox for 'resize emojis to chatroom text height.'

Describe alternatives you've considered
I may try to implement something like this myself using the CSS injection already provided by PeerTube @ Administration -> Configuration -> Advanced -> Customizations -> CSS .

Additional context

• I think that it's nice to have the option for images to maintain the size of the uploaded file; it could create lots of silly effects! Maybe the 'resize emoji to chatroom text height' checkbox could even be per-emoji so that some emoji will always scale to the text height and others could be very gigantic.

• I notice that when sending 🎵 as a message without any accompanying text, it is very large -- but when text accompanies it, it's fit to the text size. But that's not true for custom emojis. I wonder why that is? It's a pretty cool feature, although not strictly related to this feature request outside of it having to do with dynamic emoji scaling.

• Thank you so much for this awesome plug-in! I started playing with custom emojis tonight and it was SO easy. The UI is very good; better than Twitch's UI to do the same thing. The chat has been working solidly for weeks, now, and my viewers enjoy using it very much.

[JohnXLivingston]

Hi,

• I think that it's nice to have the option for images to maintain the size of the uploaded file; it could create lots of silly effects! Maybe the 'resize emoji to chatroom text height' checkbox could even be per-emoji so that some emoji will always scale to the text height and others could be very gigantic.

I have been thinking a lot about this issue when i was implementing custom emojis, and did not come with a good solution.

I take a look on twitch, and i saw some streamers using emojis that were bigger than text.

Reducing the emoji by CSS is not optimal (because the filesize will not be optimal).
Reducing the emoji file could lead to resizing artefacts.

Adding options emoji per emoji to resize or not can be complicated for the user to understand.

My conclusion was: streamers should be free to resize the emoji before uploading, but i will not try anything "smart". I just use the file as it is.

Maybe add some buttons to resize the file just after the upload (and maybe cut borders)... But i'm not 100% sure of what to do.

• I notice that when sending 🎵 as a message without any accompanying text, it is very large -- but when text accompanies it, it's fit to the text size. But that's not true for custom emojis. I wonder why that is? It's a pretty cool feature, although not strictly related to this feature request outside of it having to do with dynamic emoji scaling.

This is a special feature in ConverseJS. It only work with standard emoji. But you are right, it could be used for custom emoji, if they are designed to scale.

• Thank you so much for this awesome plug-in! I started playing with custom emojis tonight and it was SO easy. The UI is very good; better than Twitch's UI to do the same thing. The chat has been working solidly for weeks, now, and my viewers enjoy using it very much.

Thanks! Great to hear!
I never saw the Twitch UI, so glad to hear that i have done better :)

[vencabot]

Here's my opinion:

I take a look on twitch, and i saw some streamers using emojis that were bigger than text.

There is no feature on Twitch to use a different size of emoji. All emoji are slightly larger than text, which I think is a good idea, aesthetically. Basically, emoji take up nearly the entire line-height, centered to each line -- and they don't make the line they're on any bigger, which means that they don't cause alignment issues in multi-line text. Maybe what you're thinking of is that some emoji take up that entire size and others appear smaller due to transparent borders in the image.

Reducing the emoji by CSS is not optimal (because the filesize will not be optimal).

I don't expect that most users will be concerned about optimizing the file size of some small graphics. If they are, they can upload smaller or more compressed images. I think putting this burden on those users makes more sense than putting it on all users, who will likely want for emoji to look neat when inline with text.

Reducing the emoji file could lead to resizing artefacts.

Browsers use very good interpolation when resizing images. Artifacts will not be noticeable unless the uploaded image was very large, and if the user is unhappy (which very few would be), they can resize the image themselves and re-upload it. I think putting this burden on those users makes more sense than putting it on all users.

Adding options per emoji to resize or not can be complicated for the user to understand.

I can see how a checkbox for each emoji could make the UI a little more cluttered. However, I'm confident that the majority of users would rather have their emojis appear at a standard size that doesn't cause alignment issues when in-line with text. I think that including a checkbox would be nice for the edge-case of users who truly want emoji of varying sizes (so an 'advanced' option). If the option isn't presented at all for UI clarity reasons, I still think that the default behavior should be to soft-resize the graphics.

My conclusion was: streamers should be free to resize the emoji before uploading, but i will not try anything "smart". I just use the file as it is.

Users will be able to resize their emoji either-way, allowing them to limit file size and scaling artifacting if they want. With the current implementation, they are not only free to resize emoji but are forced to; even if they're using images designed to be emoji, and even if those images are small (for instance, taken from another site, like Twitch, Mastodon, etc.) they'll need to resize them very carefully to not create alignment issues with peertube-plugin-livechat.

Maybe add some buttons to resize the file just after the upload (and maybe cut borders)... But i'm not 100% sure of what to do.

I think that having an option to cut borders is a bit overkill and it wouldn't be my recommendation. Here's how Twitch handles it:

Twitch offers users two options: to upload each emoji three times, at three different sizes (28x28, 56x56, and 112x112), or to upload one emoji and have Twitch automatically scale it to those three different sizes.

Depending on the application, Twitch will use one of those three and use CSS to scale it to the appropriate size. It never uses them at their 1:1 scaling and, as of recent updates, almost never uses the 28x28 copy; it almost always uses the 56x56 copy and just scales it as needed. The 112x112 copy is used when clicking on the image, to show it in higher detail.

I also think that including hard-resizing options into peertube-plugin-livechat would be overkill; I think the way that it works right now is very simple and is probably how it should be. I also don't think that the plugin should use multiple versions of an emoji the way that Twitch does (not).

I think that the lesson to take is that Twitch ('s new UI, for the past couple of years) allows a user to upload an emoji of any size they want and Twitch scales it for them.

Twitch hard-scales the image first, to save on both storage and transmission size, and then it soft-scales with CSS on pages and in chat so that it always fits inline with text without causing issues with line-height. I don't think that the average PeerTube server is going to be concerned with the storage / transmission size of a few emoji; after all, each PeerTube server stores hundreds or thousands of user avatars (due to federation), which are about the same size as a 'large' emoji upload.

My take is that PeerTube should take 1 image upload per emoji (as it does now) and soft-scale it with CSS to fit the line-height of the chat text. If users find that the scaling looks bad, they can delete and re-upload a manually hard-scaled version. If they find that large emoji are negatively impacting storage or transmission, they can delete and re-upload a hard-scaled and/or recompressed version.

I think it's worth mentioning that every instant messenger I've used, phone texting, Discord, etc., uses emoji that are scaled (probably like Twitch, a mix of hard and soft scaling) to fit the line-height of text so that a line with emoji is the same height as lines without emoji. Even with hard-scaling, soft-scaling is still necessary because it would be trivial for a designer to decide, with some large or small UI update, to change font size or line-height; I think it's unrealistic to expect users to manually resize all of their emoji to perfectly fit alongside text each time changes to font are made or face alignment issues.

I never saw the Twitch UI, so glad to hear that i have done better :)

Without going into too much detail, the Twitch UI revolves around pop-up modals that act sort of as a little workshop for each emoji, showing what it looks like at different sizes, where you can decide on the title of it, all in the center of the screen -- and then you can confirm and close the modal to get back to the normal page flow. I really appreciate how your UI gives me access to every single emoji at the same time and is absolutely no more complicated than it needs to be: just rows of 'title' and 'the image for that title'. This is really all that's needed and is how Twitch should be, instead of treating each emoji like a big deal.

tl;dr I don't think that the UI for managing emoji should be changed; I just think that some CSS should be included in the chatbox to scale emoji so that they fit alongside text the way that emojis do in other apps. If any change to the UI is to be made, maybe it could just be a single checkbox in the plug-in settings for:

'Display chat emoji at full resolution.'

That would toggle the behavior that's being used now.

Sorry about the wall of text! Thank you for everything and for putting so much thought into this issue.

Side-note: I notice that a lot of Issues have an 'Awaiting Sponsor' tag. Is there some documentation I can read about what is expected of sponsors / how to become such a sponsor? Like everyone, my resources are limited but of course I believe very strongly that your work should be compensated and I think this 'Awaiting Sponsor' tag is a good idea, but it would be good to know how to engage with it.

[JohnXLivingston]

There is no feature on Twitch to use a different size of emoji.

Maybe it was with one of the browser extension that some streamers are using, to have more emojis than twitch allows.

I don't think that the average PeerTube server is going to be concerned with the storage / transmission size of a few emoji; after all, each PeerTube server stores hundreds or thousands of user avatars (due to federation), which are about the same size as a 'large' emoji upload.

Thinks are not so simple... We made some stress test 2 years ago, and we had surprising results.
Live streams have some unusual side effects. For example, you can have hundred (or thousands) of user that are joining in the same minute. When an emoji is posted in the chat, you can have hundreds/thousands of users that are downloading it at the exact same moment. And so on.

1000 users that downloads a 20kb image in the same second requires 20Mbps of bandwidth!!
Imagine that 100 of 1000 viewers are opening the emoji panel at the same time (saying, just after the live starts). If they are 50 emojis of ~10kb: 100 * 50 * 10kb = 50Mb to transmit!
And you have the video stream at the same time.

And for now, these emojis are not just static files on a server. To download them, you go through a backend endpoint, where Peertube must check for every single request that the plugin exists, and then send files.
This can require a lot of CPU, and a lot of file handlers (for each opened web request).

So I have to be very careful how I implement this kind of features.

Sorry about the wall of text! Thank you for everything and for putting so much thought into this issue.

Thanks for your concerns. I broadly agree with what you say. I will add this on the roadmap.

Side-note: I notice that a lot of Issues have an 'Awaiting Sponsor' tag. Is there some documentation I can read about what is expected of sponsors / how to become such a sponsor? Like everyone, my resources are limited but of course I believe very strongly that your work should be compensated and I think this 'Awaiting Sponsor' tag is a good idea, but it would be good to know how to engage with it.

Maybe I should write some documentation. For now, there is only one sentence here ("If you want to support the project financially, you can contact me by mail at …"), and the list of previous sponsors here

The "awaiting sponsor" tags has 2 purpose:

• it helps me write financing applications when i'm searching for big sponsorships (~once a year)
• it is here so that potential sponsor can spontaneously propose a sponsorship

For individuals, donation are welcome (on liberapay for example). This helps cover unforeseen last-minute work, that were not part of current grants. And there are a lot of such work, as it can take months between signing a sponsorship contract for some tasks, and the time when i can work on it.
And individual donations are good for morale :) (by the way, thank you ;) )

[vencabot]

Thinks are not so simple... We made some stress test 2 years ago, and we had surprising results.
Live streams have some unusual side effects. For example, you can have hundred (or thousands) of user that are joining in the same minute. When an emoji is posted in the chat, you can have hundreds/thousands of users that are downloading it at the exact same moment. And so on.

I'm glad that PeerTube has been doing stress tests -- that's very important! This sounds like premature optimization:

• Very few (any?) PeerTube servers have that many views on a single live-stream. On Twitch, it is extremely uncommon to have triple-digits viewers, let alone quadruple digit viewers (the absolute largest channels on the platform get this), let alone hundreds joining at the same time.
• If individual users are concerned about bandwidth, they can upload smaller files. I disagree with the implication that emoji size should be a strict limit of the plugin, itself. I think this will make life harder for many more servers / streamers that it'll make life easier for those with hundreds of viewers of their livestream (who also don't want to accidentally upload a large emoji). I've already had to manually resize / re-compress emoji to fit under the plug-ins emoji size limitations to make them smaller than they are on Twitch (where I downloaded them from). I'm certainly not concerned about my bandwidth / CPU being maxed when a hundred viewers join my stream at once -- and if I was, I would happily upload smaller emoji without being forced to.
• Maybe a simple warning would suffice at the top of the emoji management area: "Be considerate of the file-size of your emoji. They need to be downloaded by each viewer joining the stream."

This is definitely the last I have to say about this topic for now! My apologies for all of the unrequested feedback; I've been enjoying this plug-in so much and have a lot of thoughts about it, but your investment is much more than mine and I appreciate what you've done. Thank you for explaining the sponsor system and I'll be contributing more, and I wish I could be of even more help.

If you would like to have anything in particular tested, etc., I hope you'll be happy contacting me. Good luck with your roadmap!

[JohnXLivingston]

When Peertube launched the live feature, some streamers with a lot a followers have tested it.
It was not ready to scale, and they experienced some crash. Most of them have gave up Peertube.
So, we really need to make sure nothing crashes again. Otherwise, Peertube will always be a niche project.

When the streamer is also an admin, he can understand that it is in they interest to keep little file size to preserve bandwidth.
But most of streamers are not admin, and have no technical knowledge. They just want it to work. Otherwise, they will return to twitch, or any GAFAM-like product.
So, i have to set some limits.

If the file size limit is annoying, maybe i should try to resize the image directly in the emoji form, without even saying it to the user. What do you think about that?

If you would like to have anything in particular tested, etc., I hope you'll be happy contacting me. Good luck with your roadmap!

Thanks! I will.

[vencabot]

If the file size limit is annoying, maybe i should try to resize the image directly in the emoji form, without even saying it to the user. What do you think about that?

My opinion is that this would be a decent alternative, although I don't see why, for the purposes of transparency, it also couldn't be mentioned (in brief) to the user. For instance, this is what Twitch does. Like I mentioned above, it gives you the choice between uploading the image at 3 fixed resolutions, yourself, or uploading it at any resolution and having Twitch scale it to those three resolutions automatically. It isn't shy about making it clear to the user that it wants graphics at particular resolutions, and that it can scale to those resolutions for you. For instance:

"PeerTube Livechat emojis are saved at 56x56 pixels. Larger uploads will be scaled before storage." Text like that could appear just above the emoji-upload table.

If it isn't mentioned at all, some users might be confused when they want to download their emoji from PeerTube and it isn't as high a resolution as when they uploaded it.