Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unicode glyphs ruin the TUI in my terminal #420

Open
strk opened this issue Nov 25, 2023 · 7 comments
Open

Unicode glyphs ruin the TUI in my terminal #420

strk opened this issue Nov 25, 2023 · 7 comments

Comments

@strk
Copy link
Contributor

strk commented Nov 25, 2023

I'm not sure this is a problem with toot itself or with my terminal, but I'm experiencing it with toot so I'm filing it here for now: presence of unicode glyphs seem to confuse toot as to how large is the text. See screenshot:

image

This is with version v0.28.0 as packaged on Ubuntu 22.04.3 LTS

@danschwarz
Copy link
Collaborator

danschwarz commented Nov 28, 2023

This is a miserable problem. The issue is that different terminal emulators support different versions of the Unicode standard. Let's say your emulator supports Unicode 7.0. It knows the width of characters defined up to the release date of 7.0. But now we're up to Unicode 16. Any characters defined after Unicode 7.0 (lots of emojis!) your terminal will not know the width of, so it'll probably default to single-cell width. But maybe the emoji you are using is double-cell-width.

Whereas the toot app tries to be smart and uses a library (wcwidth) to find the correct cell widths for all characters. If toot and your terminal disagree, you'll see the kind of alignment errors you've posted above.

There's also ucs-detect that attempts to provide a way to detect your terminal's unicode support level. But even if we do that, I'm not sure we can provide a complete solution for using the latest Unicode emoji.

The TUI library we use (urwid) suffers from the same problem, people have tried to integrate wcwidth support in the past, but no patches have been accepted AFAIK.

Both of wcwdith and ucs-detect libraries were dormant since 2020, but have recently revived and have new releases published. I'm interested to see what they can do, but I'm not hopeful they'll provide a solution for us.

In the meantime, you can minimize the problem by using a terminal with support for a recent version of Unicode. Windows Terminal is pretty good, if you're on Windows.

@strk
Copy link
Contributor Author

strk commented Nov 29, 2023

Thanks for the in-depth explanation. Using gnome-terminal (3.44.0) does indeed fix the problem (on Ubuntu-22.04). The terminal on which I have the problem is qterminal-0.17.0

@davidbrochart
Copy link

I think that moving to Textual will solve this issue.

@danschwarz
Copy link
Collaborator

@davidbrochart after a chat with the wcwidth author I'm pretty well convinced that the only real solution to the problem is to pressure terminal emulator authors to upgrade their support to the latest Unicode version. Textual, by itself, has no solution to this problem AFAIK.

@davidbrochart
Copy link

I just tried tooi, and it seems to solve some issues. Here is toot followed by tooi (see on the right of line "It's Migrations week"):

image

image

@ihabunek
Copy link
Owner

It does not solve all the problems, e.g. here's a trans flag in foone's handle causing a black char on the next line.

foone

@danschwarz
Copy link
Collaborator

My guess is Textual renders line boxes differently, in a way that is unconnected to rendering the enclosed contents - using absolute positioning of the cursor when drawing the box sides.

It may be possible to simulate this in Urwid by obtaining the screen dimensions and using a fixed size widget of appropriate size, but I'm not sure of that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants