256 color and 24-bit true color support

[jquast]

This backports upstream issue erikrose#67
Some background: https://gist.github.com/XVilka/8346728
See also erikrose#30 (comment)

In brief: if number_of_colors is 256, we can pretty safely assume to support emitting 256 colors, and this is currently possible with term.color(196) or some such. However, as a "formatting name" it is deplorable. It would require using a reference guide to map colors-by-number. I actually spent some time trying to find a mapping and came up pretty short, I guess you would run a script that emits all 256, pick the one you like, and note it down.

As an API, supporting {t.bold_color196} is very much against the "easy to write, easy to read" philosophy that Blessings encourages. What follows is my draft proposal.

rgb method

@property
def rgb(self, red, green, blue)
   """
   Return a callable accepting arguments ``(red, green, blue)``, which emit a sequence approximated to the colorspace of the terminal bound by :attr:`number_of_colors`. The values of each color are 0-255.
   (...)
   """

This would be exactly like the existing color(n), except if given rgb(255, 50, 100) on a terminal where number_of_colors is 16, this would be "coerced" (or downsampled) as though you called rgb(255, 85, 85) which is exactly the expected color in the CGA palette, emitting the exact same sequence of term.bright_red or color(12).

true_colorspace context manager

@contextmanager.contextlib
def true_colorspace():
   """
   Context manager that enables mapping of compound color names and :attr:`rgb` values
   to a True color (24-bit, 16,777,216 colors) colorspace.
   """

This is necessary as a context manager: we cannot determine whether the given terminal emulator supports true colors: There is no xterm-16777216color terminal name, or anything like it. There is no situation that I know of where the terminal capability database for number_of_colors could return 16,777,216. A terminal emulator that supports it provides no method to query whether or not it supports it. For those who know their intended audience/emulator, they may rightly go ahead and use them, or provide configuration to enable it through the use of such context manager.

Compound Formatters as X11 color names

Use the official X11 color names, http://en.wikipedia.org/wiki/X11_color_names as compound formatters. These are mapped by their hexidecimal values to rgb() to emit the appropriate sequence. This allows one to use "deep_pink", which is (255, 20, 147), but on a 16-color terminal it would be mapped to the same sequence as term.bright_red.

[jquast]

https://github.com/magarcia/python-x256

[jquast]

Note of, https://www.w3.org/TR/REC-CSS1/#color-units

The three-digit RGB notation (#rgb) is converted into six-digit form (#rrggbb) by replicating digits, not by adding zeros. For example, #FB0 expands to #FFBB00.

[avylove]

I started working on this. Changes pushed to the color branch.

So far I have dictionaries mapping 8, 16, 88, and 256 color indexes to RGB values. There's also a dictionary mapping x11 color names to RGB values and a color translator, which supports caching and finds the color closest to the given RGB values in the supported range. In Terminal I added truecolor detection based on what I could find. It's likely to be accurate when detecting true color support, but will miss some. The nice part is, the workaround is just to set an environment variable.

Where I hit a wall was in implementing the methods/properties in Terminal. With truecolor support, \x1b[38;2;<r>;<g>;<b>m (foreground) and \x1b[48;2;<r>;<g>;<b>m (background) should provide the correct sequences. But I would think we'd want to downconvert to use color() with an index if truecolor support isn't detected. If it's flexible like that, can it still be a property? The sequence and inputs would be different depending on the situation. Not sure if there's already a way to handle that.

[jquast]

Sorry to be late in responding, I didn't want to respond until I had time to look, I'll be doing that today, I do aim to get this full color support in soon, and this looks very good so far, I'll branch along! Thanks again!

[jquast]

Will drop support for 88-colorspace, to reduce tests and such. This was unique to "rxvt", which is generally abandoned, and no significant forks exists (RIP, my first unicode terminal urxvt, and first programmable tabbed terminal mrxvt). urxvt is moderately active, but jumping on the truecolor bandwagon too.

Interesting that some of these "support" true color, but map to the 256 color space like we will also be doing, for those without COLORTERM defined

[jquast]

Will drop support for superscript and subscript, though documented as a VT100 code, is unimplemented by almost all common emulators, searches find only discussions that generally come down to 4:3 and monospace fonts being incompatible with the idea, and as unicode support was added became proffered, there are online converters ..

ᶦ ʷᵒᵘˡᵈⁿ'ᵗ ˢᵃʸ ᵗʰᵉ ʳᵉˢᵘˡᵗˢ ᵃʳᵉ ᵛᵉʳʸ ᵍᵒᵒᵈ ˡᵒᵒᵏᶦⁿᵍ, ᵗʰᵒᵘᵍʰ.

[jquast]

On caching, I would prefer the context wrapper functools.lru_cache,

but for this library, i prefer not to cache unless necessary, there were caching experiments in the past, fear of costs of dipping into curses, etc., and after writing a few downstream utils, I think its best for downstream to do a 'string builder' pattern -- gather up all the before a display/refresh, and any of those string "parts" can be cached or re-used.

[avylove]

I think memoization is necessary here because finding the closest color can be expensive. You're basically doing a three dimension distance calculation and comparing the results 256 times every time you convert from truecolor to 256 color. Chances are the end user will use a limited palette but use the same colors multiple times.

[jquast]

I understand, I won't fight it hard, I'll be working on demo apps and try functools.lru_cache, and backport it if it seems to help enough, but the potential for memoizing up to 16,777,216 results is too great for a sort of shim library, needs to have a maxsize or be backported. I wouldn't expect libraries like blessed have the kind of memory allocation potential, lru cache provides a limit.

And as a downstream user doing demoscene art stuff, calculating say more than a few dozen different colors per refresh, I would cache those strings anyway as a part of a buildup. From experience with "x/84 bbs" on gen 1 raspberry pi's, we would cache any repeatidly called string, because even a basic Terminal.magic_bold("stuff") has surprisingly large cost on a gen 1 raspberry pi, worth caching!

[avylove]

Yeah, that's a good point. I could see someone running through all the colors. It definitely needs a limit. I think we could limit it to 100 and that would max out around 24KB or so depending on the way lru_cache is implemented. Looks like backports.functools-lru-cache has been around a while and is being maintained.

I do the same thing caching lookups, but I've seen a lot of examples where people are doing formatting lookups over and over again, sometimes within the same string. Sometimes you need to protect people from themselves.

[jquast]

I've added a setter to number_of_colors, so that we can change it at runtime. I don't know, why not keep it? Because of a cache, haha, well temporarily disabled that cache for colors. Anyway i've worked through all the hardest bits, see attached for your downsampling code in works, I'll finish it up and push within the hour. Best wishes.

[jquast]

indigo and hotpink are two examples of colors that don't map to what I think people would expect by RGB distance rather than a weighted HSV.

Right now: hotpink -> magenta and indigo -> blue.
Expected? hotpink -> bright_red and indigo -> magenta.

[avylove]

Work looks great!

The number_of_colors setter could flush the cache.

Do you think we should look at HSV? There's more math there. We could try weighted RGB as described here.

[avylove]

I spent a lot of time today looking into color conversion. I implemented a weighted rgb which is better and CIE94 which is much better. CIEDE2000 is supposed to be even better, but my implementation isn't, so I must have something messed up in it. It might be worth allowing the algorithm to be changed because we may improve our algorithms in the future and there are trade-offs between resources and accuracy.

I was mainly focused on matching 24-bit colors to 256 color using the x11 colors as a sample. I think 8 and 16 color conversions may need to be statically mapped from the 256 color pallet because the numbers are small and the algorithms in small sets like that can have unexpected results.

So if we have decent dynamic mapping from 24-bit to 256 color and then statically map to 8 and 16 color, it should result in mostly expected results.

I'll try to push what I have soon.

[jquast]

looks like python colorsys module goes back to python 2.6, https://docs.python.org/2.6/library/colorsys.html so that's available to us, i know what you mean about 8 and 16-color translations being a bit overkill? I'm sure what you have will be fine.

[jquast]

sorry i touched a bit of the formatters.py, you'll probably find a conflict.

run bin/plasma.py and press tab to cycle colorspaces, the 256 colorspace is very slow compared to all others, the 4/8/16 search is perfectly fast.

[avylove]

That's pretty slick! NIce job!
Do what you need to do. I'll resolve the conflicts before I push.

[avylove]

Pushed what I have so far. Added a script, colorchart.py, which we can improve or drop later. Output is below with three different algorithms for calculating distance between two colors. These are the X11 colors. first block is the 24-color, second is down-converted to 256 color, then 16 color, and finally 8 color. The 16 and 8 are off by a lot, but I'm going to deal with that later. Just looking at the first two blocks right now. You can see there are slight differences. Still would like to get CIE2000 working, as it's considered the most accurate, but I'll have to take another look at it.

RGB (original)

RGB-weighted

CIE94

[jquast]

sounds good, i jotted down two demo utils i wanted earlier in the week, and you pretty much knocked out 1 of 2 -- an "x11 colorpicker", so we're on the same page. thanks!

I don't mind alternate algorithms in the tree that we can select the default for, the fastest I guess, and that downstream folks can derive and modify, selecting others in the tree if they like. That's how I'll try to make the API.

[avylove]

That same script has another function that lists the color names next to colors. Some colors are duplicates like grey# and gray#, so it lists all the names next to the color. If you uncomment the call in __main__ you can see what I mean. I'm sure there are nicer ways to display things, maybe columns?

I'd also like to sort the colors better, but sorting is even harder than matching. So right now they are sorted alphabetically with some additional natural sort logic so light/dark/medium color number are sorted with the same color. I think that logic could be improved, maybe with color aliases, like snow and ivory for white. Another idea I had was to group by name, again snow/ivory/white, then do an rgb or hsv sort on each group.

[avylove]

Oh, and as far as an x11 picker. Once Blessed has hyperlink support, we could make the colors clickable and have them target each color like https://www.colorhexa.com/fff8dc or https://www.google.com/search?q=%23FFF8DC

[jquast]

pushed the api i'm thinking of, and bin/x11_colorpicker.py. also pushed comments into rgb_downconvert:

Though pre-computing all 1 << 24 options is memory-intensive, a pre-computed
"k-d tree" of 256 (x,y,z) vectors of a colorspace in 3 dimensions, such as a
cone of HSV, or simply 255x255x255 RGB square, any given rgb value is just a
nearest-neighbor search of 256 points, which k-d should be much faster by
sub-dividing / culling search points, rather than our "search all 256 points
always" approach.

anyway, its good enough for a release, I'm ready to merge it if you are.

I hope that future blessed release doesn't need any color distance algorithms or selections, none of these solutions are particularly fast, as much as I appreciate this effort, I'd much rather find a way to remove all this code and complexity and be sure not to document or test or maintain it.

[jquast]

I'm going to start merging into master, we won't release to pypi for a few days yet so don't sweat lack of docs, I'd like to do the 2.0 breaking API changes, like move(y, x) -> move(x, y) and so on with this 24-bit color support, I don't wish to juggle branches !

[avylove]

Ok, sounds good! I have code for CIE2000 working, but it needs some cleanup. I can wait until you merge into master to push it. We also need to add some caching for rgb_downconvert. I think plasma illustrates the need for that well. Even with the simplest algorithm I get 1 fps with 256 color vs 30 fps in 24-color.

The color picker is awesome!

I noticed plasma always shows "paused" unless it's showing "please wait".

If you're going for 2.0, then #94 should be done at the same time too. I'm not sure what effect it will have on existing code, so best to knock it out when breaking things anyway.

[jquast]

I decided to go with move_xy and keep the original move, I guess we don't have to go 2.0 by semver, then.

[avylove]

I did some investigation into caching. The only place I could find where caching provided a significant speedup for all use cases was with a 256 item lru cache for rgb_to_lab(). It helps here because the CEI algorithms convert RGB to CEI-Lab for both colors given. One of those colors is the source color, the other is one of the colors in the 256 color palette. The 256 color palette is looked up for every color and the source color is looked up 256 times in a row. Since 0 and 16 are both (0,0,0), that makes 256 the perfect size to hold the 256 color palette and the current source color.

While caching in other places could have benefit for some use cases, most of those use cases could also be addressed by saving the result to a variable downstream like you suggested.

[jquast]

did you ever notice the open pull request for approximate_rgb()? This uses a very fast method, that is equivalent to saying, #fff is a 16-bit version of #f0f0f0 or some such, and any rgb value is trimmed to 16-bit, boxed into the 256 color pallette left, after some adjustments, I just wanted some tests and explanations there, I bet it will perform excellently. erikrose#30

[avylove]

I hadn't seen that. We could add it in and see how it compares. I'm not sure I understand it enough to document it. These seems to assume the 256 color pallette is a summary of a larger color space, but I'm not sure that's true. We also haven't implemented weighted HSV like you suggested. Doing simple HSV is easy, but I'm not sure what the weighting would need to be.

It might be worth pulling the conversion algorithms out into their own package. They probably have more utility independently and will get eyes on them from color people rather than just terminal application people. There are some packages that already do this, colour-science, colorio, and colormath, but they tend to do a lot more and have heavy requirements like numpy, scipy, and/or matplotlib.