Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the tables smaller #12

Open
kadiwa4 opened this issue Mar 4, 2024 · 1 comment
Open

Make the tables smaller #12

kadiwa4 opened this issue Mar 4, 2024 · 1 comment
Labels
breaking change good first issue Good for newcomers help wanted Extra attention is needed
Milestone

Comments

@kadiwa4
Copy link

kadiwa4 commented Mar 4, 2024

All the characters in code_table.rs are in the Basic Multilingual Plane, i.e. they fit in a u16 (and I don't think that this will ever change in the future).

The [char; 128] arrays could be replaced with u16 arrays to cut their size in half. Currently, all the decoding tables are 512 bytes in size which is quite large.

If you replace the [Option<char>; 128] arrays with [Option<u16>; 128] arrays however, that wouldn't save any space. Maybe an encoding with a special sentinel value of 0xFFFF could be chosen here to make these arrays smaller too.

I know that all of this would make the tables harder to use. Maybe the tables themselves should be hidden inside public functions that do the mapping. Or you define a new type like CompleteDecodeTable([u16; 128]); and give people accessor methods.

The encoding tables can probably be made smaller too (and could be hidden inside public functions) but I didn't look into that.

@tats-u
Copy link
Owner

tats-u commented Jun 6, 2024

I've exposed too much of the tables, so it'll be a breaking change to change the internal structure of the tables.
We probably have to wrap the raw tables into structures.
Also, additional type conversions (2 bytes u16 ↔ 4 bytes char) will be newly required. We have to perform microbench to assure the changes won't make the code too slower.

@tats-u tats-u added this to the v4 milestone Jun 6, 2024
@tats-u tats-u added breaking change good first issue Good for newcomers help wanted Extra attention is needed labels Jun 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants