lib: introduce types.hexStr #301039

Janik-Haag · 2024-04-02T20:12:39Z

Description of changes

This change introduces lib.types.hex, I thought that it's a common enough numer that it's worth the change.
I decided against adding it to other lib functions like lib.types.number, so the interface keeps stable.

not quite sure if there is anything else that needs to be done before this can get merged, or if there are any tests for lib.types I should be aware of.

I also tested if everything is working with the small repl example below

nix-repl> pkgs = import ./. { }

nix-repl> lib = pkgs.lib                                                                                                                                                                                            
                                                                                                                                                                                                                    
nix-repl> (lib.evalModules { modules = [ { options = { test = lib.mkOption { default = "a"; type = lib.types.hex; }; }; config = { }; } ]; }).config                                                                
{                                                                                                                                                                                                                   
  test = "a";                                                                                                                                                                                                       
}                                                                                                                                                                                                                   
                                                                                                                                                                                                                    
nix-repl> (lib.evalModules { modules = [ { options = { test = lib.mkOption { default = ""; type = lib.types.hex; }; }; config = { }; } ]; }).config                                                                 
{                                                                                                                                                                                                                   
  test = «error: A definition for option `test' is not of type `hexadecimal number'. Definition values:                                                                                                             
- In `<unknown-file>': ""»;                                                                                                                                                                                         
}

Things done

Add a 👍 reaction to pull requests you find important.

infinisil · 2024-04-02T20:15:38Z

Please also add docs to https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/development/option-types.section.md and tests to https://github.com/NixOS/nixpkgs/blob/master/lib/tests/modules.sh

Janik-Haag · 2024-04-02T20:40:31Z

@infinisil I added tests and updated the docs, as requested.

drupol

LGTM

roberth

This looks quite useful, but I don't think it's a number.

lib/types.nix

roberth · 2024-04-04T07:12:09Z

nixos/doc/manual/development/option-types.section.md

@@ -160,6 +160,10 @@ merging is handled.
 may result in [precision loss](https://github.com/NixOS/nix/issues/5733).
 :::

+`types.hex`
+
+: A hexadecimal number.


It seems that there's a lot more to say, and maybe move it into the section with string-like types?

Added more info, let me know what you think :D

This change introduces lib.types.hex, I thought that it's a common enough numer that it's worth the change. I decided against adding it to other lib functions like lib.types.number, so the interface keeps stable.

roberth

I think we can be a bit more clear about what's represented.
Half bytes seem like a mistake we should catch.
Sorry to drag this out, but it's important for types to have a clear meaning and to be precise about it.
Types are of such a nature that all the details will be depended on rather soon, and not just here in the nixpkgs repo. I'm not trying to be obsessive for no good reason :)

roberth · 2024-04-14T10:10:02Z

nixos/doc/manual/development/option-types.section.md

+
+: A hexadecimal value represented as string. The type only allows lowercase
+ values. The type also does not allow for hex prefixes like `0x` or `#`.
+


Suggested change

Leading zeroes are not stripped.

roberth · 2024-04-14T10:25:11Z

lib/types.nix

@@ -373,6 +373,14 @@ rec {
 merge = mergeEqualOption;
 };

+ hexStr = mkOptionType {
+ name = "hexStr";
+ description = "hexadecimal value represented as string";


I think the hexadecimal is still just a representation of something, presumably bytes.

Suggested change

description = "hexadecimal value represented as string";

description = "bytes represented as a string of hexadecimals";

That reminds me, shouldn't we check that it's an even number of hex digits?
Half bytes would be rare. The only case I can think of is short 12-bit hex color codes of three hex digits, but that should just be a distinct type (and probably a type that also allows 24-bit color codes, and a #).

Suggested change

description = "hexadecimal value represented as string";

description = "string of hexadecimal bytes";

Maybe that's a bit vague. For a more syntax oriented description:

Suggested change

description = "hexadecimal value represented as string";

description = "string of hexadecimal pairs";

Or if I'm wrong about half bytes:

Suggested change

description = "hexadecimal value represented as string";

description = "string of hexadecimal digits";

Suggested change

description = "hexadecimal value represented as string";

description = "string of hexadecimals";

In line with #301039 (comment):

Suggested change

description = "hexadecimal value represented as string";

description = "string of hexadecimal digits";

roberth · 2024-04-14T10:25:56Z

nixos/doc/manual/development/option-types.section.md

@@ -202,6 +202,12 @@ merging is handled.

 : A string. Multiple definitions are concatenated with a colon `":"`.

+`types.hexStr`
+
+: A hexadecimal value represented as string. The type only allows lowercase


Suggested change

: A hexadecimal value represented as string. The type only allows lowercase

: A sequence of bytes represented as a string of hexadecimals. The type only allows lowercase

Half bytes seem like a mistake we should catch.

Uh, I think we might be talking past each other. I this is a string of hex (as in characters representing hex numbers) and not a byte sequence of hex data. So you don't really have half bytes 👀

Yes, I did make somewhat of a leap, and I'm sorry if my presentation was messy. I'm assuming half bytes are not ok, and I think the type should check that.

Do you have a use case where the string of hexadecimal digits does not represent bytes?

I don't think it would be beneficial to talk about half-bytes here. Just saying hexadecimal sounds good

I agree half bytes are unnecessarily awkward, and I suspect they should not be allowed. If we don't allow them, we can call this a sequence of bytes in hexadecimal; no silly language, and no silly values.
If we do have a use case for odd numbers of hex digits with leading zeroes, then something like "hexadecimal string" sounds good to me.

Hmm, maybe I wasn't communicating well, because I was trying to comment about the implementation, but my suggestions are on lines about documentation.
I like to use file comments because they become threads, but it seems that I need to take into account better where I put those comments.

I'm also only getting it now: You're hinting at wanting a check to prevent against an odd number of hex digits (which would represent half-bytes), which is most likely just a bug. I've personally never encountered any programs asking for an odd number of hex digits (because indeed, hex digits are usually used to represent bytes).

I think it's fair to restrict this type to an even number of hex digits to prevent such mistakes. We can still allow an odd number of digits later if this turns out that there are use cases for an odd number of digits. Because of that though I wouldn't want this to be embedded into the documentation. So I think something like this should be good:

Suggested change

: A hexadecimal value represented as string. The type only allows lowercase

: A string of hexadecimal digits. The type only allows lowercase

infinisil · 2024-04-22T20:54:28Z

lib/types.nix

+ name = "hexStr";
+ description = "hexadecimal value represented as string";
+ descriptionClass = "noun";
+ check = x: str.check x && builtins.match "[0-9a-f]+" x != null;


In line with #301039 (comment), something like this:

Suggested change

check = x: str.check x && builtins.match "[0-9a-f]+" x != null;

check = x:

str.check x

&& builtins.match "[0-9a-f]+" x != null

# Usually hex digits are used to represent bytes.

# A single hex digit only represents half a byte,

# so if we have an odd number, it's likely a mistake.

&& mod (stringLength x) 2 == 0;

Writing this, I'm actually not sure if it's worth doing this, the error message when you get the type wrong is really not very helpful in such a case.

Right, this only outputs a bool.
What if we check it in the merge function instead, like the other, good errors?
Technically that breaks either hexStr x for odd-length definitions, but that is probably ok in practice.

Perhaps the module system could invoke the merge function instead of producing a generic error message? Or falling back to the generic message if the merge evaluates (if ! t.check v then seq merged (throw genericMsg) else merged).
(Just an idea, not a request)

I think the proper solution is to extend .check to allow custom error messages.

To move forward with this PR though, I don't really care about having this check or not. I'd like to leave this decision up to @Janik-Haag.

Janik-Haag · 2024-07-07T13:12:04Z

I quit nixpkgs and thus won't be finishing this.

roberth · 2024-07-07T16:28:48Z

All the best to you Janik, and thank you for all your contributions.

Janik-Haag requested review from infinisil and roberth as code owners April 2, 2024 20:12

github-actions bot added 6.topic: module system About "NixOS" module system internals 6.topic: lib The Nixpkgs function library labels Apr 2, 2024

Janik-Haag force-pushed the lib.types.hex branch from 405916e to 635213f Compare April 2, 2024 20:39

ofborg bot added 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Apr 2, 2024

github-actions bot added 6.topic: nixos 8.has: documentation labels Apr 2, 2024

drupol approved these changes Apr 2, 2024

View reviewed changes

wegank added the 12.approvals: 1 label Apr 3, 2024

roberth reviewed Apr 4, 2024

View reviewed changes

Janik-Haag force-pushed the lib.types.hex branch from 635213f to 7dec928 Compare April 12, 2024 19:18

lib: introduce types.hex

a8eb148

This change introduces lib.types.hex, I thought that it's a common enough numer that it's worth the change. I decided against adding it to other lib functions like lib.types.number, so the interface keeps stable.

Janik-Haag force-pushed the lib.types.hex branch from 7dec928 to a8eb148 Compare April 12, 2024 19:18

Janik-Haag requested a review from roberth April 12, 2024 19:20

wegank removed the 12.approvals: 1 label Apr 12, 2024

roberth reviewed Apr 14, 2024

View reviewed changes

infinisil changed the title ~~lib: introduce types.hex~~ lib: introduce types.hexStr Apr 15, 2024

infinisil reviewed Apr 22, 2024

View reviewed changes

Janik-Haag closed this Jul 7, 2024

Janik-Haag deleted the lib.types.hex branch July 7, 2024 13:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lib: introduce types.hexStr #301039

lib: introduce types.hexStr #301039

Janik-Haag commented Apr 2, 2024

infinisil commented Apr 2, 2024

Janik-Haag commented Apr 2, 2024

drupol left a comment

roberth left a comment

roberth Apr 4, 2024

Janik-Haag Apr 12, 2024

roberth left a comment

roberth Apr 14, 2024

roberth Apr 14, 2024

infinisil Apr 22, 2024

roberth Apr 14, 2024

Janik-Haag Apr 21, 2024

roberth Apr 21, 2024

infinisil Apr 22, 2024

roberth Apr 22, 2024

roberth Apr 22, 2024

infinisil Apr 22, 2024

infinisil Apr 22, 2024

roberth Apr 22, 2024

infinisil Apr 23, 2024

Janik-Haag commented Jul 7, 2024

roberth commented Jul 7, 2024


		: A hexadecimal value represented as string. The type only allows lowercase
		values. The type also does not allow for hex prefixes like `0x` or `#`.

	description = "hexadecimal value represented as string";
	description = "bytes represented as a string of hexadecimals";

	description = "hexadecimal value represented as string";
	description = "string of hexadecimal bytes";

	description = "hexadecimal value represented as string";
	description = "string of hexadecimal pairs";

	description = "hexadecimal value represented as string";
	description = "string of hexadecimal digits";

	description = "hexadecimal value represented as string";
	description = "string of hexadecimals";

	: A hexadecimal value represented as string. The type only allows lowercase
	: A sequence of bytes represented as a string of hexadecimals. The type only allows lowercase

	: A hexadecimal value represented as string. The type only allows lowercase
	: A string of hexadecimal digits. The type only allows lowercase

lib: introduce types.hexStr #301039

lib: introduce types.hexStr #301039

Conversation

Janik-Haag commented Apr 2, 2024

Description of changes

Things done

infinisil commented Apr 2, 2024

Janik-Haag commented Apr 2, 2024

drupol left a comment

Choose a reason for hiding this comment

roberth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

roberth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Janik-Haag commented Jul 7, 2024

roberth commented Jul 7, 2024