[Feature Request] Allow Non-ASCII characters in identifier

[constfold]

While LuaJIT can handle Non-ASCII characters correctly(http://luajit.org/extensions.html), Prometheus was only accept ASCII characters. It is inconvenient to work with many domain specific scripts that may contain a lot Russian/Chinese identifiers.

For example, this script is totally valid in LuaJIT but unable to be obfuscated by Prometheus:

Привет = 1
print(Привет)

Error while parsing Non-ASCII characters:

PROMETHEUS: Applying Obfuscation Pipeline to test.lua ...
PROMETHEUS: Parsing ...
PROMETHEUS: Lexing Error at Position 15:26, Unexpected char "\178"!
luajit: ../Prometheus/src\logger.lua:54: Lexing Error at Position 15:26, Unexpected char "\178"! 
stack traceback:
        [C]: in function 'error'
        ../Prometheus/src\logger.lua:54: in function 'errorCallback'
        ../Prometheus/src\logger.lua:57: in function 'error'
        ../Prometheus/src\prometheus\tokenizer.lua:512: in function 'next'
        ../Prometheus/src\prometheus\tokenizer.lua:518: in function 'scanAll'
        ../Prometheus/src\prometheus\parser.lua:143: in function 'parse'
        ../Prometheus/src\prometheus\pipeline.lua:178: in function 'apply'
        ../Prometheus/src\cli.lua:132: in main chunk
        [C]: in function 'require'
        ../Prometheus/cli.lua:12: in main chunk
        [C]: at 0x005f31a0

Probably related to:
https://github.com/levno-710/Prometheus/blob/e639587547077bab61047712720d88722d0c5fbd/src/prometheus/enums.lua#L33

[pnlmon]

What we could do instead is to convert non-ascii variable name to an ascii one, then convert all of them back in the output

[Brohammer5]

I think this could be a good addition. However this doesn't currently support LuaJIT but Lua51 and some of Roblox's LuaU.

[constfold]

this doesn't currently support LuaJIT but Lua51

AFAIK, there are not much differences between LuaJIT(without lua52 lang extensions) and Lua51 in syntax.

And, furthermore, in fact the lua51 language specification has never defined what exactly the identifiers could be as it describes identifiers as:
Names (also called identifiers) in Lua can be any string of letters, digits, and underscores, not beginning with a digit and not being a reserved word. See https://stackoverflow.com/questions/55671742/why-is-the-following-piece-of-lua-code-completely-valid.

Actually, identifiers already could be unicode in *NIX in UTF-8 locale without any modification to lua, accroding to UnicodeIdentifers in lua-users.