Count numerics/strings/paths from VCTP data

[Novgorod]

Hi! This is monumental work in VI file format reverse-engineering!

I'm in the process of making my own G-code tool for unlocking libraries (including lvclass), which I'll make open-source soon. I'm also re-implementing the classic VI unlocker in a cleaner and platform-independent way (e.g. no zlib.dll) and I don't want to rely on bruteforcing the Hash1 salt because of potential rare edge cases with more controls than your search space. I also can't use VI server scripting to count the controls because it has to load the VI and all its dependencies into memory, which is not feasible. So parsing the VCTP block is the only reliable way.

As I understood your code, you assume the CPC2 number (minus 1) is indexing the type definition entry in the VCTP block which is used for the salt (i.e., the connector pane), but I find this very unreliable - most of the time it points to some random entry when interpreted as an index. Do you know what the CPC2 number actually means? Has anyone come across some documentation? If the CPC2 method fails, your code apparently tries using every VCTP entry as salt source until one works.

If I'm not mistaken, the salt counts the numerics+strings+paths on the VI's connector pane, which is of type "function" (or terminal, 0xF0) - so only a function typedef in the VCTP block can ever be the salt source (right?). My question is whether there is a 100% reliable way to tell which of the function entries in the VCTP block is the VI's connector pane and therefore the salt source. I noticed something interesting here: The function typedef contains a list of client indices followed by "function flags" (2 bytes) and "connector pane pattern" (2 bytes) and then a list of client flags. Nobody has documented what the "function flags" stand for so far, but I noticed there seems to be always exactly one function with flags 0x0, which seems to be the VI's correct connector pane, and other functions (subVIs) have always higher bits set. Maybe it has something to do with their sequence in the VI hierarchy. In any case, looking for the function typedef with flags 0x0 appears to reliably identify the VI's connector pane and therefore the salt source. Do you have any more information on that or could you verify it? One exception are polymorphic VIs, which always have the salt of 0x0000... but they contain a polyVI typedef (0xF2) as an identifier.

I'm sure I'm still missing some edge cases and would be grateful for any insights...

[mefistotelis]

I'm afraid I can't remember much about the salt source. I developed the salt computation to the point where the salt could be found for all files from the group I used for verification, in reasonable time. Then I moved to the next thing.

I can't provide you with more info over what I already wrote as the in-code comments. I'm sure I also described CPC2 block there, as I remember explicitly making sure each block is commented.

The LV does use unambiguous algorithm to determine the salt. It's just that noone took time to figure it out from assembly.
At the time, I discussed some of my findings on the LAVA forum; maybe searching for posts there would help you.
Example discussion on this:
https://lavag.org/topic/21233-exe-back-to-buildable-project/page/2/#comment-129622

If you can find a 100% reliable way of getting the salt, that's interesting. Though I'm not sure if I will be ever getting back to working on pylabview and updating it.

platform-independent way (e.g. no zlib.dll)

While zlib is not platform independent, it is supported on many platforms. Using own code in place of well tested libraries in not necessarily a good direction.

[Novgorod]

Thanks for the reply! I'll give it a look. I'll also keep testing my "empirical" method (function flags) more with edge cases..

Using own code in place of well tested libraries in not necessarily a good direction.

Far be it from me to do such things :) - I'm using the built-in inflate function in the Labview executable, which should be cross-platform.

[Novgorod]

For reference, I attached a test VI (version 23.1) with a bunch of terminals of different data type. The CONP value is 1 and the CPC2 value is 8 but the correct connector pane function TD is at index 45 (decimal) in the VCTP block. If you filter out only function TDs, it's the 6th one in the filtered list (i.e. index 5), or if you filter by main type function (0xF) including typedefs, functions and polyVIs, its the 11th in the list (index 10), so I have no idea what the CPC2 value of 8 means here. The comments in your code say it's a function TD containing a list of connector pane TDs, but it's just a single number, which is probably some index but it doesn't seem to be an index into the VCTP TD list...
test.zip

[Novgorod]

Just as a quick follow-up: After some testing I have to conclude that the function/terminal flags value 0x0 does not reliably identify the parent connector pane. Some VIs have multiple function TDs with flags 0x0 and on rare occasions they have none with this flags value (but there is always at least one function TD).

So far I couldn't find a way to identify the parent connector pane TD, therefore I have to test all function TDs in the VCTP block for being the salt source (the correct one tends to be near the end). It's a tiny bit of overhead, but it seems to reliably find the salt every time (I tested it on the entire vi.lib folder) without fallback on bruteforcing. Also, every polymorphic VI I tested has a salt of 000... regardless of its function TDs (even if all of them have more than 0 of the relevant controls as clients), so the VI has to be checked for being polymorphic by looking at the VI type in the LVSR block (type 11 is polymorphic).

Anyway, I've already published my unlocker tool on my Github page because it's reliable now. If someone finds out a method to identify the parent connector pane TD, it would make it perfect ;) ...

[mefistotelis]

That's great. I've seen you took time to properly describe the algorithm:
https://github.com/Novgorod/VI-Unlocker?tab=readme-ov-file#how-it-works

VI type in the LVSR block (type 11 is polymorphic)

In pylabview code, I lack that type. But I have:

class VI_TYPE(enum.Enum):
    """ VI Type from LVSR/LVIN field
    """
[...]
    Polymorph = 4	# subVI that is an instance of a polymorphic VI

Is the type 11 a replacement for older type 4, or is type 4 for an instance and type 11 some kind of base polymorphic vi?

[Novgorod]

I did a bit of research how to identify polymorphic VIs and came across your LVSR block description (VI type is 2 bytes at offset 0x22) and it seems you assumed the values are the same as the VI-server property "VI type" published in the NI documentation. I did some quick test and the LVSR VI type stored in the file is definitely not the same as the VI-server property. Standard VIs are type 2, globals 4, controls 6 and polymorphic 11 (decimal). That's the base polymorphic VI (the menu/table GUI where you add members), while members of a poly-VI have standard type (2) but contain the poly-VI TD in their VCTP block. This seems to be consistent until version 8.0, that's as far back as I can test.

The salt calculation (version >= 12) works the normal way for members of a poly-VI, but the base poly-VI has 0 salt because it doesn't really have a connector pane (it's just a placeholder for a particular member VI). It still has a BDPW block though that is calculated the normal way, just with 0 (12 bytes) salt.

[mefistotelis]

Oh right, I see the types named in LV14 object factory:

int `vtable for'VIFactoryWithStandardErrors<ClipboardVI,(VITypes)3> = 0;
int `vtable for'VIFactoryWithStandardErrors<StatechartVI,(VITypes)14> = 0;
int `vtable for'VIFactoryWithStandardErrors<StandardVI,(VITypes)2> = 0;
int `vtable for'VIFactoryWithStandardErrors<PolyVI,(VITypes)11> = 0;
int `vtable for'VIFactoryWithStandardErrors<PxCallerVI,(VITypes)8> = 0;
int `vtable for'VIFactoryWithStandardErrors<PxCalleeVI,(VITypes)9> = 0;
int `vtable for'VIFactoryWithStandardErrors<ProbeVI,(VITypes)1> = 0;
int `vtable for'VIFactoryWithStandardErrors<MethodVI,(VITypes)13> = 0;
int `vtable for'VIFactoryWithStandardErrors<HierarchyVI,(VITypes)5> = 0;
int `vtable for'VIFactoryWithStandardErrors<GlobalVI,(VITypes)4> = 0;
int `vtable for'VIFactoryWithStandardErrors<FacadeVI,(VITypes)12> = 0;
int `vtable for'VIFactoryWithStandardErrors<DialogVI,(VITypes)0> = 0;
int `vtable for'VIFactoryWithStandardErrors<ControlVI,(VITypes)6> = 0;
int `vtable for'VIFactoryWithStandardErrors<ConfigVI,(VITypes)7> = 0;
int `vtable for'VIFactoryWithStandardErrors<SubSystemVI,(VITypes)10> = 0;

Are these used in both LVSR and LVIN?

EDIT:

Never mind - yes they are; I see the value from LVIN is copied directly to viType, in the same way as for LVSR loading:

void __cdecl LVINToVI(void **in, VirtualInstrument *instr)
{
[...]
  instr->viType = _byteswap_ushort(*((_WORD *)*in + 45));

[Novgorod]

Nice find!

I can't be of much help regarding the really old versions (LVIN was replaced around LV version 4 or so, right?) - I limited my unlocker tool to LV version 6 and later in order to keep my sanity ;) ...

[Novgorod]

Hey, last update regarding this: turned out I was just stupid :) ... Somehow I completely overlooked that the VCTP block contains a "top level type list" (or whatever you call it) after the actual TD list. I thought CPC2 and CONP are indexing into the main TD list (which is why it didn't work for me), but of course they are indexing into the "top level" list at the end of the VCTP block (DTHP does that as well), and for some reason the index starts with 1, not with 0. So essentially the connector pane TD is double-indexed.

In any case, using the CPC2 index that way works flawlessly for any VI that needs salt (version >= 12). This also enables rebuilding a broken hash in the BDPW block, i.e. if the original Hash1 got lost.

Also, it looks like both CONP and CPC2 reference a valid terminal for the salt calculation (i.e. the VI's connector pane). They are different terminals in the VCTP list with (usually) different flags, but seem to have identical client lists.