Crowdsourcing - Run test suite on different OSes and Nuke versions

[jonahfriedman]

I am currently only testing Nuke 12.0v3 on Windows 10. Cryptomatte does work around some Nuke idiosyncrasy which has been seen in the past to vary from version to version, so I'm attempting to crowdsource a bit here, If you would like to contribute and have some OSes and Nuke version combinations handy that you don't see results for here, please run the test suite and report back the results.

1. Open Nuke with a Cryptomatte installation (1.2.6 or 1.3.0-beta2)
2. Open a script editor in Nuke, clear it, and run the Python command below. See also, docs on tests
3. Report back results. You should see the Cryptomatte version, Nuke version and OS in the results (in 1.2.6 and 1.3.0-beta2). If the result looks like the sample result below in the script editor, that's a pass and you don't need to post the full results.
4. Also check the console for expression warnings between the results.

Python command:

import cryptomatte_utilities as cu
cu.tests.run_nuke_tests()

Cryptomatte 1.2.6, Nuke 12.0v3, Windows-10-10.0.18362:

# Result: test_csv_round_trip (cryptomatte_utilities_tests.CSVParsing) ... ok
test_mm3hash_float (cryptomatte_utilities_tests.CryptoHashing) ... ok
test_big_csv_through_gizmo (cryptomatte_utilities_tests.CSVParsingNuke) ... ok
test_csv_through_gizmo (cryptomatte_utilities_tests.CSVParsingNuke) ... ok
test_bug_still_exists (cryptomatte_utilities_tests.CryptomatteNodePasting)
Tests this bug still exists. We don't want to be running the fix if we don't have to. ... ok
test_paste_with_channelmerge (cryptomatte_utilities_tests.CryptomatteNodePasting)
Tests this bug has been fixed. ... ok
test_bad_names_in_nuke_layers (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_blendery_names_encryptomatte (cryptomatte_utilities_tests.CryptomatteNukeTests)
Tests that names with nuke-unfriendly layer names are mangled property by Encryptomatte. ... ok
test_blendery_names_in_metadata (cryptomatte_utilities_tests.CryptomatteNukeTests)
Tests that names with nuke-unfriendly layer names are mangled property. ... ok
test_channel_identification (cryptomatte_utilities_tests.CryptomatteNukeTests)
Makes sure multiple Cryptomattes can have sub-channels identified properly, ... ok
test_clear_button (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_crypto_channel_knobs_type (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_decrypto_basic (cryptomatte_utilities_tests.CryptomatteNukeTests)
Tests both basic decryptomatte, as well as ensuring hash_channel() is ... ok
test_decrypto_custom_channel (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_decrypto_matteonly_unpremul (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_decrypto_rmchannels_alpha (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_decrypto_rmchannels_customlayer (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_bogus_inputs (cryptomatte_utilities_tests.CryptomatteNukeTests)
Tests that when setting up layers, entering the name before pressing "setup layers" ... ok
test_encrypt_bogus_manifest (cryptomatte_utilities_tests.CryptomatteNukeTests) ... Cryptomatte: Unable to parse manifest. (Expecting property name: line 1 column 2 (char 1)).
ok
test_encrypt_channel_knobs_type (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_fresh_keyable (cryptomatte_utilities_tests.CryptomatteNukeTests)
Tests fresh Encryptomatte setup where there is no input constant. ... ok
test_encrypt_fresh_roundtrip (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_layer_forced_update_btn (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_layer_lock (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_layerselection (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_manifest (cryptomatte_utilities_tests.CryptomatteNukeTests)
Gets it into a weird state where it has a manifest but no cryptomatte. ... ok
test_encrypt_matte_name_autofill2 (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_merge_operations (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_roundtrip_keyable (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_roundtrip_setup (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_roundtrip_without_prefix (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_setup_layers_numbers (cryptomatte_utilities_tests.CryptomatteNukeTests)
Tests that when setting up layers, entering the name before pressing "setup layers" ... ok
test_file_open_doesnt_update_nodes (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_gizmo_version (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_blank_matteList (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_clear (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_manual (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_multiselect (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_nothing (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_partial_black (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_single_selection (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_stop_auto_update (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_with_removechannels (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_without_prefix (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_without_preview_channels (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_layer_bogus_manifest (cryptomatte_utilities_tests.CryptomatteNukeTests) ... Cryptomatte: Unable to parse manifest. (Expecting property name: line 1 column 2 (char 1)).
Cryptomatte: Unable to parse manifest. (Expecting property name: line 1 column 2 (char 1)).
ok
test_layer_forced_update_btn (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_layer_forced_update_func (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_layer_lock (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_layer_selection (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_layer_selection_after_keying (cryptomatte_utilities_tests.CryptomatteNukeTests)
Tests all layer selection options are still available after keying. ... ok
test_load_manifests_lazy (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_load_manifests_nonlazy (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_manifests (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_matte_list_name_modifications (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_matte_list_numeric (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_output_preview (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_output_preview_disabled (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_output_preview_multi (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok

----------------------------------------------------------------------
Ran 59 tests in 6.151s

OK
---------
Cryptomatte 1.2.6, Nuke 12.0v3, Windows-10-10.0.18362
---------
Testing passed: 0 failed, 0 errors. (59 test cases.)

[mybikeislost]

----------------------------------------------------------------------
Ran 59 tests in 6.636s

OK
---------
Cryptomatte 1.2.6, Nuke 11.3v6, Darwin-18.7.0-x86_64-i386-64bit
---------
Testing passed: 0 failed, 0 errors. (59 test cases.)

[jacksonspencer]

# Result: test_csv_round_trip (cryptomatte_utilities_tests.CSVParsing) ... ok
test_mm3hash_float (cryptomatte_utilities_tests.CryptoHashing) ... ok
test_big_csv_through_gizmo (cryptomatte_utilities_tests.CSVParsingNuke) ... ok
test_csv_through_gizmo (cryptomatte_utilities_tests.CSVParsingNuke) ... ok
test_bug_still_exists (cryptomatte_utilities_tests.CryptomatteNodePasting)
Tests this bug still exists. We don't want to be running the fix if we don't have to. ... ok
test_paste_with_channelmerge (cryptomatte_utilities_tests.CryptomatteNodePasting)
Tests this bug has been fixed. ... ok
ERROR

======================================================================
ERROR: setUpClass (cryptomatte_utilities_tests.CryptomatteNukeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:/Program Files/Nuke12.2v2/plugins\cryptomatte_utilities_tests.py", line 290, in setUpClass
    (file_path, SAMPLES_IMAGES_DIR_ENVIRON))
IOError: Could not find: C:/Program Files/Nuke12.2v2/sample_images/bunny_CryptoObject.exr. Sample image dir can be defined env variable, CRYPTOMATTE_TESTING_SAMPLES

----------------------------------------------------------------------
Ran 6 tests in 0.138s

FAILED (errors=1)
---------
Cryptomatte 1.2.8, Nuke 12.2v2, Windows-10-10.0.18362
---------
Error: _ErrorHolder.

Traceback (most recent call last):
  File "C:/Program Files/Nuke12.2v2/plugins\cryptomatte_utilities_tests.py", line 290, in setUpClass
    (file_path, SAMPLES_IMAGES_DIR_ENVIRON))
IOError: Could not find: C:/Program Files/Nuke12.2v2/sample_images/bunny_CryptoObject.exr. Sample image dir can be defined env variable, CRYPTOMATTE_TESTING_SAMPLES

---------
TESTING FAILED: 0 failed, 1 errors. (60 test cases.)

[jacksonspencer]

I'm not sure if this has anything to do with what it shows in the test I posted above or if I'm doing something incorrectly, but using the picker doesn't highlight the objects in yellow and fully create the matte, but it does select the correct identifier and add it to the list.

[jacksonspencer]

Posting back here again to say that after running tests, using Cryptomatte in Nuke with my set up listed above works correctly when the Cryptomatte options in RenderMan for Maya 23.3 are set to create mattes based on object name, and does not work when set to create mattes based on applied materials. In Nuke, the object name identifier shows up as "identifier_obect#" and the material identifier shows up as "user___materialid#"