Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[P1] Adding in constant source intervention support with new tests #59

Merged
merged 3 commits into from
Jan 17, 2024

Conversation

frankaging
Copy link
Collaborator

@frankaging frankaging commented Jan 17, 2024

Descriptions:
Currently, we assume source representations are provided at runtime to do the intervention. This is not easy to swap like zero-out certain representations, adding some noise to certain representations, and swapping with a static yet trainable representation which is fixed for all examples.

We add in a new type of intervention, ConstantSourceIntervention for this line of interventions.

Testing Done:

WARNING:root:Loading trainable intervention from intkey_layer.1.repr.block_output.unit.pos.nunit.1#0.bin.
.Directory './test_output_dir_prefix-d86b4d' created successfully.
.Directory './test_output_dir_prefix-4bf335' created successfully.
WARNING:root:Saving trainable intervention to intkey_layer.0.repr.block_output.unit.pos.nunit.1#0.bin.
WARNING:root:Saving trainable intervention to intkey_layer.1.repr.block_output.unit.pos.nunit.1#0.bin.
.Removing testing dir ./test_output_dir_prefix-19140d
Removing testing dir ./test_output_dir_prefix-d07d02
Removing testing dir ./test_output_dir_prefix-12751f
Removing testing dir ./test_output_dir_prefix-d86b4d
Removing testing dir ./test_output_dir_prefix-4bf335

----------------------------------------------------------------------
Ran 30 tests in 4.461s

OK

@frankaging frankaging merged commit 5894b18 into main Jan 17, 2024
1 check passed
@frankaging frankaging deleted the zen/constantsourceint branch January 19, 2024 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant