Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Release 2.4.0 #702

Merged
merged 2 commits into from
Aug 14, 2024
Merged

Release 2.4.0 #702

merged 2 commits into from
Aug 14, 2024

Commits on Aug 14, 2024

  1. Improve attention masking (#699) 

    * Allow attention_mask to override the default mask in HookedTransformer.forward().

* Add attention_mask argument to loss_fn() and lm_cross_entropy_loss() and adjust the cross entropy calculation to ignore masked (padding) tokens.

---------

Co-authored-by: Bryce Meyer <[email protected]>
    @UFO-101 @bryce13950
    UFO-101 and bryce13950 authored Aug 14, 2024
    Configuration menu
    Copy the full SHA
    d6ab70a View commit details
    Browse the repository at this point in the history

  2. add a demo for Patchscopes and Generation with Patching (#692) 

    * add a demo for Patchscopes and Generation with Patching

* added pathscopes generation demo to tests

* ignored a couple cells

---------

Co-authored-by: Bryce Meyer <[email protected]>
    @HenryCai11 @bryce13950
    HenryCai11 and bryce13950 authored Aug 14, 2024
    Configuration menu
    Copy the full SHA
    135adce View commit details
    Browse the repository at this point in the history