v2.4.1

Little update to the code usage, but huge update for memory consumption! TransformerLens now needs almost half the memory it needed previously to boot thanks to a change with how the TransformerLens models are loaded.

What's Changed

• removed einsum causing error when use_atten_result is enabled by @oliveradk in #660
• revised loading to recycle state dict by @bryce13950 in #706

New Contributors

• @oliveradk made their first contribution in #660

Full Changelog: v2.4.0...v2.4.1

v2.4.0

Nice little update! This gives users a little bit more control over attention masks, as well as adds a new demo.

What's Changed

• Improve attention masking by @UFO-101 in #699
• add a demo for Patchscopes and Generation with Patching by @HenryCai11 in #692

New Contributors

• @HenryCai11 made their first contribution in #692

Full Changelog: v2.3.1...v2.4.0

v2.3.1

Nice little bug fix!

What's Changed

• Update Gemma2 attention scale by @mntss in #694
• Release v2.3.1 by @bryce13950 in #701

New Contributors

• @mntss made their first contribution in #694

Full Changelog: v2.3.0...v2.3.1

v2.3.0

New models! This release adds support for Gemma 2 2B as well as Qwen2. This also removes official support for python 3.8. Python 3.8 should continue to work for a while, but there is a high risk that it will be unstable past this release. If you need python 3.8, try locking to this release or any previous release.

What's Changed

• Fix typo in embed.py docs by @ArthurConmy in #677
• Move the HookedSAE / HookedSAETransformer warning to a less prominent… by @ArthurConmy in #676
• NamesFilter can be a string by @jettjaniak in #679
• Adding RMSNorm to apply_ln_to_stack by @gaabrielfranco in #663
• added arena content as a notebook by @bryce13950 in #674
• Test arena cleanup by @bryce13950 in #681
• docs: update Main_Demo.ipynb by @eltociear in #658
• Add support for Qwen2 models by @g-w1 in #662
• Added gemma-2 2b by @curt-tigges in #687
• Python 3.8 removal by @bryce13950 in #690
• 2.3.0 by @bryce13950 in #688

New Contributors

• @gaabrielfranco made their first contribution in #663
• @eltociear made their first contribution in #658
• @g-w1 made their first contribution in #662
• @curt-tigges made their first contribution in #687

Full Changelog: v2.2.2...v2.3.0

v2.2.2

Quick little bug fix!

What's Changed

• Fix attention result projection by @callummcdougall in #666
• fix: fixing broken backward hooks change by @chanind in #673

Full Changelog: v2.2.1...v2.2.2

v2.2.1

Quick little bug fix in the Abstract Attention component shape.

What's Changed

• Fix attention result projection by @callummcdougall in #666

Full Changelog: v2.2.0...v2.2.1

v2.2.0

Here's an important one! This release adds Gemma-2, and it also greatly improves model accuracy across the board. It is highly recommended that everyone update to this version immediately to take advantage of these accuracy improvements.

What's Changed

• Fix typo in Main_Demo.ipynb by @ianand in #636
• Add comparing-to-huggingface.ipynb. by @joelburget in #637
• Add tests for gated mlp by @anthonyduong9 in #638
• Match Huggingface MLP implementation exactly. by @joelburget in #641
• Add tests for ActivationCache by @FlyingPumba in #643
• Moved mixtral weights to another module by @bryce13950 in #646
• Fixed weight conversion by @bryce13950 in #648
• Move out pretrained weight conversions by @richardkronick in #647
• Match Huggingface GPT2 implementation exactly by @joelburget in #645
• Fix Out bias not being summed in attention component when using 4 bit precision by @FlyingPumba in #654
• Mlp cleanup by @bryce13950 in #652
• Added support for Gemma-2 by @neelnanda-io in #650
• add tests for Attention by @anthonyduong9 in #639
• Release 2.2 by @bryce13950 in #656

New Contributors

• @ianand made their first contribution in #636
• @FlyingPumba made their first contribution in #643

Full Changelog: v2.1.0...v2.2.0

v2.1.0

New model support, and a handful of bug fixes to documentation!

What's Changed

• Encoder-Decoder (T5) support by @somvy in #605
• Update README links to ARENA mech interp tutorials by @gileshd in #630
• Lock datasets version by @courtney-sims in #632

New Contributors

• @somvy made their first contribution in #605
• @gileshd made their first contribution in #630
• @courtney-sims made their first contribution in #632

Full Changelog: v2.0.1...v2.1.0

v2.0.1

Minor little fix to demos. There were some bad Urls within the demos that have been fixed. Test coverage has also been increased with this release.

What's Changed

• Fix demos pip install packages from unfound repos by @anthonyduong9 in #625
• Unit tests loading from pretrained fill missing keys by @richardkronick in #623

New Contributors

• @richardkronick made their first contribution in #623

Full Changelog: v2.0.0...v2.0.1

v2.0.0

TransformerLens officially has a 2.0! The HookedSAETransformer has been removed from TransformerLens in favor of the implementation in SAELens. Along with that, a lot of cumulative changes have been added. TransformerLens also has its first official development road map! To see full details, please view the release announcement for the development roadmap, changes coming to contributors, and a few more notes on this release.

What's Changed

• Refactor components by @bryce13950 in #563
• added convenience function for unwrapping config to replace commonly … by @bryce13950 in #571
• unwrapped config by @bryce13950 in #577
• Refactor integration tests by @bryce13950 in #576
• Add Mistral 7B v0.2 Instruct by @fakerybakery in #579
• Add support for Phi-3 by @slash3g in #573
• Revert "Add Mistral 7B v0.2 Instruct" by @bryce13950 in #586
• Interactive neuroscope ci by @bryce13950 in #589
• removed Hooked SAE by @bryce13950 in #600
• Release 1.18 by @bryce13950 in #602
• More pytest fixtures by @bmillwood in #609
• (v3) Draft PR: add Pyright static typing to hook_points.py #590 by @starship006 in #607
• v1.19 by @bryce13950 in #614
• add n k v heads to model properties table by @anthonyduong9 in #610
• fixed format by @bryce13950 in #616
• Add tests for hook point add hook by @anthonyduong9 in #617
• added release blog by @bryce13950 in #618
• Fix llama demos by @bryce13950 in #619
• added news link by @bryce13950 in #620
• Release 2.0 by @bryce13950 in #582

New Contributors

• @fakerybakery made their first contribution in #579
• @slash3g made their first contribution in #573
• @bmillwood made their first contribution in #609
• @starship006 made their first contribution in #607
• @anthonyduong9 made their first contribution in #610

Full Changelog: v1.19.0...v2.0.0