We need tests.

[zkokaja]

We are updating the code a lot, it would be great to have even just really simple tests to run to ensure nothing breaks. Let's brainstorm some ideas, and what to apply it to: podcast, 247, glove, just one conversation? How do we evaluate success?

[zkokaja]

all this should be automated in a Makefile target:

what are we testing:

• generate glove, and gpt2
• one subject, 625
• two big conversations
• result: base and embedding pickles

encoding:

• run encoding for 5 good electrodes glove/gpt2 and plot
• result: average encoding

create a standard for what we will compare against in the future, but first we need to ensure current code replicates previous results for all electrodes and conversations.

hacky way to manage results folders:

mv results results-old
mkdir results
DO TEST
mv results results-test
mv results-old results

consider using another Makefile or config options

[zkokaja]

Use just 1024 tokens from each conversation?