Skip to content

Commit

Permalink
add developer doc notes (#1201)
Browse files Browse the repository at this point in the history
* add developer doc notes

* update version number
  • Loading branch information
mike-ferguson authored Aug 26, 2024
1 parent ed16fd9 commit 6196b1d
Show file tree
Hide file tree
Showing 2 changed files with 38 additions and 2 deletions.
4 changes: 2 additions & 2 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@
# -- Project information -----------------------------------------------------

project = 'Brain-Score'
copyright = '2022, Brain-Score Team'
copyright = '2024, Brain-Score Team'
author = 'Brain-Score Team'

# The full version, including alpha/beta/rc tags
release = '1.3'
release = '2.0.0'


# -- General configuration ---------------------------------------------------
Expand Down
36 changes: 36 additions & 0 deletions docs/source/modules/developer_clarifications.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
.. _interface:

************************
Developer Clarifications
************************

The Following documentation stores commonly-asked developer questions. We hope this will be useful to
anyone interested in contributing to Brain-Score's codebase or scientific workings.



1. **For a given model, are activations different on each benchmark? How?**


Activations per model are generated based on benchmark stimuli; not every benchmark has unique stimuli. For most
model-benchmark pairs, activations will be different because stimuli will be different. The exceptions to this
are the benchmarks that use the same stimuli, such as the `MajajHong20215` family of benchmarks.

2. **Result Caching**

Result Caching is a Brain-Score `repo <https://github.com/brain-score/result_caching>`_ that allows model activations (and other functions) to be cached
to disk, in order to speed up the process of rescoring models. It contains a decorator that can be attached to a function
right before it is defined. On the first run of that function, `result_caching` will save to disk the result of tha function
and will load that result from disk in future calls with the same parameters. All files are saved in the user's `~/result_caching`
folder, and they are persistent, as there is no garbage collection built in. You can deactivate
`result_caching` by simply setting the environment flag `RESULTCACHING_DISABLE` to `1`. Please see the link above
for more detailed documentation.

3. **Model Mapping Procedure**

In general, there are different methods that are used in the Brain-Score code to instruct the model to "begin recording",
observe stimuli, and to generate scores. Models follow the `ModelCommitment` to conform to the `BrainModel` API. A
`BrainModel` is any model that has a `region_layer_map`. This allows the layers in the model to be mapped to layers in
the ventral visual stream, and is chosen by scoring models on the public version of a benchmark (the private
benchmark data is heldout for the BrainModel to be scored on). See the more technical docs
`here <https://brain-score.readthedocs.io/en/latest/modules/model_interface.html>`_ for additional notes.

0 comments on commit 6196b1d

Please sign in to comment.