Valid index combinations

[ErnestJohnston]

I am training various CEBRA models, and all of the single-subject models are working fine. One example model specification is below:

combined_cebra_model = cebra.CEBRA(
    conditional = 'time_delta',
    hybrid = True,
    model_architecture = 'offset10-model',
    output_dimension = 3,
    max_iterations = 1000,
    batch_size = None, 
    temperature = 1,
    time_offsets = 10,
    verbose = True)

When I switch to multi-subject training, I am getting invalid index errors in certain cases. Some of the explanations make sense to me: For example, multi-subject training does not currently support both continuous and discrete labels.

However, some of the invalid index combinations do not make sense to me. Namely, training a multi-subject dataset with hybrid false and either continuous or discrete labels (but not both) will work as long as batch_size is specified. But if I set batch_size to None in order to train the entire dataset, it throws an error. I don't understand why batch_size is relevant here.

NotImplementedError: Invalid index combination.
Continuous: True,
Discrete: False,
Hybrid training: False,
Full dataset: True. This index combination might still be implemented in the future. Until then, please train using the PyTorch API.

Also, it seems that multi-subject models cannot be trained across time (either through conditional = 'time' or through 'time_delta' with hybrid = True) -- is this an inherent limitation of CEBRA, or is it simply not a feature that is implemented yet?

I have been unable to find documentation about what combinations are invalid (and why that is so). Finally, do you have any information on if/when hybrid training for multi-subject models will be implemented (outside of the PyTorch API)?

Any help or explanation is appreciated!

[MMathisLab]

Hi @ErnestJohnston thanks for your interest in cebra.

• time only isn't possible in this multisession training; there is nothing then to anchor across the datasets, hence an auxiliary "behavior" label is needed.

• for full combinations of every setting no we don't have s table or such, but as error message notes, more is available in the pytorch API, this is mainly due to time of my lab and resources allocated to software engineering vs research 😀. More will come but if there is a particular set up blocking you, let us know!

• batchsize is always relevant as its related to the loss value, and how data is sampled; see model description here, https://www.nature.com/articles/s41586-023-06031-6 and see the supplementary note 2 https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-023-06031-6/MediaObjects/41586_2023_6031_MOESM1_ESM.pdf
  In practice, the largest gpu you have is best to specify the largest batchsize you can without memory errors 😄

[ErnestJohnston]

Thank you for this excellent explanation! To clarify one point about the multi-session hybrid (behavior+time) training:

Is it true to say that CEBRA creates both (1) a universal embedding for all sessions/subjects based on the behavioral labels and (2) a subject-level embedding based on the time of each individual subject? Thus, each session/subject's data must be plotted in their own individual time embedding that has been somewhat molded by the universal behavioral embedding?

Apologies if this interpretation is completely wrong, and thank you for your help.

[MMathisLab]

I actually don't know if hybrid works for multi-session in sklearn api, @stes will have to confirm or deny here :)