assertion error — pybasicbayes/autoregressive error?

[ralphpeterson]

Looking into this issue and want to keep record of it here. If anyone has any hunches, let me know. Below is model output file.

Will only save the last iteration of the model
Entering modeling training
Found pcs in scores
Whitening the training data using the whiten_all function
Using model class FastARWeakLimitStickyHDPHMM
Adding data from key 05099b86-d649-45ef-acdf-2eb891eb8858
Adding data from key 08de9287-ca2b-463f-9b82-2f5a24682969
Adding data from key 0c8a0211-32de-4e30-aaa5-65b2ce68e8aa
Adding data from key 0e63d1a1-0b36-47b5-8e40-cfa42286e1a5
Adding data from key 1a2c654a-3dd0-4642-a49d-3923515166da
Adding data from key 1b68f793-f36c-4dca-b034-b3cedd2d270d
Adding data from key 32e08417-847c-42b0-9004-4a61677d130b
Adding data from key 414886c5-168c-4d34-ac44-d481de5b647a
Adding data from key 42392484-402f-4123-ba95-365b86ef0b99
Adding data from key 484882b8-7e5e-468e-b424-7b8951c81ce2
Adding data from key 5d681619-a59a-4d1e-8770-0dcb27b93ed0
Adding data from key 5d6d94d0-a921-4640-ae42-48a222af6e4b
Adding data from key 647d0675-1cc5-4c1b-b3c0-30566166f53a
Adding data from key 70f16dca-b584-4543-aad1-573a80273a97
Adding data from key 820a5c95-aeff-4176-8fb4-a7b76b9059a2
Adding data from key 86465c60-1cb6-4eab-ae8f-07cc97e24db7
Adding data from key 94556d85-c9c7-42ff-ac4a-d975c8d68842
Adding data from key 955be5ef-2c66-40f3-835f-beb17e57ce43
Adding data from key 9a6ac9dc-f72f-40f0-8861-d67eba50d4fa
Adding data from key b6ee13fd-f060-450b-8386-6a94170960e0
Adding data from key d739b489-a98e-4e42-82ab-22403054a5d7
Adding data from key d9e07f87-d374-4391-a2a1-e56a4dd7ffba
Adding data from key e436f6cc-ae5f-48a4-ab65-8fd7c6906291
Adding data from key ed1d15ec-5cc7-4fc0-8411-ecd04e979d3b
  0%|          | 0/50 [00:00<?, ?it/s]
  2%|▏         | 1/50 [00:13<11:25, 13.98s/it]
  4%|▍         | 2/50 [00:26<10:33, 13.21s/it]
  6%|▌         | 3/50 [00:39<10:16, 13.11s/it]
  8%|▊         | 4/50 [00:52<10:06, 13.19s/it]
 10%|█         | 5/50 [01:06<09:58, 13.30s/it]
 12%|█▏        | 6/50 [01:20<09:48, 13.38s/it]
 14%|█▍        | 7/50 [01:34<09:39, 13.47s/it]
 16%|█▌        | 8/50 [01:48<09:29, 13.56s/it]
 18%|█▊        | 9/50 [02:02<09:19, 13.64s/it]
 20%|██        | 10/50 [02:16<09:07, 13.69s/it]
 22%|██▏       | 11/50 [02:31<08:56, 13.75s/it]
 24%|██▍       | 12/50 [02:45<08:44, 13.80s/it]
 26%|██▌       | 13/50 [02:59<08:32, 13.85s/it]
 28%|██▊       | 14/50 [03:14<08:20, 13.89s/it]
 30%|███       | 15/50 [03:29<08:07, 13.93s/it]
 32%|███▏      | 16/50 [03:43<07:55, 13.97s/it]
 34%|███▍      | 17/50 [03:58<07:42, 14.01s/it]
 36%|███▌      | 18/50 [04:12<07:29, 14.04s/it]
 38%|███▊      | 19/50 [04:27<07:16, 14.08s/it]
 40%|████      | 20/50 [04:42<07:03, 14.13s/it]
 42%|████▏     | 21/50 [04:57<06:50, 14.15s/it]
 44%|████▍     | 22/50 [05:11<06:36, 14.18s/it]
 46%|████▌     | 23/50 [05:26<06:23, 14.20s/it]
 48%|████▊     | 24/50 [05:41<06:09, 14.22s/it]

Traceback (most recent call last):
  File "/home/rep19/anaconda2/envs/moseq2/bin/moseq2-model", line 11, in <module>
    load_entry_point('moseq2-model', 'console_scripts', 'moseq2-model')()
  File "/home/rep19/anaconda2/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/rep19/anaconda2/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/rep19/anaconda2/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/rep19/anaconda2/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/rep19/anaconda2/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/rep19/Code/moseq2-repos/moseq2-model/moseq2_model/cli.py", line 193, in learn_model
    file=sys.stdout)
  File "/home/rep19/Code/moseq2-repos/moseq2-model/moseq2_model/train/util.py", line 17, in train_model
    model.resample_model(num_procs=ncpus)
  File "/home/rep19/anaconda2/envs/moseq2/lib/python3.6/site-packages/pyhsmm/models.py", line 441, in resample_model
    self.resample_parameters()
  File "/home/rep19/anaconda2/envs/moseq2/lib/python3.6/site-packages/autoregressive/models.py", line 63, in resample_parameters
    super(_ARMixin,self).resample_parameters()
  File "/home/rep19/anaconda2/envs/moseq2/lib/python3.6/site-packages/pyhsmm/models.py", line 446, in resample_parameters
    self.resample_obs_distns()
  File "/home/rep19/anaconda2/envs/moseq2/lib/python3.6/site-packages/autoregressive/models.py", line 265, in resample_obs_distns
    o.resample(stats=statmat)
  File "/home/rep19/anaconda2/envs/moseq2/lib/python3.6/site-packages/pybasicbayes/distributions/regression.py", line 272, in resample
    *self._natural_to_standard(self.natural_hypparam + stats))
  File "/home/rep19/anaconda2/envs/moseq2/lib/python3.6/site-packages/pybasicbayes/distributions/regression.py", line 103, in _natural_to_standard
    assert np.all(0 < np.linalg.eigvalsh(S))
AssertionError

[jmarkow]

This can stem from either noisy data or how the priors are set. Currently traveling but if you point me to the data I can have a look.

[jmarkow]

What did we conclude from this? Do you still need me to look?

[ralphpeterson]

We talked about using the tAR or truncating data to only include frames where animal is moving. Haven’t the time to try either and am AFK for the next few days. Give it a go if you have bandwidth, data location in our slack convo.

…

On Jul 28, 2019, at 2:51 PM, Jeffrey Markowitz ***@***.***> wrote: What did we conclude from this? Do you still need me to look? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[rodgerlab]

Hi Guys,
I am having the same error - did you guys work out the problem in the end?

Modelled with:
moseq2-model learn-model -h TRUE -n 500 -r 20 -k 4411521 _pca/pca_scores.h5 my_model.p

Haven't noticed any problems with shorter runs

[ralphpeterson]

How does your extracted data look when you look at the movies? Is the mouse being reliably tracked with minimal noise for each session?

[rodgerlab]

Yeah, tracking seems good? and have successfully extracted the same data set without doing 20 repeats and hold outs.

There are a few blips here and there, but assumed it wouldn't be a major problem? any suggestions?