Help with error message

[jwdegee]

Hi,

After upgrading PyDDM from 0.5.0 to 0.7.0 I'm getting the below error message. I did not have this problem before. Do you know what might be going on?

Thank you,

Jan Willem

ValueError                                Traceback (most recent call last)
File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/scipy/optimize/_differentialevolution.py:1116, in DifferentialEvolutionSolver._calculate_population_energies(self, population)
   1115 try:
-> 1116     calc_energies = list(
   1117         self._mapwrapper(self.func, parameters_pop[0:S])
   1118     )
   1119     calc_energies = np.squeeze(calc_energies)

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/scipy/_lib/_util.py:407, in _FunctionWrapper.__call__(self, x)
    406 def __call__(self, x):
--> 407     return self.f(x, *self.args)

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/functions.py:350, in fit_adjust_model.<locals>._fit_model(xs)
    349     s(m, x)
--> 350 lossf = lf.loss(m)
    351 if verbose:

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/paranoid/decorators.py:114, in _wrap.<locals>._decorated(*args, **kwargs)
    113 if Settings.get("enabled", function=func) == False:
--> 114     return func(*args, **kwargs)
    115 # We only run this function once for performance reasons, and
    116 # then pass it as an argument to each check function.

File ~/repos/project_2021_attentional_effort/pyddm_components.py:490, in LossBICGonogo.loss(self, model)
    489 assert model.dt == self.dt and model.T_dur == self.T_dur
--> 490 sols = self.cache_by_conditions(model)
    491 loglikelihood = 0

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/models/loss.py:93, in LossFunction.cache_by_conditions(self, model)
     92 from ..functions import solve_all_conditions
---> 93 return solve_all_conditions(model, sample=self.sample, method=self.method)

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/functions.py:523, in solve_all_conditions(model, sample, condition_combinations, method)
    522 for c in conds:
--> 523     cache[frozenset(c.items())] = meth(conditions=c)
    524 return cache

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/paranoid/decorators.py:114, in _wrap.<locals>._decorated(*args, **kwargs)
    113 if Settings.get("enabled", function=func) == False:
--> 114     return func(*args, **kwargs)
    115 # We only run this function once for performance reasons, and
    116 # then pass it as an argument to each check function.

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/model.py:550, in Model.solve(self, conditions, return_evolution, force_python)
    549 else:
--> 550     return self.solve_numerical_implicit(conditions=conditions, return_evolution=return_evolution, force_python=force_python)

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/paranoid/decorators.py:114, in _wrap.<locals>._decorated(*args, **kwargs)
    113 if Settings.get("enabled", function=func) == False:
--> 114     return func(*args, **kwargs)
    115 # We only run this function once for performance reasons, and
    116 # then pass it as an argument to each check function.

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/model.py:913, in Model.solve_numerical_implicit(self, conditions, **kwargs)
    909 """Solve the model using the implicit method (Backward Euler).
    910 
    911 See documentation for the solve_numerical method.
    912 """
--> 913 return self.solve_numerical(method="implicit", conditions=conditions, **kwargs)

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/paranoid/decorators.py:114, in _wrap.<locals>._decorated(*args, **kwargs)
    113 if Settings.get("enabled", function=func) == False:
--> 114     return func(*args, **kwargs)
    115 # We only run this function once for performance reasons, and
    116 # then pass it as an argument to each check function.

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/model.py:727, in Model.solve_numerical(self, method, conditions, return_evolution, force_python)
    726 if method == "implicit" and HAS_CSOLVE and not force_python and not return_evolution:
--> 727     return self.solve_numerical_c(conditions=conditions)
    729 # Initial condition of decision variable

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/model.py:650, in Model.solve_numerical_c(self, conditions)
    649     xdomain = self.x_domain(t=maxt, conditions=conditions)
--> 650     drift = np.concatenate([get_drift(t=t, x=xdomain, conditions=conditions) for t in self.t_domain()])
    651 get_noise = self.get_dependence("noise").get_noise

File <__array_function__ internals>:200, in concatenate(*args, **kwargs)

ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 1 dimension(s) and the array at index 401 has 0 dimension(s)

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
File ~/repos/project_2021_attentional_effort/5_ddm_new.py:433
    431     settings = model_settings[version]
    432     print(settings)
--> 433     params = pyddm_components.fit_model_one_accumulator(df.loc[:,['start', 'duration', 'response', 'rt', 'reward', 'state']], model_settings[version], n_cpus=4)
    434     params.to_csv(os.path.join(project_dir, 'ddm_params_{}.csv'.format(version)))
    436 # groupby = ['block_id']
    437 # n_cpus = 12
    438 # version = 5
   (...)
    495 #     sns.barplot(x='subj_idx', y='value', hue='reward', data=params.loc[params['variable']==p,:], ax=ax)
    496 #     plt.ylabel(p)

File ~/repos/project_2021_attentional_effort/pyddm_components.py:550, in fit_model_one_accumulator(df, model_settings, n_cpus)
    544 # return model
    545 
    546 # fit:
    547 # model = fit_adjust_model(sample=sample, model=model, lossfunction=LossBIC, fitparams={'maxiter':5000})
    548 # model = fit_adjust_model(sample=sample, model=model, lossfunction=LossRobustBIC)
    549 shell()
--> 550 model = fit_adjust_model(sample=sample, model=model, lossfunction=LossRobustBICGonogo)
    552 # get params:
    553 # param_names = [component.required_parameters for component in model.dependencies]
    554 # param_names = [item for sublist in param_names for item in sublist]
    555 param_names = model.get_model_parameter_names()

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/functions.py:368, in fit_adjust_model(sample, model, fitparams, fitting_method, lossfunction, verify, method, verbose)
    366     if "disp" not in fitparams.keys():
    367         fitparams["disp"] = verbose
--> 368     x_fit = differential_evolution(_fit_model, constraints, **fitparams)
    369 elif fitting_method == "hillclimb":
    370     x_fit = evolution_strategy(_fit_model, x_0, **fitparams)

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/scipy/optimize/_differentialevolution.py:392, in differential_evolution(func, bounds, args, strategy, maxiter, popsize, tol, mutation, recombination, seed, callback, disp, polish, init, atol, updating, workers, constraints, x0, integrality, vectorized)
    375 # using a context manager means that any created Pool objects are
    376 # cleared up.
    377 with DifferentialEvolutionSolver(func, bounds, args=args,
    378                                  strategy=strategy,
    379                                  maxiter=maxiter,
   (...)
    390                                  integrality=integrality,
    391                                  vectorized=vectorized) as solver:
--> 392     ret = solver.solve()
    394 return ret

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/scipy/optimize/_differentialevolution.py:984, in DifferentialEvolutionSolver.solve(self)
    979     self.feasible, self.constraint_violation = (
    980         self._calculate_population_feasibilities(self.population))
    982     # only work out population energies for feasible solutions
    983     self.population_energies[self.feasible] = (
--> 984         self._calculate_population_energies(
    985             self.population[self.feasible]))
    987     self._promote_lowest_energy()
    989 # do the optimization.

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/scipy/optimize/_differentialevolution.py:1123, in DifferentialEvolutionSolver._calculate_population_energies(self, population)
   1119     calc_energies = np.squeeze(calc_energies)
   1120 except (TypeError, ValueError) as e:
   1121     # wrong number of arguments for _mapwrapper
   1122     # or wrong length returned from the mapper
-> 1123     raise RuntimeError(
   1124         "The map-like callable must be of the form f(func, iterable), "
   1125         "returning a sequence of numbers the same length as 'iterable'"
   1126     ) from e
   1128 if calc_energies.size != S:
   1129     if self.vectorized:

RuntimeError: The map-like callable must be of the form f(func, iterable), returning a sequence of numbers the same length as 'iterable'

[mwshinn]

It looks like there might be something weird with your model or your sample. Does the model gui work for this sample/model combination? A couple more things that would be helpful in debugging:

• Does it work on PyDDM 0.6.1?
• Can you post the output of pyddm.display_model?

Also, one thing that might be a quick fix: switch your imports from "import ddm" to "import pyddm as ddm" - importing pyddm by calling "import ddm" causes some weird problems, and support for this will be removed in a future version.

[jwdegee]

import pyddm as ddm I already did.

ddm.plot.model_gui(model, sample=sample)

gives:

---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/Users/jwdegee/mambaforge/envs/ddm/lib/python3.9/site-packages/multiprocess/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/Users/jwdegee/mambaforge/envs/ddm/lib/python3.9/site-packages/multiprocess/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/Users/jwdegee/mambaforge/envs/ddm/lib/python3.9/site-packages/paranoid/decorators.py", line 114, in _decorated
    return func(*args, **kwargs)
  File "/Users/jwdegee/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/model.py", line 550, in solve
    return self.solve_numerical_implicit(conditions=conditions, return_evolution=return_evolution, force_python=force_python)
  File "/Users/jwdegee/mambaforge/envs/ddm/lib/python3.9/site-packages/paranoid/decorators.py", line 114, in _decorated
    return func(*args, **kwargs)
  File "/Users/jwdegee/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/model.py", line 913, in solve_numerical_implicit
    return self.solve_numerical(method="implicit", conditions=conditions, **kwargs)
  File "/Users/jwdegee/mambaforge/envs/ddm/lib/python3.9/site-packages/paranoid/decorators.py", line 114, in _decorated
    return func(*args, **kwargs)
  File "/Users/jwdegee/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/model.py", line 727, in solve_numerical
    return self.solve_numerical_c(conditions=conditions)
  File "/Users/jwdegee/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/model.py", line 650, in solve_numerical_c
    drift = np.concatenate([get_drift(t=t, x=xdomain, conditions=conditions) for t in self.t_domain()])
  File "<__array_function__ internals>", line 200, in concatenate
ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 1 dimension(s) and the array at index 401 has 0 dimension(s)
"""

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
Cell In[1], line 1
----> 1 ddm.plot.model_gui(model, sample=sample)

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/plot.py:508, in model_gui(model, sample, data_dt, plot, conditions, verify)
    505 b.pack(expand=True, fill="both")
    507 root.update()
--> 508 set_defaults()
    509 frame_params_container.configure(scrollregion=frame_params_container.bbox('all'))
    510 tk.mainloop()

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/plot.py:496, in model_gui.<locals>.set_defaults()
    494     w.set(default)
    495     s(m, default)
--> 496 update()

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/plot.py:409, in model_gui.<locals>.update()
    407 # If there was an error, display it instead of a plot
    408 try:
--> 409     plot(model=m, fig=fig, sample=sample, conditions=current_conditions, data_dt=data_dt)
    410 except:
    411     fig.clear()

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/plot.py:204, in plot_fit_diagnostics(model, sample, fig, conditions, data_dt, method)
    202     toplabel,bottomlabel = sample.choice_names
    203 if model:
--> 204     s = solve_partial_conditions(model, sample, conditions=conditions, method=method)
    205     ax1.plot(model.t_domain(), s.pdf("_top"), lw=2, color='k')
    206     ax2.plot(model.t_domain(), s.pdf("_bottom"), lw=2, color='k')

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/paranoid/decorators.py:114, in _wrap.<locals>._decorated(*args, **kwargs)
    111 def _decorated(*args, **kwargs):
    112     # Skip verification if paranoid is disabled.
    113     if Settings.get("enabled", function=func) == False:
--> 114         return func(*args, **kwargs)
    115     # We only run this function once for performance reasons, and
    116     # then pass it as an argument to each check function.
    117     sig = inspect.Signature.from_callable(func)

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/functions.py:618, in solve_partial_conditions(model, sample, conditions, method)
    616 if not isinstance(model.get_dependence("overlay"), OverlayNone):
    617     model_undec = None
--> 618 all_conds = solve_all_conditions(model, sample=samp, method=method)
    619 for conds in samp.condition_combinations(required_conditions=model.required_conditions):
    620     subset = samp.subset(**conds)

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/pyddm/functions.py:530, in solve_all_conditions(model, sample, condition_combinations, method)
    526 if paranoid_settings.get('enabled') is False:
    527     # The *2 makes sure that this runs on all subprocesses,
    528     # since you can't broadcast commands to all processes
    529     _parallel_pool.map(lambda x : paranoid_settings.set(enabled=False), [None]*_parallel_pool.n_cpus*2)
--> 530 sols = _parallel_pool.map(meth, conds, chunksize=1)
    531 for c,s in zip(conds, sols):
    532     cache[frozenset(c.items())] = s

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/multiprocess/pool.py:364, in Pool.map(self, func, iterable, chunksize)
    359 def map(self, func, iterable, chunksize=None):
    360     '''
    361     Apply `func` to each element in `iterable`, collecting the results
    362     in a list that is returned.
    363     '''
--> 364     return self._map_async(func, iterable, mapstar, chunksize).get()

File ~/mambaforge/envs/ddm/lib/python3.9/site-packages/multiprocess/pool.py:771, in ApplyResult.get(self, timeout)
    769     return self._value
    770 else:
--> 771     raise self._value

ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 1 dimension(s) and the array at index 401 has 0 dimension(s)

Seems like there is something wrong with my model?

[mwshinn]

Interesting, one more debugging thing to help me track this down: does it work on 0.5.1, 0.5.2, and 6.0.0? I have a hunch this might be due to something in 0.5.2.

[jwdegee]

hey, both 0.5.1 and 0.5.2 work, but 0.6.0 does not.

[mwshinn]

Could you please post your get_drift function from OneAccumulatorDrift? It looks like there might be an issue with it: when the "x" is a vector, does it sometimes return a vector and sometimes return a scalar? get_drift and get_noise should always return a vector the same size as the passed x. x will (almost always) be a vector. Old versions of pyddm would have silently ignored this.

[jwdegee]

Ah, that worked! The return of get_drift was as follows:

            k_target = 10 - a
            if t < (conditions['start'] + Ter):
                return - (k * (x-k_target))
            elif (t >= (conditions['start'] + Ter)) & (t < (conditions['start']+conditions['duration'])):
                return - (k * (x-k_target)) + v
            else:
                return 0

Changing that last 0 to x*0 did the trick.

Many thanks!!

[mwshinn]

Great! Though just a heads up, if you want the equivalent code to what you had before, change the 0 to x*0 instead of just x. That way the drift is zero no matter what x position you have.

[jwdegee]

Ah yes, was a typo -- thanks again!