Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failed to load model with 1-dimension input parameter. #1194

Closed
garyhai opened this issue Sep 11, 2023 · 9 comments
Closed

Failed to load model with 1-dimension input parameter. #1194

garyhai opened this issue Sep 11, 2023 · 9 comments

Comments

@garyhai
Copy link

garyhai commented Sep 11, 2023 

fn load_model(
  chunk_size: usize,
  reader: &mut dyn Read,
) -> Result<RunnableOnnxModel> {
  let model: RunnableOnnxModel = tract_onnx::onnx()
    // .model_for_read(&mut Cursor::new(include_bytes!("silero_vad.onnx")))?
    .model_for_read(reader)?
    .with_input_names(&["input", "sr", "h", "c"])?
    .with_output_names(&["output", "hn", "cn"])?
    .with_input_fact(
      0,
      InferenceFact::dt_shape(f32::datum_type(), tvec!(1, chunk_size)),
    )?
    .with_input_fact(
      1,
      InferenceFact::dt_shape(i64::datum_type(), tvec!(1)),
    )?
    .with_input_fact(
      2,
      InferenceFact::dt_shape(f32::datum_type(), tvec!(2, 1, 64)),
    )?
    .with_input_fact(
      3,
      InferenceFact::dt_shape(f32::datum_type(), tvec!(2, 1, 64)),
    )?
    .into_optimized()?
    .into_runnable()?;
  Ok(model)
}

Failed to load silero_vad.onnx with a single dimension input of sampling rate.

thread 'main' panicked at 'failed to create vad session: Failed analyse for node #74 "If_25" If

Caused by:
    0: Infering facts
    1: Unifying shapes Unsqueezeout_dim_0,1 and batch,1
    2: Impossible to unify Sym(Unsqueezeout_dim_0) with Sym(batch).

Stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.69/src/backtrace/libunwind.rs:93:5
      backtrace::backtrace::trace_unsynchronized
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.69/src/backtrace/mod.rs:66:5
   1: backtrace::backtrace::trace
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.69/src/backtrace/mod.rs:53:14
   2: anyhow::backtrace::capture::Backtrace::create
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.75/src/backtrace.rs:216:13
   3: anyhow::backtrace::capture::Backtrace::capture
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.75/src/backtrace.rs:204:17
   4: anyhow::error::<impl anyhow::Error>::msg
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.75/src/error.rs:83:36
   5: <tract_hir::infer::factoid::GenericFactoid<T> as tract_hir::infer::factoid::Factoid>::unify
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tract-hir-0.20.18/src/infer/factoid.rs:114:18
   6: <tract_hir::infer::factoid::ShapeFactoid as tract_hir::infer::factoid::Factoid>::unify::{{closure}}
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tract-hir-0.20.18/src/infer/factoid.rs:272:31
   7: core::iter::adapters::map::map_try_fold::{{closure}}
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/iter/adapters/map.rs:91:28
   8: core::iter::traits::iterator::Iterator::try_fold
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/iter/traits/iterator.rs:2303:21
   9: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/iter/adapters/map.rs:117:9
  10: <core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::try_fold
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/iter/adapters/mod.rs:195:9
  11: core::iter::traits::iterator::Iterator::try_for_each
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/iter/traits/iterator.rs:2365:9
      <core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::next
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/iter/adapters/mod.rs:178:14
  12: <smallvec::SmallVec<A> as core::iter::traits::collect::Extend<<A as smallvec::Array>::Item>>::extend
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/smallvec-1.11.0/src/lib.rs:2061:36
  13: <smallvec::SmallVec<A> as core::iter::traits::collect::FromIterator<<A as smallvec::Array>::Item>>::from_iter
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/smallvec-1.11.0/src/lib.rs:2045:9
  14: core::iter::traits::iterator::Iterator::collect
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/iter/traits/iterator.rs:1895:9
      <core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter::{{closure}}
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/result.rs:1932:51
  15: core::iter::adapters::try_process
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/iter/adapters/mod.rs:164:17
  16: <core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/result.rs:1932:9
  17: core::iter::traits::iterator::Iterator::collect
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/iter/traits/iterator.rs:1895:9
  18: <tract_hir::infer::factoid::ShapeFactoid as tract_hir::infer::factoid::Factoid>::unify
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tract-hir-0.20.18/src/infer/factoid.rs:269:35
  19: tract_hir::infer::factoid::Factoid::unify_with_mut
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tract-hir-0.20.18/src/infer/factoid.rs:43:19
  20: <tract_onnx::ops::logic::If as tract_hir::infer::ops::InferenceOp>::infer_facts
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tract-onnx-0.20.18/src/ops/logic.rs:126:28
  21: tract_hir::infer::ops::InferenceOp::infer
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tract-hir-0.20.18/src/infer/ops.rs:29:13
  22: tract_hir::infer::analyser::Analyser<M>::analyse_one
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tract-hir-0.20.18/src/infer/analyser.rs:128:17
  23: tract_hir::infer::analyser::Analyser<M>::analyse_obstinate
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tract-hir-0.20.18/src/infer/analyser.rs:35:19
  24: <tract_core::model::graph::Graph<tract_hir::infer::fact::InferenceFact,alloc::boxed::Box<dyn tract_hir::infer::ops::InferenceOp>> as tract_hir::infer::model::InferenceModelExt>::analyse
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tract-hir-0.20.18/src/infer/model.rs:41:9
  25: <tract_core::model::graph::Graph<tract_hir::infer::fact::InferenceFact,alloc::boxed::Box<dyn tract_hir::infer::ops::InferenceOp>> as tract_hir::infer::model::InferenceModelExt>::into_typed
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tract-hir-0.20.18/src/infer/model.rs:94:9
  26: <tract_core::model::graph::Graph<tract_hir::infer::fact::InferenceFact,alloc::boxed::Box<dyn tract_hir::infer::ops::InferenceOp>> as tract_hir::infer::model::InferenceModelExt>::into_optimized
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tract-hir-0.20.18/src/infer/model.rs:148:9
  27: ican_audio::vad::load_model
             at ./audio/src/vad.rs:166:34
  28: ican_audio::vad::Vad::new
             at ./audio/src/vad.rs:144:7
  29: ican_audio::vad::VadSession::create
             at ./audio/src/vad.rs:224:24
  30: segment::main
             at ./examples/segment.rs:26:13
  31: core::ops::function::FnOnce::call_once
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/ops/function.rs:250:5
  32: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/sys_common/backtrace.rs:135:18
  33: std::rt::lang_start::{{closure}}
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/rt.rs:166:18
  34: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/ops/function.rs:284:13
      std::panicking::try::do_call
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:500:40
      std::panicking::try
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:464:19
      std::panic::catch_unwind
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panic.rs:142:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/rt.rs:148:48
      std::panicking::try::do_call
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:500:40
      std::panicking::try
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:464:19
      std::panic::catch_unwind
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panic.rs:142:14
      std::rt::lang_start_internal
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/rt.rs:148:20
  35: std::rt::lang_start
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/rt.rs:165:17
  36: _main
  37: <unknown>', examples/segment.rs:26:41
stack backtrace:
   0: rust_begin_unwind
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:593:5
   1: core::panicking::panic_fmt
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/panicking.rs:67:14
   2: core::result::unwrap_failed
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/result.rs:1651:5
   3: core::result::Result<T,E>::expect
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/result.rs:1033:23
   4: segment::main
             at ./examples/segment.rs:26:13
   5: core::ops::function::FnOnce::call_once
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
make: *** [segment] Error 101
@garyhai garyhai changed the title Failed to load model with 1 dimension. Failed to load model with 1-dimension input parameter. Sep 11, 2023
@kali
Copy link
Collaborator

kali commented Sep 11, 2023

Please give a shot at the current main branch, I have switch a default to be more lenient about ill-sized onnx network as this error was a pain. Tell me if it works better.

@garyhai
Copy link
Author

garyhai commented Sep 12, 2023

Thanks. Output with current main branch "2ea76c09678f092d00713ebbe6fdb046c0a9ad0f"

thread 'main' panicked at 'failed to create vad session: Translating node #74 "If_25" If ToTypedTranslator

Caused by:
    0: Translating node #29 "If_69" If ToTypedTranslator
    1: in output_facts invocation for If
    2: Condition failed: `self.then_body.output_fact(i)?.without_value() == self.else_body.output_fact(i)?.without_value()` (1,1,1728,F32 vs 1,1,1,1728,F32)

Stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.69/src/backtrace/libunwind.rs:93:5
      backtrace::backtrace::trace_unsynchronized
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.69/src/backtrace/mod.rs:66:5
   1: backtrace::backtrace::trace
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.69/src/backtrace/mod.rs:53:14
   2: anyhow::backtrace::capture::Backtrace::create
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.75/src/backtrace.rs:216:13
   3: anyhow::backtrace::capture::Backtrace::capture
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.75/src/backtrace.rs:204:17
   4: anyhow::error::<impl anyhow::Error>::msg
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.75/src/error.rs:83:36
   5: anyhow::ensure::render
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.75/src/ensure.rs:97:20
   6: <(A,B) as anyhow::ensure::BothDebug>::__dispatch_ensure
             at /Users/gary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.75/src/ensure.rs:20:9
   7: <tract_core::ops::logic::ite::IfThenElse as tract_core::ops::TypedOp>::output_facts
             at /Users/gary/local/tract/core/src/ops/logic/ite.rs:27:13
   8: tract_core::model::typed::<impl tract_core::model::graph::SpecialOps<tract_core::model::fact::TypedFact,alloc::boxed::Box<dyn tract_core::ops::TypedOp>> for tract_core::model::graph::Graph<tract_core::model::fact::TypedFact,alloc::boxed::Box<dyn tract_core::ops::TypedOp>>>::wire_node
             at /Users/gary/local/tract/core/src/model/typed.rs:73:32
   9: <tract_onnx::ops::logic::If as tract_hir::infer::ops::InferenceOp>::to_typed
             at /Users/gary/local/tract/onnx/src/ops/logic.rs:179:9
  10: <<tract_core::model::graph::Graph<tract_hir::infer::fact::InferenceFact,alloc::boxed::Box<dyn tract_hir::infer::ops::InferenceOp>> as tract_hir::infer::model::InferenceModelExt>::into_typed::ToTypedTranslator as tract_core::model::translator::Translate<tract_hir::infer::fact::InferenceFact,alloc::boxed::Box<dyn tract_hir::infer::ops::InferenceOp>,tract_core::model::fact::TypedFact,alloc::boxed::Box<dyn tract_core::ops::TypedOp>>>::translate_node
             at /Users/gary/local/tract/hir/src/infer/model.rs:121:35
  11: tract_core::model::translator::Translate::translate_model_with_mappings
             at /Users/gary/local/tract/core/src/model/translator.rs:34:27
  12: tract_core::model::translator::Translate::translate_model
             at /Users/gary/local/tract/core/src/model/translator.rs:23:12
  13: <tract_core::model::graph::Graph<tract_hir::infer::fact::InferenceFact,alloc::boxed::Box<dyn tract_hir::infer::ops::InferenceOp>> as tract_hir::infer::model::InferenceModelExt>::into_typed
             at /Users/gary/local/tract/hir/src/infer/model.rs:139:9
  14: <tract_onnx::ops::logic::If as tract_hir::infer::ops::InferenceOp>::to_typed
             at /Users/gary/local/tract/onnx/src/ops/logic.rs:170:25
  15: <<tract_core::model::graph::Graph<tract_hir::infer::fact::InferenceFact,alloc::boxed::Box<dyn tract_hir::infer::ops::InferenceOp>> as tract_hir::infer::model::InferenceModelExt>::into_typed::ToTypedTranslator as tract_core::model::translator::Translate<tract_hir::infer::fact::InferenceFact,alloc::boxed::Box<dyn tract_hir::infer::ops::InferenceOp>,tract_core::model::fact::TypedFact,alloc::boxed::Box<dyn tract_core::ops::TypedOp>>>::translate_node
             at /Users/gary/local/tract/hir/src/infer/model.rs:121:35
  16: tract_core::model::translator::Translate::translate_model_with_mappings
             at /Users/gary/local/tract/core/src/model/translator.rs:34:27
  17: tract_core::model::translator::Translate::translate_model
             at /Users/gary/local/tract/core/src/model/translator.rs:23:12
  18: <tract_core::model::graph::Graph<tract_hir::infer::fact::InferenceFact,alloc::boxed::Box<dyn tract_hir::infer::ops::InferenceOp>> as tract_hir::infer::model::InferenceModelExt>::into_typed
             at /Users/gary/local/tract/hir/src/infer/model.rs:139:9
  19: <tract_core::model::graph::Graph<tract_hir::infer::fact::InferenceFact,alloc::boxed::Box<dyn tract_hir::infer::ops::InferenceOp>> as tract_hir::infer::model::InferenceModelExt>::into_optimized
             at /Users/gary/local/tract/hir/src/infer/model.rs:148:9
  20: ican_audio::vad::load_model
             at ./audio/src/vad.rs:166:34
  21: ican_audio::vad::Vad::new
             at ./audio/src/vad.rs:144:7
  22: ican_audio::vad::VadSession::create
             at ./audio/src/vad.rs:221:24
  23: segment::main
             at ./examples/segment.rs:26:13
  24: core::ops::function::FnOnce::call_once
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/ops/function.rs:250:5
  25: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/sys_common/backtrace.rs:135:18
  26: std::rt::lang_start::{{closure}}
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/rt.rs:166:18
  27: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/ops/function.rs:284:13
      std::panicking::try::do_call
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:500:40
      std::panicking::try
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:464:19
      std::panic::catch_unwind
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panic.rs:142:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/rt.rs:148:48
      std::panicking::try::do_call
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:500:40
      std::panicking::try
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:464:19
      std::panic::catch_unwind
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panic.rs:142:14
      std::rt::lang_start_internal
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/rt.rs:148:20
  28: std::rt::lang_start
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/rt.rs:165:17
  29: _main
  30: <unknown>', examples/segment.rs:26:41
stack backtrace:
   0: rust_begin_unwind
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:593:5
   1: core::panicking::panic_fmt
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/panicking.rs:67:14
   2: core::result::unwrap_failed
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/result.rs:1651:5
   3: core::result::Result<T,E>::expect
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/result.rs:1033:23
   4: segment::main
             at ./examples/segment.rs:26:13
   5: core::ops::function::FnOnce::call_once
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
make: *** [segment] Error 101

@garyhai
Copy link
Author

garyhai commented Sep 12, 2023

The link of cpp version to load the onnx model:

https://github.com/snakers4/silero-vad/blob/master/examples/cpp/silero-vad-onnx.cpp

@kali
Copy link
Collaborator

kali commented Sep 14, 2023

I think I have seen this model before, and it had some structural things that were making it difficult for tract. Can you link me the onxx file so I can have a look ?

@kali
Copy link
Collaborator

kali commented Sep 14, 2023

See #1029 actually.

@garyhai
Copy link
Author

garyhai commented Sep 14, 2023

Sure. Here is the file link: https://github.com/snakers4/silero-vad/blob/master/files/silero_vad.onnx

Thanks, #1029 described it more accurately than I did.

@kali
Copy link
Collaborator

kali commented Sep 16, 2023

ok, so I guess we're looking at a duplicate of #1029 and #1062, right ?

@garyhai
Copy link
Author

garyhai commented Sep 18, 2023

Yes. I think so.

@kali
Copy link
Collaborator

kali commented Sep 18, 2023

I'm going to close this one then, as it does not bring much light to the issue.

@kali kali closed this as completed Sep 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kali @garyhai