How to install and run? Basic Tutorial?

[csimlinger]

Hello,

I have a hard time to run the program locally on my Linux machine (RTX4090).

First I tried to create a virtual env with my default CUDA Version (12.6.).
I installed the pre-requirements and the requirements.

Afterwards I run the script with "pyhton app.py". It then complained that "spaces" is not found, so I installed it via pip install spaces

Now I was able to run the script partly, but it ended with this error:

Successfully built mmpose
Installing collected packages: mmpose
  Attempting uninstall: mmpose
    Found existing installation: mmpose 0.28.0
    Uninstalling mmpose-0.28.0:
      Successfully uninstalled mmpose-0.28.0
Successfully installed mmpose-0.28.0
cp: cannot create regular file '/home/user/.pyenv/versions/3.9.20/lib/python3.9/site-packages/torchgeometry/core/conversions.py': No such file or directory
Traceback (most recent call last):
  File "/home/username/Tools/TANGO/SMPLer-X/app.py", line 127, in <module>
    from main.inference import Inferer
  File "/home/username/Tools/TANGO/SMPLer-X/main/inference.py", line 17, in <module>
    from mmdet.apis import init_detector, inference_detector
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmdet/apis/__init__.py", line 2, in <module>
    from .det_inferencer import DetInferencer
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmdet/apis/det_inferencer.py", line 22, in <module>
    from mmdet.evaluation import INSTANCE_OFFSET
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmdet/evaluation/__init__.py", line 3, in <module>
    from .metrics import *  # noqa: F401,F403
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmdet/evaluation/metrics/__init__.py", line 5, in <module>
    from .coco_metric import CocoMetric
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmdet/evaluation/metrics/coco_metric.py", line 16, in <module>
    from mmdet.datasets.api_wrappers import COCO, COCOeval, COCOevalMP
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmdet/datasets/__init__.py", line 26, in <module>
    from .utils import get_loading_pipeline
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmdet/datasets/utils.py", line 5, in <module>
    from mmdet.datasets.transforms import LoadAnnotations, LoadPanopticAnnotations
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmdet/datasets/transforms/__init__.py", line 6, in <module>
    from .formatting import (ImageToTensor, PackDetInputs, PackReIDInputs,
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmdet/datasets/transforms/formatting.py", line 11, in <module>
    from mmdet.structures.bbox import BaseBoxes
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmdet/structures/bbox/__init__.py", line 2, in <module>
    from .base_boxes import BaseBoxes
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmdet/structures/bbox/base_boxes.py", line 9, in <module>
    from mmdet.structures.mask.structures import BitmapMasks, PolygonMasks
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmdet/structures/mask/__init__.py", line 3, in <module>
    from .structures import (BaseInstanceMasks, BitmapMasks, PolygonMasks,
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmdet/structures/mask/structures.py", line 12, in <module>
    from mmcv.ops.roi_align import roi_align
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmcv/ops/__init__.py", line 3, in <module>
    from .active_rotated_filter import active_rotated_filter
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmcv/ops/active_rotated_filter.py", line 10, in <module>
    ext_module = ext_loader.load_ext(
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmcv/utils/ext_loader.py", line 13, in load_ext
    ext = importlib.import_module('mmcv.' + name)
  File "/home/username/miniconda3/envs/tango/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: /home/username/miniconda3/envs/tango/lib/python3.9/site-packages/mmcv/_ext.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi
cd ./SMPLer-X/ && python app.py --video_folder_path ../outputs/tmpvideo/ --data_save_path ../outputs/tmpdata/ --json_save_path ../outputs/save_video.json && cd ..
Traceback (most recent call last):
  File "/home/username/Tools/TANGO/./create_graph.py", line 478, in <module>
    graph = create_graph(json_path, smplx_model)
  File "/home/username/Tools/TANGO/./create_graph.py", line 129, in create_graph
    data_meta = json.load(open(json_path, "r"))
FileNotFoundError: [Errno 2] No such file or directory: './outputs/save_video.json'
Some weights of the model checkpoint at facebook/wav2vec2-base-960h were not used when initializing Wav2Vec2Model: ['lm_head.weight', 'lm_head.bias']
- This IS expected if you are initializing Wav2Vec2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing Wav2Vec2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of Wav2Vec2Model were not initialized from the model checkpoint at facebook/wav2vec2-base-960h and are newly initialized: ['wav2vec2.masked_spec_embed']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Some weights of the model checkpoint at facebook/wav2vec2-base-960h were not used when initializing Wav2Vec2Model: ['lm_head.weight', 'lm_head.bias']
- This IS expected if you are initializing Wav2Vec2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing Wav2Vec2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of Wav2Vec2Model were not initialized from the model checkpoint at facebook/wav2vec2-base-960h and are newly initialized: ['wav2vec2.masked_spec_embed']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Some weights of the model checkpoint at facebook/wav2vec2-base-960h were not used when initializing Wav2Vec2Model: ['lm_head.weight', 'lm_head.bias']
- This IS expected if you are initializing Wav2Vec2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing Wav2Vec2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of Wav2Vec2Model were not initialized from the model checkpoint at facebook/wav2vec2-base-960h and are newly initialized: ['wav2vec2.masked_spec_embed']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Some weights of the model checkpoint at facebook/wav2vec2-base-960h were not used when initializing Wav2Vec2Model: ['lm_head.weight', 'lm_head.bias']
- This IS expected if you are initializing Wav2Vec2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing Wav2Vec2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of Wav2Vec2Model were not initialized from the model checkpoint at facebook/wav2vec2-base-960h and are newly initialized: ['wav2vec2.masked_spec_embed']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Some weights of Wav2Vec2ForCTC were not initialized from the model checkpoint at facebook/wav2vec2-base-960h and are newly initialized: ['wav2vec2.masked_spec_embed']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertModel: ['cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Traceback (most recent call last):
  File "/home/username/Tools/TANGO/app.py", line 771, in <module>
    demo = make_demo()
  File "/home/username/Tools/TANGO/app.py", line 756, in make_demo
    gr.Examples(
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/gradio/helpers.py", line 81, in create_examples
    examples_obj.create()
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/gradio/helpers.py", line 340, in create
    self._start_caching()
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/gradio/helpers.py", line 391, in _start_caching
    client_utils.synchronize_async(self.cache)
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/gradio_client/utils.py", line 855, in synchronize_async
    return fsspec.asyn.sync(fsspec.asyn.get_loop(), func, *args, **kwargs)  # type: ignore
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/fsspec/asyn.py", line 103, in sync
    raise return_result
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/fsspec/asyn.py", line 56, in _runner
    result[0] = await coro
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/gradio/helpers.py", line 517, in cache
    prediction = await Context.root_block.process_api(
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/gradio/blocks.py", line 1935, in process_api
    result = await self.call_function(
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/gradio/blocks.py", line 1520, in call_function
    prediction = await anyio.to_thread.run_sync(  # type: ignore
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 2441, in run_sync_in_worker_thread
    return await future
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 943, in run
    result = context.run(func, *args)
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/gradio/utils.py", line 826, in wrapper
    response = f(*args, **kwargs)
  File "/home/username/Tools/TANGO/app.py", line 592, in tango
    result = test_fn(model, device, 0, cfg.data.test_meta_paths, test_path, cfg, audio_path, create_graph=create_graph)
  File "/home/username/Tools/TANGO/app.py", line 186, in test_fn
    graph = igraph.Graph.Read_Pickle(fname=pool_path)
  File "/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/igraph/io/files.py", line 223, in _construct_graph_from_pickle_file
    raise IOError(
OSError: Cannot load file. If fname is a file name, that filename may be incorrect.

I tried then to create another conda env with the stated CUDA version 11.8. - now even more issues appeared, for example:

(tango) username@DataCruncher:~/Tools/TANGO$ python app.py
Traceback (most recent call last):
  File "/home/username/Tools/TANGO/app.py", line 13, in <module>
    from moviepy.tools import verbose_print
ImportError: cannot import name 'verbose_print' from 'moviepy.tools' (/home/username/miniconda3/envs/tango/lib/python3.9/site-packages/moviepy/tools.py)

... could someone tell me step by step how to install and run the script run on Linux RTX4090?
My goal is to use my own audio + video in the end.
Currently I run from one problem into the next.

Thanks!

[msh1376]

I have this problem too. I run it on three different windows os with different config. finaly I saw the UI to select speaker and Audio but when I select my Audio and video from folder it generate speaker1 video automatically! no one point to this Issue!

[Longyee66]

在服务器上能够跑起来，生成视频很慢

[H-Liu1997]

hi @csimlinger you dont need to include space on your local. you may comment it out. will it work without this line?

[yikexiaoxing]

@csimlinger moviepy版本问题 请重新安装 pip install moviepy==1.0.3