Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TypeError: dist must be a Distribution instance #23

Open
kyanome opened this issue Jul 17, 2019 · 27 comments
Open

TypeError: dist must be a Distribution instance #23

kyanome opened this issue Jul 17, 2019 · 27 comments

Comments

@kyanome
Copy link

kyanome commented Jul 17, 2019

Hi, there.
Thank you for sharing.

I tried to build the cuda extension using "./build_toolbox.sh", but it failed....

(shaperecon) root@d31b5ed138c4:~/GenRe-ShapeHD# ./build_toolbox.sh           
Add -gencode to match all the GPU architectures you have.
Check 'https://en.wikipedia.org/wiki/CUDA#GPUs_supported' for list of architecture.
Check 'http://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html' for GPU compilation based on architecture.
nvcc -c -o calc_prob_kernel.cu.o calc_prob_kernel.cu -x cu -Xcompiler -std=c++0x -fPIC -I /root/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/lib/include -I /root/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/lib/include/TH -I /root/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/lib/include/THC -I /root/GenRe-ShapeHD/toolbox/calc_prob/calc_prob/src         -gencode arch=compute_30,code=sm_30         -gencode arch=compute_35,code=sm_35         -gencode arch=compute_52,code=sm_52         -gencode arch=compute_61,code=sm_61 
nvcc fatal   : Unknown option 'fPIC'
/root/GenRe-ShapeHD/toolbox/calc_prob
/root/anaconda3/envs/shaperecon/lib/python3.6/distutils/extension.py:131: UserWarning: Unknown Extension options: 'headers', 'package', 'relative_to', 'with_cuda'
  warnings.warn(msg)
Traceback (most recent call last):
  File "build.py", line 42, in <module>
    BuildExtension(ext)
  File "/root/anaconda3/envs/shaperecon/lib/python3.6/site-packages/setuptools/__init__.py", line 163, in __init__
    _Command.__init__(self, dist)
  File "/root/anaconda3/envs/shaperecon/lib/python3.6/distutils/cmd.py", line 57, in __init__
    raise TypeError("dist must be a Distribution instance")
TypeError: dist must be a Distribution instance
Add -gencode to match all the GPU architectures you have.
Check 'https://en.wikipedia.org/wiki/CUDA#GPUs_supported' for list of architecture.
......

Is there any way to solve this problem?
Thank you !
@kyanome
Copy link
Author

kyanome commented Jul 17, 2019

ubuntu16.04, CUDA Version: 9.0

@ztzhang
Copy link
Collaborator

ztzhang commented Jul 17, 2019

Would you mind sharing your gcc version as well?

@kyanome
Copy link
Author

kyanome commented Jul 18, 2019

Thank you for your confirmation.

This is gcc version.
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609

@frankshen07
Copy link

Same problem here with dist. I'm on ubuntu 18.04, CUDA 9.0 and gcc 6.5. Also, I'm using the latest version of setuptools. Not sure if that causes any problem. Thanks!

@marlinilram
Copy link

same problem here
ubuntu 18.04 CUDA 10.0 and gcc 7.4

@ztzhang
Copy link
Collaborator

ztzhang commented Jul 25, 2019 •
edited
Loading

Reproduced. fixing now. Should be fixed, tested under CUDA 10.1, gcc6.3.0 and Ubuntu 18.04.2.
Should be working with gcc >=5.x with CUDA 9.x / 10.x.
Please let me know if this commit does not fix the issue!

ztzhang added a commit that referenced this issue Jul 25, 2019
@ztzhang
fixing issue #23
62fc39b
@ztzhang ztzhang closed this as completed Jul 25, 2019
@marlinilram
Copy link

Hi @ztzhang , thanks for your quick fix. It compiles successfully now but when I run scripts/test_genre.sh I get undefined symbol: __cudaRegisterFatBinaryEnd. The environment set pytorch version 0.4.1 and cudatoolkit 9.0. Can I use pytorch 1.0.0 so that the cudatoolkit can be upgraded to 10.0 and matches the cuda runtime on my machine?

@ztzhang
Copy link
Collaborator

ztzhang commented Jul 26, 2019

@marlinilram Sorry we currently do not support python 1.0.0, as the custom C/c++ extension api has changed drastically. Currently we do not have the bandwidth to push an update for that.

@marlinilram
Copy link

Hi @ztzhang ,

I met a runtime error when running test_genre.sh
"RuntimeError:__ cublas runtime error : the GPU program failed to execute at /opt/conda/conda-bld/pytorch_1533672544752/work/aten/src/THC/THCBlas.cu:249`"

My env: Ubuntu 18.04.1 GCC 7.2 CUDA 9.2 with RTX 2080Ti

It seems related to RTX 2080Ti and CUDA version (pytorch/pytorch#13038), but I kept failing when compiling pytorch 0.4.1 with CUDA 10.0 which gives error
"cannot overload functions distinguished by return type alone in THCAtomics.cuh"

Did you compile pytorch 0.4.1 with CUDA 10 from source? If so, can you give some notes for that? Thanks!

@ztzhang ztzhang reopened this Jul 26, 2019
@ztzhang
Copy link
Collaborator

ztzhang commented Jul 26, 2019

@marlinilram I probably need more time to confirm on this. One thing might be causing this is that RTX2080TI is using sm code 75 (https://developer.nvidia.com/cuda-gpus). Yet I'm not exactly sure how this is related to pytorch, you might want to add --arch=sm_75 to the compiling scripts as a start.
I can run some test during the weekend, but as we do not have RTX 2080Ti at hand, we may not be able to reproduce this. But if I do, I'll see if I can push a fix.

@marlinilram
Copy link

Hi @ztzhang , did you manage to make pytorch 0.4.1 work with CUDA 10?

@ztzhang
Copy link
Collaborator

ztzhang commented Jul 29, 2019 via email

@duyentran1611
Copy link

duyentran1611 commented Oct 13, 2019 •
edited
Loading

Hi, there.
Thank you for sharing,
I followed the instructions. However, when running the command ./build_toolbox.sh, the error:

Add -gencode to match all the GPU architectures you have. Check 'https://en.wikipedia.org/wiki/CUDA#GPUs_supported' for list of architecture. Check 'http://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html' for GPU compilation based on architecture. nvcc -c -o calc_prob_kernel.cu.o calc_prob_kernel.cu -x cu -Xcompiler -fPIC -I /home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/lib/include -I /home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/lib/include/TH -I /home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/lib/include/THC -I /home/duyen/GenRe-ShapeHD/toolbox/calc_prob/calc_prob/src -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_61,code=sm_61 In file included from /home/duyen/anaconda3/envs/shaperecon/gcc/include/c++/type_traits:35:0, from /home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/lib/include/ATen/optional.h:19, from /home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/lib/include/ATen/Error.h:4, from /home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/lib/include/ATen/Allocator.h:6, from /home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/lib/include/TH/THAllocator.h:6, from /home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/lib/include/THC/THCGeneral.h:5, from /home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/lib/include/THC/THC.h:4, from calc_prob_kernel.cu:1: /home/duyen/anaconda3/envs/shaperecon/gcc/include/c++/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support is currently experimental, and must be enabled with the -std=c++11 or -std=gnu++11 compiler options. #error This file requires compiler and library support for the \ ^ /home/duyen/GenRe-ShapeHD/toolbox/calc_prob generating /tmp/tmp_006cgdf/_calc_prob_lib.c setting the current directory to '/tmp/tmp_006cgdf' running build_ext building '_calc_prob_lib' extension creating home creating home/duyen creating home/duyen/GenRe-ShapeHD creating home/duyen/GenRe-ShapeHD/toolbox creating home/duyen/GenRe-ShapeHD/toolbox/calc_prob creating home/duyen/GenRe-ShapeHD/toolbox/calc_prob/calc_prob creating home/duyen/GenRe-ShapeHD/toolbox/calc_prob/calc_prob/src gcc -pthread -B /home/duyen/anaconda3/envs/shaperecon/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA=True -I/home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include -I/home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/duyen/GenRe-ShapeHD/toolbox/calc_prob/calc_prob/src -I/home/duyen/anaconda3/envs/shaperecon/include/python3.6m -c _calc_prob_lib.c -o ./_calc_prob_lib.o -std=c99 gcc -pthread -B /home/duyen/anaconda3/envs/shaperecon/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA=True -I/home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include -I/home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/duyen/anaconda3/envs/shaperecon/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/duyen/GenRe-ShapeHD/toolbox/calc_prob/calc_prob/src -I/home/duyen/anaconda3/envs/shaperecon/include/python3.6m -c /home/duyen/GenRe-ShapeHD/toolbox/calc_prob/calc_prob/src/calc_prob.c -o ./home/duyen/GenRe-ShapeHD/toolbox/calc_prob/calc_prob/src/calc_prob.o -std=c99 gcc -pthread -shared -B /home/duyen/anaconda3/envs/shaperecon/compiler_compat -L/home/duyen/anaconda3/envs/shaperecon/lib -Wl,-rpath=/home/duyen/anaconda3/envs/shaperecon/lib -Wl,--no-as-needed -Wl,--sysroot=/ ./_calc_prob_lib.o ./home/duyen/GenRe-ShapeHD/toolbox/calc_prob/calc_prob/src/calc_prob.o /home/duyen/GenRe-ShapeHD/toolbox/calc_prob/calc_prob/src/calc_prob_kernel.cu.o -o ./_calc_prob_lib.so /home/duyen/anaconda3/envs/shaperecon/compiler_compat/ld: cannot find -lgcc_s /home/duyen/anaconda3/envs/shaperecon/compiler_compat/ld: cannot find -lpthread /home/duyen/anaconda3/envs/shaperecon/compiler_compat/ld: cannot find -lc /home/duyen/anaconda3/envs/shaperecon/compiler_compat/ld: cannot find -lgcc_s collect2: error: ld returned 1 exit status Traceback (most recent call last): File "/home/duyen/anaconda3/envs/shaperecon/lib/python3.6/distutils/unixccompiler.py", line 197, in link self.spawn(linker + ld_args) File "/home/duyen/anaconda3/envs/shaperecon/lib/python3.6/distutils/ccompiler.py", line 909, in spawn spawn(cmd, dry_run=self.dry_run) File "/home/duyen/anaconda3/envs/shaperecon/lib/python3.6/distutils/spawn.py", line 36, in spawn _spawn_posix(cmd, search_path, dry_run=dry_run) File "/home/duyen/anaconda3/envs/shaperecon/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix % (cmd, exit_status)) distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1 ......

I tried to fix this but it didn't work. Can you help me? Thanks.

@ztzhang
Copy link
Collaborator

ztzhang commented Oct 14, 2019 via email •
edited
Loading

@duyentran1611
Copy link

duyentran1611 commented Oct 14, 2019 •
edited
Loading

Hi @ztzhang . Thanks for your quick help. I fixed this error, but when I ran the test_shape.hd script, the error occurred:
`(shaperecon) duyen@duyen-Inspiron-15-7000-Gaming:~/GenRe-ShapeHD/scripts$ ./test_shapehd.sh 0
Testing Pipeline
==> Parsing arguments
Namespace(adam_beta1=0.5, adam_beta2=0.9, batch_size=1, canon_sup=False, classes='chair', dataset=None, epoch=0, epoch_batches=None, eval_at_start=False, eval_batches=None, expr_id=0, full_logdir=None, gan=None, gpu='0', input_mask='../downloads/data/test/shapehd/0044_mask.png', input_rgb='../downloads/data/test/shapehd/0044_rgb.png', log_batch=False, log_time=False, logdir=None, lr=0.0001, manual_seed=None, marrnet1_file='../downloads/models/marrnet1_with_minmax.pt', marrnet2=None, net='shapehd', net_file='../downloads/models/shapehd.pt', optim='adam', output_dir='../output/test', overwrite=True, resume=0, save_net=1, save_net_opt=False, sgd_dampening=0, sgd_momentum=0.9, suffix='{net}', tensorboard=False, vis_batches_train=10, vis_batches_vali=10, vis_every_train=1, vis_every_vali=1, vis_param_f=None, vis_workers=4, w_gan_loss=0, wdecay=0.0, workers=1)
==> Setting device
[Verbose] All designated GPU(s) free to use.
==> Setting up output directory
==> Setting up loggers
==> Setting up models
[Warning] Model loaded without optimizer states.
[Warning] Model loaded without optimizer states.
Testing ShapeHD

model parameters: 100,046,100

==> Setting up data loaders
[Verbose] Time spent in data IO initialization: 0.01s
[Verbose] # test points: 1
[Verbose] # test batches: 1
==> Testing
100%|█████████████████████████████████████████████| 1/1 [00:03<00:00, 3.47s/it]
/home/duyen/GenRe-ShapeHD/visualize/visualizer.py:136: RuntimeWarning: overflow encountered in exp
return 1 / (1 + np.exp(-x))
`
Do not know if the problem is due to the card? I am using an Nvidia GTX 1050Ti card? Can you help me with this? Thank you

@ztzhang
Copy link
Collaborator

ztzhang commented Oct 14, 2019 via email

@duyentran1611
Copy link

Hi @ztzhang
Thanks for your answer. I still don't really understand how to fix this error, despite following the instructions.
/GenRe-ShapeHD/visualize/visualizer.py:136: RuntimeWarning: overflow encountered in exp
return 1 / (1 + np.exp(-x))
`
Besides, why the input of shapehd must use RGB image and also masks, if it should only use RGB images. In the comment file there is a mask for bbox cropping. So what is bbox? And if using a image mask, how to get this mask? I don't really get it, even though your research is great and interesting.
Thank you...

@ztzhang
Copy link
Collaborator

ztzhang commented Oct 15, 2019

It seems to be a sigmoid underflow, which still gives you the reconstructed shape. It would be great if you could visualize the result first and see if it makes sense.
In our paper, we explicitly stated that we need the mask information so that we are focusing on the problem of reconstruction, not foreground/background segmentation. The bbox is the bounding box information derived from the mask. We use this to crop the images so that the objects are of similar scale in the image.

@duyentran1611
Copy link

It seems to be a sigmoid underflow, which still gives you the reconstructed shape. It would be great if you could visualize the result first and see if it makes sense.
In our paper, we explicitly stated that we need the mask information so that we are focusing on the problem of reconstruction, not foreground/background segmentation. The bbox is the bounding box information derived from the mask. We use this to crop the images so that the objects are of similar scale in the image.

Hi @ztzhang ,
I want to use the test data including the RGB image and its mask, so where can I get the test data with the mask image or how can I create the mask image as in your example? Can you guide me on this problem? Thank you.

@ztzhang
Copy link
Collaborator

ztzhang commented Oct 15, 2019

In our experiments, the mask information is directly rendered from the 3D shape. If you are testing the model, you can render the objects as well. Also you can use the Pix3D dataset, which has the mask information as part of the dataset.

@duyentran1611
Copy link

duyentran1611 commented Oct 16, 2019 via email

@ztzhang
Copy link
Collaborator

ztzhang commented Oct 16, 2019 •
edited
Loading

Hi,
I'm not sure what you mean by rendering 3D images. If you are referring to rendering depth, mask and RGB images, then you can use any 3D rendering softwares. We used Mitsuba for preparing the data, you can refer to issue #11 for the setup.

@duyentran1611
Copy link

duyentran1611 commented Oct 18, 2019 via email

@duyentran1611
Copy link

duyentran1611 commented Oct 22, 2019 via email

@ztzhang
Copy link
Collaborator

ztzhang commented Oct 23, 2019

I'm not sure I understand your question for surface normals. If you are asking what they are, you can refer to this. We use the camera coordinate to define those normal vectors.

As for network design, it is not a particular tuned choice. You can definitely explore other options.

For eval, you can modify the code to accommodate whatever format you like.

@lLi-M
Copy link

lLi-M commented Dec 7, 2019

Dear ztzhang：
I think you've done a perfect work！but,I don't know much about voxel . I want to ask you a question .
How to calculate IOU and CD of voxel ?
Thanks !

@lLi-M
Copy link

lLi-M commented Dec 7, 2019

Hi , @ztzhang Can you guide me how to calculate IoU and CD value of the voxel ?
I'm sorry to have bothered you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@marlinilram @frankshen07 @ztzhang @duyentran1611 @kyanome @lLi-M