Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The dataset type does not have the len() attribute #93

Open
cccccckt opened this issue Nov 30, 2023 · 9 comments
Open

The dataset type does not have the len() attribute #93

cccccckt opened this issue Nov 30, 2023 · 9 comments

Comments

@cccccckt
Copy link

I tried both the 2.0 and main branches , and produced the same error.

Part of the error code:
`
class JointLoader(torchdata.IterableDataset):

def __init__(self, loaders, key_dataset):
    dataset_names = []
    for key, loader in loaders.items():
        name = "{}".format(key.split('_')[0])
        setattr(self, name, loader)
        dataset_names += [name]
    self.dataset_names = dataset_names
    self.key_dataset = key_dataset

def __iter__(self):
    for batch in zip(*[getattr(self, name) for name in self.dataset_names]):
        yield {key: batch[i] for i, key in enumerate(self.dataset_names)}

def __len__(self):
    return len(getattr(self, self.key_dataset))`

Output error:

object of type 'AspectRatioGroupedDataset' has no len()
File "/home/ckt/projects/X-Decoder-main/datasets/build.py", line 80, in len
return len(getattr(self, self.key_dataset))
File "/home/ckt/projects/X-Decoder-main/pipeline/XDecoderPipeline.py", line 71, in get_dataloaders
logger.info(f'num of train samples: {len(dataloader)}')
File "/home/ckt/projects/X-Decoder-main/trainer/default_trainer.py", line 182, in init_train
self.train_dataloaders = self.pipeline.get_dataloaders(self, 'train', is_evaluation=False)
File "/home/ckt/projects/X-Decoder-main/trainer/default_trainer.py", line 226, in train
self.init_train()
File "/home/ckt/projects/X-Decoder-main/entry.py", line 48, in main
trainer.train()
File "/home/ckt/projects/X-Decoder-main/entry.py", line 55, in
main()
TypeError: object of type 'AspectRatioGroupedDataset' has no len()

屏幕截图 2023-11-30 161344

i dont know why the object of type 'AspectRatioGroupedDataset' has no len()

@EricZavier
Copy link

I have encountered the same problem. Have you resolved it?

@EricZavier
Copy link

logger.info(f'num of train samples: {len(dataloader)}')
TypeError: object of type 'AspectRatioGroupedDataset' has no len()

@cccccckt
Copy link
Author

cccccckt commented Dec 5, 2023

I have encountered the same problem. Have you resolved it?

Unfortunately, I have not solved this problem either, and I am waiting for the author to answer.
Now I'm going to focus on another project, and then I'll come back to it and try to solve this problem.
If you have any progress, please write your answers here, thanks a lot!

@MaureenZOU
Copy link
Collaborator

please install customized detectron2: git+https://github.com/MaureenZOU/detectron2-xyz.git

@cccccckt
Copy link
Author

cccccckt commented Dec 5, 2023

please install customized detectron2: git+https://github.com/MaureenZOU/detectron2-xyz.git

I will download detectron2 again according to your suggestion. I hope it will work, thank you.

@MaureenZOU
Copy link
Collaborator

pip install git+https://github.com/MaureenZOU/detectron2-xyz.git

@cccccckt
Copy link
Author

cccccckt commented Dec 5, 2023

please install customized detectron2: git+https://github.com/MaureenZOU/detectron2-xyz.git

Thank you very much! This is because of the detectron2 version issue. Moreover, thank you very much for your work on X-decoder, it gives me some inspiration.

@cccccckt
Copy link
Author

cccccckt commented Dec 5, 2023

I have encountered the same problem. Have you resolved it?

The author's answer has solved this problem, you can try it out.
There may be some installation issues, you can ask for help

@chenQ1114
Copy link

pip install git+https://github.com/MaureenZOU/detectron2-xyz.git

Hi, I have some issues when installing detectron2, here are the install command and the problem.

conda install pytorch=2.0.1=py3.9_cuda11.6 torchvision=0.15.2=py39_cu116 cudatoolkit=11.6 -c pytorch
pip install git+https://github.com/MaureenZOU/detectron2-xyz.git

  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/tmp/pip-req-build-x040bm7i/setup.py", line 147, in <module>
      setup(
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/__init__.py", line 104, in setup
      return distutils.core.setup(**attrs)
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 184, in setup
      return run_commands(dist)
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
      dist.run_commands()
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/dist.py", line 967, in run_command
      super().run_command(command)
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/mwp141/.local/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 343, in run
      self.run_command("build")
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
      self.distribution.run_command(command)
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/dist.py", line 967, in run_command
      super().run_command(command)
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 132, in run
      self.run_command(cmd_name)
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
      self.distribution.run_command(command)
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/dist.py", line 967, in run_command
      super().run_command(command)
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 91, in run
      _build_ext.run(self)
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
      self.build_extensions()
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 485, in build_extensions
      compiler_name, compiler_version = self._check_abi()
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 869, in _check_abi
      _, version = get_compiler_abi_compatibility_and_version(compiler)
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 337, in get_compiler_abi_compatibility_and_version
      if not check_compiler_ok_for_platform(compiler):
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 291, in check_compiler_ok_for_platform
      which = subprocess.check_output(['which', compiler], stderr=subprocess.STDOUT)
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/subprocess.py", line 424, in check_output
      return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
    File "/home/mwp141/.conda/envs/rr/lib/python3.9/subprocess.py", line 528, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['which', 'g++']' returned non-zero exit status 1.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for detectron2
Running setup.py clean for detectron2
Failed to build detectron2
ERROR: Could not build wheels for detectron2, which is required to install pyproject.toml-based projects

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants