not run #44
Comments
|
not run with accelerate,bitsandbytes Initialize the model with empty weights to save memorywith init_empty_weights(): |
|
Current code doesn't support quantization. We will consider this in the future. |
|
Apparently someone did try to implement quantization, however it is still a WIP and might be somewhat fiddly to use. Check out this PR if you are interested in using it: #29. |
|
It is working on low VRAM
|
|
2024-10-28 22:00:46.180633: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run but dont work with colab t4 |
|
!git clone https://github.com/Manni1000/OmniGen.git %cd OmniGen !pip install -e . !pip install gradio spaces !apt install net-tools -y !netstat -an | grep 7860 from google.colab import output !python /content/OmniGen/app.py |
|
!pip install -r /content/OmniGen/requirements.txt ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. |
|
2024-10-28 22:29:35.060112: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run |
|
gpu not run on colab t4 |
|
|
|
|
|
on colab tpu !python /content/OmniGen/app.py
This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run |
|
|
|
Collecting cloud-tpu-client==0.10 |
Hello, you should remove the # The pipeline will detect valid gpu device automatically
pipe = OmniGenPipeline.from_pretrained("Shitao/OmniGen-v1") # so just remove ', device_map=device' |
|
The problem is that I want to run it in Colab T4 and the RAM is 12, so I want to either quantize it and then use it after saving it in T4 or use it with acclrate device_map=device |
|
yuezewang it not run on gpu colab t4 Your session crashed after using all available RAM. from OmniGen import OmniGenPipeline pipe = OmniGenPipeline.from_pretrained("goodasdgood/OmniGen_quantization") Text to Imageimages = pipe( |
|
yuezewang where path to model quantization? |
|
in order to run i and bypass the error od device i just exposed the device to pipeline.py Define device globally (optional)device = torch.device("cuda" if torch.cuda.is_available() else "cpu") 90 then replace any sel.device with device as now is global and it will work |
|
Question: Does adding compression to the loading function not store the model on the hard disk? Is this method different from the method for converting a model to 4 bit? Like unshulesh |
|
Question: Does adding pressure in the loading function differ from the method of converting the model to 4bit like unsulsh |
|
NameError: name 'is_torch_npu_available' is not defined. Did you mean: 'is_torch_xla_available'? |
|
from OmniGen import OmniGenPipeline import torch Text to Imageimages = pipe( NameError: name 'is_torch_npu_available' is not defined. Did you mean: 'is_torch_xla_available'? |
wait, how are you getting it 30 times faster than mine? |
|
@ronfromhp , do you have a GPU? Running on CPU is very slow. You can try the latest code, and refer to https://github.com/VectorSpaceLab/OmniGen/blob/main/docs/inference.md#requiremented-resources for inference time. |
|
@staoxiao , I have a RTX 4050 laptop GPU 6gb. So it must be running slow because of that. But i tried the forked repo of the guy i was replying to #44 (comment) and it seems he's got a quantised model working that's like 50-100 times faster on my gpu |
|
Can you confirm that my fork is working fine for you and the generation is fast? Other viewers of my channel confirmed that it is working good. |
|
@nitinmukesh , upto a certain point, it is fast. But it fails at a certain point if i give two input images prompts and ask for a 1080p output for example. Then it falls back to 280 sec/step. I'd describe it as a sigmoid curve, if you exceed a certain threshold it becomes 50 ish times slower |
|
Has anyone had omnigen run on collab? |
not run on colab t4
from OmniGen import OmniGenPipeline
import torch
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
import transformers
transformers.logging.set_verbosity_error()
device = "cuda" if torch.cuda.is_available() else "cpu"
pipe = OmniGenPipeline.from_pretrained("Shitao/OmniGen-v1", device_map=device)
Text to Image
images = pipe(
prompt="A curly-haired man in a red shirt is drinking tea.",
height=768,
width=512,
guidance_scale=1,
seed=0,
separate_cfg_infer=True,
num_inference_steps=1,
num_images_per_prompt=1,
use_kv_cache=True
)
images[0].save("example_t2i.png") # save output PIL Image
Text to Image
images = pipe(
prompt="A curly-haired man in a red shirt is drinking tea.",
height=768,
width=512,
guidance_scale=1,
seed=0,
separate_cfg_infer=True,
num_inference_steps=1,
num_images_per_prompt=1,
use_kv_cache=True
)
images[0].save("example_t2i.png") # save output PIL Image
TypeError Traceback (most recent call last)
in <cell line: 8>()
6 transformers.logging.set_verbosity_error()
7 device = "cuda" if torch.cuda.is_available() else "cpu"
----> 8 pipe = OmniGenPipeline.from_pretrained("Shitao/OmniGen-v1", device_map=device)
9
10 # Text to Image
TypeError: OmniGenPipeline.from_pretrained() got an unexpected keyword argument 'device_map'
The text was updated successfully, but these errors were encountered: