Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OSError: [Errno 12] Cannot allocate memory #38

Open
emircankaaya opened this issue Aug 11, 2024 · 8 comments
Open

OSError: [Errno 12] Cannot allocate memory #38

emircankaaya opened this issue Aug 11, 2024 · 8 comments

Comments

@emircankaaya
Copy link

(SubPhaser) eck@DESKTOP-ED16O0T:/SubPhaser$ subphaser -i genome.fasta.gz -c sg.config
24-08-11 22:55:26 [WARNING] Grid computing is not available because DRMAA not installed: No module named 'drmaa'
24-08-11 22:55:26 [INFO] No DRMAA (see https://github.com/pygridtools/drmaa-python), Switching to local mode.
24-08-11 22:55:28 [WARNING] Grid computing is not available because DRMAA not installed: No module named 'drmaa'
24-08-11 22:55:28 [INFO] No DRMAA (see https://github.com/pygridtools/drmaa-python), Switching to local mode.
24-08-11 22:55:28 [INFO] Command: /home/eck/miniconda3/envs/SubPhaser/bin/subphaser -i genome.fasta.gz -c sg.config
24-08-11 22:55:28 [INFO] Version: 1.2.6
24-08-11 22:55:28 [INFO] Arguments: {'genomes': ['genome.fasta.gz'], 'sg_cfgs': ['sg.config'], 'labels': None, 'no_label': False, 'target': None, 'sg_assigned': None, 'sep': '|', 'custom_features': None, 'prefix': None, 'outdir': 'phase-results', 'tmpdir': 'tmp', 'colors': ['#f9c00c', '#00b9f1', '#7200da', '#f9320c', '#00b8a9', '#F4A460', '#009999', '#00C02E', '#980000', '#00ffff', '#0000ff', '#ff0000', '#4a86e8', '#ff9900', '#ffff00', '#00ff00', '#9900ff', '#ff00ff', '#20124d', '#274e13', '#000000', '#cccccc', '#7f6000', '#a64d79', '#6aa84f', '#fff2cc', '#47a952', '#3ea6b6', '#a5b805', '#8f9276', '#ca8d7c'], 'k': 15, 'min_fold': 2, 'min_freq': 200, 'baseline': 1, 'ratio': 1, 'lower_count': 3, 'min_prop': None, 'max_freq': 1000000000.0, 'max_prop': None, 'low_mem': None, 'by_count': False, 're_filter': False, 'nsg': None, 'replicates': 1000, 'jackknife': 50, 'max_pval': 0.05, 'test_method': 'ttest_ind', 'figfmt': 'pdf', 'heatmap_colors': ('green', 'black', 'red'), 'heatmap_options': "Rowv=T,Colv=T,scale='col',dendrogram='row',labCol=F,trace='none',key=T,key.title=NA,density.info='density',main=NA,xlab='Differential kmers',margins=c(2.5,12)", 'just_core': False, 'disable_ltr': False, 'ltr_detectors': ['ltr_harvest'], 'ltr_finder_options': '-w 2 -D 15000 -d 1000 -L 7000 -l 100 -p 20 -C -M 0.8', 'ltr_harvest_options': '-seqids yes -similar 80 -vic 10 -seed 20 -minlenltr 100 -maxlenltr 7000 -mintsd 4 -maxtsd 6', 'tesorter_options': '-db rexdb -dp2', 'all_ltr': False, 'intact_ltr': False, 'exclude_exchanges': False, 'non_specific': False, 'mu': 1.3e-08, 'disable_ltrtree': False, 'subsample': 1000, 'ltr_domains': ['INT', 'RT', 'RH'], 'trimal_options': '-automated1', 'tree_method': 'FastTree', 'tree_options': '', 'ggtree_options': "branch.length='none', layout='circular'", 'disable_circos': False, 'window_size': 1000000, 'disable_blocks': False, 'aligner': 'minimap2', 'aligner_options': '-x asm20 -n 10', 'min_block': 100000, 'alt_cfgs': None, 'chr_ordered': None, 'ncpu': 4, 'max_memory': '11.5G', 'cleanup': False, 'overwrite': False}
24-08-11 22:55:28 [INFO] Target chromosomes: ['chr01a', 'chr01b', 'chr01c', 'chr01d', 'chr02a', 'chr02b', 'chr02c', 'chr02d', 'chr03a', 'chr03b', 'chr03c', 'chr03d', 'chr04a', 'chr04b', 'chr04c', 'chr04d', 'chr05a', 'chr05b', 'chr05c', 'chr05d', 'chr06a', 'chr06b', 'chr06c', 'chr06d', 'chr07a', 'chr07b', 'chr07c', 'chr07d']
24-08-11 22:55:28 [INFO] Splitting genomes by chromosome into /home/eck/SubPhaser/tmp/
24-08-11 22:55:28 [INFO] Check point file: /home/eck/SubPhaser/tmp/split.ok exists; skip this step
24-08-11 22:55:29 [INFO] Chromosomes: ['chr01a', 'chr01b', 'chr01c', 'chr01d', 'chr02a', 'chr02b', 'chr02c', 'chr02d', 'chr03a', 'chr03b', 'chr03c', 'chr03d', 'chr04a', 'chr04b', 'chr04c', 'chr04d', 'chr05a', 'chr05b', 'chr05c', 'chr05d', 'chr06a', 'chr06b', 'chr06c', 'chr06d', 'chr07a', 'chr07b', 'chr07c', 'chr07d']
24-08-11 22:55:29 [INFO] Chromosome Number: 28
24-08-11 22:55:29 [INFO] CONFIG: [[['chr01a'], ['chr01b'], ['chr01c'], ['chr01d']], [['chr02a'], ['chr02b'], ['chr02c'], ['chr02d']], [['chr03a'], ['chr03b'], ['chr03c'], ['chr03d']], [['chr04a'], ['chr04b'], ['chr04c'], ['chr04d']], [['chr05a'], ['chr05b'], ['chr05c'], ['chr05d']], [['chr06a'], ['chr06b'], ['chr06c'], ['chr06d']], [['chr07a'], ['chr07b'], ['chr07c'], ['chr07d']]]
24-08-11 22:55:29 [INFO] Genome size: 1,892,584,152 bp
24-08-11 22:55:29 [INFO] ###Step: Kmer Count
24-08-11 22:55:29 [INFO] Counting kmer by jellyfish
24-08-11 22:55:29 [INFO] Start Pool with 4 process(es)
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr01a.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr01c.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr02a.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr02c.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr01b.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr01d.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr02b.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr02d.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr03a.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr03c.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr04a.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr04c.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr03b.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr03d.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr04b.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr04d.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr05a.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr05c.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr06a.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr06c.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr05b.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr05d.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr06b.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr06d.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr07a.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr07c.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr07b.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr07d.fasta_15.fa.ok exists; skip this step
24-08-11 22:55:29 [INFO] Loading kmer matrix from jellyfish
24-08-11 22:55:29 [INFO] Start Pool with 4 process(es)
24-08-12 00:29:31 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr01a.fasta_15.fa
24-08-12 00:30:23 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr01b.fasta_15.fa
24-08-12 00:31:03 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr01c.fasta_15.fa
24-08-12 00:31:15 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr01d.fasta_15.fa
24-08-12 00:32:18 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr02a.fasta_15.fa
24-08-12 00:34:03 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr02b.fasta_15.fa
24-08-12 00:34:12 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr02c.fasta_15.fa
24-08-12 00:35:55 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr02d.fasta_15.fa
24-08-12 00:36:03 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr03a.fasta_15.fa
24-08-12 00:36:07 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr03b.fasta_15.fa
24-08-12 00:36:11 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr03c.fasta_15.fa
24-08-12 00:38:07 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr03d.fasta_15.fa
24-08-12 00:38:10 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr04a.fasta_15.fa
24-08-12 00:38:14 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr04b.fasta_15.fa
24-08-12 00:38:17 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr04c.fasta_15.fa
24-08-12 00:38:20 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr04d.fasta_15.fa
24-08-12 00:38:23 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr05a.fasta_15.fa
24-08-12 00:38:27 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr05b.fasta_15.fa
24-08-12 00:38:43 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr05c.fasta_15.fa
24-08-12 00:38:46 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr05d.fasta_15.fa
24-08-12 00:42:05 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr06a.fasta_15.fa
24-08-12 00:42:17 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr06b.fasta_15.fa
24-08-12 00:42:34 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr06c.fasta_15.fa
24-08-12 00:42:48 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr06d.fasta_15.fa
24-08-12 00:43:12 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr07a.fasta_15.fa
24-08-12 00:43:41 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr07b.fasta_15.fa
24-08-12 00:44:07 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr07c.fasta_15.fa
24-08-12 00:44:49 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr07d.fasta_15.fa
24-08-12 00:46:21 [INFO] 28129798 kmers in total
24-08-12 00:46:21 [INFO] Filtering differential kmers
Traceback (most recent call last):
File "/home/eck/miniconda3/envs/SubPhaser/bin/subphaser", line 33, in
sys.exit(load_entry_point('subphaser==1.2.6', 'console_scripts', 'subphaser')())
File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/site-packages/subphaser-1.2.6-py3.8.egg/subphaser/main.py", line 797, in main
pipeline.run()
File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/site-packages/subphaser-1.2.6-py3.8.egg/subphaser/main.py", line 422, in run
d_mat = dumps.filter(d_mat, lengths, self.sgs, outfig=histfig, #d_targets=d_targets,
File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/site-packages/subphaser-1.2.6-py3.8.egg/subphaser/Jellyfish.py", line 491, in filter
for kmer, freqs, tot_freq in pool_func(_filter_kmer, args, self.ncpu,
File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/site-packages/subphaser-1.2.6-py3.8.egg/subphaser/RunCmdsMP.py", line 336, in pool_func
pool = multiprocessing.Pool(processors)
File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/context.py", line 119, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/pool.py", line 212, in init
self._repopulate_pool()
File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/pool.py", line 303, in _repopulate_pool
return self._repopulate_pool_static(self._ctx, self.Process,
File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/pool.py", line 326, in _repopulate_pool_static
w.start()
File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/popen_fork.py", line 19, in init
self._launch(process_obj)
File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/popen_fork.py", line 70, in _launch
self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
(SubPhaser) eck@DESKTOP-ED16O0T:/SubPhaser$
@zhangrengang
Copy link
Owner

zhangrengang commented Aug 12, 2024 via email

@emircankaaya
Copy link
Author

Unfortunately, the same error was raised. Do you know any alternative way?

@zhangrengang
Copy link
Owner

zhangrengang commented Aug 12, 2024 via email

@emircankaaya
Copy link
Author

Solved

@LikeLucasLab
Copy link

@emircankaaya I have the same problem how have you solved the problem ?

@emircankaaya
Copy link
Author

emircankaaya commented Aug 16, 2024 via email

@emircankaaya
Copy link
Author

emircankaaya commented Aug 16, 2024 via email

@LikeLucasLab
Copy link

@emircankaaya thanks your suggestions worked now it is running

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@zhangrengang @LikeLucasLab @emircankaaya