Not enough memory to keep dbreader/write in memory!

[htran76]

Expected Behavior

mmseqs createindex "colabfold_envdb_202108_db"

Current Behavior

Not enough memory to keep dbreader/write in memory!
Memory limit: 14845796352 dbreader/writer need: 22753642488
Error: indexdb died

Steps to Reproduce (for bugs)

executing setup_databases.sh

MMseqs Output (for bugs)

• ARIA_NUM_CONN=8
• WORKDIR=database/
• PDB_SERVER=rsync.wwpdb.org::ftp
• PDB_PORT=33444
• PDB_AWS_DOWNLOAD=
• PDB_AWS_SNAPSHOT=20240101
• UNIREF30DB=uniref30_2302
• MMSEQS_NO_INDEX=
• cd database/
• STRATEGY=
• hasCommand aria2c
• command -v aria2c
• STRATEGY=' ARIA'
• hasCommand curl
• command -v curl
• STRATEGY=' ARIA CURL'
• hasCommand wget
• command -v wget
• STRATEGY=' ARIA CURL WGET'
• '[' ' ARIA CURL WGET' = '' ']'
• '[' -n '' ']'
• export MMSEQS_FORCE_MERGE=1
• MMSEQS_FORCE_MERGE=1
• '[' '!' -f UNIREF30_READY ']'
• '[' '!' -f COLABDB_READY ']'
• mmseqs tsv2exprofiledb colabfold_envdb_202108 colabfold_envdb_202108_db
  colabfold_envdb_202108_db exists and will be overwritten
  tsv2exprofiledb colabfold_envdb_202108 colabfold_envdb_202108_db

MMseqs Version: 15.6f452
Verbosity 3

• '[' -z '' ']'
• mmseqs createindex colabfold_envdb_202108_db tmp2 --remove-tmp-files 1 --split 1
  createindex colabfold_envdb_202108_db tmp2 --remove-tmp-files 1 --split 1

MMseqs Version: 15.6f452
Seed substitution matrix aa:VTML80.out,nucl:nucleotide.out
k-mer length 0
Alphabet size aa:21,nucl:5
Compositional bias 1
Compositional bias 1
Max sequence length 65535
Max results per query 300
Mask residues 1
Mask residues probability 0.9
Mask lower case residues 0
Spaced k-mers 1
Spaced k-mer pattern
Sensitivity 7.5
k-score seq:0,prof:0
Check compatible 0
Search type 0
Split database 1
Split memory limit 0
Index subset 0
Verbosity 3
Threads 12
Min codons in orf 30
Max codons in length 32734
Max orf gaps 2147483647
Contig start mode 2
Contig end mode 2
Orf start mode 1
Forward frames 1,2,3
Reverse frames 1,2,3
Translation table 1
Translate orf 0
Use all table starts false
Offset of numeric ids 0
Create lookup 0
Compressed 0
Add orf stop false
Overlap between sequences 0
Sequence split mode 1
Header split mode 0
Strand selection 1
Remove temporary files true

createindex colabfold_envdb_202108_db tmp2 --remove-tmp-files 1 --split 1

MMseqs Version: 15.6f452
Seed substitution matrix aa:VTML80.out,nucl:nucleotide.out
k-mer length 0
Alphabet size aa:21,nucl:5
Compositional bias 1
Compositional bias 1
Max sequence length 65535
Max results per query 300
Mask residues 1
Mask residues probability 0.9
Mask lower case residues 0
Spaced k-mers 1
Spaced k-mer pattern
Sensitivity 7.5
k-score seq:0,prof:0
Check compatible 0
Search type 0
Split database 1
Split memory limit 0
Index subset 0
Verbosity 3
Threads 12
Min codons in orf 30
Max codons in length 32734
Max orf gaps 2147483647
Contig start mode 2
Contig end mode 2
Orf start mode 1
Forward frames 1,2,3
Reverse frames 1,2,3
Translation table 1
Translate orf 0
Use all table starts false
Offset of numeric ids 0
Create lookup 0
Compressed 0
Add orf stop false
Overlap between sequences 0
Sequence split mode 1
Header split mode 0
Strand selection 1
Remove temporary files true

indexdb colabfold_envdb_202108_db colabfold_envdb_202108_db --seed-sub-mat 'aa:VTML80.out,nucl:nucleotide.out' -k 0 --alph-size aa:21,nucl:5 --comp-bias-corr 1 --comp-bias-corr-scale 1 --max-seq-len 65535 --max-seqs 300 --mask 1 --mask-prob 0.9 --mask-lower-case 0 --spaced-kmer-mode 1 -s 7.5 --k-score seq:0,prof:0 --check-compatible 0 --search-type 0 --split 1 --split-memory-limit 0 --index-subset 0 -v 3 --threads 12

Not enough memory to keep dbreader/write in memory!
Memory limit: 14845796352 dbreader/writer need: 22753642488
Error: indexdb died

Context

I am trying to setup my laptop for localcolab but keep running into "Not enough memory to keep dbreader/write in memory!"
I've tried --split-memory-limit 10G as my system only has 16GB of RAM but the same message appears. I have also tried splitting the index using --split 1 but the same message appears.

I've tried changing wsl.config memory to greater than 16GB but --free m shows that I still have only 16GB. I have also tried increasing swap to 16GB but this too does not resolve the issue.

Your Environment

WSL2 Ubuntu 24.04.1 LTS
16GB RAM
NVIDIA GeForce 1050Ti
Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz 2.21 GHz (6 cores/ 12 logical processor)

[milot-mirdita]

Please use the MSA server for this set up. The local database search will likely not work at with less than 64-128GB RAM.