Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Search result error occurs after docker creation of Rest VV version 2.2.0 #105

Open
yebean-k opened this issue Jul 17, 2023 · 5 comments
Open

Comments

@yebean-k
Copy link

yebean-k commented Jul 17, 2023

Hi,

I'm trying to install rest VV (version 2.2.0).
Since the Docker build finished without errors, I tested the Variant Validator inside Docker.
Then I got this error message.
I think the postgresql connection is the problem.
Can you help me??

Thank you.

query
variant_validator.py -v 'NC_000017.11:g.50198002C>A' -g GRCh38 -t mane -s individual -f json -m -o stdout

error message
Lost connection to postgresql://uta_admin:[email protected]:54320/vvta/vvta_2022_11; attempting reconnect
Reconnected to postgresql://uta_admin:[email protected]:54320/vvta/vvta_2022_11
Lost connection to postgresql://uta_admin:[email protected]:54320/vvta/vvta_2022_11; attempting reconnect
Reconnected to postgresql://uta_admin:[email protected]:54320/vvta/vvta_2022_11
Lost connection to postgresql://uta_admin:[email protected]:54320/vvta/vvta_2022_11; attempting reconnect
Reconnected to postgresql://uta_admin:[email protected]:54320/vvta/vvta_2022_11
Lost connection to postgresql://uta_admin:[email protected]:54320/vvta/vvta_2022_11; attempting reconnect
Reconnected to postgresql://uta_admin:[email protected]:54320/vvta/vvta_2022_11
Lost connection to postgresql://uta_admin:[email protected]:54320/vvta/vvta_2022_11; attempting reconnect
Reconnected to postgresql://uta_admin:[email protected]:54320/vvta/vvta_2022_11
CRITICAL: <class 'RuntimeError'> generator didn't stop after throw()
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/vvhgvs/dataproviders/uta.py", line 585, in _get_cursor
yield cur
File "/usr/local/lib/python3.10/site-packages/vvhgvs/dataproviders/uta.py", line 220, in _fetchall
cur.execute(sql, *args)
File "/usr/local/lib/python3.10/site-packages/psycopg2/extras.py", line 146, in execute
return super().execute(query, vars)
psycopg2.errors.ObjectNotInPrerequisiteState: materialized view "tx_exon_aln_mv" has not been populated
HINT: Use the REFRESH MATERIALIZED VIEW command.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/VariantValidator/modules/vvMixinCore.py", line 1343, in validate
lifted_response = liftover(genomic_position_info[g_p_key]['hgvs_genomic_description'],
File "/usr/local/lib/python3.10/site-packages/VariantValidator/modules/liftover.py", line 167, in liftover
options = validator.hdp.get_tx_mapping_options(tx)
File "/usr/local/lib/python3.10/site-packages/vvhgvs/decorators/lru_cache.py", line 186, in wrapper
result = user_function(*args, **kwds)
File "/usr/local/lib/python3.10/site-packages/vvhgvs/dataproviders/uta.py", line 453, in get_tx_mapping_options
rows = self._fetchall(self._queries['tx_mapping_options'], [tx_ac])
File "/usr/local/lib/python3.10/site-packages/vvhgvs/dataproviders/uta.py", line 219, in _fetchall
with self._get_cursor() as cur:
File "/usr/local/lib/python3.10/contextlib.py", line 185, in exit
raise RuntimeError("generator didn't stop after throw()")
RuntimeError: generator didn't stop after throw()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/bin/variant_validator.py", line 48, in
output = validator.validate(variant, args.genome, args.transcripts)
File "/usr/local/lib/python3.10/site-packages/VariantValidator/modules/vvMixinCore.py", line 1442, in validate
raise fn.VariantValidatorError('Validation error')
VariantValidator.modules.utils.VariantValidatorError: Validation error

@Peter-J-Freeman
Copy link
Contributor

Hi @yebean-k . This sounds like the postgres docker container did not build correctly. I think the best thing to do is re-install and see if this fixes the issue.

@Peter-J-Freeman
Copy link
Contributor

I also recommend building from master. We do periodic bug fixes between major releases

@yebean-k
Copy link
Author

Hi,

I recently cloned GIT and tried to install it, so it seems to be the latest version.
And, I rebuilding vvta docker as you recommended but got the same error.
I think it installed well without any errors, but I don't know where the problem occurred.
For reference, I am creating a container after �building an image without using Docker compile.

- query
docker build --tag vv_ta:2.2.0 -f vvta_docker.df .

- log
Sending build context to Docker daemon 2.829MB
Step 1/12 : FROM postgres:14.7
14.7: Pulling from library/postgres
9e3ea8720c6d: Pull complete
7782b3e1be4b: Pull complete
247ec4ff783a: Pull complete
f7ead6900700: Pull complete
e7afdbe9a191: Pull complete
3ef71fe7cece: Pull complete
1459ebb56be5: Pull complete
3595124f6861: Pull complete
b4486f48017f: Pull complete
a63c7ed11505: Pull complete
0146526465ec: Pull complete
039e22ce4453: Pull complete
58942c7b9396: Pull complete
Digest: sha256:5ac16ee311340b09e3670d660c76f77a611202fd07b05d486e934eece99bea7c
Status: Downloaded newer image for postgres:14.7
---> 1c159088a878
Step 2/12 : ENV POSTGRES_DB=vvta
---> Running in 55ff09b36963
Removing intermediate container 55ff09b36963
---> d698ce14931e
Step 3/12 : ENV POSTGRES_USER=uta_admin
---> Running in 0f1770d0b7b0
Removing intermediate container 0f1770d0b7b0
---> 686736ac5f3e
Step 4/12 : ENV POSTGRES_PASSWORD=uta_admin
---> Running in f1d404659769
Removing intermediate container f1d404659769
---> 7f883d55a5c7
Step 5/12 : RUN apt-get update && apt-get install -y wget && rm -rf /var/lib/apt/lists/*
---> Running in 66fdfe7e5f7d
Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 Packages [8,183 kB]
Get:5 http://apt.postgresql.org/pub/repos/apt bullseye-pgdg InRelease [123 kB]
Get:6 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [252 kB]
Get:7 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [14.8 kB]
Get:8 http://apt.postgresql.org/pub/repos/apt bullseye-pgdg/14 amd64 Packages [2,563 B]
Get:9 http://apt.postgresql.org/pub/repos/apt bullseye-pgdg/main amd64 Packages [282 kB]
Fetched 9,066 kB in 3s (2,999 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
ca-certificates libpsl5 publicsuffix
The following NEW packages will be installed:
ca-certificates libpsl5 publicsuffix wget
0 upgraded, 4 newly installed, 0 to remove and 7 not upgraded.
Need to get 1,307 kB of archives.
After this operation, 4,392 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bullseye/main amd64 ca-certificates all 20210119 [158 kB]
Get:2 http://deb.debian.org/debian bullseye/main amd64 libpsl5 amd64 0.21.0-1.2 [57.3 kB]
Get:3 http://deb.debian.org/debian bullseye/main amd64 wget amd64 1.21-1+deb11u1 [964 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 publicsuffix all 20220811.1734-0+deb11u1 [127 kB]
ESC[91mdebconf: delaying package configuration, since apt-utils is not installed
ESC[0mFetched 1,307 kB in 0s (3,186 kB/s)
Selecting previously unselected package ca-certificates.
(Reading database ... ^M(Reading database ... 5%^M(Reading database ... 10%^M(Reading database ... 15%^M(Reading database ... 20%^M(Reading database ... 25%^M(Reading database ... 30%^M(Reading database ... 35%^M(Reading database ... 40%^M(Readi
ng database ... 45%^M(Reading database ... 50%^M(Reading database ... 55%^M(Reading database ... 60%^M(Reading database ... 65%^M(Reading database ... 70%^M(Reading database ... 75%^M(Reading database ... 80%^M(Reading database ... 85%^M(Reading
database ... 90%^M(Reading database ... 95%^M(Reading database ... 100%^M(Reading database ... 12116 files and directories currently installed.)
Preparing to unpack .../ca-certificates_20210119_all.deb ...
Unpacking ca-certificates (20210119) ...
Selecting previously unselected package libpsl5:amd64.
Preparing to unpack .../libpsl5_0.21.0-1.2_amd64.deb ...
Unpacking libpsl5:amd64 (0.21.0-1.2) ...
Selecting previously unselected package wget.
Preparing to unpack .../wget_1.21-1+deb11u1_amd64.deb ...
Unpacking wget (1.21-1+deb11u1) ...
Selecting previously unselected package publicsuffix.
Preparing to unpack .../publicsuffix_20220811.1734-0+deb11u1_all.deb ...
Unpacking publicsuffix (20220811.1734-0+deb11u1) ...
Setting up libpsl5:amd64 (0.21.0-1.2) ...
Setting up wget (1.21-1+deb11u1) ...
Setting up ca-certificates (20210119) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
Updating certificates in /etc/ssl/certs...
129 added, 0 removed; done.
Setting up publicsuffix (20220811.1734-0+deb11u1) ...
Processing triggers for libc-bin (2.31-13+deb11u6) ...
Processing triggers for ca-certificates (20210119) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Removing intermediate container aecb671473b6
---> d5040de90791
Step 6/12 : RUN echo "shared_buffers = 2GB" > /docker-entrypoint-initdb.d/postgresql.conf
---> Running in 97f4b3829f83
Removing intermediate container 97f4b3829f83
---> 1beed96b7534
Step 7/12 : RUN wget https://www528.lamp.le.ac.uk/vvdata/vvta/vvta_2022_11_no_seq.sql.gz -O input_file.sql.gz
---> Running in 86a0284cccbb
ESC[91m--2023-07-19 00:26:42-- https://www528.lamp.le.ac.uk/vvdata/vvta/vvta_2022_11_no_seq.sql.gz
ESC[0mESC[91mResolving www528.lamp.le.ac.uk (www528.lamp.le.ac.uk)... ESC[0mESC[91m143.210.56.21
Connecting to www528.lamp.le.ac.uk (www528.lamp.le.ac.uk)|143.210.56.21|:443... ESC[0mESC[91mconnected.
ESC[0mESC[91mHTTP request sent, awaiting response... ESC[0mESC[91m200 OK
Length: 201476885 (192M) [application/x-gzip]
Saving to: ‘input_file.sql.gz’
ESC[0mESC[91m
0K ........ESC[0mESC[91m.. ......ESC[0mESC[91m.... .......... ..ESC[0mESC[91m........ .......... 0% 89.0K 36m51s
50K ......ESC[0mESC[91m.... .......... ..ESC[0mESC[91m........ ........ESC[0mESC[91m.. .......... 0% 178K 27m37s
100K .......... ..ESC[0mESC[91m........ ........ESC[0mESC[91m.. ......ESC[0mESC[91m.ESC[0mESC[91m... ....ESC[0mESC[91m...... 0% 48.0M 18m25s
150K ..ESC[0mESC[91m........ESC[0mESC[91m .......... ......ESC[0mESC[91m.... .......... ..ESC[0mESC[91m........ 0% 179K 18m23s
200K .......... ......ESC[0mESC[91m.... .......... ..ESC[0mESC[91m........ESC[0mESC[91m .......... 0% 60.3M 14m43s
.
.
.
196550K .........ESC[0mESC[91m. .......... .....ESC[0mESC[91m..... .......... .ESC[0mESC[91m......... 99% 64.2M 0s
196600K .......... .....ESC[0mESC[91m..... .......... .ESC[0mESC[91m......... .......... 99% 142M 0s
196650K .....ESC[0mESC[91m..... ...ESC[0mESC[91m....... .ESC[0mESC[91m......... .......... .....ESC[0mESC[91m..... 99% 146M 0s
196700K ....ESC[0mESC[91m...... ..ESC[0mESC[91m........ .......... ......ESC[0mESC[91m.... ....ESC[0mESC[91m...... 99% 105M 0s
196750K .... 100% 65.8M=38s

ESC[0mESC[91m2023-07-19 00:27:21 (5.07 MB/s) - ‘input_file.sql.gz’ saved [201476885/201476885]

ESC[0mRemoving intermediate container 86a0284cccbb
---> 61e2d96cfabf
Step 8/12 : RUN gzip -dq input_file.sql.gz
---> Running in 87c0e69e629e
Removing intermediate container 87c0e69e629e
---> e37a0543bdbd
Step 9/12 : RUN sed 's/anyarray/anycompatiblearray/g' input_file.sql > modified_file.sql
---> Running in b29fc489c3e2
Removing intermediate container b29fc489c3e2
---> dd25147dbaab
Step 10/12 : RUN rm input_file.sql
---> Running in 5b09c17e1be9
Removing intermediate container 5b09c17e1be9
---> 12fbbd1b36ac
Step 11/12 : RUN gzip modified_file.sql
---> Running in 10a8da640fa9
Removing intermediate container 10a8da640fa9
---> 65810ba1eea7
Step 12/12 : RUN mv modified_file.sql.gz /docker-entrypoint-initdb.d/vvta_2022_11_noseq.sql.gz
---> Running in ce8901aa5f24
Removing intermediate container ce8901aa5f24
---> f04a916facf4
Successfully built f04a916facf4
Successfully tagged vv_ta:2.2.0

@yebean-k
Copy link
Author

When I installed Flask 2.3.2 version written in REQUIREMENTS.txt, I got a message that "propagate_exceptions" does not exist.

So I installed Flask 2.2.5 version because "propagate_exceptions" disappeared from Flask 2.3.0 version.

Could this be a problem?

@Peter-J-Freeman
Copy link
Contributor

OK, a couple of things.

Firstly you are building from the tagged version

docker build --tag vv_ta:2.2.0 -f vvta_docker.df .

Ideally build from master as we regularly test and update. Also, this command will not build the databases needed by VV.

Instead, the build instructions are here https://github.com/openvar/rest_variantValidator/blob/master/docs/DOCKER.md

Follow these through carefully. Once you have the API running, then you will be able to make url requests and get a JSON response, just as if you were hitting the hosted API at https://rest.variantvalidator.org/.

Feel free to give any feedback on the build.

Please let me know how you get on

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants