Skip to content

Commit

Permalink
chore(deps): Remove unused Python requirements (#21580)
Browse files Browse the repository at this point in the history 
* chore(deps): Remove unused Python requirements

* Also remove `dnspython`

* Recompile txts

* Revert "Also remove `dnspython`"

This reverts commit 7eaf899.

* Re-recompile txts

* Remove more deps based on `deptry`

* Restore `phonenumberslite`

* Restore `geoip2`

* Restore `python3-saml`

* Move `prance` and `openapi-spec-validator` to dev reqs

* Reorder `geoip2`

* Force HTTP 1.1 in downloading MMDB

Should fix CI failures caused by `curl: (92) HTTP/2 stream 0 was not closed cleanly: INTERNAL_ERROR (err 2)` (e.g. https://github.com/PostHog/posthog/actions/runs/8741861825/job/23988870688?pr=21580).
  • Loading branch information
@Twixes
Twixes authored Apr 22, 2024
1 parent eb3f8be commit 233d3a9
Show file tree
Hide file tree
Showing 7 changed files with 71 additions and 95 deletions.
2 changes: 1 addition & 1 deletion .github/actions/run-backend-tests/action.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ runs:
touch frontend/dist/index.html
touch frontend/dist/layout.html
touch frontend/dist/exporter.html
[ ! -f ./share/GeoLite2-City.mmdb ] && ( curl -L "https://mmdbcdn.posthog.net/" | brotli --decompress --output=./share/GeoLite2-City.mmdb )
[ ! -f ./share/GeoLite2-City.mmdb ] && ( curl -L "https://mmdbcdn.posthog.net/" --http1.1 | brotli --decompress --output=./share/GeoLite2-City.mmdb )
- name: Wait for Clickhouse & Kafka
shell: bash
Expand Down
2 changes: 1 addition & 1 deletion bin/start
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ service_warning() {
nc -z localhost 9092 || ( service_warning 'Kafka'; bin/check_kafka_clickhouse_up )
curl -s 'http://localhost:8123/ping' || ( service_warning 'ClickHouse'; bin/check_kafka_clickhouse_up )

[ ! -f ./share/GeoLite2-City.mmdb ] && ( curl -L "https://mmdbcdn.posthog.net/" | brotli --decompress --output=./share/GeoLite2-City.mmdb )
[ ! -f ./share/GeoLite2-City.mmdb ] && ( curl -L "https://mmdbcdn.posthog.net/" --http1.1 | brotli --decompress --output=./share/GeoLite2-City.mmdb )

./bin/start-worker &
./bin/start-backend &
Expand Down
2 changes: 1 addition & 1 deletion production.Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ RUN apt-get update && \
&& \
rm -rf /var/lib/apt/lists/* && \
mkdir share && \
( curl -s -L "https://mmdbcdn.posthog.net/" | brotli --decompress --output=./share/GeoLite2-City.mmdb ) && \
( curl -s -L "https://mmdbcdn.posthog.net/" --http1.1 | brotli --decompress --output=./share/GeoLite2-City.mmdb ) && \
chmod -R 755 ./share/GeoLite2-City.mmdb


Expand Down
3 changes: 3 additions & 0 deletions requirements-dev.in
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ fakeredis[lua]==2.11.0
freezegun==1.2.2
packaging==23.1
black~=22.8.0
boto3-stubs[s3]
types-markdown==3.3.9
types-PyYAML==6.0.1
types-freezegun==1.1.10
Expand All @@ -48,3 +49,5 @@ responses==0.23.1
syrupy~=4.6.0
flaky==3.7.0
aioresponses==0.7.6
prance==23.06.21.0
openapi-spec-validator==0.7.1 # Needed for prance as a validation backend
61 changes: 61 additions & 0 deletions requirements-dev.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,16 +33,24 @@ attrs==23.2.0
# via
# -c requirements.txt
# aiohttp
# jsonschema
# referencing
black==22.8.0
# via
# -r requirements-dev.in
# datamodel-code-generator
boto3-stubs[s3]==1.34.84
# via -r requirements-dev.in
botocore-stubs==1.34.84
# via boto3-stubs
build==0.10.0
# via pip-tools
certifi==2019.11.28
# via
# -c requirements.txt
# requests
chardet==5.2.0
# via prance
charset-normalizer==2.1.0
# via
# -c requirements.txt
Expand Down Expand Up @@ -122,6 +130,20 @@ jinja2==3.1.3
# via
# coreschema
# datamodel-code-generator
jsonschema==4.20.0
# via
# -c requirements.txt
# openapi-schema-validator
# openapi-spec-validator
jsonschema-path==0.3.2
# via openapi-spec-validator
jsonschema-specifications==2023.12.1
# via
# -c requirements.txt
# jsonschema
# openapi-schema-validator
lazy-object-proxy==1.10.0
# via openapi-spec-validator
lupa==1.14.1
# via fakeredis
markupsafe==2.1.5
Expand All @@ -138,20 +160,29 @@ mypy==1.8.0
# djangorestframework-stubs
mypy-baseline==0.6.1
# via -r requirements-dev.in
mypy-boto3-s3==1.34.65
# via boto3-stubs
mypy-extensions==1.0.0
# via
# -r requirements-dev.in
# black
# mypy
openapi-schema-validator==0.6.2
# via openapi-spec-validator
openapi-spec-validator==0.7.1
# via -r requirements-dev.in
packaging==23.1
# via
# -c requirements.txt
# -r requirements-dev.in
# build
# datamodel-code-generator
# prance
# pytest
parameterized==0.9.0
# via -r requirements-dev.in
pathable==0.4.3
# via jsonschema-path
pathspec==0.9.0
# via black
pip-tools==7.3.0
Expand All @@ -164,6 +195,8 @@ pluggy==0.13.1
# via pytest
pprintpp==0.4.0
# via pytest-icdiff
prance==23.6.21.0
# via -r requirements-dev.in
pydantic[email]==2.5.3
# via
# -c requirements.txt
Expand Down Expand Up @@ -212,25 +245,47 @@ pyyaml==6.0.1
# via
# -c requirements.txt
# datamodel-code-generator
# jsonschema-path
# responses
redis==4.5.4
# via
# -c requirements.txt
# fakeredis
referencing==0.31.1
# via
# -c requirements.txt
# jsonschema
# jsonschema-path
# jsonschema-specifications
requests==2.31.0
# via
# -c requirements.txt
# coreapi
# djangorestframework-stubs
# jsonschema-path
# prance
# responses
responses==0.23.1
# via -r requirements-dev.in
rfc3339-validator==0.1.4
# via openapi-schema-validator
rpds-py==0.16.2
# via
# -c requirements.txt
# jsonschema
# referencing
ruamel-yaml==0.18.6
# via prance
ruamel-yaml-clib==0.2.8
# via ruamel-yaml
ruff==0.3.7
# via -r requirements-dev.in
six==1.16.0
# via
# -c requirements.txt
# prance
# python-dateutil
# rfc3339-validator
sortedcontainers==2.4.0
# via
# -c requirements.txt
Expand All @@ -253,6 +308,8 @@ tomli==1.2.3
# pip-tools
# pyproject-hooks
# pytest
types-awscrt==0.20.9
# via botocore-stubs
types-freezegun==1.1.10
# via -r requirements-dev.in
types-markdown==3.3.9
Expand All @@ -273,16 +330,20 @@ types-requests==2.26.1
# via -r requirements-dev.in
types-retry==0.9.9.4
# via -r requirements-dev.in
types-s3transfer==0.10.1
# via boto3-stubs
types-tzlocal==5.1.0.1
# via -r requirements-dev.in
typing-extensions==4.7.1
# via
# -c requirements.txt
# asgiref
# boto3-stubs
# django-stubs
# django-stubs-ext
# djangorestframework-stubs
# mypy
# mypy-boto3-s3
# pydantic
# pydantic-core
uritemplate==4.1.1
Expand Down
14 changes: 0 additions & 14 deletions requirements.in
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,13 @@ aiohttp>=3.9.0
aioboto3==12.0.0
aiokafka>=0.8
antlr4-python3-runtime==4.13.1
amqp==5.1.1
boto3==1.28.16
boto3-stubs[s3]
brotli==1.1.0
celery==5.3.4
celery-redbeat==2.1.1
clickhouse-driver==0.2.6
clickhouse-pool==0.5.3
cryptography==37.0.2
defusedxml==0.6.0
dj-database-url==0.5.0
Django~=4.2.11
django-axes==5.9.0
Expand All @@ -41,29 +38,20 @@ dnspython==2.2.1
drf-exceptions-hog==0.4.0
drf-extensions==0.7.0
drf-spectacular==0.27.1
gevent==23.9.1
geoip2==4.6.0
google-cloud-bigquery==3.11.4
gunicorn==20.1.0
idna==2.8
importlib-metadata==6.8.0
infi-clickhouse-orm@ git+https://github.com/PostHog/infi.clickhouse_orm@9578c79f29635ee2c1d01b7979e89adab8383de2
kafka-python==2.0.2
kafka-helper==0.2
kombu==5.3.2
lzstring==1.0.4
natsort==8.4.0
numpy==1.23.3
openapi-spec-validator==0.7.1
openpyxl==3.1.2
orjson==3.9.10
pandas==2.2.0
parso==0.8.1
pexpect==4.7.0
pickleshare==0.7.5
Pillow==10.2.0
posthoganalytics==3.5.0
prance==23.06.21.0
psycopg2-binary==2.9.7
psycopg[binary]==3.1.13
pyarrow==15.0.0
Expand All @@ -75,7 +63,6 @@ pytz==2023.3
redis==4.5.4
retry==0.9.2
requests~=2.31.0
requests-oauthlib==1.3.0
s3fs==2023.10.0
stripe==7.4.0
selenium==4.1.5
Expand Down Expand Up @@ -103,4 +90,3 @@ openai==1.10.0
tiktoken==0.6.0
nh3==0.2.14
hogql-parser==1.0.7
urllib3[secure,socks]==1.26.18
Loading

0 comments on commit 233d3a9

Please sign in to comment.