Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pesacheck Meedan Bridge #549

Merged
merged 33 commits into from
Feb 23, 2024
Merged

Pesacheck Meedan Bridge #549

merged 33 commits into from
Feb 23, 2024

Conversation

koechkevin
Copy link
Contributor

@koechkevin koechkevin commented Feb 2, 2024

Description

This PR adds Pesacheck Meedan Bridge script to upload fact checks as per this conversation.

Implementation approach is as outlined in #547.

This script is expected to be deployed as a docker image.

  • Create a .env file under pesacheck_meedan_bridge folder as per .env.example.
    The variables are:
 PESACHECK_URL=https://pesacheck.org/feed
 PESACHECK_RSS2JSON_API_KEY=
 PESACHECK_CHECK_URL=https://check-api.checkmedia.org/api/graphql
 PESACHECK_CHECK_TOKEN=
 PESACHECK_CHECK_WORKSPACE_SLUG=pesacheck-tipline-sandbox
 PESACHECK_SENTRY_DSN=
 PESACHECK_SENTRY_TRACES_SAMPLE_RATE=1.0
 PESACHECK_SENTRY_ENVIRONMENT=local
 # Only for Dokku to allow app to be deployed from docker image
 DOKKU_DOCKERFILE_START_CMD="tail -f /dev/null"
  • A docker-compose script has been added to run this docker-compose up pesacheck_meedan_bridge --build -d.
  • To run binary, pants run pesacheck_meedan_bridge/docker:pesacheck_meedan_bridge
  • To deploy cron to dokku, app.json has been added with a daily command to execute ./pex daily.

Development

.env

  • Execute pesacheck_meedan_bridge/py/main.py

Fixes #547

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Screenshots

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation

Add Dockerfile
Test git image deployment on Dokku
@koechkevin koechkevin requested a review from kilemensi February 2, 2024 06:14
@koechkevin koechkevin self-assigned this Feb 2, 2024
Copy link
Member

@kilemensi kilemensi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏽


Make sure you've run format and lint on the code (I think they've already been configured)

.gitignore Outdated Show resolved Hide resolved
.vscode/settings.json Outdated Show resolved Hide resolved
Dockerfile Outdated Show resolved Hide resolved
app.json Outdated Show resolved Hide resolved
docker-compose.yml Outdated Show resolved Hide resolved
pesacheck_meedan_bridge/.env.example Outdated Show resolved Hide resolved
pesacheck_meedan_bridge/.env.example Outdated Show resolved Hide resolved
pesacheck_meedan_bridge/output.json Outdated Show resolved Hide resolved
pesacheck_meedan_bridge/service.py Outdated Show resolved Hide resolved
pesacheck_meedan_bridge/mutation_queries.py Outdated Show resolved Hide resolved
pesacheck_meedan_bridge/Dockerfile Outdated Show resolved Hide resolved
pesacheck_meedan_bridge/Dockerfile Outdated Show resolved Hide resolved
dependabot bot added 6 commits February 8, 2024 13:45
Bumps [boto3](https://github.com/boto/boto3) from 1.34.31 to 1.34.32.
- [Release notes](https://github.com/boto/boto3/releases)
- [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst)
- [Commits](boto/boto3@1.34.31...1.34.32)

---
updated-dependencies:
- dependency-name: boto3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Bumps [boto3](https://github.com/boto/boto3) from 1.34.32 to 1.34.33.
- [Release notes](https://github.com/boto/boto3/releases)
- [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst)
- [Commits](boto/boto3@1.34.32...1.34.33)

---
updated-dependencies:
- dependency-name: boto3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Bumps [boto3](https://github.com/boto/boto3) from 1.34.33 to 1.34.34.
- [Release notes](https://github.com/boto/boto3/releases)
- [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst)
- [Commits](boto/boto3@1.34.33...1.34.34)

---
updated-dependencies:
- dependency-name: boto3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Bumps [boto3](https://github.com/boto/boto3) from 1.34.34 to 1.34.35.
- [Release notes](https://github.com/boto/boto3/releases)
- [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst)
- [Commits](boto/boto3@1.34.34...1.34.35)

---
updated-dependencies:
- dependency-name: boto3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Bumps [boto3](https://github.com/boto/boto3) from 1.34.35 to 1.34.36.
- [Release notes](https://github.com/boto/boto3/releases)
- [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst)
- [Commits](boto/boto3@1.34.35...1.34.36)

---
updated-dependencies:
- dependency-name: boto3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Bumps [sentry-sdk](https://github.com/getsentry/sentry-python) from 1.40.0 to 1.40.2.
- [Release notes](https://github.com/getsentry/sentry-python/releases)
- [Changelog](https://github.com/getsentry/sentry-python/blob/master/CHANGELOG.md)
- [Commits](getsentry/sentry-python@1.40.0...1.40.2)

---
updated-dependencies:
- dependency-name: sentry-sdk
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
@kilemensi
Copy link
Member

Haven't gone through the code yet @koechkevin but linting throws errors:

pants lint ::

@koechkevin
Copy link
Contributor Author

Haven't gone through the code yet @koechkevin but linting throws errors:

pants lint ::

@kilemensi running pant lint, I get this error

image

And also exists on main branch, not sure what causes this error.

@kilemensi
Copy link
Member

It does tell you what is wrong @koechkevin, no?

11:06:07.35 [ERROR] Completed: Lint with Hadolint - hadolint failed (exit code 1).
pesacheck_meedan_bridge/docker/Dockerfile:12 DL3008 warning: Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
pesacheck_meedan_bridge/docker/Dockerfile:12 DL3009 info: Delete the apt-get lists after installing something
pesacheck_meedan_bridge/docker/Dockerfile:14 DL4006 warning: Set the SHELL option -o pipefail before RUN with a pipe in it. If you are using /bin/sh in an alpine image or if your shell is symlinked to busybox then consider explicitly setting your SHELL to /bin/ash, or disable this check
pesacheck_meedan_bridge/docker/Dockerfile:17 DL3009 info: Delete the apt-get lists after installing something
pesacheck_meedan_bridge/docker/Dockerfile:17 DL3015 info: Avoid additional packages by specifying `--no-install-recommends`
pesacheck_meedan_bridge/docker/Dockerfile:17 DL3008 warning: Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
pesacheck_meedan_bridge/docker/Dockerfile:18 SC2016 info: Expressions don't expand in single quotes, use double quotes for that.
pesacheck_meedan_bridge/docker/Dockerfile:19 DL3059 info: Multiple consecutive `RUN` instructions. Consider consolidation.
pesacheck_meedan_bridge/docker/Dockerfile:21 DL3059 info: Multiple consecutive `RUN` instructions. Consider consolidation.

Copy link
Member

@kilemensi kilemensi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏽


I think there should be just one instance of db loaded and locked (probably when main start). If main can not load db (e.g. in use), app exits.

main would then pass db to all function that need db access, etc.

.gitignore Outdated Show resolved Hide resolved
.gitignore Show resolved Hide resolved
3rdparty/py/requirements-all.txt Outdated Show resolved Hide resolved
pesacheck_meedan_bridge/.env.example Outdated Show resolved Hide resolved
pesacheck_meedan_bridge/py/VERSION Outdated Show resolved Hide resolved
pesacheck_meedan_bridge/py/main.py Outdated Show resolved Hide resolved
pesacheck_meedan_bridge/py/main.py Outdated Show resolved Hide resolved
- Set version to 0.1.0
- Remove uneeded files in gitignore
- Move post to check to check_api
@koechkevin koechkevin requested a review from kilemensi February 19, 2024 11:50
cfa_vpn/py/credentials.json Outdated Show resolved Hide resolved
docker-compose.yml Outdated Show resolved Hide resolved
pesacheck_meedan_bridge/.env.example Outdated Show resolved Hide resolved
pesacheck_meedan_bridge/py/check_api.py Outdated Show resolved Hide resolved
pesacheck_meedan_bridge/py/main.py Outdated Show resolved Hide resolved
@koechkevin koechkevin requested a review from kilemensi February 20, 2024 12:00
Copy link
Contributor

@thepsalmist thepsalmist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@kilemensi kilemensi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If language issue is fixed @koechkevin, lets deploy this as well.

@koechkevin koechkevin merged commit 56a38f3 into main Feb 23, 2024
1 check passed
@koechkevin koechkevin deleted the feature/pesacheck branch February 23, 2024 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upload fact checks
3 participants