Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] version 1.3 #79

Open
wants to merge 88 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
a20ae65
upgrade base code dependencies fixes #74
Feb 28, 2017
3d71e77
add docker closes #78
Apr 18, 2017
931298a
consolidate requirements to 1 file
Apr 18, 2017
9c28552
rename pillbox-engine folder to engine
Apr 18, 2017
19ea391
use latest version of celery
Apr 18, 2017
afca215
remove unused files
Apr 18, 2017
f331317
remove unused files
Apr 18, 2017
d7d8f6b
move stuff to config folder
Apr 18, 2017
932ba86
move around files
Apr 18, 2017
bd5cb49
update celery import
Apr 19, 2017
53f6a71
fix how boxes are shown on dashboard
Apr 19, 2017
b595478
simplify docker-compose
Apr 19, 2017
3c98c60
separate date/time for download/unzip operations
Apr 19, 2017
13fee9c
show two decimal places
Apr 19, 2017
8853ccb
update readonly file list
Apr 19, 2017
db4e130
refactor download and unzip
Apr 19, 2017
0d2e9c8
adjust percentage weights
Apr 19, 2017
f6022f6
better download and zip status update
Apr 19, 2017
138044f
fix #62
Apr 19, 2017
cddd23e
fix #68
Apr 19, 2017
3dd8d32
uncomment app
Apr 19, 2017
2bee100
move around docs
Apr 19, 2017
2155f3d
add raw
Apr 19, 2017
7d09344
relative path
Apr 19, 2017
40923fa
update image paths
Apr 19, 2017
596a14b
refresh from db
Apr 19, 2017
a83db8e
more commands
Apr 19, 2017
fb4e2f3
change log level to info
Apr 19, 2017
a1ee05d
reorganize docs
Apr 19, 2017
96db5b6
change index to readme
Apr 19, 2017
578caa0
update image paths
Apr 19, 2017
d4e607b
Move dockerfile to outer layer
Apr 19, 2017
4225fad
fixes #76
Apr 19, 2017
965c702
switch to version 2
Apr 19, 2017
af478b7
make medicine_name optional
Apr 19, 2017
356d27b
improve percentage reporting
Apr 19, 2017
2337222
update print
Apr 19, 2017
575c0f6
update how status is updated
Apr 19, 2017
5270347
remove extra colon
Apr 19, 2017
8fbec00
improve total handling
Apr 20, 2017
33480d5
update status class
Apr 20, 2017
581d756
update counts
Apr 20, 2017
e55f1b1
fix typo
Apr 20, 2017
518e173
not be undefined
Apr 20, 2017
16bd9f5
added not add
Apr 20, 2017
755fdd5
fix env variable name
Apr 20, 2017
effa561
change from_sis to epc_match fixes #72
Apr 20, 2017
7db1fd0
fixes #69 fixes #70 fixes #71
Apr 20, 2017
c2d5010
fixes #35
Apr 20, 2017
d26db05
epc migration
Apr 20, 2017
a78a97d
updated installation docs
Apr 24, 2017
5150ce2
update xnorm url
Apr 26, 2017
93669e9
remove postgres .keep
Apr 26, 2017
031ed82
update install.md
Apr 26, 2017
6286516
disable huge_tree flag
Apr 27, 2017
da3fa0c
Merge branch 'develop' of github.com:HHS/pillbox-engine into develop
Apr 27, 2017
48778cb
catch missing title
Apr 27, 2017
b95eea0
add value not provided
Apr 27, 2017
51e26dd
Update spl_sources.json
Apr 27, 2017
4074db6
if product is missing skip
Jun 30, 2017
d4c46cc
remove print
Jun 30, 2017
4cea68a
update how rxnorm is handled with better handling
Jul 2, 2017
4836d9d
add rxupdatetime
Jul 2, 2017
c8a1604
add shell commands for transfer and compare
Jul 2, 2017
45d6c13
add rx time to admin
Jul 2, 2017
9dedd27
add shell command for export
Jul 2, 2017
ebbbee9
read filename from name property
Jul 2, 2017
d04242d
handle the case where time is missing
Jul 2, 2017
1a38bda
add export link to list page
Jul 2, 2017
e81d26f
mark SPL images as SPL
Jul 2, 2017
68e0938
add missing fields to pillbox fixes #68
Jul 2, 2017
c2c1fbd
clean up command for pillbox data fixes #69 #79 #71
Jul 2, 2017
6952e41
Merge pull request #82 from HHS/fixes
Jul 2, 2017
c82e58a
better handle multiple space cleanup
Jul 27, 2017
0531971
add rx updatetime to export
Jul 27, 2017
d364e92
allow running tasks < 24 hours
Jul 27, 2017
9b23a01
reverse previous commit
Jul 27, 2017
84c615a
custom command
Jul 27, 2017
c910e59
command for products
Jul 27, 2017
b9a5912
remove debug comments
Jul 27, 2017
c9c2ae2
add has_pillbox_value flag
Jul 27, 2017
f565451
add migration
Jul 27, 2017
5e2063a
fix images shown on admin
Jul 27, 2017
c9b69a3
add special logger for cleanup work
Jul 27, 2017
7fa95b9
better logging
Jul 27, 2017
7275c4d
fix #35
Aug 16, 2017
0420a78
add command for sync has_pillbox_value field. fixes #63
Aug 16, 2017
527d9d3
add missing command #63
Aug 16, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
postgres
17 changes: 0 additions & 17 deletions .editorconfig

This file was deleted.

1 change: 0 additions & 1 deletion .gitattributes

This file was deleted.

4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,9 @@ node_modules/
.settings
.env

pillbox_engine.sublime-project
tmp-unzipped
celerybeat-schedule
staticfiles
media
downloads
config/db/postgres/*
!config/db/postgres/.keep
1 change: 0 additions & 1 deletion CONTRIBUTORS.txt

This file was deleted.

7 changes: 7 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
FROM python:2.7.13-wheezy

Add . /pillbox
WORKDIR /pillbox

RUN pip install -r requirements.txt

4 changes: 2 additions & 2 deletions Procfile
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
web: gunicorn --pythonpath="$PWD/pillbox-engine" wsgi:application
worker: celery worker --app=pillbox-engine._celery -B --loglevel=DEBUG --concurrency=1
web: gunicorn --pythonpath="$PWD/engine" config.wsgi:application
worker: celery worker -A config.celery --loglevel=INFO --concurrency=1
File renamed without changes.
22 changes: 22 additions & 0 deletions config/celery.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
from __future__ import absolute_import, unicode_literals
import os
import sys

BASE_DIR = os.path.dirname(os.path.dirname(__file__))
sys.path.append(BASE_DIR + '/engine')

from celery import Celery
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings.production')

from django.conf import settings # noqa

app = Celery('pillbox')
app.config_from_object('django.conf:settings', namespace='CELERY')

# Load task modules from all registered Django app configs.
app.autodiscover_tasks()

@app.task(bind=True)
def debug_task(self):
print('Request: {0!r}'.format(self.request))

File renamed without changes.
277 changes: 277 additions & 0 deletions config/settings/common.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,277 @@
# -*- coding: utf-8 -*-
"""
Django settings for pillbox-engine project.

For more information on this file, see
https://docs.djangoproject.com/en/dev/topics/settings/

For the full list of settings and their values, see
https://docs.djangoproject.com/en/dev/ref/settings/
"""

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
# import os


from __future__ import absolute_import, unicode_literals
from os.path import join, dirname
import environ

BASE_DIR = environ.Path(__file__) - 3 # (/a/myfile.py - 2 = /)

env = environ.Env()


# DEBUG
DEBUG = env.bool('DEBUG', default=True)
# END DEBUG

# APP CONFIGURATION
DJANGO_APPS = (
# Default Django apps:
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',

# Useful template tags:
'django.contrib.humanize',

# Admin
'django.contrib.admin',
)
THIRD_PARTY_APPS = (
'xadmin',
# 'reversion',
'crispy_forms',
# 'kombu.transport.django',
'rest_framework',
)

# Apps specific for this project go here.
LOCAL_APPS = (
# Your stuff: custom apps go here
'spl',
'pillbox',
'compare',
# 'djcelery_pillbox'
)

# See: https://docs.djangoproject.com/en/dev/ref/settings/#installed-apps
INSTALLED_APPS = DJANGO_APPS + LOCAL_APPS + THIRD_PARTY_APPS
# END APP CONFIGURATION

# MIDDLEWARE CONFIGURATION
MIDDLEWARE_CLASSES = (
# Make sure djangosecure.middleware.SecurityMiddleware is listed first
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
)
# END MIDDLEWARE CONFIGURATION

# MIGRATIONS CONFIGURATION
# MIGRATION_MODULES = {
# 'sites': 'contrib.sites.migrations'
# }
# END MIGRATIONS CONFIGURATION

# See: https://docs.djangoproject.com/en/dev/ref/settings/#template-debug
TEMPLATE_DEBUG = DEBUG
# END DEBUG

# SECRET CONFIGURATION
# See: https://docs.djangoproject.com/en/dev/ref/settings/#secret-key
# Note: This key only used for development and testing.
# In production, this is changed to a values.SecretValue() setting
SECRET_KEY = "changemetosomethingsecure!"
# END SECRET CONFIGURATION

# FIXTURE CONFIGURATION
# See: https://docs.djangoproject.com/en/dev/ref/settings/#std:setting-FIXTURE_DIRS
FIXTURE_DIRS = (
str(BASE_DIR.path('config/fixtures')),
)
# END FIXTURE CONFIGURATION

# MANAGER CONFIGURATION
# See: https://docs.djangoproject.com/en/dev/ref/settings/#admins
ADMINS = (
("""Change ME""", '[email protected]'),
)

# See: https://docs.djangoproject.com/en/dev/ref/settings/#managers
MANAGERS = ADMINS
# END MANAGER CONFIGURATION

# DATABASE CONFIGURATION
# See: https://docs.djangoproject.com/en/dev/ref/settings/#databases
DATABASES = {
# Raises ImproperlyConfigured exception if DATABASE_URL not in os.environ
'default': env.db("DATABASE_URL", default='sqlite:///%s' % str(BASE_DIR.path('config/db/db.sqlite3'))),
}
# END DATABASE CONFIGURATION

# GENERAL CONFIGURATION
# See: https://docs.djangoproject.com/en/dev/ref/settings/#time-zone
TIME_ZONE = 'America/New_York'

# See: https://docs.djangoproject.com/en/dev/ref/settings/#language-code
LANGUAGE_CODE = 'en-us'

# See: https://docs.djangoproject.com/en/dev/ref/settings/#site-id
SITE_ID = 1

# See: https://docs.djangoproject.com/en/dev/ref/settings/#use-i18n
USE_I18N = True

# See: https://docs.djangoproject.com/en/dev/ref/settings/#use-l10n
USE_L10N = True

# See: https://docs.djangoproject.com/en/dev/ref/settings/#use-tz
USE_TZ = True
# END GENERAL CONFIGURATION

# TEMPLATE CONFIGURATION
TEMPLATES = [{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'APP_DIRS': True,
'DIRS': [
str(BASE_DIR.path('config/templates')),
],
'OPTIONS': {
'debug': DEBUG,
'context_processors': [
'django.contrib.auth.context_processors.auth',
'django.core.context_processors.debug',
'django.core.context_processors.i18n',
'django.core.context_processors.media',
'django.core.context_processors.static',
'django.core.context_processors.tz',
'django.contrib.messages.context_processors.messages',
'django.core.context_processors.request',
],
},
}]

# STATIC FILE CONFIGURATION
# See: https://docs.djangoproject.com/en/dev/ref/settings/#static-root
STATIC_ROOT = str(BASE_DIR.path('config/staticfiles'))

# See: https://docs.djangoproject.com/en/dev/ref/settings/#static-url
STATIC_URL = '/static/'

# See: https://docs.djangoproject.com/en/dev/ref/contrib/staticfiles/#std:setting-STATICFILES_DIRS
STATICFILES_DIRS = (
str(BASE_DIR.path('config/static')),
)

# See: https://docs.djangoproject.com/en/dev/ref/contrib/staticfiles/#staticfiles-finders
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
)
# END STATIC FILE CONFIGURATION

# MEDIA CONFIGURATION
# See: https://docs.djangoproject.com/en/dev/ref/settings/#media-root
MEDIA_ROOT = str(BASE_DIR.path('config/media'))

# See: https://docs.djangoproject.com/en/dev/ref/settings/#media-url
MEDIA_URL = '/media/'
# END MEDIA CONFIGURATION

# URL Configuration
ROOT_URLCONF = 'config.urls'

# See: https://docs.djangoproject.com/en/dev/ref/settings/#wsgi-application
WSGI_APPLICATION = 'config.wsgi.application'
# End URL Configuration

# AUTHENTICATION CONFIGURATION
AUTHENTICATION_BACKENDS = (
"django.contrib.auth.backends.ModelBackend",
)

# Custom user app defaults
# Select the correct user model
# AUTH_USER_MODEL = "users.User"
# END Custom user app defaults

# LOGGING CONFIGURATION
# See: https://docs.djangoproject.com/en/dev/ref/settings/#logging
# A sample logging configuration. The only tangible logging
# performed by this configuration is to send an email to
# the site admins on every HTTP 500 error when DEBUG=False.
# See http://docs.djangoproject.com/en/dev/topics/logging for
# more details on how to customize your logging configuration.
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'formatters': {
'standard': {
'format' : "[%(asctime)s] %(levelname)s [%(name)s:%(lineno)s] %(message)s",
'datefmt' : "%d/%b/%Y %H:%M:%S"
},
},
'handlers': {
'mail_admins': {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler'
},
'file': {
'level': 'INFO',
'class': 'logging.FileHandler',
'filename': '/pillbox/downloads/cleanup.log',
'formatter': 'standard'
}
},
'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
'cleanup': {
'handlers': ['file'],
'level': 'INFO'
}
}
}
# END LOGGING CONFIGURATION

# Your common stuff: Below this line define 3rd party library settings
# Celery settings

CELERY_BROKER_URL = 'amqp://guest@broker//'
CELERY_IMPORTS = (
'spl.tasks',
)
# CELERY_RESULT_BACKEND = values.Value('djcelery_pillbox.database:DatabaseBackend')
# CELERY_TASK_RESULT_EXPIRES = values.IntegerValue(3600)
# CELERY_DISABLE_RATE_LIMITS = values.BooleanValue(True)
# CELERYD_CONCURRENCY = values.IntegerValue(1)
# CELERY_ACCEPT_CONTENT = values.ListValue(['json', 'msgpack', 'yaml'])
# CELERY_TASK_SERIALIZER = values.Value('json')
# CELERY_RESULT_SERIALIZER = values.Value('json')
# CELERY_TRACK_STARTED = values.BooleanValue(True)
# CELERY_STORE_ERRORS_EVEN_IF_IGNORED = True

DAILYMED_FTP_SITE = env.str('DAILYMED_FTP_SITE', 'public.nlm.nih.gov')
DAILYMED_FTP_PATH = env.str('DAILYMED_FTP_PATH', 'nlmdata/.dailymed/')
DAILYMED_FTP_USER = env.str('DAILYMED_FTP_USER', 'anonymous')
DAILYMED_FTP_PASS = env.str('DAILYMED_FTP_PASS', '')
DOWNLOAD_PATH = str(BASE_DIR.path('downloads/zip'))
SOURCE_PATH = str(BASE_DIR.path('downloads/unzip'))

34 changes: 34 additions & 0 deletions config/settings/local.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# -*- coding: utf-8 -*-
'''
Local Configurations

- Runs in Debug mode
- Uses console backend for emails
- Use Django Debug Toolbar
'''
import environ
BASE_DIR = environ.Path(__file__) - 1 # (/a/myfile.py - 2 = /)

from .common import * # noqa


# DEBUG
DEBUG = env.bool('DEBUG', True)
TEMPLATE_DEBUG = DEBUG
# END DEBUG

# django-debug-toolbar
MIDDLEWARE_CLASSES = MIDDLEWARE_CLASSES + ('debug_toolbar.middleware.DebugToolbarMiddleware',)
INSTALLED_APPS += ('debug_toolbar',)

INTERNAL_IPS = ('127.0.0.1',)

DEBUG_TOOLBAR_CONFIG = {
'DISABLE_PANELS': [
'debug_toolbar.panels.redirects.RedirectsPanel',
],
'SHOW_TEMPLATE_CONTEXT': True,
}
# end django-debug-toolbar

# Your local stuff: Below this line define 3rd party libary settings
Loading