mtg card price visualization project
run server undaemonized:
sudo /home/mig/anaconda3/envs/mtg/bin/mod_wsgi-express start-server ~/mtg/app/app/app.wsgi --port=80 --user www-data --group www-data --log-to-terminal
run server daemonized:
sudo /home/mig/anaconda3/envs/mtg/bin/mod_wsgi-express setup-server ~/mtg/app/app/app.wsgi --port=80 --user www-data --group www-data --server-root=/etc/mod_wsgi-express-80 --log-to-terminal
then to start daemon:
sudo /etc/mod_wsgi-express-80/apachectl start
then stop daemon:
sudo /etc/mod_wsgi-express-80/apachectl stop
ERRORS:
(to view errors, stop daemon server and start an undaemonized one)
ModuleNotFoundError: No module named 'app':
- check if apahe has permissions to read and execute files in the project folder
- to check apache user, run: apachectl -S
- So, in order to make a directory writable by the webserver we have to set the directory’s owner or group to Apache’s owner or group and enable the write permission for it. Usually, we set the directory to belong to the Apache group (apache or `www-data or whatever user is used to launch the child processes) and enable the write permission for the group.
chgrp www-data ~/mtg/app/
chmod g+rwx ~/mtg/app/
- also check if any imports are failing
- make sure the paths written in python strings are correct
psql [database_name] [user_name]
psql mtg mig
https://www.freecodecamp.org/news/how-to-get-started-with-postgresql-9d3bc1dd1b11/
CREATE DATABASE mtg;
sudo su - postgres
psql
\c DBNAME <- to access the created db
create database mtg;
\l # list databases, then press q to go back to db console
\q # to exit:
\dt # list tables of the public schema
CREATE TABLE card_listings (
card_name varchar(100),
ts timestamp,
avg_sell_price float,
seller_name varchar(30),
seller_sales int,
seller_available_items int,
item_price real,
item_amount int,
item_location varchar(30),
item_condition char(2),
item_language varchar(20),
item_is_playset boolean,
item_is_foil boolean
);
create user mig with password 'password';
GRANT ALL PRIVILEGES ON DATABASE mtg to mig;
GRANT CONNECT ON DATABASE mtg TO mig;
GRANT USAGE ON SCHEMA public TO mig;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO mig;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO mig;
GRANT USAGE ON SCHEMA public TO mig;
GRANT SELECT, UPDATE, INSERT, DELETE ON ALL TABLES IN SCHEMA public TO mig;
ALTER TABLE card_listings DROP COLUMN list_order;
ALTER TABLE card_listings ADD COLUMN list_order int;
-- Firstly, remove PRIMARY KEY attribute of former PRIMARY KEY ALTER TABLE card_listings DROP CONSTRAINT card_listings_pkey;
-- Lastly set your new PRIMARY KEY ALTER TABLE card_listings ADD PRIMARY KEY (card_name, ts, list_order);
https://askubuntu.com/questions/420981/how-do-i-save-terminal-output-to-a-file
https://askubuntu.com/questions/420981/how-do-i-save-terminal-output-to-a-file
crontab -e crontab -u mig -e
SHELL=/bin/bash
#regular prototype_scraping job
0 * * * * export DISPLAY=:0.0 ; /home/mig/anaconda3/envs/mtg/bin/python ~/mtg/prototype_scraping.py |& tee -a ~/mtg/logs/regular_log.txt
#regular prototype_web_plot_update_data job
30 * * * * /home/mig/anaconda3/envs/mtg/bin/python ~/mtg/prototype_web_plot_update_data.py |& tee -a ~/mtg/logs/plot_update_log.txt
#experimental prototype_scraping job
25 * * * * export DISPLAY=:0.0 ; /home/mig/anaconda3/envs/mtg/bin/python ~/mtg/prototype_scraping.py |& tee -a ~/mtg/logs/experimental_log.txt
https://askubuntu.com/questions/213678/how-to-install-x11-xorg
do this on mig-pc:
~/.ssh/config
Host mig-server
HostName mtgdata.ml
User mig
Port 49000
do this on mig-server:
https://pt.godaddy.com/help/alterar-a-porta-ssh-para-o-seu-servidor-com-o-linux-7306
ssh -X -p 49000 user@hostname
ssh -X -p 49000 [email protected]
ssh -X -p 49000 [email protected]
ssh -X -p 49000 [email protected]
check if port is open or closed: nmap -p 49000 79.168.14.53 nmap -p 49000 mtgdata.ml
allow connections on specific port: sudo ufw allow 49000
https://superuser.com/questions/307820/how-do-i-properly-test-whether-my-port-forwarding-works
https://serverfault.com/questions/554359/postgresql-timezone-does-not-match-system-timezone
The solution for your case is quite simple, just change the TimeZone setting on postgresql.conf to the value you want:
TimeZone = 'Europe/Vienna'
After that you need to reload the service:
sudo su - postgres -c "psql mtg -c 'SELECT pg_reload_conf()'"
https://stackoverflow.com/questions/3602450/where-are-my-postgres-conf-files
ask your database:
$ psql -U postgres -c 'SHOW config_file'
SET TIME ZONE 'UTC';
select now();
show timezone;
use this setting in pandas as well (UTC)
ALTER TABLE card_listings
ALTER COLUMN item_price TYPE real;