minetest · ShadowNinja · Jun 13, 2021 · Jul 10, 2021 · Jul 10, 2021 · Jul 10, 2021
diff --git a/.editorconfig b/.editorconfig
@@ -0,0 +1,7 @@
+root = true
+
+[*]
+indent_style = tab
+indent_size = 4
+trim_trailing_whitespace = true
+insert_final_newline = true
diff --git a/.gitignore b/.gitignore
@@ -1,7 +1,11 @@
 *~
-node_modules
-__pycache__
-static/list.json
-static/servers.js
-config.py
 *.mmdb
+*.sqlite
+node_modules/
+__pycache__/
+/server_list/static/list.json
+/server_list/static/servers.js
+/config.py
+/celerybeat-schedule
+/package-lock.json
+/Pipfile.lock
diff --git a/Pipfile b/Pipfile
@@ -0,0 +1,18 @@
+[[source]]
+url = "https://pypi.org/simple"
+verify_ssl = true
+name = "pypi"
+
+[packages]
+maxminddb = ">=2.0.0"
+Flask = "~=2.0"
+flask-sqlalchemy = "~=3.0"
+flask-migrate = "~=4.0"
+celery = "~=5.0"
+
+[dev-packages]
+pylint = "*"
+rope = "*"
+
+[requires]
+python_version = "3"
diff --git a/README.md b/README.md
@@ -1,131 +1,203 @@
-Minetest server list
+Minetest Server List
 ====================
 
-Setting up the webpage
-----------------------
+Webpage Setup
+---
 
-You will have to install node.js, doT.js and their dependencies to compile
-the server list webpage template.
+You will have to install node.js, doT.js and their dependencies to compile the server list webpage template.
 
 First install node.js, e.g.:
 
-	# apt-get install nodejs
-	# # OR:
-	# pacman -S nodejs
+```sh
+sudo pacman -S nodejs
+# OR:
+sudo apt-get install nodejs
+```
 
 Then install doT.js and its dependencies:
 
-	$ npm install dot commander mkdirp
+```sh
+npm install
+```
 
 And finally compile the template:
 
-	$ cd static
-	$ ../node_modules/dot/bin/dot-packer -s .
+```sh
+cd server_list/static
+../../node_modules/dot/bin/dot-packer -s .
+```
 
-You can now serve the webpage by copying the files in static/ to your web root, or by [starting the server list](#setting-up-the-server).
+You can now serve the webpage by copying the files in `server_list/static/` to your web root, or by [starting the server list](#server-setup).
 
+Embedding in a Webpage
+---
 
-Embedding the server list in a page
------------------------------------
+```html
+<head>
+	...
+	<script>
+		var master = {
+			root: 'https://servers.minetest.net/',
+			limit: 10,
+			clients_min: 1,
+			no_flags: 1,
+			no_ping: 1,
+			no_uptime: 1
+		};
+	</script>
+	...
+</head>
+<body>
+	...
+	<div id="server_list"></div>
+	...
+</body>
+<script src="list.js"></script>
+```
 
-	<head>
-		...
-		<script>
-			var master = {
-				root: 'http://servers.minetest.net/',
-				limit: 10,
-				clients_min: 1,
-				no_flags: 1,
-				no_ping: 1,
-				no_uptime: 1
-			};
-		</script>
-		...
-	</head>
-	<body>
-		...
-		<div id="server_list"></div>
-		...
-	</body>
-	<script src="list.js"></script>
+Server Setup
+---
 
+ 1. Install Python 3 and Pipenv:
 
-Setting up the server
----------------------
+    ```sh
+    sudo pacman -S python python-pipenv
+    # OR:
+    sudo apt-get install python3 python3-pip && pip install pipenv
+    ```
 
-  1. Install Python 3 and pip:
+ 2. Install required Python packages:
 
-	pacman -S python python-pip
-	# OR:
-	apt-get install python3 python3-pip
+    ```sh
+    pipenv sync
+    ```
 
-  2. Install required Python packages:
+ 3. Set up Celery message broker.  Pick a Celery backend (Redis or RabbitMQ are recommended), and install and enable the required packages.  For example:
 
-	# You might have to use pip3 if your system defaults to Python 2
-	pip install -r requirements.txt
+    ```sh
+    # Redis support requires an additional package
+    pipenv run pip install redis
+    sudo pacman -S redis # or sudo apt-get install redis
+    sudo systemctl enable --now redis
+    ```
 
-  3. If using in production, install uwsgi and it's python plugin:
+ 4. Configure the server by adding options to `config.py`.
+    See `server_list/config.py` for defaults.
 
-	pacman -S uwsgi uwsgi-plugin-python
-	# OR:
-	apt-get install uwsgi uwsgi-plugin-python
-	# OR:
-	pip install uwsgi
+ 5. Start the server for development:
 
-  4. Configure the server by adding options to `config.py`.
-       See `config-example.py` for defaults.
+    ```sh
+    pipenv run flask run
+    ```
 
-  5. Start the server:
+ 6. Start the celery background worker:
 
-	$ ./server.py
-	$ # Or for production:
-	$ uwsgi -s /tmp/minetest-master.sock --plugin python -w server:app --enable-threads
-	$ # Then configure according to http://flask.pocoo.org/docs/deploying/uwsgi/
+    ```sh
+    pipenv run celery --app server_list:celery worker --beat
+    ```
 
-  7. (optional) Configure the proxy server, if any.  You should make the server
-	load static files directly from the static directory.  Also, `/list`
-	should be served from `list.json`.  Example for nginx:
+Running in Production
+---
 
-	root /path/to/server/static;
-	rewrite ^/list$ /list.json;
-	try_files $uri @uwsgi;
-	location @uwsgi {
-		uwsgi_pass ...;
-	}
+When running in production you should set up a proxy server that calls the server list through WSGI.
 
-Setting up the server (Apache version)
----------------------
+These examples assume that the server list is installed to `/srv/http/serverlist`.
 
-If you wish to use Apache to host the server list, do steps 1-2, 4, above. Additionally install/enable mod_wsgi and an Apache site config like the following:
+### Nginx
 
-		# This config assumes you have the server list at DocumentRoot.
-		# Visitors to the server list in this config would visit http://local.server/ and
-		# apache would serve up the output from server.py. Static resources would be served
-		# from http://local.server/static.
+First [set up uWSGI](#uwsgi), then update the Nginx configuration to proxy to uWSGI.  You should make the server load static files directly from the static directory.  Also, `/list` should be aliased to `list.json`.
 
-		# Where are the minetest-server files located?
-		DocumentRoot /var/games/minetest/serverlist
+Here's an example configuration:
 
-		# Serve up server.py at the root of the URL.
-		WSGIScriptAlias / /var/games/minetest/serverlist/server.py
+```nginx
+root /srv/http/serverlist/server_list/static;
+rewrite ^/list$ /list.json;
+try_files $uri @uwsgi;
+location @uwsgi {
+	uwsgi_pass unix:/run/uwsgi/server_list.sock;
+}
+```
 
-		# The name of the function that we call when we invoke server.py
-		WSGICallableObject app
+Also see [the Flask uwsgi documentation](https://flask.palletsprojects.com/en/2.0.x/deploying/uwsgi/).
 
-		# These options are necessary to enable Daemon mode. Without this, you'll have strange behavior
-		# with servers dropping off your list! You can tweak threads as needed. See mod_wsgi documentation.
-		WSGIProcessGroup minetest-serverlist
-		WSGIDaemonProcess minetest-serverlist threads=2
+### Apache
 
+There are two options for Apache, you can use either `mod_wsgi` or `mod_proxy_uwsgi`.
 
-		<Directory /var/games/minetest/serverlist>
-			Require all granted
-		</Directory>
+Note: both of these example configurations serve static through WSGI, instead of bypassing WSGI for performance.
 
-	</VirtualHost>
+#### mod_wsgi
+
+First install/enable `mod_wsgi`.
+
+Then create `wsgi.py` in the directory containing `server_list` with the following contents:
+
+```py
+import os, sys
+sys.path.append(os.path.dirname(__file__))
+from server_list import app
+```
+
+Then configure the Apache VirtualHost like the following:
+
+```apache
+WSGIDaemonProcess server_list python-home=<output of pipenv --venv>
+
+WSGIProcessGroup server_list
+WSGIApplicationGroup %{GLOBAL}
+
+WSGIScriptAlias / /srv/http/serverlist/wsgi.py
+WSGICallableObject app
+
+<Directory /srv/http/serverlist>
+	<Files wsgi.py>
+		Require all granted
+	</Files>
+</Directory>
+```
+
+#### mod_proxy_uwsgi
+
+First [set up uWSGI](#uwsgi), then install/enable `mod_proxy` and `mod_proxy_uwsgi` and add the following to your VirtualHost:
+
+```apache
+ProxyPass / unix:/run/uwsgi/server_list.sock|uwsgi://localhost/
+```
+
+Note: this requires at least Apache 2.4.7 for the unix socket syntax.  If you have an older version of Apache you'll have to use IP sockets.
+
+### uWSGI
+
+First, install uWSGI and its python plugin.
+
+```sh
+pacman -S uwsgi uwsgi-plugin-python
+# OR:
+apt-get install uwsgi uwsgi-plugin-python
+# OR:
+pip install uwsgi
+```
+
+Then create a uWSGI config file.  For example:
+
+```ini
+[uwsgi]
+socket = /run/uwsgi/server_list.sock
+plugin = python
+virtualenv = <output of pipenv --venv>
+python-path = /srv/http/serverlist
+module = server_list
+callable = app
+```
+
+You can put the config file in `/etc/uwsgi/server_list.ini`.  Make sure that uWSGI is configured to start as the appropriate user and group for your distro (e.g. http:http) and then start and enable uWSGI.
+
+```sh
+systemctl enable --now uwsgi@server_list.service
+```
 
 License
--------
+---
 
 The Minetest server list code is licensed under the GNU Lesser General Public
 License version 2.1 or later (LGPLv2.1+).  A LICENSE.txt file should have been

diff --git a/config-example.py b/config-example.py