Skip to content

Host server setup

Jump to bottom
Irene Vagionakis edited this page Mar 10, 2023 · 21 revisions

Host server setup (step-by-step tutorial for Mac OS X / Unix)

1. Get a host server

Usually the host server is provided by your institution. If you have not been provided with a host server, you can get one for example by using the Amazon Web Services (AWS) or DigitalOcean. AWS allow you to create also a free instance, which could be useful for testing purposes. On AWS:

  • Create a free EC2 instance, for example with OS Amazon Linux AMI 2; choose a region; set a security group with inbound rule with type SSH, protocol TCP, port range 22, source Anywhere and a security group with inbound rule with type HTTP, protocol TCP, port range 80, source Anywhere; enable SSH connections on your computer, or disable the Firewall.

  • Create a static IP: Elastic IP > allocate new address > associate to instance.

  • Create an SSH private key (key.pem) and save it wherever you want in your computer.

At this point you should have the following information:

  • user: your username for accessing the host server
  • IP: the host server IP
  • EFES: the name of your EFES folder; in this example we assume that it is initially located on the Desktop of your computer
  • key.pem: the name of your private key (if you need one)
  • path-to-folder-containing-key.pem: the path to the folder where the private key is saved (if you have one)

NB: If you are going to use a private key, before executing scp and ssh commands you will need to enter the computer folder in which you have saved the private key, by typing in the Terminal the command cd path-to-folder-containing-key.pem; also, you will need to replace each ssh with ssh -i key.pem, each scp -r with scp -ri key.pem and each rsync -re ssh with rsync -re 'ssh -i key.pem'.

2. Install Oracle Java Development Kit 8 (Oracle JDK 8) on the server and set JAVA_HOME

scp -r ~/Desktop/jdk-8u321-linux-x64.tar.gz user@IP:~
ssh user@IP
sudo mv jdk-8u321-linux-x64.tar.gz /opt/
cd /opt
sudo tar xzf jdk-8u321-linux-x64.tar.gz
cd jdk1.8.0_321
sudo update-alternatives --install /usr/bin/java java /opt/jdk1.8.0_321/bin/java 2
sudo update-alternatives --config java
sudo update-alternatives --install /usr/bin/jar jar /opt/jdk1.8.0_321/bin/jar 2
sudo update-alternatives --set jar /opt/jdk1.8.0_321/bin/jar
sudo update-alternatives --set javac /opt/jdk1.8.0_321/bin/javac
sudo nano /etc/environment

add JAVA_HOME="/opt/jdk1.8.0_321/" and then save and close (ctrl+O, enter, ctrl+X)

export JAVA_HOME="/opt/jdk1.8.0_321/"
export JRE_HOME="/opt/jdk1.8.0_321/jre"
export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
source /etc/environment
echo $JAVA_HOME

3. Install Apache on the server and configure it as a proxy to Jetty

NB: this step may vary significantly depending on the used OS. The current instructions work for Linux Debian and Ubuntu. If you are using Linux CentOS, replace every occurrence of apache2 with httpd.

  • Install Apache:
sudo apt install apache2
sudo service apache2 start

or:

sudo yum -y install httpd
sudo service httpd start
  • Remove the default homepage index.html:
unlink /var/www/html/index.html
  • Enable the Apache modules for reverse proxying:
cd /usr/lib/apache2/modules
sudo a2enmod proxy_html
sudo a2enmod proxy_http

(or: sudo nano /etc/httpd/conf.modules.d/00-proxy.conf and uncomment LoadModule proxy_module modules/mod_proxy.so and LoadModule proxy_http_module modules/mod_proxy_http.so)

  • Create an admin password:
sudo htpasswd -c /etc/apache2/.htpasswd admin
  • Edit the Apache configuration file to configure it as a proxy to Jetty and to restrict the access to the admin interface only to authorized users:
sudo nano /etc/apache2/sites-available/000-default.conf

(or: sudo nano /etc/httpd/sites-available/000-default.conf; or: sudo nano /etc/httpd/httpd.conf; or: sudo nano /usr/local/etc/httpd/httpd.conf)

and add the following inside <VirtualHost *:80>:

    ServerName localhost
    ProxyVia Off
    ProxyPreserveHost On
    ProxyRequests Off
    ProxyPass / http://localhost:9999/
    ProxyPassReverse / http://localhost:9999/

    <Location ~ "/(admin|solr|openrdf-workbench)">
        AuthType Basic
        AuthName "Restricted Content"
        AuthUserFile /etc/apache2/.htpasswd
        Require user admin
        Order allow,deny
        Allow from IP
        Satisfy Any
    </Location>

If you need to restrict the access to the site to authorized users only during construction, add also the following:

    <Location />
        AuthType Basic
        AuthName "Restricted Content"
        AuthUserFile /etc/apache2/.htpasswd
        Require user admin
        Order allow,deny
        Allow from IP
        Satisfy Any
    </Location>

If you prefer to restrict the access to the site to authorized users only during construction BUT also making certain pages public, add the following instead, specifying in SetEnvIf their URIs:

<Location ~ "/(.+?)/($|inscriptions|texts|indices|search|concordances)">
    SetEnvIf Request_URI /inscriptions/0001.html noauth=1
    AuthType Basic
    AuthName "Restricted Content"
    AuthUserFile /etc/apache2/.htpasswd
    Require env noauth
    Require user admin
    Order allow,deny
    Allow from IP
    Satisfy any
</Location>

then save and close (ctrl+O, enter, ctrl+X)

sudo systemctl restart apache2

(or: systemctl restart httpd)

4. Copy or clone the EFES repository into the server

You can either copy the EFES repository from your computer (4a) or clone it from GitHub (4b).

4a. Copy the EFES repository into the server and move it into /var/www/html

scp -r ~/Desktop/EFES user@IP:~
ssh user@IP
sudo mv ~/EFES /var/www/html

4b. Clone the EFES repository from GitHub into /var/www/html

cd /var/www/html
sudo git clone https://github.com/EpiDoc/EFES.git

(or the address of your fork/copy of the repository on GitHub)

5. If necessary, require permissions for executing build.sh and ant

chmod 777 /var/www/html/EFES/build.sh
chmod 777 /var/www/html/EFES/sw/ant/bin/ant

6. Execute build.sh to start the EFES site

ssh user@IP
cd /var/www/html/EFES
screen
./build.sh

ctrl+A, ctrl+D

Your EFES site is running now! You can now close the Terminal, because screen allows you to close it without stopping the process that is keeping the EFES site running.

NB: once the site is running, you will have to re-create a new Sesame repository through the Admin interface (Admin > Components > Sesame > Admin > New repository > create new repository with Type 'In Memory Store RDF Schema and Direct Type Hierarchy' and ID 'kiln'), right after typing http://localhost:9999/openrdf-sesame (or http://IP:80/openrdf-sesame, or http://IP:80/rdf4j-server if using Kiln) in the 'Change Server' field and clicking on 'Change'.

7. Stop the EFES site (if needed)

ssh user@IP
screen -r

ctrl+C

exit

8. Update the EFES site with rsync or git pull

8a. Update the EFES site with rsync

rsync -re ssh --progress --delete --exclude={'.git/','*.log','sw/jetty/temp'} ~/Desktop/EFES user@IP:~
ssh user@IP
screen -r

ctrl+C

rsync -r --progress --delete ~/EFES /var/www/html
./build.sh

ctrl+A, ctrl+D

8b. Update the EFES site with git pull (if cloned from GitHub)

ssh user@IP
screen -r

ctrl+C

sudo git pull
./build.sh

ctrl+A, ctrl+D

9. Update the site automatically from the GitHub repository (if cloned from GitHub)

9a. Set up a GitHub Action to update the site

  • In the GitHub repository settings go to 'Secrets' and add three new repository settings: HOST (the host server IP), USERNAME and PASSWORD
  • Go to the GitHub Actions section and set up a new workflow. This will create the file .github/workflows/main.yml, in which to put the following content:
name: CI
on:
  push:
    branches: [ master ]
  workflow_dispatch:
jobs:
  build:
    name: Build
    runs-on: ubuntu-latest
    steps:
      - name: Update site
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.HOST }}
          username: ${{ secrets.USERNAME }}
          password: ${{ secrets.PASSWORD }}
          script: cd /var/www/html/EFES && sudo git pull

If you want also to automate the harvesting and indexing, at the end of the last line you can add && sudo curl http://IP/admin/rdf/harvest/all.html && sudo curl http://IP/admin/solr/index/all.html (replacing IP with the host server IP); if curl is not installed, you can install it with sudo apt-get install curl.

NB: this setup works with servers that can be accessed simply with username + password via SSH; if you use SSH keys or other more secure access modes, the setup of the GitHub Action will be more complex.

9b. Use crontab to update the site

  • Install crontab
  • Run sudo crontab -e and add the following:

/5 * * * * cd /var/www/html/EFES && sudo git pull (to pull changes from GitHub, e.g. every 5 minutes)

1 1 * * * curl http://USERNAME:PASSWORD@IP/admin/rdf/harvest/all.html (to reharvest once a day e.g. at 1:01)

3 1 * * * curl http://USERNAME:PASSWORD@IP/admin/solr/index/all.html (to reindex once a day e.g. at 1:03)

@reboot sudo sh /var/www/html/EFES/build.sh (restart after reboot)

sudo may not be required. If required, to allow crontab to execute commands as sudo without inserting the password, run sudo visudo and add at end $USER ALL=(ALL) NOPASSWD: ALL.

10. Configure SSL

See an example here

Clone this wiki locally