Open Remote Jupyter Notebook

Here we are setting up the steps to open an jupyter notebook that is located on the cluster remotely on your local machine (i.e., your laptop or desktop). This will allow the functionality and convenience of running an jupyter notebook on your laptop browser while accessing data and computational power afforded by the cluster.

Step 1: ssh onto the cluster like you normally would.

ssh [email protected]

Example: ssh [email protected]

Step 2: Activate the environment you would like to use.

projectname_env

Example: emu_env

Step 3: Navigate to the directory that has the jupyter notebook that you want to work on.

Example: cd /home/data/madlab/scripts/wmaze

Step 4: Type on the command line the following command to run jupyter without a browser directing it to a specific port, with XXXX replaced with a valid port number.

jupyter notebook --no-browser --port=XXXX

Example: jupyter notebook --no-browser --port=7000

Important: 7000 was chosen randomly. If multiple users will be utilizing this functionality they'll need to direct their separate jupyter notebook sessions to distinct ports.

Step 5: Open up a terminal on your local machine (e.g., laptop) and access the port using an SSH tunnel with port forwarding. The command for this looks like:

ssh -N -f -L localhost:8888:localhost:7000 [email protected]

-N flag is used to tell SSH to NOT execute a remote command -f flag tells SSH to go into the background after the port forwards are established -L sets up the port forwards --> In the example above the first pair of localhost:8888 says to forward port 8888 on the local host (e.g. laptop) to the SSH port 22. The second pari of localhost:7000 says to have the ssh port 22 on the remote machine (e.g. cluster) be forward to port 7000 which is where the jupyter notebook is running.

The overall idea of the command is to tell SSH to connect to the remote machine, not read data from stdin and go into the background successfully after establishing the port forwards.

Important: 5000 and 7000 were arbitrarily chosen. If multiple users are using this functionality they'll need to select different port numbers.

STEP 6: On your local machine (e.g., laptop) open your preferred web browser (e.g., chrome) and type in the URL field:

http://localhost:8888

You should now see your jupyter notebooks on your local machine in your web browser. If password protection has been enabled on the remote side you may need to type in a password.

Check out here for original description on how to set this up.

NOTE

If you get the following error after shutting down a session:

bin: Address already in use

You may need to kill whatever is using the port with the following command:

lsof -ti:#### | xargs kill -9 (replacing #### with the occupied port)