Open Remote Jupyter Notebook

Here we are setting up the steps to open an jupyter notebook that is located on the cluster remotely on your local machine (i.e., your laptop or desktop). This will allow the functionality and convenience of running an jupyter notebook on your laptop browser while accessing data and computational power afforded by the cluster.

Step 1: ssh onto the cluster like you normally would.

ssh [email protected]

Example: ssh [email protected]

Step 2: Activate the environment you would like to use.

projectname_env

Example: emu_env

Step 3: Navigate to the directory that has the jupyter notebook that you want to work on.

Example: cd /home/data/madlab/scripts/wmaze

Step 4: Type on the command line the following command to run jupyter without a browser directing it to a specific port, with XXXX replaced with a valid port number.

jupyter notebook --no-browser --port=XXXX

Example: jupyter notebook --no-browser --port=7000

_Notes: 7000 was chosen randomly. If multiple users will be utilizing this functionality they'll need to direct their separate jupyter notebook sessions to distinct ports.

If you want to reconnect to the JN in the remote cluster, you can use the same pair of ports you used last time._

Step 5: Open up a terminal on your local machine (e.g., laptop) and access the port using an SSH tunnel with port forwarding. The command for this looks like:

ssh -N -f -L localhost:8888:localhost:7000 [email protected]

Note: if the above command does not work, try the following ones: ssh -NfL 8888:localhost:7000 [email protected] ssh -X -N -f -L localhost:8888:localhost:7000 [email protected]

"8888" and "7000" were arbitrarily chosen.

-N flag is used to tell SSH to NOT execute a remote command -f flag tells SSH to go into the background after the port forwards are established -L sets up the port forwards --> In the example above the first pair of localhost:8888 says to forward port 8888 on the local host (e.g. laptop) to the SSH port 22. The second pair of localhost:7000 says to have the ssh port 22 on the remote machine (e.g. cluster) be forward to port 7000 which is where the jupyter notebook is running.

The overall idea of the command is to tell SSH to connect to the remote machine, not read data from stdin and go into the background successfully after establishing the port forwards.

Notes: 5000 and 7000 were arbitrarily chosen. If multiple users are using this functionality they'll need to select different port numbers. Again, if you want to reconnect to the JN in the remote cluster, you can use the same two pairs of ports you used last time.

STEP 6: On your local machine (e.g., laptop) open your preferred web browser (e.g., chrome) and type in the URL field:

http://localhost:8888

Note: 8888 was arbitrarily chosen. You can use the same first port you used to forward your port. For instance, if you used: ssh -N -f -L localhost:8010:localhost:7000 [email protected], in the URL field you can type: http://localhost:8010

You should now see your jupyter notebooks on your local machine in your web browser. Sometimes JN will ask you for a token/password. If asked, refer back to the original terminal where you typed in jupyter notebook --no-browser --port=7000. Underneath, in the links offered to open the JN you will find the token requested (e.g., token=1631393e136025fd9d36a3d18e7edcf517918bb24764357e). If password protection has been enabled on the remote side you may need to type in a password. This can be set up on the same page.

Check out here for original description on how to set this up.

NOTE

If you get the following error after shutting down a session:

bin: Address already in use

You may need to kill whatever is using the port with the following command:

lsof -ti:#### | xargs kill -9 (replacing #### with the occupied port)