SSH tunnel for using web app on the cluster¶

Some apps like Jupyter or Anvi'o are operated via a web interface. When run on a cluster node, it seems to be impossible to access to their web interface. It is possible to use a ssh tunnel to access them.

For pedagogic purpose, we will run, on a compute node, jupyter-lab from a conda environment we installed.

Set the environment¶

Optional

This section is only needed if you need a custom environment. If the software or your environment is already installed, there is nothing to do.

We create a jupyter-lab environment with conda this way:

# We connect to the cluster# Replace <login> with your Genotoul-Bioinfo login and enter your password when asked.ssh <login>@genobioinfo.toulouse.inrae.fr# We enable conda on the clustermodule load devel/Miniforge/Miniforge3# We create a conda env containing jupyter-lab and the python kernel# in ~/work/tuto-jupyterconda create -p ~/work/tuto-jupyter jupyterlab ipykernel...

Run the software on a node¶

Nothing special

Here we will run jupyter-lab on a compute node in an interactive job. The approach is the same as usual.

First we run an interactive job with srun:

# You must be connected on the cluster beforehand# We run an interactive session using 2 cores, 2GiB RAM and with 1h duration:srun -c 2 --mem 2G --time 01:00:00 --pty bash# We check on which compute node we are. We will be called it `<compute-node>`.hostnamen001
# We enable conda on the nodemodule load devel/Miniforge/Miniforge3# We activate the `tuto-jupyter` environmentsource activate ~/work/tuto-jupyter# We run jupyter-lab where options:# --no-browser : don't start the browser# --ip 0.0.0.0 : accept connection from any computer (access will be controlled with a token)jupyter lab --no-browser --ip 0.0.0.0 ...

To access the server, open this file in a browser:
file:///.../jupyter/runtime/jpserver-711529-open.html
Or copy and paste one of these URLs:
http://0.0.0.0:8888/lab?token=...
http://127.0.0.1:8888/lab?token=...
...

From this point, jupyter-lab is running. Next step it to create the ssh tunnel to access it.

Create the SSH tunnel¶

Prerequisite

To create a SSH tunnel, two things are required:

The node where the software, here jupyter-lab, is running. It is n001 in the example. We name it <compute-node>.
The port number on which the software is listening. In example, jupyter-lab displays the url http://0.0.0.0:8888/lab?token=... The number 8888 is the port number. We name it <remote-port>.

Any software must provide this information, or you must set it when running it.

On your computer, open a second terminal while keeping the fist one opened. Connect to cluster with the following command:

ssh -N -L 8889:n001:8888 <login>@genobioinfo.toulouse.inrae.fr

Replace <login> with your login and enter your Genotoul-Bioinfo password when asked.

The command gives the impression to never finish and it is something normal. You must keep it in this state in order to connect to jupyter-lab.

You can stop it by hitting the Ctrl+C combination. If you stop it by mistake, run the command again.

Adapt it to your needs

The usual pattern for this command is:

ssh -N -L <local-port>:<compute-node>:<remote-port> <login>@genobioinfo.toulouse.inrae.fr

The value <local-port> can be any value between 1024 and 65535 (even the same value than <remote-port>).

The option -L is called local port redirection. Everything from your computer that goes in the network port <local-port> will by transferred to the <remote-port> port of <compute-node> through the ssh connection it created.

Connect to the software on compute node¶

Finally, run a browser on your own computer, and enter the following web address in navigation field.

http://localhost:8889

The jupyter-lab page must appears and a connection token may be asked. The token is available in the url http://127.0.0.1:8888/lab?token=... displayed by jupter-lab. It is the ... part after token=.

Note

If you use another value for <local-port> in place of 8889, please use your own value.

Stop everything¶

There are two things to do:

Stop jupyter-lab by hitting Ctrl+C combination in the terminal where you run srun and then stop the interactive session itself as usual (exit or Ctrl+D )
Stop the tunnel by hitting Ctrl+C in its dedicated terminal