SSH and VSCode Setup

Recommendations for Remote Development using SSH and VSCode

The Visual Studio Code Remote – SSH extension allows you to open a remote folder on any remote machine, virtual machine, or container with a running SSH server and take full advantage of VS Code’s feature set. Once connected to a server, you can interact with files and folders anywhere on the remote filesystem.

Users Gabriel Sarch and Nicholas Blauch hypothesize that the resources used by a large number of users running Visual Studio code (or other remote development tools) on the head node is contributing to some of the slowdowns users have been seeing. The following is a solution they found to connect these processes to the compute nodes instead of the head node. If their hypothesis is correct, we expect users adopting this method will reduce resources used by VSCode on the head node and improve its performance for all users.

How to use VsCode interactive Python capabilities on a compute node

The following content is from this Stack Overflow solution.

ssh to the remote cluster ssh username@mind.cs.cmu.edu
Once on the cluster, add your public key to the authorized keys. You will typically append the content of ~/.ssh/id_rsa.pub (on your local machine) to ~/.ssh/authorized_keys (on the cluster) Note: To configure SSH authorized keys you may need to generate an SSH key pair on your local computer (i.e. terminal for unix, powershell for windows). To do this, use ssh-keygen, which is included with the standard OpenSSH suite of tools. You will need to find the the .pub file based on the output path of the ssh-keygen tool.
Allocate some resources inside the cluster (srun --pty bash). You can do this in a screen session so that the job stays running even after you log out.
Get the name of the compute node, typically visible in the command line as username@mind-x-x).
On your local machine, modify the ~/.ssh/config file. Using Gabriel’s example, modify your ssh config file, replacing Gabriel’s userid with yours :

Host mind.cs.cmu.edu
   HostName mind.cs.cmu.edu
   User gsarch

Host mind-*
    HostName %h
    ProxyJump mind.cs.cmu.edu
    User gsarch

Since the compute nodes are not directly accessible from outside the cluster, we are configuring a proxy jump.

A proxy jump in SSH (Secure Shell) is a way to connect to a remote server through an intermediary server. This is particularly useful when the target server is not directly accessible from your local machine but can be reached through another server that you have access to.

(For this explanation I am going to use compute node mind-1-1 but this applies to all compute nodes)

When you set up a proxy jump, your SSH connection first goes to the jump host (mind.cs.cmu.edu) and then from there, it connects to the target server mind-1-1. This is done seamlessly, so you only need to issue a single command to reach the target machine. (Your local machine connects to mind.cs.cmu.edu and mind.cs.cmu.edu then connects to the target machine (in this case we are using, mind-1-1)

Once the changes to the ssh config file are applied, you will be able to access the node directly from VSCode by:

CTRL+P > Remote-SSH: Connect to Host…
Type the compute node name mind-x-x (e.g. mind-1-1) at the prompt (do NOT select mind.cs.cmu.edu).
You will get connected to the node, now every interactive python you run (including jupyter and jupytext) will have access to your allocated resources

The node can also be directly accessed with “ssh mind-x-x”.

Useful tip:

Command to find the GPU ID slurm allocated to your job.

$ scontrol show job -d 12345 | grep -i 'Gres=gpu' Nodes=mind-1-30 CPU_IDs=26-29 Mem=14336 GRES=gpu(IDX:1)

Where IDX:1 == /dev/nvidia1

Step-by-Step Guide to Using SSH, SLURM and VSCode on mind.cs.cmu.edu

Download and Install PuTTY (Windows Users Only)
- If you’re using Windows, you’ll need an SSH client like PuTTY. You can download it from here.
Log onto the Head Node
- For Windows Users:
  - Open PuTTY.
  - Enter mind.cs.cmu.edu as the Host Name and click “Open”.
  - Log in with your username and password.
- For macOS Users:
  - Open the Terminal application.
  - Connect to the head node by entering the following command:
```
ssh your_username@mind.cs.cmu.edu
```
  - Replace your_username with your actual username.
Request a Compute Node Using SLURM
- Once logged in, request a compute node by running the following command:
```
srun -p cpu --mem=30GB --time=4:00:00 --pty bash
```
- This command requests a node from the cpu partition with 30GB of memory for 4 hours.
Identify Your Assigned Compute Node
- After running the command, you’ll see a prompt similar to this:
```
[dpane@mind ~]$ srun -p cpu --mem=30GB --time=4:00:00 --pty bash
[dpane@mind-0-37-1 ~]$
```
- This indicates that SLURM has assigned you the compute node mind-0-37-1.
Connect to the Compute Node Using VSCode
- Open VSCode and use the Remote – SSH extension to connect to the assigned compute node.
- In VSCode, go to the Command Palette (Ctrl+Shift+P) and select Remote-SSH: Connect to Host....
- Enter the compute node address, e.g., mind-0-37-1.

Important Note

Dynamic Node Assignment: Each time you request a compute node, you may be assigned a different one. Always check the node assigned to you before connecting with VSCode.

Problem solving

(Vladislav Ayzenberg provided information for this section)

Solving Initial Connection issues

When initially configuring VSCode, pay close attention to the permissions of the files and directories (i.e. private and public certificates and .ssh directory).

The following solution partially taken from this stack exchange solution [https://superuser.com/questions/215504/permissions-on-private-key-in-ssh-folder/215506#215506 ]

MacOS/UNIX File/Directory File Permissions
(Windows computer private/public key management and permissions differ from these instructions. However, they are equally as important to configure correctly.)

This help page has general information about File and Directory Permissions.

On your local machine, open a terminal window and confirm you are in your home directory.

imac:~ jim$ pwd
/Users/jim

If you are not in your home directory, you can use the following `cd` command to change to it.

imac:~ jim$ cd ~

Confirm that the .ssh directory has the correct permission. Running the following command should give you permission results similar to this..

imac:~ jim$ ls -la | grep .ssh
drwx------   6 jim  staff    192 Oct 11  2019 .ssh</pre

If the permissions aren't set correctly (drwx------ ) on your ~/.ssh directory, use the following command to set it:

imac:~ jim$  chmod 700 ~/.ssh/

You can check the file permission in the .ssh directory and should confirm they have the following permissions.

imac:~ jim$ ls -l ~/.ssh/
-rw-------  1 jim  staff  0 Jun 29 11:43 authorized_keys
-rw-------  1 jim  staff  0 Jun 29 11:43 id_rsa.pub
-rw-r--r--  1 jim  staff  0 Jun 29 11:42 id_rsa

If they don’t have the correct permission, use the following commands to change them to the appropriate settings.

	chmod 644 ~/.ssh/id_rsa.pub
	chmod 600 ~/.ssh/id_rsa
	chmod 600 ~/.ssh/authorized_keys

VSCode connection suddenly stops working

If you suddenly experience issues connecting to the cluster from VSCode after it had been working (regardless of what node you use). Try resolving the issue by simply deleting the .vscode-server/ directory in your home directory and then try reconnecting. This forces VScode to re-install the dependencies it needs the next time it reconnects.
(Note: The time to delete the .vscode-server/ directory may take a while. You may need to manually reinstall some extensions next time you connect.)

Updated on August 29, 2024

Was this article helpful?

Yes No

Need Help?

Can't find the answer you're looking for?

Contact NI Support