Difference between revisions of "Tips and tricks cjc73"

From CAC Documentation wiki
Jump to navigation Jump to search
Line 67: Line 67:
  
 
<pre>
 
<pre>
% nano ~/.ssh/config
+
% nano ~/.zshrc
 
</pre>
 
</pre>
  
Line 88: Line 88:
 
source ~/.zshrc
 
source ~/.zshrc
 
</pre>
 
</pre>
 
  
 
== Connecting to Jupyter Lab with a configured client ==
 
== Connecting to Jupyter Lab with a configured client ==

Revision as of 18:37, 7 January 2022

Prevent accidental writes to mount point folders

Suppose you have a folder at /mnt/mountpoint that you would like to use as a mount point for a volume. Because the mount point is a valid path to a location on the boot volume, it is possible to write data to the mount point even when the volume is not mounted. Data written to the mount point will not be on the external volume --- it will be on the boot volume. It can a source of confusion, especially because copying data into the mount point and then later properly mounting a volume will hide any data on the boot drive that is in the mountpoint directory.

To avoid this, make the mountpoint directory unwritable so attempting to write data to the mountpoint when the volume is not mounted will generate an error.

sudo chattr +i /mnt/mountpoint

Configure screen with .screenrc

Using a program like screen to keep your session active makes your computation robust to network interruption and disconnections. This means you can keep a process running on an instance even if your local machine is turned off or not connected to the internet.

Screen also has a number of optional features that make working from a remote terminal more pleasant. One feature is the "hard status" bar across the bottom the screen, which is roughly analogous to a tab bar. It will show the open screen windows as numbered "tabs" and the currently open window will be highlighted. A window can be renamed by entering the command sequence ctrl-a, shift-a. The new name will show in the tab bar.

You can enable the hard status bar (and extended scroll back history) by using nano to create a file called .screenrc in your home directory.

% nano ~/.screenrc

Copy and paste the following into the nano editor:

#termcapinfo xterm* ti@:te@
autodetach on # Autodetach session on hangup instead of terminating screen completely
startup_message off # Turn off the splash screen
defscrollback 30000 # Use a 30000-line scrollback buffer
hardstatus on
hardstatus alwayslastline
hardstatus string "%{.bW}%-w%{.rW}%n %t%{-}%+w %=%{..G} %H %{..Y} %m/%d %C%a "

Use ctrl-X to stop editing and follow the prompts to save the file. The next time you launch screen, it will show the status bar along the bottom.


macOS client config for remote Jupyter access

You can set up configuration files, aliases, and functions to make remote Jupyter work more streamlined. In this section, we configure ssh and create three commands to manage the Jupyter connection via ssh.

Note, the % at the start of a line indicates that the rest of the text is a command to be entered in the command line (Terminal.app or iTerm). The % is not part of the command. The term "VM" means "the Red Cloud virtual machine instance" you want to connect to. If you have multiple VMs, you can make additional entries.

The examples in this section assume:

  1. your VM username is netid42
  2. your SSH/Cloud key-pair is in ~/.ssh/id_rsa4096
  3. your VM IP address is 128.84.YY.XXX
  4. Google Chrome is installed in your /Applications folder (for app mode, optional)


First edit or create ~/.ssh/config. In Terminal.app enter the following to open the ssh config file in the nano text editor. Use the arrow keys to move the cursor as needed.

% nano ~/.ssh/config

Once nano opens, add an entry like the following:

Host 128.84.YY.XXX                   # your VM IP          
   User netid42                      # your user name on the VM  
   IdentityFile ~/.ssh/id_rsa4096    # the path to your ssh key file

Use ctrl-X to stop editing and follow the prompts to save the file.

Next, add an alias to your shell profile. For recent versions of macOS, this is in a file called ~/.zshrc. If you still have bash as your main shell, try ~/.bash_profile.

% nano ~/.zshrc

Add entries like the following three lines, making the appropriate substitutions for netid42, IP address, and key file. Replace "myVM" with any short name you would like to use as the alias name (no spaces or special characters). If your VM address changes in the future, be sure to update these entries.

alias myVM='ssh netid42@128.84.10.222'
myVM_nb() { ssh -N -L "$1":localhost:"$2" netid42@128.84.YY.XXX; }

Optionally, add this line to enable opening jupyter lab in Chrome's "app mode". It presents a less cluttered browser window. No changes are needed unless Chrome is not installed in the Applications folder.

lab_appp() { /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --app="http://localhost:""$1""/?token=""$2"";"; }

Use ctrl-X to stop editing and follow the prompts to save the file.

To make the changes active, close and reopen Terminal.app or run the following command in each active local Terminal.

source ~/.zshrc

Connecting to Jupyter Lab with a configured client

Starting the notebook server

Once the above configuration is in place, launching and connecting to jupyter on the vm takes three steps:

1. Connect via SSH, launch screen, and then launch jupyter

  1. Open a terminal and enter the command to connect to your VM. In the example above, the command is named "myVM". Substitute the name you chose.
% myVM

Once connected to the VM, enter the following via SSH (choose something descriptive in place of "myproject", but do not use spaces or special characters):

% screen -DR myproject

(screen will open) Now change directory (cd) to the project folder and launch jupyter. Substitue the correct path to your project in place of "/project/directory/".

% cd /project/directory/
% jupyter lab --no-browser

Jupyter will launch and print some output to the terminal screen. The last lines will be similar to:

    Or copy and paste one of these URLs:
        http://localhost:8888/lab?token=c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f
     or http://127.0.0.1:8888/lab?token=c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f

The 4 to 5 numbers following http://localhost: in the first URL are the remote port number. In this example, the remote port number is 8888. Note the remote port number and substitute this value in place of RRRR in the commands below.

The token is the alphanumeric string after token=. Copy this value to your system clipboard.

2. Use a local terminal to set up ssh port forwarding

In a local terminal (not SSH-ed into the VM), set up the port forwarding between the remote and local ports. If you don't have jupyter lab running locally, you can choose 8888 as the local port number. Otherwise, any open port will work (if it is not open, the command below will generate an error to that effect). Local ports in the 8888 - 8899 range tend to work well on macOS. In the command below, substitute the local port number in place of the LLLL and the remote port number from the step above in place of RRRR. (If you choose a different name for your myVM_nb command, substitute it below.)

% myVM_nb LLLL RRRR

If this command is successful, the cursor will move to the start of the next line and no messages will print. As long as this command is running, the ssh port forwarding will be enabled. Later, type Ctrl-C to stop the forwarding.

3. Connect via a web browser

Finally, connect your web browser to the local port. To use Chrome's app mode, use the command we defined earlier:

% lab_appp LLLL <paste token here>

For example, a complete command might look like:

% lab_appp 8889 c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f

Chrome will open in app mode and load jupyter from the VM via the ssh tunnel. It may prompt you to enter the token once more. If so, simply paste the token into the dialog.

Alternatively, you can edit the url from jupyter to open in any browser by replacing the remote port (RRRR) with the local port number (LLLL). Jupyter will generate a random token each time it opens so be sure you are editing your URL and not this example: Edit

http://127.0.0.1:RRRR/lab?token=c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f

to

http://127.0.0.1:LLLL/lab?token=c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f

and paste into a web browser.

Reconnecting to the running server

Since the jupyter server is running in a screen session, it will stay running when you disconnect. To reconnect, you only need to re-establish the local port forwarding (step 2). If you closed your local browser window, you will also need to reopen the browser (step 3). If jupyter is still loaded in the browser it might reconnect automatically once port forwarding is re-established, otherwise, select "Reconnect to Kernel" from the Kernel menu in the Jupyter web interface.


Jupyter lab extensions

To enable extensions, choose the View > Activate Command Palette menu item and then type Enable Extension Manager and press return. Choose "enable" on the notification. Be sure to install only reputable extensions.


  1. jupytext
  2. jupyterlab_spellchecker
  3. to be continued....