Difference between revisions of "Tips and tricks cjc73"
(15 intermediate revisions by the same user not shown) | |||
Line 6: | Line 6: | ||
<!-- # <code>sudo chmod a-rwx /mnt/mountpoint</code> --> | <!-- # <code>sudo chmod a-rwx /mnt/mountpoint</code> --> | ||
− | <code>sudo chattr +i/mnt/mountpoint</code> | + | <code>sudo chattr +i /mnt/mountpoint</code> |
== Configure screen with .screenrc == | == Configure screen with .screenrc == | ||
Line 36: | Line 36: | ||
− | == macOS client config == | + | == macOS client config for remote Jupyter access == |
− | You can set up configuration files, aliases, and functions to make remote work more streamlined. Note, the <code>%</code> at the start of a line indicates that the rest of the text is a command to be entered in the command line (Terminal.app or iTerm). The <code>%</code> is not part of the command. The term "VM" means "the Red Cloud virtual machine instance" | + | You can set up configuration files, aliases, and functions to make remote Jupyter work more streamlined. In this section, we configure ssh and create three commands to manage the Jupyter connection via ssh. |
+ | |||
+ | Note, the <code>%</code> at the start of a line indicates that the rest of the text is a command to be entered in the command line (Terminal.app or iTerm). The <code>%</code> is not part of the command. The term "VM" means "the Red Cloud virtual machine instance" you want to connect to. If you have multiple VMs, you can make additional entries. | ||
The examples in this section assume: | The examples in this section assume: | ||
Line 46: | Line 48: | ||
# Google Chrome is installed in your /Applications folder (for app mode, optional) | # Google Chrome is installed in your /Applications folder (for app mode, optional) | ||
− | + | ==== 1. Add your VM information to SSH config ==== | |
First edit or create <code>~/.ssh/config</code>. In Terminal.app enter the following to open the ssh config file in the nano text editor. Use the arrow keys to move the cursor as needed. | First edit or create <code>~/.ssh/config</code>. In Terminal.app enter the following to open the ssh config file in the nano text editor. Use the arrow keys to move the cursor as needed. | ||
<pre> | <pre> | ||
Line 52: | Line 54: | ||
</pre> | </pre> | ||
− | Once nano opens add an entry like the following: | + | Once nano opens, add an entry like the following: |
<pre> | <pre> | ||
Line 62: | Line 64: | ||
Use <code>ctrl-X</code> to stop editing and follow the prompts to save the file. | Use <code>ctrl-X</code> to stop editing and follow the prompts to save the file. | ||
− | Next, | + | ==== 2. Add shortcuts (alias and functions) to your shell profile ==== |
+ | Next, edit your shell profile in nano to add a few commands. For recent versions of macOS, the shell profile is called <code>~/.zshrc</code>. If you still have bash as your main shell, try editing <code>~/.bash_profile</code>. | ||
<pre> | <pre> | ||
− | % nano ~/. | + | % nano ~/.zshrc |
</pre> | </pre> | ||
− | + | Use the nano editor to add entries like those shown in following two code cells, '''making the appropriate substitutions for netid42, IP address, and key file'''. Replace "myVM" with any short name you would like to use as the alias name (no spaces or special characters). If your VM address changes in the future, be sure to update these entries. | |
<pre> | <pre> | ||
+ | # Add these lines to your shell profile, with appropriate substitutions: | ||
+ | # shortcut to connect to server: | ||
alias myVM='ssh netid42@128.84.10.222' | alias myVM='ssh netid42@128.84.10.222' | ||
+ | # shortcut to create an SSH tunnel | ||
myVM_nb() { ssh -N -L "$1":localhost:"$2" netid42@128.84.YY.XXX; } | myVM_nb() { ssh -N -L "$1":localhost:"$2" netid42@128.84.YY.XXX; } | ||
</pre> | </pre> | ||
Line 77: | Line 83: | ||
Optionally, add this line to enable opening jupyter lab in Chrome's "app mode". It presents a less cluttered browser window. No changes are needed unless Chrome is not installed in the Applications folder. | Optionally, add this line to enable opening jupyter lab in Chrome's "app mode". It presents a less cluttered browser window. No changes are needed unless Chrome is not installed in the Applications folder. | ||
<pre> | <pre> | ||
+ | # Add this to your shell profile so you can use Chrome in app mode: | ||
lab_appp() { /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --app="http://localhost:""$1""/?token=""$2"";"; } | lab_appp() { /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --app="http://localhost:""$1""/?token=""$2"";"; } | ||
</pre> | </pre> | ||
Use <code>ctrl-X</code> to stop editing and follow the prompts to save the file. | Use <code>ctrl-X</code> to stop editing and follow the prompts to save the file. | ||
+ | |||
+ | ==== 3. Activate your profile changes ==== | ||
+ | To make the changes active, close and reopen Terminal.app or run the following command in each active local Terminal. | ||
+ | <pre> | ||
+ | source ~/.zshrc | ||
+ | </pre> | ||
+ | |||
+ | == Connecting to Jupyter Lab with a configured client == | ||
+ | |||
+ | === Starting the notebook server === | ||
+ | |||
+ | Once the above configuration is in place, launching and connecting to jupyter on the vm takes three steps: | ||
+ | |||
+ | ==== 1. Connect via SSH, launch <code>screen</code>, and then launch jupyter ==== | ||
+ | |||
+ | # Open a terminal and enter the command to connect to your VM. In the example above, the command is named "myVM". Substitute the name you chose. | ||
+ | |||
+ | <pre> | ||
+ | % myVM | ||
+ | </pre> | ||
+ | |||
+ | Once connected to the VM, enter the following via SSH (choose something descriptive in place of "myproject", but do not use spaces or special characters): | ||
+ | <pre> | ||
+ | % screen -DR myproject | ||
+ | </pre> | ||
+ | (screen will open) | ||
+ | Now change directory (cd) to the project folder and launch jupyter. Substitue the correct path to your project in place of "/project/directory/". | ||
+ | <pre> | ||
+ | % cd /project/directory/ | ||
+ | % jupyter lab --no-browser | ||
+ | </pre> | ||
+ | |||
+ | Jupyter will launch and print some output to the terminal screen. The last lines will be similar to: | ||
+ | |||
+ | <pre> | ||
+ | Or copy and paste one of these URLs: | ||
+ | http://localhost:8888/lab?token=c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f | ||
+ | or http://127.0.0.1:8888/lab?token=c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f | ||
+ | </pre> | ||
+ | |||
+ | The 4 to 5 numbers following <code>http://localhost:</code> in the first URL are the remote port number. In this example, the remote port number is 8888. Note the remote port number and substitute this value in place of RRRR in the commands below. | ||
+ | |||
+ | The token is the alphanumeric string after <code>token=</code>. Copy this value to your system clipboard. | ||
+ | |||
+ | ==== 2. Use a local terminal to set up ssh port forwarding ==== | ||
+ | |||
+ | In a local terminal (not SSH-ed into the VM), set up the port forwarding between the remote and local ports. If you don't have jupyter lab running locally, you can choose 8888 as the local port number. Otherwise, any open port will work (if it is not open, the command below will generate an error to that effect). Local ports in the 8888 - 8899 range tend to work well on macOS. In the command below, substitute the local port number in place of the LLLL and the remote port number from the step above in place of RRRR. | ||
+ | (If you choose a different name for your <code>myVM_nb</code> command, substitute it below.) | ||
+ | |||
+ | <pre> | ||
+ | % myVM_nb LLLL RRRR | ||
+ | </pre> | ||
+ | |||
+ | If this command is successful, the cursor will move to the start of the next line and no messages will print. As long as this command is running, the ssh port forwarding will be enabled. Later, type Ctrl-C to stop the forwarding. | ||
+ | |||
+ | ==== 3. Connect via a web browser ==== | ||
+ | |||
+ | Finally, connect your web browser to the local port. | ||
+ | To use Chrome's app mode, use the command we defined earlier: | ||
+ | <pre> | ||
+ | % lab_appp LLLL <paste token here> | ||
+ | </pre> | ||
+ | For example, a complete command might look like: | ||
+ | <pre> | ||
+ | % lab_appp 8889 c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f | ||
+ | </pre> | ||
+ | |||
+ | Chrome will open in app mode and load jupyter from the VM via the ssh tunnel. It may prompt you to enter the token once more. If so, simply paste the token into the dialog. | ||
+ | |||
+ | Alternatively, you can edit the url from jupyter to open in any browser by replacing the remote port (RRRR) with the local port number (LLLL). Jupyter will generate a random token each time it opens so be sure you are editing your URL and not this example: | ||
+ | |||
+ | Edit | ||
+ | <pre>http://127.0.0.1:RRRR/lab?token=c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f</pre> | ||
+ | to | ||
+ | <pre>http://127.0.0.1:LLLL/lab?token=c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f</pre> | ||
+ | and paste into a web browser. | ||
+ | |||
+ | === Reconnecting to the running server === | ||
+ | |||
+ | Since the jupyter server is running in a <code>screen</code> session, it will stay running when you disconnect. To reconnect, you only need to re-establish the local port forwarding (step 2). If you closed your local browser window, you will also need to reopen the browser (step 3). If jupyter is still loaded in the browser it might reconnect automatically once port forwarding is re-established, otherwise, select "Reconnect to Kernel" from the Kernel menu in the Jupyter web interface. | ||
+ | |||
+ | |||
+ | == Jupyter lab extensions == | ||
+ | |||
+ | To enable extensions, choose the View > Activate Command Palette menu item and then type Enable Extension Manager and press return. Choose "enable" on the notification. Be sure to install only reputable extensions. | ||
+ | |||
+ | |||
+ | # jupytext | ||
+ | # jupyterlab_spellchecker | ||
+ | # to be continued.... |
Latest revision as of 21:00, 7 January 2022
Prevent accidental writes to mount point folders
Suppose you have a folder at /mnt/mountpoint that you would like to use as a mount point for a volume. Because the mount point is a valid path to a location on the boot volume, it is possible to write data to the mount point even when the volume is not mounted. Data written to the mount point will not be on the external volume --- it will be on the boot volume. It can a source of confusion, especially because copying data into the mount point and then later properly mounting a volume will hide any data on the boot drive that is in the mountpoint directory.
To avoid this, make the mountpoint directory unwritable so attempting to write data to the mountpoint when the volume is not mounted will generate an error.
sudo chattr +i /mnt/mountpoint
Configure screen with .screenrc
Using a program like screen to keep your session active makes your computation robust to network interruption and disconnections. This means you can keep a process running on an instance even if your local machine is turned off or not connected to the internet.
Screen also has a number of optional features that make working from a remote terminal more pleasant. One feature is the "hard status" bar across the bottom the screen, which is roughly analogous to a tab bar. It will show the open screen windows as numbered "tabs" and the currently open window will be highlighted. A window can be renamed by entering the command sequence ctrl-a, shift-a
. The new name will show in the tab bar.
You can enable the hard status bar (and extended scroll back history) by using nano
to create a file called .screenrc
in your home directory.
% nano ~/.screenrc
Copy and paste the following into the nano editor:
#termcapinfo xterm* ti@:te@ autodetach on # Autodetach session on hangup instead of terminating screen completely startup_message off # Turn off the splash screen defscrollback 30000 # Use a 30000-line scrollback buffer hardstatus on hardstatus alwayslastline hardstatus string "%{.bW}%-w%{.rW}%n %t%{-}%+w %=%{..G} %H %{..Y} %m/%d %C%a "
Use ctrl-X
to stop editing and follow the prompts to save the file. The next time you launch screen
, it will show the status bar along the bottom.
macOS client config for remote Jupyter access
You can set up configuration files, aliases, and functions to make remote Jupyter work more streamlined. In this section, we configure ssh and create three commands to manage the Jupyter connection via ssh.
Note, the %
at the start of a line indicates that the rest of the text is a command to be entered in the command line (Terminal.app or iTerm). The %
is not part of the command. The term "VM" means "the Red Cloud virtual machine instance" you want to connect to. If you have multiple VMs, you can make additional entries.
The examples in this section assume:
- your VM username is
netid42
- your SSH/Cloud key-pair is in
~/.ssh/id_rsa4096
- your VM IP address is 128.84.YY.XXX
- Google Chrome is installed in your /Applications folder (for app mode, optional)
1. Add your VM information to SSH config
First edit or create ~/.ssh/config
. In Terminal.app enter the following to open the ssh config file in the nano text editor. Use the arrow keys to move the cursor as needed.
% nano ~/.ssh/config
Once nano opens, add an entry like the following:
Host 128.84.YY.XXX # your VM IP User netid42 # your user name on the VM IdentityFile ~/.ssh/id_rsa4096 # the path to your ssh key file
Use ctrl-X
to stop editing and follow the prompts to save the file.
2. Add shortcuts (alias and functions) to your shell profile
Next, edit your shell profile in nano to add a few commands. For recent versions of macOS, the shell profile is called ~/.zshrc
. If you still have bash as your main shell, try editing ~/.bash_profile
.
% nano ~/.zshrc
Use the nano editor to add entries like those shown in following two code cells, making the appropriate substitutions for netid42, IP address, and key file. Replace "myVM" with any short name you would like to use as the alias name (no spaces or special characters). If your VM address changes in the future, be sure to update these entries.
# Add these lines to your shell profile, with appropriate substitutions: # shortcut to connect to server: alias myVM='ssh netid42@128.84.10.222' # shortcut to create an SSH tunnel myVM_nb() { ssh -N -L "$1":localhost:"$2" netid42@128.84.YY.XXX; }
Optionally, add this line to enable opening jupyter lab in Chrome's "app mode". It presents a less cluttered browser window. No changes are needed unless Chrome is not installed in the Applications folder.
# Add this to your shell profile so you can use Chrome in app mode: lab_appp() { /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --app="http://localhost:""$1""/?token=""$2"";"; }
Use ctrl-X
to stop editing and follow the prompts to save the file.
3. Activate your profile changes
To make the changes active, close and reopen Terminal.app or run the following command in each active local Terminal.
source ~/.zshrc
Connecting to Jupyter Lab with a configured client
Starting the notebook server
Once the above configuration is in place, launching and connecting to jupyter on the vm takes three steps:
1. Connect via SSH, launch screen
, and then launch jupyter
- Open a terminal and enter the command to connect to your VM. In the example above, the command is named "myVM". Substitute the name you chose.
% myVM
Once connected to the VM, enter the following via SSH (choose something descriptive in place of "myproject", but do not use spaces or special characters):
% screen -DR myproject
(screen will open) Now change directory (cd) to the project folder and launch jupyter. Substitue the correct path to your project in place of "/project/directory/".
% cd /project/directory/ % jupyter lab --no-browser
Jupyter will launch and print some output to the terminal screen. The last lines will be similar to:
Or copy and paste one of these URLs: http://localhost:8888/lab?token=c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f or http://127.0.0.1:8888/lab?token=c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f
The 4 to 5 numbers following http://localhost:
in the first URL are the remote port number. In this example, the remote port number is 8888. Note the remote port number and substitute this value in place of RRRR in the commands below.
The token is the alphanumeric string after token=
. Copy this value to your system clipboard.
2. Use a local terminal to set up ssh port forwarding
In a local terminal (not SSH-ed into the VM), set up the port forwarding between the remote and local ports. If you don't have jupyter lab running locally, you can choose 8888 as the local port number. Otherwise, any open port will work (if it is not open, the command below will generate an error to that effect). Local ports in the 8888 - 8899 range tend to work well on macOS. In the command below, substitute the local port number in place of the LLLL and the remote port number from the step above in place of RRRR.
(If you choose a different name for your myVM_nb
command, substitute it below.)
% myVM_nb LLLL RRRR
If this command is successful, the cursor will move to the start of the next line and no messages will print. As long as this command is running, the ssh port forwarding will be enabled. Later, type Ctrl-C to stop the forwarding.
3. Connect via a web browser
Finally, connect your web browser to the local port. To use Chrome's app mode, use the command we defined earlier:
% lab_appp LLLL <paste token here>
For example, a complete command might look like:
% lab_appp 8889 c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f
Chrome will open in app mode and load jupyter from the VM via the ssh tunnel. It may prompt you to enter the token once more. If so, simply paste the token into the dialog.
Alternatively, you can edit the url from jupyter to open in any browser by replacing the remote port (RRRR) with the local port number (LLLL). Jupyter will generate a random token each time it opens so be sure you are editing your URL and not this example:
Edit
http://127.0.0.1:RRRR/lab?token=c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f
to
http://127.0.0.1:LLLL/lab?token=c23ab79e182775d3b987b74b7c52a358aa8c38493d42ee1f
and paste into a web browser.
Reconnecting to the running server
Since the jupyter server is running in a screen
session, it will stay running when you disconnect. To reconnect, you only need to re-establish the local port forwarding (step 2). If you closed your local browser window, you will also need to reopen the browser (step 3). If jupyter is still loaded in the browser it might reconnect automatically once port forwarding is re-established, otherwise, select "Reconnect to Kernel" from the Kernel menu in the Jupyter web interface.
Jupyter lab extensions
To enable extensions, choose the View > Activate Command Palette menu item and then type Enable Extension Manager and press return. Choose "enable" on the notification. Be sure to install only reputable extensions.
- jupytext
- jupyterlab_spellchecker
- to be continued....