Difference between revisions of "Red Cloud Linux Instances"
(Replaced several occurrences of Eucalyptus with OpenStack, refined language to fit OpenStack, more needed) |
|||
(38 intermediate revisions by 6 users not shown) | |||
Line 1: | Line 1: | ||
− | + | Linux Instances can be created and maintained on [[Red_Cloud|Red Cloud]] using the [https://redcloud.cac.cornell.edu OpenStack Web Interface]. This documentation assumes a basic understanding of [[OpenStack]], so please review that page as needed. If you are '''new to Linux''', you may want to read the [[Linux Tutorial]] first. It is also a good idea to be familiar with the [[Linux Tutorial]] if you have not previously done '''Linux system administration''', which is an assumed prerequisite to managing Linux Instances. Additionally, you may find the [https://cvw.cac.cornell.edu/Linux/ Introduction to Linux] topic on the [https://cvw.cac.cornell.edu/topics Cornell Virtual Workshop] helpful. | |
− | |||
− | |||
− | Linux Instances can be created and maintained on [[Red_Cloud|Red Cloud]] using the [https://redcloud.cac.cornell.edu OpenStack Web Interface]. This documentation assumes a basic understanding of [[OpenStack]], so please review that page as needed. If you are '''new to Linux''', you may want to read the [[Linux Tutorial]] first. It is also a good idea to be familiar with the [[Linux Tutorial]] if you have not previously done '''Linux system administration''', which is an assumed prerequisite to managing Linux Instances. | ||
__TOC__ | __TOC__ | ||
Line 15: | Line 12: | ||
=== Steps === | === Steps === | ||
# Log in to the [https://redcloud.cac.cornell.edu OpenStack Web Interface] (check out [[OpenStack#Logging_In|how to log in]] if you need to) | # Log in to the [https://redcloud.cac.cornell.edu OpenStack Web Interface] (check out [[OpenStack#Logging_In|how to log in]] if you need to) | ||
− | + | # If you have not already, [[OpenStack Key Pairs#Creating_a_Key_Pair|create a key pair]] | |
− | + | # If you have not already, [[OpenStack Security Groups#Creating a Security Group|create a security group]]. Note that your security group should include the inbound SSH rule so you can connect to it. | |
− | # If you have not already, | + | # '''Optional:''' [[Networks#Private Networks|Set up a Private Network]] |
− | # If you have not already, | + | # Select <tt>Launch Instance</tt> from the [https://redcloud.cac.cornell.edu/dashboard/project/instances/ Instances] page |
− | # '''Optional:''' [[OpenStack# | + | # Follow the instructions about [[OpenStack#Launch an Instance|launching a new instance]], and select one of the a CentOS or Ubuntu [[Images|images]] under the <tt>Source</tt> tab |
− | # [[ | + | # '''Optional:''' [[Volumes#Create and Attach a Volume|Create and attach a Volume]] |
+ | # '''Optional:''' [[Networks#Floating IP Addresses|Associate a Floating IP address]] | ||
− | Now that you have created and launched an instance, your next steps will be to [[ | + | Now that you have created and launched an instance, your next steps will be to [[#Accessing_Instances|connect to it]] and set up a new user account. See the [[Linux_Tutorial#Initial_User_Setup_2|CentOS steps]] or [[Linux_Tutorial#Initial_User_Setup|Ubuntu steps]] for more information on how to set up a new user, update, and install software for each distribution. |
== Accessing Instances == | == Accessing Instances == | ||
− | + | First, establish access to your instance using [[Connect_to_Linux#Using_Secure_Shell|Secure Shell (ssh)]], possibly including [[Connect_to_Linux#Using_X-Windows|X Windows]] for remote graphical display. If you are at all unfamiliar with Linux, we recommend following the [[Linux Tutorial]]. If you would like to have a desktop environment (not available by default for Linux instances), then you may want to follow the steps for [[XFCE Desktop on CentOS]]. Establishing an SSH connection is a prerequisite for creating a Linux desktop environment such as XFCE. | |
+ | |||
+ | === Secure Shell (SSH) === | ||
+ | |||
+ | The main requirements for logging in to your instance using secure shell are: | ||
+ | :* The [[OpenStack Security Groups|security group]] for your instance must permit SSH connections (TCP port 22) from your current IP address. | ||
+ | |||
+ | :* You must use the private key that matches the public key in the [[OpenStack Key Pairs|key pair]] you specified when launching the instance. | ||
+ | |||
+ | :* You must log in to your instance using the correct initial username: | ||
+ | :** For CentOS 6 and 7, the username is <tt>centos</tt>, | ||
+ | :** For CentOS 8, the username is <tt>cloud-user</tt>, and | ||
+ | :** For Ubuntu, it is <tt>ubuntu</tt>. | ||
+ | |||
+ | To log in through SSH, specify the key pair file (for example <tt>my_key.pem</tt>), | ||
+ | username and IP address as follows: | ||
+ | |||
+ | ssh -i my_key.pem <username>@<IP address of your instance> | ||
+ | |||
+ | For more information on how to use SSH, see the [[Connect to Linux]] page. | ||
Note: Transferring files can also be done over <code>ssh</code>. See the <code>scp</code> and <code>sftp</code> commands, or programs like [//winscp.net/eng/index.php WinSCP] and [//apple.stackexchange.com/questions/25661/whats-a-good-graphical-sftp-utility-for-os-x similar software for Mac OS X]. | Note: Transferring files can also be done over <code>ssh</code>. See the <code>scp</code> and <code>sftp</code> commands, or programs like [//winscp.net/eng/index.php WinSCP] and [//apple.stackexchange.com/questions/25661/whats-a-good-graphical-sftp-utility-for-os-x similar software for Mac OS X]. | ||
+ | |||
+ | === Troubleshooting === | ||
+ | |||
+ | There are several common error messages you may see when trying to access your Linux instance using SSH. | ||
+ | These are the most likely causes for each common message. | ||
+ | |||
+ | :* '''"Connection timed out"''' means that your SSH command can't communicate at all with the instance. | ||
+ | :** Has the instance been started? Check the instance's console in the [[OpenStack|OpenStack Web Interface]]. Did the instance boot successfully? | ||
+ | :** Do the [[OpenStack Security Groups|security group's]] rules allow incoming ssh connections (TCP port 22) from ''the IP address of your computer''? | ||
+ | :** Is your instance on the [[Networks#Public Network|public network]]? If not, does it have an associated [[Networks#Floating IP Addresses|floating IP address]] and are you using the correct address? | ||
+ | |||
+ | :* An error like '''"Permissions <4-digits> for <key-file-name> are too open"''' means you can reach the instance, but your private key file has improper permissions. | ||
+ | :** Make sure your private key file is saved as a ".pem" extension and that it has the proper permissions: <p><code>chmod 600 <key name>.pem</code></p> | ||
+ | |||
+ | :* '''"Permission denied (<some details>)"''' means that the combination of the username and SSH key you are providing are not correct for this instance. | ||
+ | :** Make sure you are using the correct username: | ||
+ | :*** If you are using Ubuntu, did you login as the '''ubuntu''' user? For more information on that, see the [[Linux_Tutorial#The_.22ubuntu.22_User|Linux Tutorial]]. | ||
+ | :*** If you are using CentOS 6 or 7, did you login as the '''centos''' user? | ||
+ | :*** If you are using CentOS 8, did you login as the '''cloud-user''' user? | ||
+ | :*** If you are using the CentOS MATLAB instance, did you login as the '''root''' user? | ||
+ | :** Did you [[OpenStack_Key_Pairs#Creating_a_Key_Pair|create a key pair]] and make sure to [[OpenStack_Key_Pairs#Selecting_a_Key_Pair_When_Creating_an_Instance|select it when creating the instance]]? | ||
+ | :** Are you supplying your key pair in the command? See the [[#Secure_Shell_.28SSH.29|SSH]] instructions above for an example. | ||
+ | :* If you get an '''unexpected password prompt''': | ||
+ | :** Did you use the correct user name? See suggestions above. | ||
+ | :** Make sure your private key matches the public key of the [[OpenStack Key Pairs|key pair]]. | ||
+ | :**# On your computer, run: <code>ssh-keygen -y -f <private key file></code> | ||
+ | :**# Confirm that the output matches the public key on Red Cloud at [https://redcloud.cac.cornell.edu/dashboard/project/key_pairs https://redcloud.cac.cornell.edu/dashboard/project/key_pairs]/<your keypair name>. | ||
=== XFCE Desktop on CentOS === | === XFCE Desktop on CentOS === | ||
Line 36: | Line 80: | ||
==== XFCE Setup ==== | ==== XFCE Setup ==== | ||
− | # Log in as root | + | # Log in as root via ssh as [[#Secure_Shell_.28ssh.29 | described above]]. |
# Install needed packages: | # Install needed packages: | ||
#* <code>yum groupinstall xfce</code> | #* <code>yum groupinstall xfce</code> | ||
Line 47: | Line 91: | ||
==== VNC User Setup ==== | ==== VNC User Setup ==== | ||
− | For each user that will want to use the XFCE Desktop, you will need to set up VNC | + | For each user that will want to use the XFCE Desktop, you will need to set up VNC (Virtual Network Computing) cability. To do this, follow the directions below. Alternatively, there is also an [https://linuxtechlab.com/secure-vnc-server-tls-encryption/ Easy guide to secure VNC server with TLS encryption]. |
# Open a shell as that user | # Open a shell as that user | ||
Line 72: | Line 116: | ||
# run <code>~/bin/start_vncserver.sh</code> | # run <code>~/bin/start_vncserver.sh</code> | ||
− | You can '''stop''' the NVC server by running <code>vncserver -kill :1</code> | + | To find the port, you can run e.g. <code>ps gxu | grep vnc</code>. |
+ | |||
+ | You can '''stop''' the NVC server by running <code>vncserver -kill :1</code>. If this doesn't work, | ||
+ | you can try <code>pkill Xvnc</code>. | ||
To '''change or reset the VNC password''', you can simply run <code>vncpassword</code> | To '''change or reset the VNC password''', you can simply run <code>vncpassword</code> | ||
Line 92: | Line 139: | ||
'''Note for Windows users:''' You can also find useful instructions on the [[VNC Tunnel Windows]] page. You may also want to look into [https://fossbytes.com/enable-built-windows-10-openssh-client/ OpenSSH] or [https://www.windowscentral.com/how-install-bash-shell-command-line-windows-10 Bash on Ubuntu on Windows]. | '''Note for Windows users:''' You can also find useful instructions on the [[VNC Tunnel Windows]] page. You may also want to look into [https://fossbytes.com/enable-built-windows-10-openssh-client/ OpenSSH] or [https://www.windowscentral.com/how-install-bash-shell-command-line-windows-10 Bash on Ubuntu on Windows]. | ||
+ | |||
+ | == Instance Maintenance == | ||
+ | |||
+ | All self-managed desktops, laptops, servers, and Red Cloud instances, both Windows and Linux, should be updated with Operating System, Adobe Flash, Acrobat, and Reader critical and security updates on a '''''monthly''''' basis. For Linux instances: | ||
+ | # Check for updates | ||
+ | #* Ubuntu: <code>sudo apt update</code> | ||
+ | #* CentOS: <code>yum check-update</code> | ||
+ | # Install updates | ||
+ | #* Ubuntu: <code>sudo apt upgrade</code> | ||
+ | #* CentOS: <code>sudo yum update</code> | ||
+ | # Reboot the instance with <code>sudo reboot</code> on both Ubuntu and CentOS | ||
+ | |||
+ | Before rebooting make sure to save all active work. Rebooting will disconnect you from the instance. Wait a minute or two to allow the instance to restart before reconnecting. When you reconnect, you should verify that the updates were applied by repeating step 1. | ||
== Initialize and Mount a Volume == | == Initialize and Mount a Volume == | ||
− | The instructions here are for formatting and mounting | + | The instructions here are for formatting and mounting [[Volumes|attached volumes]], though steps like these can only be performed if you have [[Volumes#Create_and_Attach_a_Volume|allocated and attached the volume]] through OpenStack, which can be done while the instance is running. |
− | + | '''Note:''' These instructions assume you are the [[Linux_Tutorial#Definitions|root user]]. If you are not (such as on [[Linux_Tutorial#The_.22ubuntu.22_User|Ubuntu]]), then you may need to prepend <code>sudo</code> where appropriate. | |
− | # | + | |
− | # | + | # Set up file system: |
− | + | #* <code>mkfs.ext4 /dev/vdb</code> | |
− | # Set up file system | + | # Make a directory where device will be mounted, for example "<tt>scratch</tt>": |
− | #* <code>mkfs.ext4 /dev/ | + | #* <code>mkdir /scratch</code> |
− | # Make a directory where device will be mounted | + | # Mount the device: |
− | #* <code>mkdir / | + | #* <code>mount /dev/vdb /scratch</code> |
− | # Mount the device | ||
− | #* <code>mount /dev/ | ||
# To have the mount automatically renewed after reboot, add an <code>fstab</code> entry (this is a little dangerous) | # To have the mount automatically renewed after reboot, add an <code>fstab</code> entry (this is a little dangerous) | ||
#* <code>vim /etc/fstab</code> | #* <code>vim /etc/fstab</code> | ||
#* Add a line with tab separations between four fields: disk device, mounted location, "ext4", "defaults": | #* Add a line with tab separations between four fields: disk device, mounted location, "ext4", "defaults": | ||
− | #** <code>/dev/ | + | #** <code>/dev/vdb /scratch ext4 defaults</code> |
− | |||
− | |||
− | |||
− |
Revision as of 15:09, 4 November 2020
Linux Instances can be created and maintained on Red Cloud using the OpenStack Web Interface. This documentation assumes a basic understanding of OpenStack, so please review that page as needed. If you are new to Linux, you may want to read the Linux Tutorial first. It is also a good idea to be familiar with the Linux Tutorial if you have not previously done Linux system administration, which is an assumed prerequisite to managing Linux Instances. Additionally, you may find the Introduction to Linux topic on the Cornell Virtual Workshop helpful.
Creating a New Linux Instance
You can boot an instance with most modern Linux distributions. Currently Red Cloud offers pre-made VM images running the following Linux distributions:
- CentOS
- Ubuntu (including LTS)
Steps
- Log in to the OpenStack Web Interface (check out how to log in if you need to)
- If you have not already, create a key pair
- If you have not already, create a security group. Note that your security group should include the inbound SSH rule so you can connect to it.
- Optional: Set up a Private Network
- Select Launch Instance from the Instances page
- Follow the instructions about launching a new instance, and select one of the a CentOS or Ubuntu images under the Source tab
- Optional: Create and attach a Volume
- Optional: Associate a Floating IP address
Now that you have created and launched an instance, your next steps will be to connect to it and set up a new user account. See the CentOS steps or Ubuntu steps for more information on how to set up a new user, update, and install software for each distribution.
Accessing Instances
First, establish access to your instance using Secure Shell (ssh), possibly including X Windows for remote graphical display. If you are at all unfamiliar with Linux, we recommend following the Linux Tutorial. If you would like to have a desktop environment (not available by default for Linux instances), then you may want to follow the steps for XFCE Desktop on CentOS. Establishing an SSH connection is a prerequisite for creating a Linux desktop environment such as XFCE.
Secure Shell (SSH)
The main requirements for logging in to your instance using secure shell are:
- The security group for your instance must permit SSH connections (TCP port 22) from your current IP address.
- You must use the private key that matches the public key in the key pair you specified when launching the instance.
- You must log in to your instance using the correct initial username:
- For CentOS 6 and 7, the username is centos,
- For CentOS 8, the username is cloud-user, and
- For Ubuntu, it is ubuntu.
- You must log in to your instance using the correct initial username:
To log in through SSH, specify the key pair file (for example my_key.pem), username and IP address as follows:
ssh -i my_key.pem <username>@<IP address of your instance>
For more information on how to use SSH, see the Connect to Linux page.
Note: Transferring files can also be done over ssh
. See the scp
and sftp
commands, or programs like WinSCP and similar software for Mac OS X.
Troubleshooting
There are several common error messages you may see when trying to access your Linux instance using SSH. These are the most likely causes for each common message.
- "Connection timed out" means that your SSH command can't communicate at all with the instance.
- Has the instance been started? Check the instance's console in the OpenStack Web Interface. Did the instance boot successfully?
- Do the security group's rules allow incoming ssh connections (TCP port 22) from the IP address of your computer?
- Is your instance on the public network? If not, does it have an associated floating IP address and are you using the correct address?
- "Connection timed out" means that your SSH command can't communicate at all with the instance.
- An error like "Permissions <4-digits> for <key-file-name> are too open" means you can reach the instance, but your private key file has improper permissions.
- Make sure your private key file is saved as a ".pem" extension and that it has the proper permissions:
chmod 600 <key name>.pem
- Make sure your private key file is saved as a ".pem" extension and that it has the proper permissions:
- An error like "Permissions <4-digits> for <key-file-name> are too open" means you can reach the instance, but your private key file has improper permissions.
- "Permission denied (<some details>)" means that the combination of the username and SSH key you are providing are not correct for this instance.
- Make sure you are using the correct username:
- If you are using Ubuntu, did you login as the ubuntu user? For more information on that, see the Linux Tutorial.
- If you are using CentOS 6 or 7, did you login as the centos user?
- If you are using CentOS 8, did you login as the cloud-user user?
- If you are using the CentOS MATLAB instance, did you login as the root user?
- Did you create a key pair and make sure to select it when creating the instance?
- Are you supplying your key pair in the command? See the SSH instructions above for an example.
- Make sure you are using the correct username:
- If you get an unexpected password prompt:
- Did you use the correct user name? See suggestions above.
- Make sure your private key matches the public key of the key pair.
- On your computer, run:
ssh-keygen -y -f <private key file>
- Confirm that the output matches the public key on Red Cloud at https://redcloud.cac.cornell.edu/dashboard/project/key_pairs/<your keypair name>.
- On your computer, run:
- "Permission denied (<some details>)" means that the combination of the username and SSH key you are providing are not correct for this instance.
XFCE Desktop on CentOS
In case one would prefer a desktop environment over a command-line, this section details the steps to setting up an XFCE Desktop Environment on a CentOS 7.4 instance. Other typical Linux desktop environments, such as Gnome, are also available but XFCE is used as an example here. Additionally, setting up a desktop environment may work on Ubuntu instances as well, with some differences. Once the environment is set up, you can run a VNC server and connect to it through an ssh tunnel.
XFCE Setup
- Log in as root via ssh as described above.
- Install needed packages:
yum groupinstall xfce
yum install tigervnc-server
- Install some additional software that most users will want. These are only suggestions, and this is not a comprehensive list
yum install gedit
yum install firefox
- LibreOffice
VNC User Setup
For each user that will want to use the XFCE Desktop, you will need to set up VNC (Virtual Network Computing) cability. To do this, follow the directions below. Alternatively, there is also an Easy guide to secure VNC server with TLS encryption.
- Open a shell as that user
vncpasswd
- Sets the user's VNC password
- This step is not necessary for read-only VNC
- This creates a ~/.vnc folder
vim ~/.vnc/xstartup
- Paste this text into the file:
#!/bin/bash
xrdb $HOME/.Xresources
startxfce4 &
- Paste this text into the file:
chmod 775 ~/.vnc/xstartup
mkdir ~/bin
vim ~/bin/start_vncserver.sh
- Paste this text into the file:
#!/bin/bash
vncserver -geometry 1680x1050
- Paste this text into the file:
chmod 775 ~/bin/start_vncserver.sh
Using VNC
A brief overview for users is provided here, and for more information please see the Using VNC section on our Getting Started page.
Manage the VNC Server
Whenever an instance gets rebooted, you can restart the VNC server by doing the following
- ssh into the instance
- run
~/bin/start_vncserver.sh
To find the port, you can run e.g. ps gxu | grep vnc
.
You can stop the NVC server by running vncserver -kill :1
. If this doesn't work,
you can try pkill Xvnc
.
To change or reset the VNC password, you can simply run vncpassword
To change the screen resolution:
- Permanently: edit the script
~/.vnc/xstartup
- For this session only: use Applications->Display
- Permanently: edit the script
View the Desktop
- Open a local shell and ssh to establish the tunnel (recommended local port 10000):
ssh -L 10000:localhost:5901 <remote ip>
- Note: When this shell is closed, the VNC viewer will have to close, too, though VNC is still running.
- In a VNC viewer app, connect to the VNC server
localhost:10000
(or whichever port you chose above).- This should open a window showing the desktop
- The first time you do this:
- For the remote computer, you may have to dismiss a warning dialog
- You will need to initialize a "panel". Click "Use default config"
Note for Windows users: You can also find useful instructions on the VNC Tunnel Windows page. You may also want to look into OpenSSH or Bash on Ubuntu on Windows.
Instance Maintenance
All self-managed desktops, laptops, servers, and Red Cloud instances, both Windows and Linux, should be updated with Operating System, Adobe Flash, Acrobat, and Reader critical and security updates on a monthly basis. For Linux instances:
- Check for updates
- Ubuntu:
sudo apt update
- CentOS:
yum check-update
- Ubuntu:
- Install updates
- Ubuntu:
sudo apt upgrade
- CentOS:
sudo yum update
- Ubuntu:
- Reboot the instance with
sudo reboot
on both Ubuntu and CentOS
Before rebooting make sure to save all active work. Rebooting will disconnect you from the instance. Wait a minute or two to allow the instance to restart before reconnecting. When you reconnect, you should verify that the updates were applied by repeating step 1.
Initialize and Mount a Volume
The instructions here are for formatting and mounting attached volumes, though steps like these can only be performed if you have allocated and attached the volume through OpenStack, which can be done while the instance is running.
Note: These instructions assume you are the root user. If you are not (such as on Ubuntu), then you may need to prepend sudo
where appropriate.
- Set up file system:
mkfs.ext4 /dev/vdb
- Make a directory where device will be mounted, for example "scratch":
mkdir /scratch
- Mount the device:
mount /dev/vdb /scratch
- To have the mount automatically renewed after reboot, add an
fstab
entry (this is a little dangerous)vim /etc/fstab
- Add a line with tab separations between four fields: disk device, mounted location, "ext4", "defaults":
/dev/vdb /scratch ext4 defaults