Red Cloud

From CAC Documentation wiki
Jump to: navigation, search

NOTE: this documentation is for Red Cloud with HPE Helion Eucalyptus 4.2, in service on January 25th, 2016.

Who Should Use this Documentation

The instructions on this page only apply to users who have been specifically designated as a Red Cloud subscription holder. Such individuals can create, manage, and delete virtual servers and storage in Red Cloud. These instructions do not apply to users of virtual machines running in Red Cloud.

Tips for New Users

New users would be best served by reading this complete document. At an absolute minimum, read the following sections before using your subscription:

First Time Login

Access Red Cloud via HPE Helion Eucalyptus User Console

Accounting, aka Don't use up your Subscription by Accident!

Virtual machines come in various configurations, or instance types. The user has complete control over access to the system, i.e., defining the firewall and setting up users and their permissions. This means that the primary user of a Linux system must be familiar with Linux system administration, which aside from the basics of using the command line, includes at a minimum, familiarity with creating and modifying users, installing software, remote login software and configuration, and managing/transferring data.

For users that want to use Red Cloud, but do not have much system administration experience, we've written a Basic Linux Tutorial that should work for RedHat/CentOS and Ubuntu Linux systems. Consulting is also available to answer general questions about systems administration, or for help on specific software and research problems.

HPE Helion Eucalyptus 4 Introduction

Red Cloud is an on-demand research computing service. This wiki provides documentation for “Red Cloud”, an Infrastructure as a Service (IaaS) that runs HPE Helion Eucalyptus, the open source cloud computing platform.

"Red Cloud" was updated to Eucalyptus 4.2.1 in January 2016. [HPE Helion Eucalyptus Documentation]

“Red Cloud” with Eucalyptus has two components:

  • Virtual Server Instances: Users can create virtual servers with up to 28 CPU cores. Each instance gets 4 to 8 GB RAM/core, depending on the instance type and which cloud is hosting it. Users have root access to their virtual servers and complete flexibility in their configurations.
  • Volumes: Users can create virtual disks that can be attached to and detached from their virtual servers on demand for permanent data storage.

There are two Red Cloud installations: Red Cloud Ithaca located in Ithaca, NY and Red Cloud NYC located in New York City. Either of these can be used with your Red Cloud subscription, but projects on each cloud are not linked.

The current “Red Cloud Ithaca” configuration is hosted on 10 x 28-core Dell C6320 and 8 x 12-core Dell C6100 cloud servers in Ithaca for a total of 376 cores. The current “Red Cloud NYC” configuration is hosted on 4 24-core Dell C6220 for a total of 96 cores in NYC. Instances share 10Gb Ethernet connectivity with the other instances running on the same server. See the Working with Red Cloud Instances section for available instance types in Red Cloud Ithaca and Red Cloud NYC. We do not over-subscribe our servers to guarantee each instance exclusive access to the CPU cores and RAM with which it is configured.

  • In Red Cloud Ithaca, smaller instance types receive 4 GB/core, and larger instance types receive ~7 GB/core.
  • In Red Cloud NYC, all instance types receive 8 GB/core.

As with any IaaS cloud, users are expected to possess working knowledge and expertise in system administration to effectively secure, configure, and use their virtual server instances. Please refer to this wiki for helpful documentation. If you still have a question, contact us and we’ll be happy to answer a quick question or two. If you or your project team need additional help getting started, anticipate having lots of questions along the way, or you’d like us to do some work for you (such as building your image so that it’s all set to go), consulting is available as an option when you start your subscription. We can help you estimate the number of consulting hours to include with your subscription. Consulting may also be added after starting your subscription.

First Time Login

When you are added to a CAC project, you will receive an e-mail confirming your Red Cloud access. You must change the automatically generated password immediately for security reasons and to access computing resources. If you are a PI or a PI's proxy for a new project, verify that you have added a subscription to your project; see the Manage Projects Form URL on the How to Start a Project page. After waiting up to 4 hours for account information to propagate, you will then be ready to download the Eucalyptus credentials and start managing Red Cloud resources.

How to Access Red Cloud Resources?

Manage Red Cloud instances, images, volumes using any of the following tools:

Note: Regardless which method you choose (Graphical User Interface or Command Line Interface), you must follow the First Time Login instructions above.

To access running instances, most users will use ssh to access their Linux instances, and Remote Desktop (RDP) to access their Microsoft Windows instances.

Access Red Cloud via HPE Helion Eucalyptus User Console

While there are several ways to manage Red Cloud resources, most users will use the HPE Helion Eucalyptus User Console. This web-based console can be used to manage Red Cloud resources (instances and volumes) as follows:

  • Using an up-to-date web browser, log onto Eucalyptus User Console
    • the account is your CAC project with underscore removed, e.g. your account is dal160234 if you are on CAC project dal16_0234
    • the username and password are your CAC username and CAC password.
  • Click on "Create key pair" in the "Key pairs" box.


Select create keypair.png


  • Name the keypair "netid-key" (substitute your username for netid) and click on the "Create and Download" button.


CreateKeyPair Euca4.png


  • Your web browser will download a file named "netid-key.pem."
    • IT IS IMPORTANT to store this key in a safe place as you will need it to access your instances.
  • Click here for the Eucalyptus User Console documentation.

Access Red Cloud via Linux euca2ools CLI command-line tools

You may use euca2ools either locally or from an instance in Red Cloud itself:

  • To install euca2ools locally on your Linux host follow the installation instructions here.
  • As a quicker alternative, use the Eucalyptus User Console to launch a Red Cloud instance using the "centos-6.8-ebs-image/centos-6.8-ebs-image.img" (emi-81894f07) image and run the euca2ools CLI tools from this virtual machine to manage your Red Cloud instances.

Users wishing to use euca2ools, or other tools that employ euca2ools and the Eucalyptus API, will need to obtain credentials; see How do I download credentials to use with euca2ools/API?.

Working with Red Cloud Instances

Each instance is a virtual machine (VM) in the cloud. You can select CPU/RAM/disk configurations for the VM. The available VM configurations are:

Type CPUs RAM Disk
(ephemeral storage)
Red Cloud ITH (https://euca4.cac.cornell.edu)
m1.small 1 4 GB 100 GB
c1.medium 2 8 GB 200 GB
m1.large 4 16 GB 400 GB
m1.xlarge 8 32 GB 800 GB
c1.xlarge 12 48 GB 1000 GB
m2.xlarge 14 96 GB 2000 GB
m2.2xlarge 28 192 GB 6000 GB
Red Cloud NYC (https://euca4-nyc.cac.cornell.edu)
m1.small 1 8 GB 100 GB
c1.medium 2 16 GB 200 GB
m1.large 4 32 GB 400 GB
m1.xlarge 8 64 GB 800 GB
c1.xlarge 12 96 GB 1000 GB
m2.2xlarge 24 192GB 6000 GB

Note that the virtual cores map to individual physical cores, with hyperthreading disabled.

Ephemeral storage is typically available from an unmounted disk device /dev/vdb when you start a Linux instance. Note that any partitions and data you add to /dev/vdb will be lost when the instance is stopped or terminated.

Disk and network I/O requests are answered FIFO and are not limited based on instance type; Red Cloud is different from Amazon and Google Compute Engine in this regard.

You can boot an instance with most modern Linux distributions. Currently Red Cloud offers VM images running the following Linux distributions:

  • CentOS 6 (EBS and instance-store root devices),
  • CentOS 7 (EBS and instance-store root devices),
  • Ubuntu 12.04 LTS (EBS and instance-store root devices),
  • Ubuntu 14.04 LTS (EBS and instance-store root devices), and
  • Ubuntu 16.04 LTS (EBS and instance-store root devices)
  • For Windows version available see Red Cloud Windows instances

Please see the Working with Images section for information on EBS and instance-store images. After the instance is running, you can ssh into the instance as root using the private ssh key of the keypair you specified when you launched the instance.

The following operations can be performed on each instance:

  • Run: create a new instance.
  • Terminate: shuts down an instance. Note: everything on instance's local (ephemeral) storage, including the root file system if the instance is running an image with an instance-store root device, will be lost when an instance is terminated.
  • Reboot: reboot an instance. Unlike stopping and then starting (see below), this should preserve the currently registered public IP address.
  • Stop: This feature is available to only instances running EBS images. When you stop an instance, the virtual server is shut down gracefully. However, the instance ID, the contents and attached state of the EBS volume are preserved. euca-describe-instances and euca-describe-volumes will show the instance in "stopped" state with the EBS volume attached. Note: a stopped instance does NOT incur CPU-hour charges toward your subscription. However, the EBS storage charge will continue as the volume is still attached to the stopped instance.
  • Start: This feature is available to only stopped instances with EBS images. When you start a stopped instance, the virtual server is booted off the attached EBS volume. All the data on the EBS volume are preserved from when the instance was stopped. When an instance is started, the CPU-hour charges for the instance will resume.

How to Manage Instances via User Console | Command Line Quick Start

Persistent Versus Ephemeral Instances: Working with Elastic Block Storage (EBS) Volumes

Virtual machine instances are created based on images. There are two types of images: instance-store and EBS. Unless you know you want instance-store, please choose EBS, as instances created from EBS images will have their file-system persisted after the instance is stopped. Both types of instances can be terminated (permanently deleted), but only EBS can be stopped and restarted. Using EBS instances charge against the storage on the project.


EBS Volumes provides persistent storage where data can be saved after an instance is terminated. EBS volumes can be attached to any one instance and detached in real time, regardless on which node the instance is hosted. When an EBS volume is attached to an instance, it appears as a block device (/dev/vdX) in the instance. To attach a volume within a Linux instance, find a non-existent /dev/vdX (where X should be a lower case letter; a-z) and attach your volume to this using euca-attach-volume or the Eucalyptus web management console. Afterwards, the /dev/vdX file will be created and can be mounted as usual.

Currently the size of an EBS volume is limited to 15TB. However, LVM can be used to concatenate EBS volumes to create volumes larger than 15TB in Red Cloud.

The following operations can be performed on the volume:

  • Create: create a new volume. EBS volume charge will apply for all created volumes until they are deleted.
  • Attach: attach an existing volume to a running instance.
  • Detach: detach an attached volume from a running instance.
  • Delete: delete an existing volume.
  • Create a snapshot: create a snapshot of a volume.
  • Delete a snapshot: delete a snapshot of a volume.

Instructions: Manage Volumes via User Console | euca2ools CLI

Managing Network Access

Each instance is assigned to a security group when it is created. Instances in each security group are on the same VLAN, isolating them from other instances in the cloud as well as the Internet. Use security groups to control access to your instances. Note that instances cannot be assigned a new security group; however, you can edit access policies for the security group if needed.

By default a security group does not allow inbound access from outside the VLAN/security group. However, instances can access the public Internet. A rule must be added to a security group to allow inbound access to instances in the security group. Each rule is comprised of protocol, port number, and source IP. For example,

  • the rule "tcp, 22, 0.0.0.0/0" allows ssh access to all instances in the security group from anywhere on the Internet.
  • the rule "icmp, 0, 128.84.8.0/24" will enable instances in the security group to respond to pings from 128.84.8.x network.

The following operations can be performed on a security group:

  • Create: create a security group


CreateSecurityGroup Euca4.png


  • Delete: delete a security group
  • Authorize: add a rule to the security group
  • Revoke: delete a rule from the security group

Modifying an existing security group is a common occurrence as you may add services to your instance; this does not require a reboot of the instance, but may take a few seconds for the change to occur.

Instructions: Manage Security Groups via User Console | euca2ools CLI

Managing IP Addresses

Each instance has a public IP address and a private address. The private address is used by other instances in the cloud whereas the public address is used to access the instance from the Internet. The private address is assigned by the system. The public address can be either assigned by the system or user.

By default an instance gets a randomly assigned public IP address when it launches or starts from the stopped state. You can allocate an elastic IP address and assign it to the instance so the instance can be reached at a pre-determined IP address from the Internet. When you no longer need the elastic IP address, you can release it back to the system.

The following operations are available for managing (public) IP addresses:

  • Allocate: reserve an elastic IP address. You must first reserve an IP address before it can be assigned to one of your instances.
  • Describe: list all elastic IP addresses allocated by your project.
  • Associate: associate an elastic IP address to a running instance. This can be helpfully scripted from an instance using euca2ools: euca-associate-address -i i-12345678 <allocated ip> --region=account@redcloud-ith
  • Disassociate: dissociate an elastic IP address from a running instance. The system will assign the running instance a new public IP address from the pool of unreserved IP addresses.
  • Release: release an elastic IP address back to the cloud.

Instructions: Manage IP addresses via User Console | euca2ools CLI

Accessing Instances

To log in to a Linux instance, you will need an ssh client. On Linux, Mac, and most Unix systems, you can use OpenSSH's ssh command:

 ssh -Y -i userid-key.pem root@<public IP address>

e.g.

 ssh -Y -i beb82-mykey.pem root@128.84.8.42 

To login to a Linux instance from Windows, you can also use OpenSSH's ssh command as above if you install it in the Cygwin environment, but many find it more pleasant, quicker, and more space-efficient to use a stand-alone client like PuTTY. Note that with PuTTY, you will also need to convert the .pem file mentioned above to a .ppk file using PuTTYgen. Transferring files can also be done over ssh; see the sftp and scp commands, or programs like WinSCP and similar software for Mac OS X.

To log in to a Windows instance, wait for the Euca User Console to say it is running, then select instance -> Actions -> Connect to instance. Click the link under Password and open the .pem file from the key pair that you associated with this instance. The password for user Administrator will be displayed. You can then log in to the Windows instance using Microsoft's Remote Desktop program for Windows or Mac (it should be installed by default in Windows, and is available from the Mac App Store for macOS users). Linux/Unix users can connect using rdesktop. The name of the computer in Red Cloud is simply the public IP address of the instance (e.g., 128.84.8.42).

Working with Images

Two types of images can be used to boot Red Cloud instances:

  • [instance-store]: An instance-store image is stored in an S3 (object store) bucket. When you boot an instance-store, the image is first downloaded to the instance's local(ephemeral) disk and the instance is booted off ephemeral disk.
  • [EBS] (Elastic Block Storage): An EBS based image is stored in a volume snapshot. When you boot an EBS based image, an EBS volume is created from the snapshot, the volume is then attached to the instance and the instance is booted off the volume. Inside a virtual server, each attached EBS volume is a block device (e.g. /dev/vdc). Users can create EBS volumes up to 1TB in increments of 1GB. 50GB is included with each initial subscription. EBS volumes connect to Red Cloud via iSCSI for best performance (300+ MB/sec observed in testing). Cloud virtual servers can serve storage to other cloud virtual servers or clients external to the Red Cloud (e.g. NFS or CIFS shares, WebDAV, databases). Cloud virtual servers can access network storage external to Red Cloud.

Advantages of EBS based images include:

  • Ability to allocate a very large root file system (up to 1 TB). 50GB is included with subscription, usage greater than 50GB is billed at CAC storage rates.
  • Ability to run any kernel of your choice.
  • Ability to customize EBS based images easily:
    1. Boot an instance off an EBS image.
    2. Customize the instance to your liking.
    3. Take a snapshot of the EBS volume connected to the customized instance to create a newly customized EBS-based image.
  • Ability to stop (pause) and start (resume) an instance running an EBS image.

Advantages/disadvantages of instance-store images include:

  • File access is faster because an instance running instance-store image uses ephemeral disk for root file system.
  • A running instance-store image does not incur storage charges as long as you don't explicitly allocate and attach an EBS volume.
  • Can not be stopped and started

You need to use euca2ools CLI tool to create/modify/manage your custom images with the following operations (Note: currently, the Euca web interface does not correctly create Images from snapshots):

  • Using EBS: [How to use EBS] | instance-store image
  • Delete an image
  • Create an image from a snapshot: euca-register --name somename --description "describe me" --architecture x86_64 --snapshot snap-blahblah -b /dev/sdb=ephemeral0 --region=account-name@region-name
  • Modify Image ACL: When you create an customized image, by default the image is visible only to users in your project. If you want to make an image public (i.e. accessible to all Red Cloud users), do:
euca-modify-image-attribute -l -a all <image ID>
  • Creating an Image for a Custom OS
    • You'll need an installation of KVM on a local system and be sure to set USE_VIRTIO_DISK and USE_VIRTIO_NET equal to 1.
    • For detailed procedures see [Prepare a Linux System for Eucalyptus]

When you create an image base on an existing instance's root volume, either directly in one step or by first creating a snapshot, be certain you first delete /etc/udev/rules.d/70-persistent-net.rules, otherwise instances created from your new image will not boot! This file can be safely deleted without affecting the instance, and snapshots can be created of running instances, allowing you to create new instances based off of existing instances at almost any time. But be aware the some distributions might recreate this file while you are creating the snapshot, so you may want to probe for it a few times during the snapshot creation process. Also be sure to temporarily disable any boot-time configurations that may cause problems on the new instance, for instance, comment out any custom volumes as appropriate in /etc/fstab.

It can be a good idea to mount a volume partition based on a UUID so that your mount won't fail if Eucalyptus decides to attach or volume as a different device name. When mounting an LVM volume or using UUIDs for mounts, you need to be careful with cloning volumes, as cloning a volume will result in an identical UUID. This will affect it being used properly by mount or by LVM, as LVM also uses UUIDs behind the scenes. You can change a UUID like so ('man tune2fs' for more info):

tune2fs /dev/{device} -U {uuid}

Red Cloud Windows Instances

Creating and Working with Windows Instances

Both EBS and instance-store images are available for the following:

  • Windows Server 2008 R2 Datacenter with SP1
  • Windows Server 2012 R2 Datacenter
  • Windows 7 Enterprise with SP1

Most users will launch Windows instances using the Euca User Console. Be sure to get your <key>.pem file as outlined in the section Access Red Cloud via HPE Helion Eucalyptus User Console, which can then be used to obtain an administrative password to login to the instance as described in Accessing Instances.

Windows Instance Tips and Best Practices

  • Make sure your security group allows RDP port (3389) from your current IP address.
  • To log in to a Windows instance, wait for the Euca User Console says it is running, then select instance -> Actions -> Connect to instance. Click the link under Password and open the .pem file from the key pair that you associated with this instance. The password for user Administrator will be displayed. You can then log in to the Windows instance using Microsoft's Remote Desktop program for Windows or Mac, or with rdesktop for Linux. The name of the computer in Red Cloud is simply the public IP address of the instance (e.g., 128.84.8.42).
  • The first time you start a Windows instance it may take up to 10 minutes or more from pushing the start button, depending on the size of the instance, before you can log in.
  • After your instance is running you should be able to Remote Desktop into it.
  • We highly recommend you create a new Administrative account, this will persist if you stop and restart an instance. The default Administrator account password will change each time the instance is stopped and restarted. Accounts for users can be created in the Users and Groups control panel, as you would normally do in Windows.

Windows Activation

  • The Windows instance is not activated by default. If you attempt to activate you will most likely receive the following error:
    • Activation Error: Code 0x8007232b
    • To find a kms server, from a windows computer in your current domain (not the instance), start the command prompt as administrator and issue:
nslookup -type=all _vlmcs._tcp>kms.txt
  • The kms.txt file contains the information you need to activate windows in Red Cloud; there should be one or more entries for KMS servers. Pick one entry and use the svr hostname and port in the next step. Open a command prompt as administrator on the Windows instance and use the information from the kms.txt as follows:
cscript \windows\system32\slmgr.vbs /skms <svr hostname>:<port>
For example, you might enter the following:
cscript \windows\system32\slmgr.vbs /skms kms2.xyz.cornell.edu:1688
  • At this point, you should right click on Computer and select 'Properties' in your Windows instance, then activate it. Alternatively, you can enter the following command:
cscript \windows\system32\slmgr.vbs /ato
  • Please refer to this help page for more information on activating Windows:

EBS Windows instance notes

  • If you stop the instance, the default Administrator account password gets reset when you start the instance. You can get the password the same way as previously described.
  • If you create a new Admin or user account on the machine, the account and password are retained when stopping and starting the instance.
  • Anything installed or stored on the C: drive will be retained. If using the ephemeral secondary disk, this is not retained so perhaps think of this as temporary storage. If you want data other than the C: drive kept then use an EBS storage volume.
  • If you have Activated Windows it does stay activated on stop and start of EBS instance.

Creating a Windows EBS instance with a larger C: drive

If you're running Windows, you probably know the C: drive can fill up quickly with Windows security patches, etc. so you may want your Windows EBS instance to have a larger C: drive than the default, which is 50GB.

This may be done in the Eucalyptus web portal when launching a new instance on the Advanced Tab.

OR

This may be done using the command line tools. Complete the prerequisites in the following sections to launch a linux instance with the command line tools installed:

From the Euca User Console

  • Create and Edit a new security group, add RDP port and ssh port for linux machine, you'll use this security group below
    • SSH port: tcp (22), 128.84.3.0/24
    • RDP port: tcp (3389), xxx.xxx.xxx.xxx/32

From the linux instance where the command line tools are installed:

  • Create a 30GB C: partition
  • issue the following command (substitute your userid-key, security_group)
   euca-run-instances emi-1234567 -k <userid-key> -g <security_group> -t <m1.small> -b /dev/sda=:30:

Obtain Windows administrator password from linux instance using euca tools

From the linux instance where the command line tools are installed issue (substitute your userid-key):

    euca-get-password <instance ID> -k <.euca/userid-key.pem>  
  • This provides the administrator password to log into your Windows instance. Your user account is 128.84.8.xxx\Administrator, replacing the xxx with whatever the correct IP address is for this instance. You should now be able to use this information to Remote Desktop to access your instance.
  • Important: For Security reasons, create a new administrative account for yourself with a strong password.
  • You can/should terminate the Linux instance now if you are done using the command line tools.

Accounting, aka Don't use up your Subscription by Accident!

To understand how billing works, it is necessary to understand a bit about how Red Cloud works. Red Cloud allows the user to start, stop, and terminate system virtual machines. Since starting a virtual machine allocates memory and CPU resources on a physical machine to that virtual machine, users are billed based on the length of time a virtual machine is running, even if it is idle and doing NO work for the user. Of course, the number of CPU cores and amount of memory also factor in to the billing rate, so it is usually best to do development on a smaller virtual machine instance. For instance, if you have an exploratory account, you have 50 core hours to start. If you leave a 1-core node running, you will use up the allocated 50 hours in just over 2 days. Another possibility is that you are on a CAC project with a Red Cloud subscription (8,585 core hours). If you start up an instance with 4 cores (sometimes called CPUs in Eucalyptus), and you leave the instance running for a week, or 168 hours, you will use up (168 hours)*(4 cores) or 672 core hours, or 8% of the subscription.

This is true for Windows instances as well; also note, Cornell users do not need to pay for a Windows license.

It is worth pointing out now that an upcoming release of Red Cloud will allow the node size (instance type) to be changed if the virtual machine is stopped (i.e. shut down), which will allow the user to easily grow the system size when needed.

We recommend you check your balance frequently using pages provided for Cornell or external users.

Frequently Asked Questions/Tips/Troubleshooting

  1. Eucalyptus Known Issues
  2. euca-describe-instances or the web console says my instance is running, but why is it not responding to ping or ssh connections?
  3. Red Cloud FAQ
  4. Basic Linux Tutorial for systems administration tasks. Not at all comprehensive, but may help you get up and running.
  5. Understanding and resizing Instance type (the size of the virtual machine). Resizing volumes is a separate issue, and is somewhat more involved.
  6. An example of Installing R, a commonly used software package.
  7. Backups: Data stored on Red Cloud is not backed up by default; users are responsible for their own backups. For a CIT solution, please see EZ-backup. For a very basic backup solution, see the discussion above on transferring files.
  8. If you have more questions, see here