Difference between revisions of "Red Cloud"

From CAC Documentation wiki
Jump to navigation Jump to search
m (→‎Accounting: Don't Use Up Your Subscription by Accident!: Added clearer recommendation about starting with a small instance)
(→‎Important Pages: added a link to the MATLAB Parallel Server instructions)
Line 37: Line 37:
#** [[OpenStack#Instance_States|changing instance state]]
#** [[OpenStack#Instance_States|changing instance state]]
# Either instructions for [[Red_Cloud_Linux_Instances|Linux Instances]] OR [[Red_Cloud_Windows_Instances|Windows Instances]]
# Either instructions for [[Red_Cloud_Linux_Instances|Linux Instances]] OR [[Red_Cloud_Windows_Instances|Windows Instances]]
#* There are special instructions if you intend to use [[MATLAB Parallel Server in Red Cloud]]
== New Users ==
== New Users ==

Revision as of 14:04, 28 October 2019

This wiki provides documentation for Red Cloud, an on-demand research cloud computing service maintained and supported by the CAC. At present, Red Cloud is an Infrastructure as a Service (IaaS) based on OpenStack.

Instructions on these pages apply to users who have a Red Cloud subscription they are managing, though some instructions may also apply to users of subscriptions managed by someone else. Individuals who manage a Red Cloud subscription can create, administer, and delete virtual servers and storage in Red Cloud.

How To Read This Documentation

  • Exploratory Account Users
    • Read through all the sections on this page targeting New Users.
    • Look through the Important Pages listed below to help you get started managing Red Cloud resources.
    • Pay particular attention to the Accounting section on this page, as your exploratory project ends when you have exhausted your subscription.
    • An important point to remember is that you are not the PI on your account, so you can ignore any instructions targeting PIs on a project.
  • New Users
    • Read through all the sections on this page targeting New Users.
    • Look through the Important Pages listed to help you get started managing Red Cloud resources.
    • If you are not the PI on your account, you can ignore any instructions targeting PIs.
  • Returning Users
    • Check out the section dedicated to all users.
    • You may also want to look through the Important Pages listed to help you manage Red Cloud resources.
    • If you are not the PI on your account, you can ignore any instructions targeting PIs.

Note: All links on our wiki are colored red and underlined. Each of the external links will have dotted underlining and an icon next to them that looks like an arrow pointing out of a box, whereas internal links have a solid underline and do not have any icon. This can help you navigate by knowing that the external links are not part of our documentation or "how to" instructions.

Important Pages

Here is a suggested list of pages to look over to help with getting started managing resources.

  1. This page - includes information about:
  2. OpenStack - a highly recommended quick-start page including instructions for:
  3. Either instructions for Linux Instances OR Windows Instances

New Users

New users would be best served by reading this complete page first, then reading through the pages listed in the Important Pages section. New users are also encouraged to explore the Getting Started page, which includes a lot more general information on using CAC resources beyond Red Cloud.

Note for new Linux users: As the root user, you will have complete control over access to the system, such as setting up users and their permissions, defining the firewall, and more. This means that the primary user of a Linux system must be familiar with Linux system administration. Aside from the basics of using the command line, this includes familiarity with: creating and modifying users, installing software, configuring software for remote logins, and managing/transferring data. For users that want to use Red Cloud, but do not have much system administration experience, we've written a Linux Tutorial that should work for RedHat/CentOS and Ubuntu Linux systems. Consulting is also available to answer general questions about systems administration, or for help on specific software and research problems.

First Time Login

When you are added to a CAC project, you will receive an e-mail confirming your Red Cloud access. You must change the automatically generated password immediately for security reasons and to access computing resources. Refer to the instructions for managing your password as needed.

If you are a PI or a PI's proxy for a new project, verify that you have added a subscription to your project; see the Manage Projects page. After waiting up to an hour for account information to propagate, you will then be ready to download the OpenStack credentials and start managing Red Cloud resources.

How to Access Red Cloud Resources

Red Cloud is a private research cloud with an OpenStack backend. Interacting with OpenStack is how resources can be managed. In this case, resources can refer to instances (or virtual machines), images, and volumes. There are two ways to interact with OpenStack:

Note: Regardless which method you choose (Web Interface or Command Line Interface), you must follow the First Time Login instructions.

Depending on which operating system you are planning on running on your instances, you should also refer to one of the following pages:

Accounting: Don't Use Up Your Subscription by Accident!

To understand how billing works, it is necessary to understand a bit about how Red Cloud operates. Red Cloud enables the user to control the state of system virtual machines, such as start, pause, suspend, shelve, and delete (see Instance States for a full list). Since starting a virtual machine allocates memory and CPU resources on a physical machine to that virtual machine, subscriptions are billed based on the length of time a virtual machine is running, even if it is idle and doing NO work for the user. This is fair because your running instance will prevent others from using the hardware, even if the hardware is idle.

Thus, the best way to avoid using up your subscription needlessly is to make sure you shelve your Red Cloud instance any time you are not using it. It is very simple to do this via the menu in the OpenStack Web Interface. You can always start the instance again later, and the disk contents will be unchanged. It is just like shutting down your laptop.

Whenever you have one or more instances that are up and running, the amount that is deducted from your Red Cloud subscription is: the length of time that your instances are running, multiplied by the number of cores that you are occupying with those instances. This implies that you should also take advantage of the various instance sizes available. For example, it is usually best to choose a small instance type to do your development work.

It is worth pointing out that Red Cloud allows the instance type to be changed if the virtual machine is stopped (i.e. shut down). This allows you to "scale up" an instance at any time by stopping it, choosing a larger size for it, and starting it back up. You can shrink an instance in the same way. If you intend to use a large instance, we recommend that you start with the smallest instance size you can to install software and get used to your instance before resizing your instance to the full size you would like.

Here are a couple of motivating examples for you. Let's say you have an exploratory account, with just 165 core hours to start. If you leave a 1-core node running around the clock, you will use up the entire account in a little less than a week. Similarly, let's say you are on a CAC project with a Red Cloud subscription (8,585 core hours). If you start up an instance with 4 cores (sometimes called CPUs in OpenStack), and you leave the instance running for a week, or 168 hours, you will use up (168 hours)*(4 cores) or 672 core hours, or 8% of the subscription.

All of the above is true for Linux instances and Windows instances; note that Cornell users do not need to pay for a Windows license in Red Cloud.

We recommend you check your balance frequently using pages provided for Cornell or external users.

All Users

Please refer to the OpenStack page for more in-depth guidance on how to use Red Cloud, and read either Linux instances or Windows instances based on what systems will be used.

The current Red Cloud System Status can be checked anytime.

Common Tasks

Here are some links to help you with particular aspects of using Red Cloud:

  • Linux Tutorial - This may help you get up and running with some basic systems administration tasks. It is not intended to be comprehensive.
  • Information on choosing instance type (the size of the virtual machine). Resizing volumes is a separate issue, and is somewhat more involved.
  • An example of Installing R, a commonly used software package.
  • EZ-backup - a CIT solution for backups. Data stored on Red Cloud is not backed up by default; users are responsible for their own backups.

Acknowledging CAC

When you publish a paper, make presentations, or are interviewed by the Cornell Chronicle, national news media, etc., please acknowledge the Center by including:

"This research was conducted with support from the Cornell University Center for Advanced Computing."

Alternatively, the full acknowledgement is:

"This research was conducted with support from the Cornell University Center for Advanced Computing, which receives funding from Cornell University, the National Science Foundation, and members of its Partner Program."