13 CloudLab OpenStack Tutorial

This tutorial will walk you through the process of creating a small cloud on CloudLab using OpenStack. Your copy of OpenStack will run on bare-metal machines that are dedicated for your use for the duration of your experiment. You will have complete administrative access to these machines, meaning that you have full ability to customize and/or configure your installation of OpenStack.

13.1 Objectives

In the process of taking this tutorial, you will learn to:

• Log in to CloudLab

• Create your own cloud by using a pre-defined profile

• Access resources in a cloud that you create

• Use administrative access to customize your cloud

• Clean up your cloud when finished

• Learn where to get more information

13.2 Prerequisites

This tutorial assumes that:

• You have an existing account on either:

  • CloudLab (Instructions for getting an account can be found here.)

  • The GENI portal. (Instructions for getting an account can be found here.)

14.4 Logging In

The first step is to log in to CloudLab. CloudLab is available to all researchers and educators who work in cloud computing. If you have an account at one of its federated facilities, like Emulab or GENI, then you already have an account at CloudLab.

14.4.1 Using a CloudLab Account

If you have signed up for an account at the CloudLab website, simply open https://www.cloudlab.us/ in your browser, click the “Log In” button, enter your username and password, and skip to the next part of the tutorial.

14.4.2 Using a GENI Account

If you have an account at the GENI portal, you will use the single-sign-on features of the CloudLab and GENI portal.

1. Open the web interface
   
   To log in to CloudLab using a GENI account, start by visiting https://www.cloudlab.us/ in your browser and using the “Log In” button in the upper right.
   
   

2. Click the "GENI User" button
   
   On the login page that appears, you will use the “GENI User” button. This will start the GENI authorization tool, which lets you connect your GENI account with CloudLab.
   
   

3. Select the GENI portal
   
   You will be asked which facility your account is with. Select the GENI icon, which will take you to the GENI portal. There are several other facilities that are federated with CloudLab, which can be found in the drop-down list.
   
   

4. Log into the GENI portal
   
   You will be taken to the GENI portal, and will need to select the institution that is your identity provider. Usually, this is the university you are affiliated with. If your university is not in the list, you may need to log in using the “GENI Project Office” identity provider. If you have ever logged in to the GENI portal before, your identity provider should be pre-selected; if you are not familiar with this login page, there is a good chance that you don’t have a GENI account and need to apply for one.
   
   

5. Log into your institution
   
   This will bring you to your institution’s standard login page, which you likely use to access many on-campus services. (Yours will look different from this one.)
   
   

6. Authorize CloudLab to use your GENI account
   

   What’s happening in this step is that your browser uses your GENI user certificate (which it obtained from the GENI portal) to cryptographically sign a “speaks-for” credential that authorizes the CloudLab portal to make GENI API calls on your behalf.

   Click the “Authorize” button: this will create a signed statement authorizing the CloudLab portal to speak on your behalf. This authorization is time-limited (see “Show Advanced” for the details), and all actions the CloudLab portal takes on your behalf are clearly traceable.

   If you’d like, you can click “Remember This Decision” to skip this step in the future (you will still be asked to log in to the GENI portal).
   
   

7. Set up an SSH keypair in the GENI portal
   
   Though not strictly required, many parts of this tutorial will work better if you have an ssh public key associated with your GENI Portal account. To upload one, visit portal.geni.net and use the “SSH Keys” item on the menu that drops down under your name.
   If you are not familiar with ssh keys, you may skip this step. You may find GitHub’s ssh key page helpful in learning how to generate and use them.

   When you create a new experiment, CloudLab grabs the set of ssh keys you have set up at the time; any ssh keys that you add later are not added to experiments that are already running.

   
   

13.4 Building Your Own OpenStack Cloud

Once you have logged in to CloudLab, you will “instantiate” a “profile” to create an experiment. (An experiment in CloudLab is similar to a “slice” in GENI.) Profiles are CloudLab’s way of packaging up configurations and experiments so that they can be shared with others. Each experiment is separate: the experiment that you create for this tutorial will be an instance of a profile provided by the facility, but running on resources that are dedicated to you, which you have complete control over. This profile uses local disk space on the nodes, so anything you store there will be lost when the experiment terminates.

The OpenStack cloud we will build in this tutorial is very small, but CloudLab has additional hardware that can be used for larger-scale experiments.

For this tutorial, we will use a basic profile that brings up a small OpenStack cloud. The CloudLab staff have built this profile by capturing disk images of a partially-completed OpenStack installation and scripting the remainder of the install (customizing it for the specific machines that will get allocated, the user that created it, etc.) See this manual’s section on profiles for more information about how they work.

1. Start Experiment
   
   
   
   After logging in, you are taken to your main status dashboard. Select “Start Experiment” from the “Experiments” menu.

2. Select a profile
   
   
   
   The “Start an Experiment” page is where you will select a profile to instantiate. We will use the OpenStack profile; if it is not selected, follow this link or click the “Change Profile” button, and select “OpenStack” from the list on the left.
   Once you have the correct profile selected, click “Next”
   
   

3. Set parameters
   
   Profiles in CloudLab can have parameters that affect how they are configured; for example, this profile has parameters that allow you to set the size of your cloud, spread it across multiple clusters, and turn on and off many OpenStack options.
   For this tutorial, we will leave all parameters at their defaults and just click “next”.
   
   

4. Select a cluster
   
   CloudLab has multiple clusters available to it. Some profiles can run on any cluster, some can only run on specific ones due to specific hardware constraints, etc.
   Note: If you are at an in-person tutorial, the instructor will tell you which cluster to select. Otherwise, you may select any cluster.

   The dropdown menu for the clusters shows you both the health (outer ring) and available resources (inner dot) of each cluster. The “Check Cluster Status” link opens a page (in a new tab) showing the current utilization of all CloudLab clusters.

   
   

5. Click Finish!
   
   When you click the “finish” button, CloudLab will start provisioning the resources that you requested on the cluster that you selected.

   You may optionally give your experiment a name—this can be useful if you have many experiments running at once.

   
   

6. CloudLab instantiates your profile
   
   CloudLab will take a few minutes to bring up your copy of OpenStack, as many things happen at this stage, including selecting suitable hardware, loading disk images on local storage, booting bare-metal machines, re-configuring the network topology, etc. While this is happening, you will see this status page:
   
   

   Provisioning is done using the GENI APIs; it is possible for advanced users to bypass the CloudLab portal and call these provisioning APIs from their own code. A good way to do this is to use the geni-lib library for Python.

   As soon as a set of resources have been assigned to you, you will see details about them at the bottom of the page (though you will not be able to log in until they have gone through the process of imaging and booting.) While you are waiting for your resources to become available, you may want to have a look at the CloudLab user manual, or use the “Sliver” button to watch the logs of the resources (“slivers”) being provisioned and booting.

7. Your cloud is ready!
   
   When the web interface reports the state as “Booted”, your cloud is provisioned, and you can proceed to the next section.
   Important: A “Booted” status indicates that resources are provisioned and booted; this particular profile runs scripts to complete the OpenStack setup, and it will be a few more minutes before OpenStack is fully ready to log in and create virtual machine instances. You will be able to tell that this has finished when the status changes from “Booted” to “Ready”. For now, don’t attempt to log in to OpenStack, we will explore the CloudLab experiment first.
   
   

13.5 Exploring Your Experiment

Now that your experiment is ready, take a few minutes to look at various parts of the CloudLab status page to help you understand what resources you’ve got and what you can do with them.

13.5.1 Experiment Status

The panel at the top of the page shows the status of your experiment—you can see which profile it was launched with, when it will expire, etc. The buttons in this area let you make a copy of the profile (so that you can customize it), ask to hold on to the resources for longer, or release them immediately.

Note that the default lifetime for experiments on CloudLab is less than a day; after this time, the resources will be reclaimed and their disk contents will be lost. If you need to use them for longer, you can use the “Extend” button and provide a description of why they are needed. Longer extensions require higher levels of approval from CloudLab staff. You might also consider creating a profile of your own if you might need to run a customized environment multiple times or want to share it with others.

You can click the title of the panel to expand or collapse it.

13.5.2 Profile Instructions

Profiles may contain written instructions for their use. Clicking on the title of the “Profile Instructions” panel will expand (or collapse) it; in this case, the instructions provide a link to the administrative interface of OpenStack, and give you passwords to use to log in. (Don’t log into OpenStack yet—for now, let’s keep exploring the CloudLab interface.)

13.5.3 Topology View

At the bottom of the page, you can see the topology of your experiment. This profile has three nodes connected by a LAN, which is represented by a gray box in the middle of the topology. The names given for each node are the names assigned as part of the profile; this way, every time you instantiate a profile, you can refer to the nodes using the same names, regardless of which physical hardware was assigned to them. The green boxes around each node indicate that they are up; click the “Refresh Status” button to initiate a fresh check.

If an experiment has “startup services” (programs that run at the beginning of the experiment to set it up), their status is indicated by a small icon in the upper right corner of the node. You can mouse over this icon to see a description of the current status. In this profile, the startup services on the compute node(s) typically complete quickly, but the control node may take much longer.

It is important to note that every node in CloudLab has at least two network interfaces: one “control network” that carries public IP connectivity, and one “experiment network” that is isolated from the Internet and all other experiments. It is the experiment net that is shown in this topology view. You will use the control network to ssh into your nodes, interact with their web interfaces, etc. This separation gives you more freedom and control in the private experiment network, and sets up a clean environment for repeatable research.

13.5.4 List View

The list view tab shows similar information to the topology view, but in a different format. It shows the identities of the nodes you have been assigned, and the full ssh command lines to connect to them. In some browsers (those that support the ssh:// URL scheme), you can click on the SSH commands to automatically open a new session. On others, you may need to cut and paste this command into a terminal window. Note that only public-key authentication is supported, and you must have set up an ssh keypair on your account before starting the experiment in order for authentication to work.

13.5.5 Manifest View

The third default tab shows a manifest detailing the hardware that has been assigned to you. This is the “request” RSpec that is used to define the profile, annotated with details of the hardware that was chosen to instantiate your request. This information is available on the nodes themselves using the geni-get command, enabling you to do rich scripting that is fully aware of both the requested topology and assigned resources.

Most of the information displayed on the CloudLab status page comes directly from this manifest; it is parsed and laid out in-browser.

13.5.6 Graphs View

The final default tab shows a page of CPU load and network traffic graphs for the nodes in your experiment. On a freshly-created experiment, it may take several minutes for the first data to appear. After clicking on the “Graphs” tab the first time, a small reload icon will appear on the tab, which you can click to refresh the data and regenerate the graphs. For instance, here is the load average graph for an OpenStack experiment running this profile for over 6 hours. Scroll past this screenshot to see the control and experiment network traffic graphs. In your experiment, you’ll want to wait 20-30 minutes before expecting to see anything interesting.

Here are the control network and experiment network packet graphs at the same time. The spikes at the beginning are produced by OpenStack setup and configuration, as well as the simple OpenStack tasks you’ll perform later in this profile, like adding a VM.

13.5.7 Actions

In both the topology and list views, you have access to several actions that you may take on individual nodes. In the topology view, click on the node to access this menu; in the list view, it is accessed through the icon in the “Actions” column. Available actions include rebooting (power cycling) a node, and re-loading it with a fresh copy of its disk image (destroying all data on the node). While nodes are in the process of rebooting or re-imaging, they will turn yellow in the topology view. When they have completed, they will become green again. The shell and console actions are described in more detail below.

13.5.8 Web-based Shell

CloudLab provides a browser-based shell for logging into your nodes, which is accessed through the action menu described above. While this shell is functional, it is most suited to light, quick tasks; if you are going to do serious work, on your nodes, we recommend using a standard terminal and ssh program.

This shell can be used even if you did not establish an ssh keypair with your account.

Two things of note:

• Your browser may require you to click in the shell window before it gets focus.

• Depending on your operating system and browser, cutting and pasting into the window may not work. If keyboard-based pasting does not work, try right-clicking to paste.

13.5.9 Serial Console

CloudLab provides serial console access for all nodes, which can be used in the event that normal IP or ssh access gets intentionally or unintentionally broken. Like the browser-based shell, it is launched through the access menu, and the same caveats listed above apply as well. In addition:

• If you look at the console for a node that is already booted, the console may be blank due to a lack of activity; press enter several times to get a fresh login prompt.

• If you need to log in, do so as root; your normal user account does not allow password login. There is a link above the console window that reveals the randomly-generated root password for your node. Note that this password changes frequently for security reasons.

13.6 Bringing up Instances in OpenStack

Now that you have your own copy of OpenStack running, you can use it just like you would any other OpenStack cloud, with the important property that you have full root access to every machine in the cloud and can modify them however you’d like. In this part of the tutorial, we’ll go through the process of bringing up a new VM instance in your cloud.

We’ll be doing all of the work in this section using the Horizon web GUI for OpenStack, but you could also ssh into the machines directly and use the command line interfaces or other APIs as well.

1. Check to see if OpenStack is ready to log in
   
   As mentioned earlier, this profile runs several scripts to complete the installation of OpenStack. These scripts do things such as finalize package installation, customize the installation for the specific set of hardware assigned to your experiment, import cloud images, and bring up the hypervisors on the compute node(s).
   If exploring the CloudLab experiment took you more than ten minutes, these scripts are probably done. You can be sure by checking that all nodes have completed their startup scripts (indicated by a checkmark on the Topology view); when this happens, the experiment state will also change from “Booting” to “Ready”
   
   
   If you continue without verifying that the setup scripts are complete, be aware that you may see temporary errors from the OpenStack web interface. These errors, and the method for dealing with them, are generally noted in the text below.

2. Visit the OpenStack Horizon web interface
   
   On the status page for your experiment, in the “Instructions” panel (click to expand it if it’s collapsed), you’ll find a link to the web interface running on the ctl node. Open this link (we recommend opening it in a new tab, since you will still need information from the CloudLab web interface).
   
   

3. Log in to the OpenStack web interface
   
   Log in using the username admin and the password shown in the instructions for the profile. Use the domain name default if prompted for a domain.

   This profile generates a new (random) password for every experiment.

   Important: if the web interface rejects your password, wait a few minutes and try again. If it gives you another type of error, you may need to wait a minute and reload the page to get login working properly.
   
   

4. Launch a new VM instance
   
   Click the “Launch Instance” button on the “Instances” page of the web interface.
   
   

5. Set Basic Settings For the Instance
   
   There a few settings you will need to make in order to launch your instance. These instructions are for the launch wizard in the OpenStack Mitaka, but the different release wizards are all similar in function and required information. Use the tabs in the column on the left of the launch wizard to add the required information (i.e., Details, Source, Flavor, Networks, and Key Pair), as shown in the following screenshots:
   
   

   • Pick any “Instance Name” you wish

   
   

   • For the “Image Name”, select “trusty-server”

   • For the “Instance Boot Source”, select “Boot from image”

   Important: If you do not see any available images, the image import script may not have finished yet; wait a few minutes, reload the page, and try again.
   
   

   • Set the “Flavor” to m1.small—the disk for the default m1.tiny instance is too small for the image we will be using, and since we have only one compute node, we want to avoid using up too many of its resources.

6. Add a Network to the Instance
   
   In order to be able to access your instance, you will need to give it a network. On the “Networking” tab, add the tun0-net to the list of selected networks by clicking the “+” button or dragging it up to the blue region. This will set up an internal tunneled connection within your cloud; later, we will assign a public IP address to it.

   The tun0-net consists of EGRE tunnels on the CloudLab experiment network.

   Important: If you are missing the Networking tab, you may have logged into the OpenStack web interface before all services were started. Unfortunately, reloading does not solve this, you will need to log out and back in.
   
   

7. Set an SSH Keypair
   
   On the “Key Pair” tab (or “Access & Security” in previous versions), you will add an ssh keypair to log into your node. If you configured an ssh key in your GENI account, you should find it as one of the options in this list. You can filter the list by typing your CloudLab username, or a portion of it, into the filter box to reduce the size of the list. (By default, we load in public keys for all users in the project in which you created your experiment, for convenience – thus the list can be long.) If you don’t see your keypair, you can add a new one with the red button to the right of the list. Alternately, you can skip this step and use a password for login later.
   
   

8. Launch, and Wait For Your Instance to Boot
   
   Click the “Launch” button on the “Key Pair” wizard page, and wait for the status on the instances page to show up as “Active”.

9. Add a Public IP Address
   
   
   
   At this point, your instance is up, but it has no connection to the public Internet. From the menu on the right, select “Associate Floating IP”.

   Profiles may request to have public IP addresses allocated to them; this profile requests four (two of which are used by OpenStack itself.)

   On the following screen, you will need to:

   1. Press the red button to set up a new IP address

   2. Click the “Allocate IP” button—this allocates a new address for you from the public pool available to your cloud.

   3. Click the “Associate” button—this associates the newly-allocated address with this instance.

   The public address is tunneled by the ctl (controller) node from the control network to the experiment network. (In older OpenStack profile versions, or depending the profile parameters specified to the current profile version, the public address may instead be tunneled by the nm (network manager) node, in a split controller/network manager OpenStack deployment.)

   You will now see your instance’s public address on the “Instances” page, and should be able to ping this address from your laptop or desktop.
   
   

10. Log in to Your Instance
    
    You can ssh to this IP address. Use the username ubuntu; if you provided a public key earlier, use your private ssh key to connect. If you did not set up an ssh key, use the same password you used to log in to the OpenStack web interface (shown in the profile instructions.) Run a few commands to check out the VM you have launched.

13.7 Administering OpenStack

Now that you’ve done some basic tasks in OpenStack, we’ll do a few things that you would not be able to do as a user of someone else’s OpenStack installation. These just scratch the surface—you can upgrade, downgrade, modify or replace any piece of your own copy of OpenStack.

13.7.1 Log Into The Control Nodes

If you want to watch what’s going on with your copy of OpenStack, you can use ssh to log into any of the hosts as described above in the List View or Web-based Shell sections. Don’t be shy about viewing or modifying things; no one else is using your cloud, and if you break this one, you can easily get another.

Some things to try:

• Run ps -ef on the ctl to see the list of OpenStack services running

• Run ifconfig on the ctl node (or the nm node, if your experiment has one), to see the various bridges and tunnels that have been brought to support the networking in your cloud

• Run sudo virsh list --all on cp1 to see the VMs that are running

13.7.2 Reboot the Compute Node

Since you have this cloud to yourself, you can do things like reboot or re-image nodes whenever you wish. We’ll reboot the cp1 (compute) node and watch it through the serial console.

1. Open the serial console for cp1
   
   On your experiment’s CloudLab status page, use the action menu as described in Actions to launch the console. Remember that you may have to click to focus the console window and hit enter a few times to see activity on the console.

2. Reboot the node
   
   On your experiment’s CloudLab status page, use the action menu as described in Actions to reboot cp1. Note that in the topology display, the box around cp1 will turn yellow while it is rebooting, then green again when it has booted.

3. Watch the node boot on the console
   
   Switch back to the console tab you opened earlier, and you should see the node starting its reboot process.
   
   

4. Check the node status in OpenStack
   
   You can also watch the node’s status from the OpenStack Horizon web interface. In Horizon, select “Hypervisors” under the “System” menu, and switch to the “Compute Host” tab.
   
   
   Note: This display can take a few minutes to notice that the node has gone down, and to notice when it comes back up. Your instances will not come back up automatically; you can bring them up with a “Hard Reboot” from the “Admin -> System ->Instances” page.

13.8 Terminating the Experiment

Resources that you hold in CloudLab are real, physical machines and are therefore limited and in high demand. When you are done, you should release them for use by other experimenters. Do this via the “Terminate” button on the CloudLab experiment status page.

Note: When you terminate an experiment, all data on the nodes is lost, so make sure to copy off any data you may need before terminating.

If you were doing a real experiment, you might need to hold onto the nodes for longer than the default expiration time. You would request more time by using the “Extend” button the on the status page. You would need to provide a written justification for holding onto your resources for a longer period of time.

13.9 Taking Next Steps

Now that you’ve got a feel for for what CloudLab can do, there are several things you might try next:

• Create a new project to continue working on CloudLab past this tutorial

• Try out some profiles other than OpenStack (use the “Change Profile” button on the “Start Experiment” page)

• Read more about the basic concepts in CloudLab

• Try out different hardware

• Learn how to make your own profiles