Overview
This post covers the basics of managing Amazon Web Services (AWS) Elastic Cloud Computing (EC2) instances. Amazon EC2 service is the most well known cloud service operator providing computing instances and cloud storage in both Europe and the US. The service is charged by the hour of usage for the computing resources and by the amount of storage space used for the cloud storage. It is possible to choose from a number of different types of computing resources, which are priced according to several factors, for example, number of CPU’s or the amount of memory available to the node. The price increases with performance, for example, the price of a medium instance with 4 times the CPU power is 4 times the price of the small instance.
If the computing requirements are known beforehand, it is also possible to reserve the instances for 1 year or more paying an initial fee and getting lower monthly costs in return.
The service itself
Amazon provides a very nice service for people wanting to start using cloud computing resources with it’s easily accessible EC2 service. Getting starting with EC2 is quick once you know what you want to do with the nodes. The service is cheap enough to just start an instance and try out different options before making the decision on how to use it, too.
How to manage instances
As the whole point of a cloud service is the possibility to scale up to even a high number of instances based on the current demand, one of the most important topics is how to manage all of these nodes in an efficient way. It should be quick to start new ones when there is demand to do so, and also shutdown the nodes that are not needed thus saving unnecessary costs.
We will now take a look at different tools to manage the nodes and then go through the basic steps needed to setup your own instance.
EC2 Management tools
There is a number of options for managing the EC2 nodes. The most commonly used tools are:
- AWS Management Console
- Firefox Elasticfox Addon
- Amazon EC2 tools
- Custom scripts
AWS Management Console
The first one that the user comes in contact with is the AWS Management Console, which is part of Amazon AWS website that is accessible after signing up.
AWS Management Console
The dashboard can manage all aspects of the instances as can be expected so there are lots of different options available. The best was is to start it up, start some instances (Note: charges will apply) and try it out.
Elasticfox
Another way to manage the nodes is to use the Mozilla Firefox Elasticfox addon. The following screenshot does not really show the multitude of options the addon has. This view will show you a list of instances you have either starting up, running or stopped state. It will also show the terminated instances for an hour after termination.
Elasticfox Firefox addon
Both of these tools are nice for starting up and instance once in a while or getting an overview of your nodes, but for the power-user or for managing a high number of nodes, they will be too cumbersome. And that’s where the command line tools come into the picture.
Amazon AMI Tools
The tools are available from Amazon, EC2 command line tools. First thing to do is to download the tools and unzip the package or install the RPM. To be able to use the tools, you need to get the private key and certificate from Amazon and set a number of environment variables. A good start is to create an .ec2 directory in your home directory and store the keys there.
Once that is done, you need to set the environment variables. On a Mac, you can add these lines to your .bash_login file:
export EC2_HOME=<directory where the tools are installed> export EC2_URL=https://eu-west-1.ec2.amazonaws.com # your location export EC2_PRIVATE_KEY=/Users/user/.ec2/pk-<YOUR KEY>.pem export EC2_CERT=/Users/user/.ec2/cert-<YOUR CERT>.pem
The tool set consists of more than a 100 tools, which we are not going to go through here. The full documentation is available at CLI reference
Starting up an EC2 instance
The previous introduction will of course not suffice for understanding the tools, but is rather a starting point for the work.
The easiest way to start an instance is to click on the Launch Instance button on the dashboard.
The next step is to choose the image type you need. There are a number of different images types available, some of them from Amazon but most from other people that have been kind enough to provide their images for other peoples use.
Launch a new instance wizard
The next step is to select the number of instances and the type of “hardware” to run on and the availability zone.
After that you need to choose a key pair to use for connecting to the instance from the ones you have already or create a new one. This is the key you need to use when connecting to the machine through SSH if running a UNIX/Linux instance. Last thing to do is to select a suitable firewall configuration. After that, review the instance parameters and start it. This is when Amazon starts charging you, so remember to shutdown instances you don’t need.
Next steps
While it is nice to have your own EC2 instance running, it’s usually not sufficient. When you terminate an instance, everything you have installed on it will be gone.
What you need is to build an AMI with all of the software you need, bundle it to an AMI and register it. Once that is done, you can just launch more instances of the same all having the correct software for your needs. You will also need some S3 storage for your files that you want to keep between the runs.
This can be done in several ways. The easiest one is to pick a publicly available image that is as close to your needs as possible. For example, one of the very nice Ubuntu images provided by Alestic. They have provided images for most Ubuntu versions and you can just pick one. Alestic images contain, for example, recent releases of Python and Ruby (1.8.7). Installing other software can be done with the usual Ubuntu tools.
It is also possible to build an image from scratch and then bundle it with the EC2 AMI tools.
Either way you do it, the image will be uploaded to S3 service for storage. Then the image can be registered to make it available for use in the dashboard or the other tools. You will need to choose the option “My AMI’s” from the EC2 dashboard when launching in an instance to use it.
And then you’re all set for some more serious work on the EC2.
