Erik Howard

Today Canonical, the company behind Ubuntu, released Ubuntu 9.04.

What makes this release stand out from previous Ubuntu releases is the new Ubuntu Enterprise Cloud services.

Ubuntu Enterprise Cloud services is the first commercially-supported Linux distribution that enables businesses to build private cloud environments inside their firewalls. Companies will now be able to create their own private clouds pretty easily.

The Ubuntu Enterprise Cloud is powered by open source system known as Eucalyptus.  The Eucalyptus API matches Amazons EC2 API. If you really are dying to create your own private cloud, check out the Ubuntu Eucalyptus Getting Started documentation.

Besides supporting creating private clouds, Ubuntu 9.04 will also be fully available on Amazon Elastic Compute Cloud (EC2). Thanks to the work of Alestic and others, creating an Ubuntu EC2 instance was quite simple. With Ubuntu 9.04, Canonical is taking more of a leading role in helping customers deploy Ubuntu EC2 instances.

Erik Howard Ubuntu, cloud computing cloud computing, Ubuntu

Today, Amazon annouced the availability of it’s newest web servrice – Elastic MapReduce.

Amazon Elastic MapReduce is a web service that enables businesses, researchers, data analysts, and developers to easily and cost-effectively process vast amounts of data. It utilizes a hosted Hadoop framework running on the web-scale infrastructure of Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3).

Amazon Elastic MapReduce automatically spins up a Hadoop implementation of the MapReduce framework on Amazon EC2 instances, sub-dividing the data in a job flow into smaller chunks so that they can be processed (the “map” function) in parallel, and eventually recombining the processed data into the final solution (the “reduce” function). Amazon S3 serves as the source for the data being analyzed, and as the output destination for the end results.

Amazon Elastic MapReduce applications can be authored in Java, Ruby, Perl, Python, PHP, R, or C++.  Elastic MapReduce applications only run on Linux/Unix instances running in the US region on EC2. Both reserved and on-demand instances are supported. If you have any on-demand instances, they will be used first.

Amazon Elastic MapReduce pricing is in addition to standard EC2 rates.

Amazon EC2 and Amazon S3 charges are billed separately. Pricing for Amazon Elastic MapReduce is per instance-hour consumed for each instance type, from the time job flow began processing until it is terminated. Each partial instance-hour consumed will be billed as a full hour.

Erik Howard MapReduce cloud computing, ec2, hadoop, MapReduce

Amazon Elastic Compute Cloud (EC2) is a commercial web service offering that allows you to rent your own server in which to deploy your applications. The infrastructure is highly scalable and allows you to increase or decrease your computing horsepower based on demand. Think of it as your own elastic data center where you only pay for the resources that you use.

EC2 uses Xen virtualization for its instances (servers). EC2 instances come in 3 sizes Small, Large, and Extra large. Amazon sizes instances on what’s called “EC2 Compute Units”. One EC2 Compute Unit (ECU) provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.

EC2 Standard Instances
Most applications are well suited for these types of instances:

• Small Instance: 1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of instance storage, 32-bit platform
• Large Instance: 7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each), 850 GB of instance storage, 64-bit platform
• Extra Large Instance: 15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each), 1690 GB of instance storage, 64-bit platform

EC2 High-CPU Instances
This family of instances has proportionally more CPU resources than memory and is better suited for computational-intensive applications.

• High-CPU Medium Instance: 1.7 GB of memory, 5 EC2 Compute Units (2 virtual cores with 2.5 EC2 Compute Units each), 350 GB of instance storage, 32-bit platform
• High-CPU Extra Large Instance: 7 GB of memory, 20 EC2 Compute Units (8 virtual cores with 2.5 EC2 Compute Units each), 1690 GB of instance storage, 64-bit platform

For most blogs, company websites and low-traffic ecommerce sites, a Small Instance will be fine. You can always migrate up to a larger instance.

Getting Started
For this article we are going to launch a 32-bit Ubuntu 8.04 LTS Hardy AMI (Amazon Machine Instance) from Alestic. Amazon provides a few AMI’s designed to get you up and running quickly. They also have over 1,700 Community AMI’s. You can select from Fedora, Ubuntu, Windows, CentOS and other flavors of Linux.

Before you can create your first instance, you will have to Sign Up for Amazon EC2.

After signing up, visit the AWS Management Console page. Click on the Go to Amazon EC2 Console button.

Before you create and launch your EC2 instance. You should first create a Key Pair. You will use this key pair to login to your instance after it’s been created.
To create your Key Pair, click Key Pairs in the left Navigation menu. Next, click the Create Key Pair button in the Key Pairs window. Enter a descriptive name for your new Key Pair in the Create Key Pair dialog window.

Next you will be prompted to download your newly created Key Pair. Download your newly created Key Pair; you will need this to login to your new instance later.

NOTE: AMAZON DOES NOT STORE YOUR PRIVATE KEY PAIR. IF YOU DO NOT DOWNLOAD YOUR KEY FILE AT THIS POINT YOU WILL NOT BE ABLE TO LOGIN TO ANY INSTANCES THAT WERE CREATED USING THIS PRIVATE KEY PAIR.

Now let’s go ahead and find our Amazon Ubuntu AMI and fire it up. Click the Dashboard link in the left hand navigation menu and then click the Launch Instances button.

Select the Community AMI’s tab. Enter ami-71fd1a18 in the search field. This is the AMI id of the Alestic 32-bit Ubuntu Hardy server. Click the Select button to continue.

Now we are going to set some options for our Ubuntu AMI. First type in the number of instances you want to create, we are only going to create 1 for this article.
Next, select the Small instance type. In the Key Pair Name drop-down, select the new Key Pair we just created. In the Security Groups listbox, select default. We will skip the Advanced Options for now.

Click Launch to start you new Ubuntu Amazon instance!

Amazon will now attempt to start you instance. This may take anywhere from 30 seconds to 5 minutes. To check on the status of your instance click the Instance link in the Navigation window.

You can connect to your instance with any SSH client. Find the Key Pair file you created previously for this instance. Make note of the Public DNS address of your new instance. For example, to connect to my test instance I just created, I would use the following SSH command:

ssh -i Web1.pem root@ec2-67-202-37-5.compute-1.amazonaws.com

Web1.pem is the private Key Pair that I created for this instance. If you did not save your Key Pair, then you will have to destroy this instance and create a new Key Pair and start a new instance with the new Key Pair.

Once you have finished playing around with your instance, make sure you terminate the instance so you won’t be charged for the idle instance.
In future articles, I will show you how to setup Elastic IP’s, Elastic Block Storage, run shell scripts to setup your new instance and creating an instance using only Amazon’s EC2 API.

Erik Howard cloud computing, ec2 amazon, cloud computing, ec2

While startups, consultants and individuals are rushing to put their applications in the cloud large enterprises are, for the most part, are still sitting on the fence.

What is preventing adoption of cloud computing for enterprises. It basically comes down to a few issues – security, accountability and standards.

Most cloud vendors, but not all, are mum (publicly at least) on the exact details of their cloud infrastructure. Large enterprises want assurances that their applications and their data will be secure in the cloud.

If I had to pick one cloud vendor who I trusted the most in the security arena, I would go with Joyent. They are going to be a little more expensive than your commodity cloud vendors, but you are getting rock-solid service, security and support for your money.

As a potential corporate consumer of cloud services, I would want to know how my usage is being tracked and billed. “Just trust us” is not a good answer.

Right now there are no standards when it comes to deploying applications into the cloud. Applications deployed on Google App Engine need to be modified for App Engine’s environment. Same is true for Microsoft’s Azure platform.

There are no standard API’s for managing instances and services in the cloud. If you decide to move your application from one cloud vendor to another. You will probably have to re-write any custom applications you developed to manage your cloud presence.

Cloud computing can and will play a big part in enterprise architectures. Cloud vendors need to figure out how to address the concerns that potential enterprise customers may have.

Erik Howard cloud computing cloud computing

Should you host your next application in the cloud or should you continue to host your application on dedicated servers?

If you were itching to put your next application in the cloud, you may want to hold off on that move. In some cases it makes sense to start your application in the cloud, but in most cases it doesn’t. In the past, reliability was a big concern with hosting in the cloud. Your instance could disappear at any time without notice. There were no up-time guarantees or solid SLA’s. Most of these concerns are melting away as the cloud computing industry starts to mature.

If looking strictly at price points, cloud computing may seem just a little more cost effective than renting a dedicated server. But it’s not an apples-to-apples comparison. For cloud servers, you are paying for the resources that your virtualized instance uses. With a dedicated server, you pay the same amount regardless how much that server is used.

Don’t fall for the cloud computing marketing hype of “pay for only what you use”. Who in their right mind is only going to let their servers run from 9am to 9pm to save costs? What they really mean is if your application needs to scale, create a few more virtualized instances to handle the load. When the load subsides, you can destroy those instances since you no longer need those resources. If you had experienced high loads with a dedicated server, you would have to quickly have your vendor provision new hardware. That process could take anywhere from 2-24 hours. That doesn’t include the time you would spend getting the server ready for production.

Most virtualized cloud instances are un-managed, meaning it’s your responsibility fore the care and feeding of your instance. Dedicated servers come in managed and un-managed flavors. A managed dedicated server is usually more expensive than an un-managed server. So if your computer administration skills are on the week side, you’d probably want to go with a managed dedicated server. There are a few companies, such as Enki, offer managed cloud computing services.

There are also other companies (Rackspace, GoGrid) that offer a hybrid strategy of dedicated servers combined with a cloud to handle spikes in traffic or temporary large computing projects.

I’ve managed dedicated servers now for many years with many different vendors. Each experience has been different from the next. From 99% reliability to outright fraud. My experiences with cloud computing have been favorable. But don’t expect much hand holding.

As far as cost, again – it depends on the type of application you want to host. If you just want to host a blog, you will probably be better off with getting a virtual private server (VPS) than a dedicated server or putting your application in the cloud.

If you find yourself transcoding thousands of videos, running a social network, need your content distributed globally, hosting a Facebook application, or have some other unpredictable high-volume application, then the cloud is the place for you.

If you know what your bandwidth requirements will be month-to-month and you can plan your growth, then it will probably be more cost effective to stick with dedicated servers.

Erik Howard cloud computing cloud computing, dedicated server

Sun recently announced that they are entering the cloud computing business. They were short on details, but long on creating communities, OS virtualization, API’s, and being a better value than Amazon’s EC2 service.

It remains to see how much traction Sun will gain in the growing cloud computing services market. I’m very satisfied with Amazon’s service. Joyent and GoGrid are also great vendors.

Consultants, small businesses and startups have been quick to latch onto cloud computing. For them, it’s a big win. No data center to maintain, an on-demand scale-out infrastructure and pay for the resources that you actually use.

There are downsides to putting your data in the cloud. At the top of the list is security. Followed closely by data persistence, resource availability, backup and recovery.

A new issue that has cropped up lately is regulatory compliance. All cloud vendors are pretty mum about giving away details of their cloud infrastruce. So it makes it very difficult for large corporate enterprises to assert that what they put out in the cloud is secure and in compliance.

Erik Howard cloud computing cloud computing