Erik Howard

Lets rewind to about a month ago. As I usually do, I skipped past the README files and installed Erlang with apt-get. Ten seconds later, I had Erlang up and running on my Ubuntu development VM. Like a bad teenage horror movie, I’m sure you can see where this is going.

Fast forward to present day. I was running into another serious compiler error with Erlang. A few of the Erlang web frameworks were refusing to compile – again. As usual, the error message being thrown by Erlang was as cryptic as Aramaic spoken with a lisp. Nothing turned up in Google about the error. I even hopped on the #erlang channel on IRC hopping to get a few pointers.

No one had a direct answer, but I did get a clue. One person mentioned that Erlang is very good telling you vary far in advance if there will be any breaking changes in future releases. I also found out that I was using an Alpha release. Not good, not good at all.

So armed with this new information and a few hunches, I un-installed Erlang (again) and download the last stable release – R12B-5 at the time of this article. Here’s what I did.

sudo apt-get libncurses5 libncurses5-dev
tar xvzf otp_src_R12B-5.tar.gz
cd otp_src_R12B-5/
./configure
make
sudo make install

I had to install libncurses5 and libncurses5-dev since it wasn’t installed on my Ubuntu development VM.

After building Erlang from source, I went back to compile my code and everything worked. Erlang is definitely death by 1,000 cuts. It’s enough to make a grown man cry.

I’ve been teaching myself Erlang.  It’s a great Functional Programming language. I’ve also dabbled a little with Scala.

Besides wanting to learn a new computer language, I’ve also wanted to port some of my high-traffic Ruby On Rails sites into Erlang. Erlang will be able to handle 3x-4x the traffic using less resources. Less EC2 instances up and running means more money in my pocket.

While playing around with mochiweb I ran into a compile error that didn’t make sense. At least it didn’t make sense to my limited knowledge of Erlang. The error was this:

Eshell V5.5.5  (abort with ^G)
1> c("mochijson2.erl").
./mochijson2.erl:38: illegal atom
./mochijson2.erl:2615: no module definition
error

I opened up the file and went to the offending line-numbers and could not find anything out of the ordinary. After a little Googling, I found a post on the Erlang-Questions mailing list from someone who had the same error as I did.

Turns out that there was nothing wrong with the file other than being encoded as UTF-16, which Erlang does not like at all. It would have been nice if the compiler threw an error saying that your source file is in the wrong encoding. Illegal atom / no module definition just doesn’t cut it for me.

Cryptic error messages have been my biggest stumbling block with learning Erlang. I take two steps forward, then one epic step backwards. So the moral of the story is to make sure that all of your Erlang files are encoded using Latin-1 (ISO-8859-1) and not UTF-16.

• More News of Google Buying Twitter
  Posted on Thursday, April 2nd, 2009 in acquisitions – Comments: (0)
  More rumors today from TechCrunch about Google and Twitter.  Michael Arrington from TechCrunch goes on to say that Google and Twitter are in early talks of some type of partnership or an outright acquisition.Last year Facebook was in talks to buy Twitter for over a half billion dollars. Twitters current valuation is around $250 million dollars. If such a deal would go through this would be the second time that Evan Williams has sold a company he co-founded to Google.
• Amazon Introduces Elastic MapReduce
  Posted on Thursday, April 2nd, 2009 in MapReduce – Comments: (0)

  Amazon Elastic MapReduce

  [/caption]Today, Amazon annouced the availability of it’s newest web servrice – Elastic MapReduce.Amazon Elastic MapReduce is a web service that enables businesses, researchers, data analysts, and developers to easily and cost-effectively process vast amounts of data. It utilizes a hosted Hadoop framework running on the web-scale infrastructure of Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3).

More rumors today from TechCrunch about Google and Twitter.  Michael Arrington from TechCrunch goes on to say that Google and Twitter are in early talks of some type of partnership or an outright acquisition.

Last year Facebook was in talks to buy Twitter for over a half billion dollars. Twitters current valuation is around $250 million dollars. If such a deal would go through this would be the second time that Evan Williams has sold a company he co-founded to Google. He previously sold Blogger to Google.

Here’s hoping that the rumors are true. The fail whale is starting to appear a little too frequently for me.

Amazon Elastic MapReduce

Today, Amazon annouced the availability of it’s newest web servrice – Elastic MapReduce.

Amazon Elastic MapReduce is a web service that enables businesses, researchers, data analysts, and developers to easily and cost-effectively process vast amounts of data. It utilizes a hosted Hadoop framework running on the web-scale infrastructure of Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3).

Amazon Elastic MapReduce automatically spins up a Hadoop implementation of the MapReduce framework on Amazon EC2 instances, sub-dividing the data in a job flow into smaller chunks so that they can be processed (the “map” function) in parallel, and eventually recombining the processed data into the final solution (the “reduce” function). Amazon S3 serves as the source for the data being analyzed, and as the output destination for the end results.

Amazon Elastic MapReduce applications can be authored in Java, Ruby, Perl, Python, PHP, R, or C++.  Elastic MapReduce applications only run on Linux/Unix instances running in the US region on EC2. Both reserved and on-demand instances are supported. If you have any on-demand instances, they will be used first.

Amazon Elastic MapReduce pricing is in addition to standard EC2 rates.

Current Amazon MapReduce Pricing

Amazon EC2 and Amazon S3 charges are billed separately. Pricing for Amazon Elastic MapReduce is per instance-hour consumed for each instance type, from the time job flow began processing until it is terminated. Each partial instance-hour consumed will be billed as a full hour.

• Should Google Buy Twitter?
  Posted on Wednesday, March 25th, 2009 in twitter – Comments: (0)
  The Motley Fool has a new article on why Google should buy Twitter.They assert that Google would be the most suitable match for Twitter. They could afford to buy the non-revenue generating Twitter in this bad economic environment.They would have no problems hosting Twitter. Google has the horsepower and bandwidth to prevent Tweet Overload. The culture and vision of both companies would not clash as much as a Microsoft Twitter tie-up.
• Getting Started with Amazon EC2
  Posted on Saturday, March 28th, 2009 in cloud computing – Comments: (0)
  Amazon Elastic Compute Cloud (EC2) is a commercial web service offering that allows you to rent your own server in which to deploy your applications. The infrastructure is highly scalable and allows you to increase or decrease your computing horsepower based on demand. Think of it as your own elastic data center where you only pay for the resources that you use.
• Enterprise Adoption of Cloud Computing
  Posted on Monday, March 23rd, 2009 in cloud computing – Comments: (0)
  While startups, consultants and individuals are rushing to put their applications in the cloud large enterprises are, for the most part, are still sitting on the fence.What is preventing adoption of cloud computing for enterprises. It basically comes down to a few issues – security, accountability and standards.Most cloud vendors, but not all, are mum (publicly at least) on the exact details of their cloud infrastructure.

Amazon Elastic Compute Cloud (EC2) is a commercial web service offering that allows you to rent your own server in which to deploy your applications. The infrastructure is highly scalable and allows you to increase or decrease your computing horsepower based on demand. Think of it as your own elastic data center where you only pay for the resources that you use.

EC2 uses Xen virtualization for its instances (servers). EC2 instances come in 3 sizes Small, Large, and Extra large. Amazon sizes instances on what’s called “EC2 Compute Units”. One EC2 Compute Unit (ECU) provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.

EC2 Standard Instances
Most applications are well suited for these types of instances:

• Small Instance: 1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of instance storage, 32-bit platform
• Large Instance: 7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each), 850 GB of instance storage, 64-bit platform
• Extra Large Instance: 15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each), 1690 GB of instance storage, 64-bit platform

EC2 High-CPU Instances
This family of instances has proportionally more CPU resources than memory and is better suited for computational-intensive applications.

• High-CPU Medium Instance: 1.7 GB of memory, 5 EC2 Compute Units (2 virtual cores with 2.5 EC2 Compute Units each), 350 GB of instance storage, 32-bit platform
• High-CPU Extra Large Instance: 7 GB of memory, 20 EC2 Compute Units (8 virtual cores with 2.5 EC2 Compute Units each), 1690 GB of instance storage, 64-bit platform

For most blogs, company websites and low-traffic ecommerce sites, a Small Instance will be fine. You can always migrate up to a larger instance.

Getting Started
For this article we are going to launch a 32-bit Ubuntu 8.04 LTS Hardy AMI (Amazon Machine Instance) from Alestic. Amazon provides a few AMI’s designed to get you up and running quickly. They also have over 1,700 Community AMI’s. You can select from Fedora, Ubuntu, Windows, CentOS and other flavors of Linux.

Before you can create your first instance, you will have to Sign Up for Amazon EC2.

Amazon EC2 Signup

After signing up, visit the AWS Management Console page. Click on the Go to Amazon EC2 Console button.

Launch EC2 Management Console

Before you create and launch your EC2 instance. You should first create a Key Pair. You will use this key pair to login to your instance after it’s been created.
To create your Key Pair, click Key Pairs in the left Navigation menu. Next, click the Create Key Pair button in the Key Pairs window. Enter a descriptive name for your new Key Pair in the Create Key Pair dialog window.

Create Amazon EC2 Key Pair

Next you will be prompted to download your newly created Key Pair. Download your newly created Key Pair; you will need this to login to your new instance later.

Save EC2 Key Pair

NOTE: AMAZON DOES NOT STORE YOUR PRIVATE KEY PAIR. IF YOU DO NOT DOWNLOAD YOUR KEY FILE AT THIS POINT YOU WILL NOT BE ABLE TO LOGIN TO ANY INSTANCES THAT WERE CREATED USING THIS PRIVATE KEY PAIR.

Now let’s go ahead and find our Amazon Ubuntu AMI and fire it up. Click the Dashboard link in the left hand navigation menu and then click the Launch Instances button.

Launch Amazon EC2 Instance

Select the Community AMI’s tab. Enter ami-71fd1a18 in the search field. This is the AMI id of the Alestic 32-bit Ubuntu Hardy server. Click the Select button to continue.

Alestic Ubuntu Hardy AMI

Now we are going to set some options for our Ubuntu AMI. First type in the number of instances you want to create, we are only going to create 1 for this article.
Next, select the Small instance type. In the Key Pair Name drop-down, select the new Key Pair we just created. In the Security Groups listbox, select default. We will skip the Advanced Options for now.

Amazon EC2 AMI Options

Click Launch to start you new Ubuntu Amazon instance!

Amazon will now attempt to start you instance. This may take anywhere from 30 seconds to 5 minutes. To check on the status of your instance click the Instance link in the Navigation window.

Amazon EC2 Instance Status

You can connect to your instance with any SSH client. Find the Key Pair file you created previously for this instance. Make note of the Public DNS address of your new instance. For example, to connect to my test instance I just created, I would use the following SSH command:

ssh -i Web1.pem root@ec2-67-202-37-5.compute-1.amazonaws.com

Web1.pem is the private Key Pair that I created for this instance. If you did not save your Key Pair, then you will have to destroy this instance and create a new Key Pair and start a new instance with the new Key Pair.

Once you have finished playing around with your instance, make sure you terminate the instance so you won’t be charged for the idle instance.
In future articles, I will show you how to setup Elastic IP’s, Elastic Block Storage, run shell scripts to setup your new instance and creating an instance using only Amazon’s EC2 API.