Thursday, January 24, 2013

Ran a cloud server today for the first time.

--What is this Cloud Computing Anyway?--

Is it just another buzz word, or is there really something to it all?  I had to find hosting for a couple of e-commerce classes I am taking this semester, so I decided to try to kill two birds with one stone.  Can I get the cloud computer to host my site, save some money, and become proficient in using the Amazon Cloud Computing resources?  That is what this series of articles will explore. 

The virtual server I ran today was a free EC2 instance running in the cloud.  For 12 months you can get 750 hours of compute time for free using one of the starred containers to create your instance.   750 hours over 31 days is just over 24 hours a day, so you should be good leaving the server on all the time.

You also get a bunch of other stuff for free, according to this page:

Free Tier*
As part of AWS’s Free Usage Tier, new AWS customers can get started with Amazon EC2 for free. Upon sign-up, new AWS customers receive the following EC2 services each month for one year:
  • 750 hours of EC2 running Linux/Unix Micro instance usage
  • 750 hours of EC2 running Microsoft Windows Server Micro instance usage
  • 750 hours of Elastic Load Balancing plus 15 GB data processing
  • 30 GB of Amazon EBS Standard volume storage plus 2 million IOs and 1 GB snapshot storage
  • 15 GB of bandwidth out aggregated across all AWS services
  • 1 GB of Regional Data Transfer

 You can find out more information and the other free stuff you can get access to by following the link to here: AWS’s Free Usage Tier

And there is a lot more info here:

There is also another free service offer:

60-day Free Trial
In addition to the Free Usage Tier we also offer 60-day Free Trial for all AWS customers that are new to Amazon ElastiCache. AWS customers can avail of this Free Trial in addition to the Free Usage Tier offerings.  For more information on the free trial please visit Amazon ElastiCache free trial.

--Creating an Account-- 

In order to get into this you must have an Amazon account.  I used my account to log into the AWS console at, it prompted me to sign up for the 12 months of free service.  There is a second log in because this service can quickly add up in cost if you do something wrong.  Or you become wildly successful and build a website that performs beyond all expectations.  I had to give them a credit card, my address and my phone number.  My phone number was called and I had to say a pin number to authenticate the number.  Then you have to wait a few minutes and they send you a final email clearing you for the service.

-- Starting an Instance --

This is why we are here.  It is easy to start an instance, you just click a big button on the AWS web console labelled "Launch Instance" here:

There are just a couple of free ones, the one I choose is described here:

When you create the instance you download its private key so that you can log in latter using the management tools and ssh to log in.   Don't lose this, because it is not saved anywhere else and you cannot download it again later.

You can also configure a few things, and one of them most important things to configure is if you are using persistent storage, or just storage in that instance. This matters later when you stop or terminate the process.

-- Security --

By default your server is blind to the world.  All incoming ports are blocked.  You have to select the instance and switch to the ports tab in the bottom section add in the ports you want to open to the world, and then and only then can you connect to the server using ssh or a web browser.
SSH is port 22.
Web is ports 80 and 443.

--Connect to the instance--

Then I installed the management tools to my Linux box following these directions:

I found that link in the documentation at the lower right of the EC2 management console.

You basically have to set 4 environmental variables to tell it where you put the binary files you downloaded, where java is located, and two strings of text from the AWS site under your name where it says "Security Credentials."

There are hundreds of commands in the management directory that I am going to have to explore.

I have not yet been able to figure out how to log into the instance, but I did get a connection refused, not a valid user, so I think I cam close.   I just found this:

For an Amazon Linux instance, the default user name is ec2-user. For Ubuntu, the default user name is ubuntu. Some AMIs allow you to log in as root. So in this example, you may need to change the user name from ec2-user to the appropriate user name.
So that might be what I was doing wrong.  I need to put ec2-user@ in front of the ssh host name on the command line. :)

Update:  I did use `ssh -i myprivatekey.pem ec2-user@mynewamidynamicdnsname` and I went right into my box as the ec2-user who had full sudo permissions.     More in a new article!

--To Stop or Terminate--

OK, you connected, you looked around, maybe saw a cool thing or two,  now you want to stop the instance.

I looked online for some way to turn off the instance, but there is no way that I saw to do it from the AWS console.  If anyone reading this knows how to manage an instance inside the console, I am all eyes.  As far as I can tell you must stop the instance using the command line utilities.

But wait, there is stop, and also terminate.  What is the difference between stopping and terminating an instance?  Terminate certainly sounds worse than just stopping. 

If you terminate then not only is the virtual instance eliminated, but also the storage space it was using.

If you stop the instance, then the virtual server goes away, but the storage you attached to it stays around.  Unfortunately you will get billed for this storage.

Right clicking on the instance in the management console will allow you to turn on protection so that it can't accidently be terminated, just stopped.

--Cost of Storage--

You decided to keep a disk full of data around, for sentimental reasons, so how much does this cost?  This is from the pricing page.

Amazon Elastic Block Store

Region:  Virginia East

Amazon EBS Standard volumes

  • $0.10 per GB-month of provisioned storage
  • $0.10 per 1 million I/O requests

Amazon EBS Provisioned IOPS volumes

  • $0.125 per GB-month of provisioned storage
  • $0.10 per provisioned IOPS-month

Amazon EBS Snapshots to Amazon S3

  • $0.095 per GB-month of data stored
10 GB of storage would cost you a cool buck every month.

I have no clue what my I/O rate is for a web server, not something I have ever tracked before, so I don't what  ten cents per million read writes will cost in a month.

But you get so much free for the first year as well:
So as long as you stay under this limit, you are good.

I'm wondering if you could have a single drive boot multiple instances, and then store all their local data on their own instance drive to save a couple of GB per instance.  I am not sure how fast we are going to go through those 2 million I/Os, something to keep an eye on.

Not sure what snapshots or IOPS is yet, something to look into later.

If you want to know more about stopping, terminating when you get charged, this site breaks it down with graphs and charts so that it is very clear:


And finally here is the reason to have a web server on the Internet.  The requests going to the server and the pages you send back.  With the free service you get a total of 15GB of data, up and down.  This sounds like a lot, but that is just 5.6KB per second of continuous data transfer up and down combined for the full month.  There are a lot of seconds in a month so watch this and stop the server before you hit the limit. 


Under My Accounts/Usage there is something called Usage Reports.  The reporting looks very detailed, I had the box on for less than an hour, but  the begin and end times show a full hour increment for the usage:

Service      Operation       UsageType     Resource    StartTime            EndTime              UsageValue
AmazonEC2    RunInstances    DataTransfer-In-Bytes     01/24/13 16:00:00    01/24/13 17:00:00    4864
AmazonEC2    RunInstances    BoxUsage:t1.micro         01/24/13 16:00:00    01/24/13 17:00:00    1
AmazonEC2    EBS:IO-Read     EBS:VolumeIOUsage         01/24/13 16:00:00    01/24/13 17:00:00    2195
AmazonEC2    EBS:IO-Write    EBS:VolumeIOUsage         01/24/13 16:00:00    01/24/13 17:00:00    5622
AmazonEC2    CreateVolume    EBS:VolumeUsage           01/24/13 16:00:00    01/24/13 17:00:00    8589934592
AmazonEC2    RunInstances    DataTransfer-Out-Bytes    01/24/13 16:00:00    01/24/13 17:00:00    4864
AmazonEC2    RunInstances    DataTransfer-In-Bytes     01/24/13 17:00:00    01/24/13 18:00:00    15710
AmazonEC2    RunInstances    DataTransfer-Out-Bytes    01/24/13 17:00:00    01/24/13 18:00:00    14314
AmazonEC2    EBS:IO-Write    EBS:VolumeIOUsage         01/24/13 17:00:00    01/24/13 18:00:00    262
AmazonEC2    EBS:IO-Read     EBS:VolumeIOUsage         01/24/13 17:00:00    01/24/13 18:00:00    5414 

There are also account activity reports under the same drop down show how much this is costing me.  There are also billing alerts in this sections so that if I misconfigure anything it will send me a notice so that I can fix whatever is running wild.

--Further Research--

So, I definitely want to get a LAMP server up and running as an instance for the 750 free hours a month.  This will save having to buy hosting on the Internet.  It is also much nicer to have your own instance of a virtual machine, rather than sharing a box with a lot of other people.  Free helps a lot too.

 I would also like to explore setting up a web site using C# and Dot Net on a windows server for free as well, and AWS will let me do that too.

And what are all these other free services that Amazon offers? 
  • Amazon S3 - Highly-scalable, reliable, and low-latency data storage.
  • Amazon RDS - Managed MySQL, Oracle and SQL Server databases.
  • Amazon CloudWatch - Monitoring for AWS cloud resources and applications.
  • AWS Data Pipeline - Orchestration for data-driven workflows.
  • DynamoDB Icon - Fully managed NoSQL database service with seamless scalability.
  • Amazon EBS - Highly available, highly reliable, predictable storage volumes.
  • Amazon ELB - Web service that provides scalability and high availability.
  • Amazon SNS - Web service to set up, operate, and send notifications from the cloud.
  • Simple Queue Service - Scalable queue for storing messages as they travel between computers.
  • Amazon SWF - Workflow service for building scalable, resilient applications.

Each of these is an area to explore in its own right. And there are even more services to explore beyond the free tiers.

Planning to get that LAMP server up and running tomorrow, so stay tuned for more info!

No comments:

Post a Comment