aws

Drug simulations on Amazon Web Servers

From the New York Times "Bits" column:

"The supercomputer game has traditionally been the playground of giants — government agencies and a handful of major corporations.

Get Started with SciDB

 

SciDB is run on an Amazon Machine Image (AMI), which you will need to create in order to follow the procedures here. The AMI ID for SciDB is: ami-ba9668d3.

If you need help setting up an AMI, see Get Started with Amazon Web Services.

System Requirements

Cloud Economics: Visualizing AWS Prices over Time

This post is the first in a series on cloud economics.

The key value proposition of cloud computing is that the economies of scale earned inside massive data centers, coupled with the flexibility and elasticity of virtualization, make it possible to outsource your computing at a lower cost than running your infrastructure locally.

Needless to say, "your mileage may vary," but for many circumstances (for example, highly variable usage, non-subsidized acquisition and operating costs), the evidence supporting this value proposition is already widely available.

Here we focus on an aspect that has received less attention: the steady decrease in prices offered by Amazon for their services. Over time, the price to rent 1 unit of resources for three years of continuous usage has fallen dramatically as Amazon offered new instance types, offered new long-term pricing plans, and lowered prices outright across the board. This makes cloud computing even more attractive financially. (Presumably other vendors of cloud computing will show a similar trend over time.)

To visualize these effects, we can plot the price of renting various AWS resources over time, as affected by the announcements made by Amazon. To normalize the effect of different kind of price adjustments, we consider one resource at a time, independent of the others. For example, this plot shows the lowest possible price to rent a single EC2 Compute Unit, ignoring the exact number of cores, the amount of memory, the amount of local storage, etc. In this plot, each price drop (vertical segment) is flagged with a letter that indicates an announcement on Amazon's website that affected the price of that resource.

Lowest Total Price (USD) for Continuous 3 Year Rental of 1 EC2 Compute Unit
(Amount of memory, number of cores, etc., is allowed to vary to achieve the lowest possible price)

We can also group multiple types of resources into a single plot to see that many announcements affected the price of multiple resources. For example, the figure below plots the drop in price for a three-year deployment of resources on EC2 over the last few years. Resources are normalized to 1 EC2 compute unit (blue line) - approximately equivalent to a 1.0 GHz CPU - and to 1 GB of RAM (red line). In all plots, the events that influenced the price are linked at the right and labeled in the graph.

We see that the price of one EC2 compute unit has dropped over 80% since 2007, which does not quite track Moore's law (100% every 18 months), but it comes close. Of course, Moore's law only considers transistors per chip and therefore instructions per second, which rarely dominates costs in realistic applications.

The price of 1 GB of RAM has dropped about 75% over the same period.

For these purposes, the price of 1 EC2 unit is defined as the lowest possible price regardless of instance type. For example, a High CPU instance offers 20X the compute power, but only costs 8X the amount. Similarly, a High Memory instance type offers 10X the main memory, but only costs about 6X the price. So the cost per unit (either compute or memory) dropped when these new instance types were announced.

For the long-term deployment, this price includes the up-front reservation charge plus continuous hourly usage fees for 3 years (once reserved pricing was available).

These prices are in fact overstated, since existing customers benefit from an immediate price reduction in hourly charges, just like new customers. So the total cost for a 3-year deployment beginning in 2007 dropped when prices were reduced in 2008.

Lowest Possible Price (USD) for a 3-year Rental of AWS Resources
Price per EC2 Compute Unit (amount of RAM, scratch storage, etc. varies)
Price per GB of RAM (number of EC2 compute units, total amount of RAM on the instance, amount of scratch storage, etc. varies)

We can perform a similar analysis for a short-term deployment, say, 10 days. The cost in this case does not benefit from reserved pricing. Specifically, the cost is determined from hourly usage fees for 240 hours of use at on-demand prices.

The results are similar: a 65% drop in the cost of 1 EC2 compute unit, and a 50% drop in the cost of 1 GB of RAM.

Lowest Possible Price (USD) for a 10-day Rental of AWS Resources
Price per EC2 Compute Unit (amount of RAM, scratch storage, etc. varies)
Price per GB of RAM (number of EC2 compute units, total amount of RAM on the instance, amount of scratch storage, etc. varies)

Finally, we can look at storage costs. Over time, the price to rent 1PB and 1TB of storage has fallen as Amazon offers new pricing tiers, additional redundancy offerings, and across the board price drops. In this third plot, we see the cost to store 1 petabyte (orange line) for one month has dropped by 45% since 2008, and the cost to store 1 terabyte (green line) for one month has dropped by 38%. We are assuming that Amazon's reduced redundancy storage, which still provides 99.99% availability, is acceptable in this scenario, motivated by the fact that researchers rarely want, need, or can afford the seven nines of availability demanded by industry.

Price to Rent 1 PB and 1 TB of storage for one month

Overall, these prices are not dropping as precipitously as the cost of the raw hardware, but the hardware cost is only a fraction of the total cost of deploying computing infrastructure.

Other Resources:

How are the economics of computing affecting your research? Leave a comment and let us know!

Cloud Computing Taxonomy

Cloud Computing Taxonomy

Understanding Cloud Computing for Research and Teaching

Overview: Cloud Computing for eScience, Research, and Teaching

Cloud computing refers to computing facilities provided on demand, served over the Internet from shared data centers that exploit enormous economies of scale. Rather than purchasing a cluster of computers, finding space in your local lab, hiring an administrator, and then letting the facility sit idle when not needed, you can outsource your computing to remote facilities in the cloud, and pay only for what you use.
Syndicate content