No doubt, cloud computing is hot at the moment. Everyone is jumping onto the bandwagon before it become too late for them. Currently data clouds seems to be a major focus for most of the companies and institutions adopting cloud computing in their long term strategy. These organizations are using data clouds for both on demand computation and to persist and manage the data. Distributed and replicated data clouds not only enable the faster access to resources but they also ensure the higher availability, scalability and fault-tolerance. Data clouds have proven to be highly attractive for scientific community as well. Large scale genomic analysis on the cloud is one of the many examples where community is enjoying best of on demand computing and storage technologies. Most of use cases in life sciences community are focused around mining the huge amount of data produced by third generation sequencers.

Another niche area where cloud computing is making its way is simulation based science and engineering. Compared to data clouds modeling and simulation of various science and engineering problems using scalable cloud computing environments is still in fancy stage. There is some excitement with Amazon’s recently announced High Performance Computing (HPC) cloud services, but there is lot of uncertainty to what extent cloud based HPC clusters can compete with on-premise HPC clusters or in-house dedicated machines. For instance, it remains to be seen  how multi-tenancy in the cloud will react to the HPC performance. Exclusive access to a cloud computing node are way too expensive for both cloud infrastructure providers and users especially when scientific applications require large numbers of nodes. In addition dedicated or exclusive nodes don’t fit very well with economies of cloud computing, in fact multi-tenancy is a prerequisite for the cloud computing. There are some concerns over processing, memory, storage, and network usage patterns in shared multi-tenancy environments. This is an open and unexplored area for both scientific community as well as cloud infrastructure providers. Before people start adopting the cloud based HPC services,  these concerns need to be addressed and explored through the various benchmarking studies. As Mohamed Ahmed suggests,

Cloud infrastructure is still lucrative if comparing its economics to building in-house HPC machines. However, cloud for HPC has to be efficient enough to reach proper performance ceilings without disappointing customers who probably experienced at a certain point to run their HPC applications on dedicated machines.

I could not agree more. Lack of performance guarantees against shared cloud infrastructure is major issue which cloud computing users are facing on regular basis irrespective of type of application they are running. Currently most of cloud infrastructure provider guarantee only the uptime of their nodes and in some cases they provide persistent access to the resources by fully reserved RAM and storage allocations without over-subscription. Some of them guarantee a minimum CPU availability proportional to reserved size but often there are huge gaps between what is promised and what is delivered. There is growing demand for Service Level Agreement (SLA) which should cover both performance and availability. Compared to HPC applications, for the data clouds performance is not a major issue. HPC applications are computationally intensive and they can be highly demanding in a given time period while data clouds behave uniformly.

In next few weeks through a series of blog posts we will focus on some interesting modelling and simulations applications for high-throughput computational science built around cloud based HPC clusters. So stay tuned.