Sun recently announced the availability of the Sun Grid Compute Utility for public use. This is a service that I seem to recall HP and IBM talking about for some time. Looks like Sun is giving it a try. From the Sun Grid website: "Sun Grid provides an easy and affordable access to an enormous computing resource for the predictable and all-inclusive price of $1/CPU-hr."
This is basically a homogenous compute farm running Solaris 10 with the Sun Grid Engine for job control. As it is open for diverse public use, it accepts only self-contained programs specifically compiled for the Solaris 10 (32 or 64-bit) platform. This is an interesting development at a time when many companies and academic institutions alike are considering the benefits of outsourcing to data centers. Passing over maintenance and administration of a likewise massive computational center has several advantages; there’s some benefit in Sun (or any centralized compute center, like the Los Alamos Blue Mountain computer cluster ) being able to scale up easily, and heat removal and power consumption spring quickly to mind. However, it's not clear that bioinformatics compute cluster needs mesh well with such a vanilla implementation such as Sun's. Bioinformatics programs, at least on the level I run, tend to be development style programs, which benefit from a customizable compute environment (if you want to install and freeze some versions of software). From an academic standpoint, many grants don’t allow for this type of computing model yet. The pay as you go model does not play well with grant expirations - transferring funds from your grant to a PayPal account may raise a few eyebrows.
That being said, Sun has brought out a few early success stories, as would be expected, and one in particular is sure to draw attention to the bioinformatics crowd: Applied Biosystems seems to have adopted use of the Sun Grid Compute Utility.
The question I'm wondering is whether or not the benefits outweight the costs in terms of farming out compute centers for bioinformatics? Is the flexibility of constant provisioning cycles necessary in pure bioinformatics research? Is the data center paradigm only useful for enterprise-level software deployment? Or, as bioinformaticists, are we necessarily going to face having to develop matured and self-contained programs and leave behind many of the scripted languages that we so much cherish?
On another note, this is a master stroke for Sun's Solaris 10 platform. Since all jobs submitted to the Sun Grid Compute Utility have to be compiled for Solaris, I expect at least a small blip in the Sol10 downloads and at least a meager jump in the number of ported applications to Sol10. Notwithstanding Java apps, of course.
As anybody who's built or administrated a computational cluster can attest, there are some serious advantages towards outsourcing compute center needs. Centralized data centers have been around for years. Unless power becomes free, heat becomes cool and fileservers don't fail, I expect to see a very large industry emerging along the same lines as the data centers, but centered on computational power, such as this initiative by Sun.
Comments
Interesting points
I'm still recovering from building and administrating a compute cluster. It was so bad, I had to leave the state.
I guess for very large clusters (10s or 100s of nodes), there are definite advantages to outsourcing in terms of power and cooling costs. I don't necessarily agree that this leads to more enterprise-type software and less scripting. What cluster administrators need to realise is that a cluster for biological applications is quite different to a cluster for more traditional HPC research such as physics or engineering, where people tend to write their own parallelised code for MPI or whatever. Biological apps by their nature involve glue scripts to generate data analysis pipelines. We also require local database storage with automated incremental updates and efficient distribution to nodes.
What it boils down to is: make friends with the admins on an outsourced cluster and explain your needs or work in an environment where others share your "can do" attitude, so as you get it done. Many academic environments are a nightmare when it comes to this kind of endeavour, not because it's hard but because they lack IT staff with sufficient understanding and because of stifling beaurocracy, restrictions, rules and regulations surrounding simple tasks such as buying and installing an air conditioner.