I was doing a bit of Googling on the topic of grid computing. Specifically, I was looking for good examples of universities using idle computer resources to good effect. Our uni has heaps of machines that are idle much of the time and I'm considering raising awareness of the issue.
Does anyone have a good example, maybe from their own uni or research institute that they'd like to share?
On a separate but related note, I'm not sure what to make of PatriotGrid. At the least, it's a bloody awful name.
Comments
Grid Computing
I am doing my Master project on Grid Compuitng, so I am still developing my project whcih might be interested. So, I will inform you about my results and detailes as soon as I finish my practical work. I am almost finished 80% of my job.
Best regards
Rafid Al-KHannak
Bolton University - UK
University for Applied Science, South Westphalia.Campus Soest -Germany
Condor in the UK
Hi there,
At Newcastle Uni (in the UK) we have decided to go with Condor for our campus grid. There are also large deployments at Cambridge, UCL and Bristol. Condor is almost the "solution of choice" for this in the UK as it has been pushed very heavily by certain people in our e-science programmes. Personally I think its great, having done small deployments before - it's very easy to get up and running and nicely cross-platform (ok binaries are clipped on Windows and MacOS X so no checkpointing), and after struggling with OpenPBS (yuck) and having some SGE experience I still rate Condor.
The Condor community is very friendly and the developers open to dialogue with users, it has the best mailing list of any free software I've ever used..
Our deployment, well we're aiming for 10,000 machines, tying in our HPC facilities as well as the idle Windows desktops scattered in labs all across the Uni. At the moment we're in the testing phase - slowly rolling out and testing. Please feel free to get in touch.
Actual experience
Recently I've been working on grabbing as many computing resources as possible at my uni. After some research, it looks like "Condor" is the best approach for using PCs (public and "owned").
But when I approached the sys admins of the public PCs on my campus, they really weren't bothered about any issues, as long as I assured them that the jobs are working at low priority (cycle-stealing).
So basically, I have >100 WinXP which at startup time run a process that pulls a job off my server, process it, return the results, rinse and repeat.
My experience is that sometimes the computation authority just thinks it's cool to have their PC's used for something other than Excel and MP3 listening...
hello... I think condor's d
hello...
I think condor's days are over.
One might wanna look at GridIron or GigaSpaces or Jgrid as very easy to use Grid toolkits and if one wants to achieve the same but has a lot of admin and programming expertise at hand they would like Globus, personally I think Grid is all about simplicity and Globus just does'nt have it.
Thanks
ps :I don't wanna leave my email online for spammers, but if you want to reach me leave a comment....
commercial
All of which are commercial packages for sale, apart from Globus. I would also point out that the latter two are Java-based (ick!).
Dunno about condor's days being numbered, but this looks awfully like a plug to me...
suspicious indeed
Dunno about condor's days being numbered, but this looks awfully like a plug to me...
Seconded. Condor looks very much alive and kicking from their website.
What kind of distributed applications ?
What kind of distributed applications are you using or planning to use Condor for ?
Would it be possible to run distributed BLAST for example ? I'm interested in the idea of using idle desktops computers for this kind of thing but my only experience is with dedicated cluster nodes under Linux/PBS/LAM etc.
I haven't been able to get it clear in my mind how grid platforms like Condor work with respect to bioinformatics applications like BLAST.
Pointers appreciated :)
Condor docs
My very limited understanding is that Condor supports MPI and PVM, and also has its own "vanilla" message passing system. I need to read the documentation - it's quite extensive and the website is very good.
Condor is a bit limited on windows
It has all sorts of goodies to make it easy writing parallel stuff on all sorts of unixen, just not on Microsoft's OSs....
condor usage
I was planning on using condor for "embarassingly parallel" jobs, where the total workload can be divided into separate work-units which do not require interaction between themselves. Also, I wasn't counting on having significant amount of local storage on the distributed nodes.
I don't know if it's suitable for distributed BLAST, which requires having the BLAST databases residing on the nodes locally.
The best pointer I can give on this subject is this: join the bioclusters mailing list, or browse the archives - there is a ton of traffic on distributed computing, BLAST in particular.
Makes sense
Embarassingly parallel jobs makes sense for something like Condor. I guess what I had in mind was a windows version of mpiBlast packaged as a screensaver (cf. SETI@HOME). It would be nice to hook up all the high end windows desktops in the building that sit idle overnight for BLAST processing.
I'm sure there would be various technical challenges with an application like this (limited storage, start/stop of BLAST processing with the screensaver).
In fact most of these boxen are in accademic's offices (Damn it, I need a dual processor boxen for email), so that means they are probably sitting idle during the day too (I don't consider email a serious application of computing power).
Distributed Computing
I would say
WebCom-G (http://www.cuc.ucc.ie)
Legion
NetSolve
Condor
or P2P such as
Seti@home
Folding@home
and many others
Cheers,
Adarsh
(http://www.adarshpatil.com)