nodalpoint.org - Data management - Comments

User-friendly, High compatibility but low requirements

amejat — Tue, 06 May 2008 17:37:22 -0400

Hi Neil,

And thanks for your interest in the project. I fixed the mistake in "invaluable".

I also completely agree on the facts that:
- we need a free open source Laboratory Information Management System
- to be adopted, it needs to be user-friendly, cross-platform, highly compatible with existing open source solutions.

I had that in mind when I joined the eyeOS team, three years ago and I always kept it in mind during eyeOS development. And it is because eyeOS fits these needs that I really think it provides the perfect platform for a LIMS:
- For users, eyeOS provides a really intuitive and user-friendly desktop-like environment, accessible from anywhere.
- For developers, eyeOS provides a complete Toolkit and set of libraries to easily and quickly develop new eyeOS applications.
- For network manager, eyeOS has very low requirements. It only needs PHP and XML (javascript being provided by the navigator). It does NOT require a MySQL (or other) database or a Java server (such as Tomcat). For this reason, it can be installed on any basic hosting plan. However, it allows the easy integration of Flash (swf), Java applets or MySQL applications. Importantly, it constitutes a secured platform to store and share documents with a Virtual File System controlling the user access to files.

To conclude, the last but not the least, eyeOS is Free and Open Source. To my knowledge, it is unique.

I hope you will take time to discover this platform and agree with me on its potential.

Best regards,

Alexandre

www.eyeLIMS.com - Free Open Source Laboratory Information Management System powered by eyeOS

Value

Neil — Tue, 06 May 2008 02:55:35 -0400

provides an unvaluable system

*Invaluable*, surely. I hope?

I'll be interested to see how this develops: the research world is in desperate need of a quality, open-source LIMS. However, I'm concerned that this LIMS is tied to an existing product. Everyone has a web browser, but not everyone will want to sign up for eyeOS and learn how to use it. A true web application is something more like Google Documents, where you just open a page and start work.

Remember, biologists will look for any excuse not to learn how to use an application; you need to keep the entry barrier as low as possible.

Yeah, need to update my

ejain — Tue, 22 Jan 2008 19:47:42 -0500

Yeah, need to update my blog...

700?

nuin — Tue, 22 Jan 2008 13:18:11 -0500

At 700 a pop for an Action Figure?

Dude, Where's My NAR?

Duncan — Tue, 22 Jan 2008 13:09:27 -0500

Hello Maximillian, I wasn't attacking NAR, was just wondering about what proportion of the data is dead. It is an interesting technical challenge to find this out. It would also be useful to measure the cost of gathering noisy, redundant and poorly understood data in terms of wasted resources (people, time, money, computers, false positives etc). Perhaps somebody has done something like this already? Especially with all the irresponsible sequencing just for sake of it that goes on...

As for the physicists, I mentioned them for comparison. Like you, I doubt they will use 100% of their data either, but will probably use much more of it. However, they keep delaying switching their big machine on, which means they are still waiting for the data. That is, unless when they finally flip the switch on the LHC, we all disappear into a black hole, tombs and all :)

It doesn't matter, they got published

lbbros — Tue, 22 Jan 2008 06:46:50 -0500

Does it matter that large quantities of this data will probably never be used?

Probably not, as their authors are already happy enough to have those published, and then they can ~~stop mantaining~~ move on to another ~~project~~ publication. It is indeed a sad state of affairs.

Swiss Prot Databases / Action Figures

Duncan — Mon, 21 Jan 2008 18:12:33 -0500

Hi Eric, yeah even more redundancy there. Talking of SWISS Prot people, I'm just wondering when your Amos Bairoch action figure will be available in shops?!?! What are you up to now that you've moved on from SWISS Prot to Seattle?

Why NAR

maximilianh — Sun, 20 Jan 2008 10:59:14 -0500

I add the usual defense of the NAR databases: At least they are indexed by pubmed, so as there are biologists that are not used to checking in google if there are websites about their subject, they will find them quicker via NAR. The data is not dead, websites can always be exported. (I've made the experience that for smaller databses it's usually quicker to scrape the data from an html page with something like HTTP::Recorder than writing to the person responsible for the data to send you an sql dump.) In addition, people get papers for their databases like this and other people can cite the database properly, so NAR makes the web citable and advances someone's career a little bit. It already eases the transition from a traditional paper-based science to a more web-orientated world. Databases are peer-reviewed, so they don't contain complete crap, a paper in NAR assures some minimal quality. And a write-only database is better than none at all, at least someone has collected something and you can scrape it from its tomb.

I would just love to see a minimal requirement for a publication in NAR: They should all offer some simple text-based export, e.g. tab-delimited flatfiles. That could save me a lot of time...

Of course, large quantities of these data are never used and never read. But, heck, this is research, right? 90% will not be used in the end. It's not too different from those 500 alignments algorithms, 200 genomic analyses, hundreds of papers that describe "new" cloning strategies, yet another new gene, etc. I don't believe that the physisists use 100% of their 1,5 GB / sec either.

Also, think of all the

ejain — Fri, 18 Jan 2008 20:12:16 -0500

Also, think of all the redundant effort that goes into setting up the basic technical infrastructure each database needs (data storage, user interfaces etc)...

You might try out MedWorm too

frankie — Mon, 19 Nov 2007 00:17:08 -0500

A nice introduction to Yahoo Pipes. You might also try out MedWorm which allows you to create RSS feeds from queries on its sources - which currently includes over 5000 medically related RSS feeds (sorry, I too am recommending my own site - which is where I came across this posting, since I like to keep a check on blog postings tagged with 'RSS' - I hope you may excuse this since the site is highly relevant to this posting).

Scintilla

alf — Fri, 16 Nov 2007 05:43:28 -0500

For the science side of things you could give Scintilla a try (sorry for plugging my own site, but it is designed for this kind of thing: if it doesn't suit the way you work then let me know).

that works too,,,

agbiotec — Thu, 15 Nov 2007 13:58:15 -0500

,,, but on a second thought, with the Pipes you can mashup and filter RSS from any source,,, so in addition to having keyword-filtered articles from papers, you can also have blog posts, newgroups and whatever else RSS content cleansed and delivered all in a single feed,,, I guess you could also do that by searching within Google reader after you aggregate your feeds there,,,

What about PubMed?

dbtodd — Thu, 15 Nov 2007 12:32:41 -0500

If your articles of interest are there of course (and a big if here). I have a couple saved searches in PubMed and subscribe to the feeds for those searches. One of these is a long list of scientists in my field for example. I subscribe using Google Reader so new papers come through every day and scanning through them in Reader is much easier than using PubMed directly. I then use a set of labels to prioritize articles for follow up and importing into the excellent pdf manager Papers.

Pipes are pretty nifty

pwnedd — Thu, 15 Nov 2007 10:33:28 -0500

Pipes are pretty nifty indeed. I created one for the same purpose myself a little while back for filtering items from a few different journals. I've since thought of an even easier way to search through a large number of journal feeds, but have not had the time to blog about it lately because I've been so caught-up in school work. Basically though, the idea is to combine google reader + yahoo pipes. Yahoo pipes is powerful, but it is a pain to handle a high number of feeds. It is very easy to add feeds to google reader however, so rather than adding the feeds directly to yahoo pipes, you could instead create a google reader account just for journal feeds. Give them all a tag like "article," and then feed your google reader "article" feed (which will include all items from all of the journals) into Yahoo pipes. This way you will only have to deal with a single feed in Yahoo pipes, and adding new journals becomes trivial.

Not bad, huh? :)
Keith

some points

maximilianh — Thu, 11 Oct 2007 13:01:13 -0400

Thanks Neil. I think there is no limit to the number of sequences you can download but a limit to the number of keywords you can search for. I did "id1 or id2 or id3 ..." and hit the limit there. There should be probably a better way to interface with NCBI but I don't know it.

Why 1 GB? Most EST-projects don't give their sequences genbank-compatible ids. They start with something homegrown, of course, then put the clones onto plates and distribute them. This is similar for mouse clones, zebrafish clones and also in my case ascidian cDNA clones. You see, a word in the description can be a real ID, if a sequence historically always had 2 different IDs. My EST sequences were just selected sequences, there is no way to filter from the sequence.

I think Badboys is right in that very often plain old Perl is more than enough. With hindsight, I could have hacked a python/perl script together in 20 minutes. It's so simple: Slurp the ids into a hash, iterate over the lines of the fasta and output everything that has a fasta id that contains a word in the hash. My problem was that I assumed that there was already a solution for this.

This happends to me all the time: I find programs on the internet that claim to do something and end up hacking something together 3 days later because the tools a) don't come with source code b) won't run at all c) do something slightly different which compromises the whole application for which I need them for.