nodalpoint.org - NAR - Comments

Yeah, need to update my

ejain — Tue, 22 Jan 2008 19:47:42 -0500

Yeah, need to update my blog...

700?

nuin — Tue, 22 Jan 2008 13:18:11 -0500

At 700 a pop for an Action Figure?

Dude, Where's My NAR?

Duncan — Tue, 22 Jan 2008 13:09:27 -0500

Hello Maximillian, I wasn't attacking NAR, was just wondering about what proportion of the data is dead. It is an interesting technical challenge to find this out. It would also be useful to measure the cost of gathering noisy, redundant and poorly understood data in terms of wasted resources (people, time, money, computers, false positives etc). Perhaps somebody has done something like this already? Especially with all the irresponsible sequencing just for sake of it that goes on...

As for the physicists, I mentioned them for comparison. Like you, I doubt they will use 100% of their data either, but will probably use much more of it. However, they keep delaying switching their big machine on, which means they are still waiting for the data. That is, unless when they finally flip the switch on the LHC, we all disappear into a black hole, tombs and all :)

It doesn't matter, they got published

lbbros — Tue, 22 Jan 2008 06:46:50 -0500

Does it matter that large quantities of this data will probably never be used?

Probably not, as their authors are already happy enough to have those published, and then they can ~~stop mantaining~~ move on to another ~~project~~ publication. It is indeed a sad state of affairs.

Swiss Prot Databases / Action Figures

Duncan — Mon, 21 Jan 2008 18:12:33 -0500

Hi Eric, yeah even more redundancy there. Talking of SWISS Prot people, I'm just wondering when your Amos Bairoch action figure will be available in shops?!?! What are you up to now that you've moved on from SWISS Prot to Seattle?

Why NAR

maximilianh — Sun, 20 Jan 2008 10:59:14 -0500

I add the usual defense of the NAR databases: At least they are indexed by pubmed, so as there are biologists that are not used to checking in google if there are websites about their subject, they will find them quicker via NAR. The data is not dead, websites can always be exported. (I've made the experience that for smaller databses it's usually quicker to scrape the data from an html page with something like HTTP::Recorder than writing to the person responsible for the data to send you an sql dump.) In addition, people get papers for their databases like this and other people can cite the database properly, so NAR makes the web citable and advances someone's career a little bit. It already eases the transition from a traditional paper-based science to a more web-orientated world. Databases are peer-reviewed, so they don't contain complete crap, a paper in NAR assures some minimal quality. And a write-only database is better than none at all, at least someone has collected something and you can scrape it from its tomb.

I would just love to see a minimal requirement for a publication in NAR: They should all offer some simple text-based export, e.g. tab-delimited flatfiles. That could save me a lot of time...

Of course, large quantities of these data are never used and never read. But, heck, this is research, right? 90% will not be used in the end. It's not too different from those 500 alignments algorithms, 200 genomic analyses, hundreds of papers that describe "new" cloning strategies, yet another new gene, etc. I don't believe that the physisists use 100% of their 1,5 GB / sec either.

Also, think of all the

ejain — Fri, 18 Jan 2008 20:12:16 -0500

Also, think of all the redundant effort that goes into setting up the basic technical infrastructure each database needs (data storage, user interfaces etc)...

Databases in Peril

Duncan — Fri, 12 Jan 2007 10:48:39 -0500

Thanks for all your comments, here are some thoughts...

Quantity is a significant problem we're not just talking about individual databases getting bigger and bigger like GenBank, we're talking about more different types of databases. Potentially we want to allow the combination of data from any of these different databases and others that will appear in the future. Obviously, any given researcher probably isn't going to want to search all 900+ databases, but it would be beneficial to the wider scientific community if all these databases can easily interoperate. The more databases there are, the more challenging easy interoperation becomes, because there is more heterogeneity, more API's, more schemas etc.

Peer-reviewed publication can help assess quality this is what peer-review is for. The editors of this issue claim to look for good quality data as well as a good quality interface. As pointed out in the comments above “anyone with a modicum of knowledge can put a database or web app online”. By itself, this is not enough for publication. It is no good having great data with an awkward non-standard interface and vice versa. The NAR database issue may well be an “easy” publication, but it doesn't make it any less important. The Databases in Peril article, wouldn't have been possible if NAR hadn't been faithfully recording all this information in the first place. I suspect publication in the NAR database issue is harder than some suggested, it's not just a case of shoving a database on the web then writing a paper about it, you have to convince the reviewers the database is worthy: novel, useful and usable.

Churn is inevitable but the overall trend is still upward Databases (and tools) are not immortal, some are bound to wither and die eventually. Since last year 11 databases have gone this way, and the article, discusses why. The general trend is still upward and will probably keep going. In the long run, the longevity of database can be an indicator of its quality because somebody cares and is skilled enough to maintain and fund it for a long period of time. As for the databases that are “struggling financially” (according to Nature) how is this news? Struggling could mean anything. Haven't you always had to fight for sustained funding of any scientific project?

Standards are boring (but important) it can be difficult to get standards work funded, done and published, what John Quackenbush calls Blue-collar science. It is unglamorous but essential work, and nobody is going to win a nobel prize for creating a standard schema, ontology or whatever. What is the research contribution of creating a standard? Novelty? Discovery of new knowledge? This is partly why we have chaos, creating standards, in itself is often not considered “science” or “research”. But without them, science is much harder.

Integrated Search is hard We would all like integrated search “from one box”, but the way to do this is still very much an open research question, not just in bioinformatics, but for computer science also. What is more, this is not merely an “IT problem”, there are novel and serious scientific challenges in achieving this. If it was easy and straight forward to provide integrated search to all these databases, don't you think somebody would have done it by now? Until that time, we have Entrez Global Query...

Quantity not the problem

Neil — Sun, 07 Jan 2007 08:10:00 -0500

Not drowning. More data are good - the more, the better. Only a few of those databases are relevant to an individual researcher.

As others have already commented, the problems are (1) the quality of the databases, (2) their diverse, "higgledy-piggledy" nature (no standards, APIs, integration) and (3) their longevity, or lack thereof. Frankly, anyone with a modicum of SQL and CGI knowledge can put a database or web app online. So they do. You can't legislate against bad web resources.

I would question whether these annual issues still serve any useful purpose, other than to make the journal appear authoritative or provide an avenue for an easy publication. If I'm looking for an online resource I start with Google, not an outdated journal article.

Need standards

Animesh — Sun, 07 Jan 2007 05:02:36 -0500

Great article Duncan, thanks for bringing this on.
I have seen lot of people accessing low quality data from many well-known db's and high quality data in not-so-well-known ones.
For eg, GBK files mostly does not talk about quality while it's ASN.1 counterpart might offer it [ http://getentry.ddbj.nig.ac.jp/cgi-bin/get_entry.pl?AF207953 ].
Regarding algorithms to analyze these data, any comment will be like a troll.
To minimize this, I feel, something like Bioinformatics oriented DIGG will be great.

Vaporware

lindenb — Fri, 05 Jan 2007 16:23:41 -0500

Each time a new annual issue of NAR is published I remember this paper from Nature.

http://www.nature.com/nature/journal/v435/n7045/full/4351010a.html

Databases in peril
Zeeya Merali and Jim Giles
Nature 435, 1010-1011 (23 June 2005)
doi: 10.1038/4351010a

Nature contacted 89 databases listed in the Molecular Biology Database Collection (Nucl. Acids Res.28 1−7; 2000) to see how many still have funding five years on. Of these, 51 reported that they are struggling financially. Seven of these have closed; the rest are being updated sporadically in their owners' spare time.

Pierre

Drowning!!!

mndoci — Fri, 05 Jan 2007 16:12:25 -0500

I think that we're going to drown at this rate. Not because there are too many databases. Those can, and perhaps should be spread far and wide. My concerns

Quality. How do we know whether the results in our hands are any good? Can we glean meaningful knowledge from them?
Integrated search. I don't want to go to every database and search there. I want to search from one box
Standards. I want the data to follow certain minimum standards.

What was that about airlines :-)?

My Blog: http://mndoci.com