nodalpoint.org - Lincoln Stein - Comments

-10 Ooops

Duncan — Fri, 10 Aug 2007 05:05:19 -0400

Ooops, my semantic web agent did the sums wrong (bad ontology) and made Jim aged 89 not 79....corrected it now!

+10 ?

radmap — Thu, 09 Aug 2007 23:30:12 -0400

Man, that sequence must have been pretty scary ... seeing his own genome sequence instantly aged Lucky Jim an extra 10 years! ;-)

Databases in Peril

Duncan — Fri, 12 Jan 2007 10:48:39 -0500

Thanks for all your comments, here are some thoughts...

Quantity is a significant problem we're not just talking about individual databases getting bigger and bigger like GenBank, we're talking about more different types of databases. Potentially we want to allow the combination of data from any of these different databases and others that will appear in the future. Obviously, any given researcher probably isn't going to want to search all 900+ databases, but it would be beneficial to the wider scientific community if all these databases can easily interoperate. The more databases there are, the more challenging easy interoperation becomes, because there is more heterogeneity, more API's, more schemas etc.

Peer-reviewed publication can help assess quality this is what peer-review is for. The editors of this issue claim to look for good quality data as well as a good quality interface. As pointed out in the comments above “anyone with a modicum of knowledge can put a database or web app online”. By itself, this is not enough for publication. It is no good having great data with an awkward non-standard interface and vice versa. The NAR database issue may well be an “easy” publication, but it doesn't make it any less important. The Databases in Peril article, wouldn't have been possible if NAR hadn't been faithfully recording all this information in the first place. I suspect publication in the NAR database issue is harder than some suggested, it's not just a case of shoving a database on the web then writing a paper about it, you have to convince the reviewers the database is worthy: novel, useful and usable.

Churn is inevitable but the overall trend is still upward Databases (and tools) are not immortal, some are bound to wither and die eventually. Since last year 11 databases have gone this way, and the article, discusses why. The general trend is still upward and will probably keep going. In the long run, the longevity of database can be an indicator of its quality because somebody cares and is skilled enough to maintain and fund it for a long period of time. As for the databases that are “struggling financially” (according to Nature) how is this news? Struggling could mean anything. Haven't you always had to fight for sustained funding of any scientific project?

Standards are boring (but important) it can be difficult to get standards work funded, done and published, what John Quackenbush calls Blue-collar science. It is unglamorous but essential work, and nobody is going to win a nobel prize for creating a standard schema, ontology or whatever. What is the research contribution of creating a standard? Novelty? Discovery of new knowledge? This is partly why we have chaos, creating standards, in itself is often not considered “science” or “research”. But without them, science is much harder.

Integrated Search is hard We would all like integrated search “from one box”, but the way to do this is still very much an open research question, not just in bioinformatics, but for computer science also. What is more, this is not merely an “IT problem”, there are novel and serious scientific challenges in achieving this. If it was easy and straight forward to provide integrated search to all these databases, don't you think somebody would have done it by now? Until that time, we have Entrez Global Query...

Quantity not the problem

Neil — Sun, 07 Jan 2007 08:10:00 -0500

Not drowning. More data are good - the more, the better. Only a few of those databases are relevant to an individual researcher.

As others have already commented, the problems are (1) the quality of the databases, (2) their diverse, "higgledy-piggledy" nature (no standards, APIs, integration) and (3) their longevity, or lack thereof. Frankly, anyone with a modicum of SQL and CGI knowledge can put a database or web app online. So they do. You can't legislate against bad web resources.

I would question whether these annual issues still serve any useful purpose, other than to make the journal appear authoritative or provide an avenue for an easy publication. If I'm looking for an online resource I start with Google, not an outdated journal article.

Need standards

Animesh — Sun, 07 Jan 2007 05:02:36 -0500

Great article Duncan, thanks for bringing this on.
I have seen lot of people accessing low quality data from many well-known db's and high quality data in not-so-well-known ones.
For eg, GBK files mostly does not talk about quality while it's ASN.1 counterpart might offer it [ http://getentry.ddbj.nig.ac.jp/cgi-bin/get_entry.pl?AF207953 ].
Regarding algorithms to analyze these data, any comment will be like a troll.
To minimize this, I feel, something like Bioinformatics oriented DIGG will be great.

Vaporware

lindenb — Fri, 05 Jan 2007 16:23:41 -0500

Each time a new annual issue of NAR is published I remember this paper from Nature.

http://www.nature.com/nature/journal/v435/n7045/full/4351010a.html

Databases in peril
Zeeya Merali and Jim Giles
Nature 435, 1010-1011 (23 June 2005)
doi: 10.1038/4351010a

Nature contacted 89 databases listed in the Molecular Biology Database Collection (Nucl. Acids Res.28 1−7; 2000) to see how many still have funding five years on. Of these, 51 reported that they are struggling financially. Seven of these have closed; the rest are being updated sporadically in their owners' spare time.

Pierre

Drowning!!!

mndoci — Fri, 05 Jan 2007 16:12:25 -0500

I think that we're going to drown at this rate. Not because there are too many databases. Those can, and perhaps should be spread far and wide. My concerns

Quality. How do we know whether the results in our hands are any good? Can we glean meaningful knowledge from them?
Integrated search. I don't want to go to every database and search there. I want to search from one box
Standards. I want the data to follow certain minimum standards.

What was that about airlines :-)?

My Blog: http://mndoci.com

agreed

Neil — Mon, 06 Nov 2006 20:47:06 -0500

IMHO, screen-scraping this kind of data is a mugs game

I'm with you there. Scraping seems to be the basis of Zotero, which everyone is talking about just now. Broken scraping seems to be the reason for its current inability to import from PubMed/HubMed.

Having said that, I just wrote a scraper to pull hundreds of fasta files from an online genome database. It was a one-off though, honest.

Screen Scraping Hell

Duncan — Mon, 06 Nov 2006 11:57:45 -0500

IMHO, screen-scraping this kind of data is a mugs game, or as Lincoln Stein once put it: mediaeval torture, best left as a task for the sado-masochists, who actually like having to repeatedly rewrite their code!

Rise of BMC

Neil — Sat, 04 Nov 2006 04:09:29 -0500

...young upstart BioMed Central Bioinformatics...

I like all of the BMC Journals and on the whole, rate BMC Bioinformatics over Bioinformatics. The latter journal has become almost solely a methods/algorithms journal, whereas the former has a lot more interest in terms of application to real biological problems.

And yet - look at the problems that the BMC journals had initially in getting established - all because noone could figure out how impact factors applied to free, online articles! The nonsense that is impact factors, once again.

Digestability

chris — Fri, 03 Nov 2006 15:02:14 -0500

Probably because a single, simple metric has intuitive appeal. Eugene Garfield, the guy behind IFs, has commented on their use and abuse in JAMA and on the Thomson website (pdf). There's quite a large literature critical of whole concept, too.

What a mess

Duncan — Thu, 02 Nov 2006 05:58:49 -0500

Yeah, impact factors are always horribly out of date, are difficult to get hold of, often misleading and in the hands of a for-profit organisation that is responsible primarily to its shareholders. How did research assessment become so overly dependent on such figures?

Good work

Neil — Wed, 01 Nov 2006 21:58:16 -0500

Everything about impact factors is ridiculous - the difficulty in obtaining what should be freely-available and up to date data is perhaps the most ridiculous aspect. Thanks for the list.

Given that many journal webpages now display their impact factor, I wonder how many could be obtained using scrapers?