nodalpoint.org - XML - Comments

User-friendly, High compatibility but low requirements

amejat — Tue, 06 May 2008 17:37:22 -0400

Hi Neil,

And thanks for your interest in the project. I fixed the mistake in "invaluable".

I also completely agree on the facts that:
- we need a free open source Laboratory Information Management System
- to be adopted, it needs to be user-friendly, cross-platform, highly compatible with existing open source solutions.

I had that in mind when I joined the eyeOS team, three years ago and I always kept it in mind during eyeOS development. And it is because eyeOS fits these needs that I really think it provides the perfect platform for a LIMS:
- For users, eyeOS provides a really intuitive and user-friendly desktop-like environment, accessible from anywhere.
- For developers, eyeOS provides a complete Toolkit and set of libraries to easily and quickly develop new eyeOS applications.
- For network manager, eyeOS has very low requirements. It only needs PHP and XML (javascript being provided by the navigator). It does NOT require a MySQL (or other) database or a Java server (such as Tomcat). For this reason, it can be installed on any basic hosting plan. However, it allows the easy integration of Flash (swf), Java applets or MySQL applications. Importantly, it constitutes a secured platform to store and share documents with a Virtual File System controlling the user access to files.

To conclude, the last but not the least, eyeOS is Free and Open Source. To my knowledge, it is unique.

I hope you will take time to discover this platform and agree with me on its potential.

Best regards,

Alexandre

www.eyeLIMS.com - Free Open Source Laboratory Information Management System powered by eyeOS

Value

Neil — Tue, 06 May 2008 02:55:35 -0400

provides an unvaluable system

*Invaluable*, surely. I hope?

I'll be interested to see how this develops: the research world is in desperate need of a quality, open-source LIMS. However, I'm concerned that this LIMS is tied to an existing product. Everyone has a web browser, but not everyone will want to sign up for eyeOS and learn how to use it. A true web application is something more like Google Documents, where you just open a page and start work.

Remember, biologists will look for any excuse not to learn how to use an application; you need to keep the entry barrier as low as possible.

Content

alf — Thu, 19 Apr 2007 05:34:45 -0400

If you can define more precisely the types of content you'll want to log, that would make it easier to point to useful plugins/modules.

use a blog to follow a project of bioinformatics

dalloliogm — Wed, 18 Apr 2007 09:18:02 -0400

I would like to do the same thing as you with notepress: use a blog to follow a project of bioinformatics.

I'm a last-year degree student and I have to follow a short period of practice of six-months in a research laboratory; the project I will work on is an analysis of the secundary structure of some genes in yeast.
Since I'm fond of Internet and the web 2.0 stuff, my head laboratory agreed to let me work slower on my project, allowing me to spend more time on deciding how to organize it and how to present it.
I've also had had to present a seminar a few weeks ago, you can find it here: http://genome.imim.es/~giovanni/ .

so I'm trying with a wordpress blog running on my local computer: I'm using it to collect all the resources on Internet, the workflows I'm using and to take note of the progresses I make.
Actually I'm not very good with the technical part, as I'm not sure I've understood very well what COinS is and I've not tried Drupal yet (I will study ;) ).

So I was looking for some interesting plugins for wordpress: in particular, I would like to find something to manage reviews on articles and resources on Internet, and maybe to implement a connotea/citeulike/other(zotero when it will have social bookmarking utilities?) account, but I believe I won't find very much :(

What kind of thing are you

alf — Tue, 17 Apr 2007 05:13:20 -0400

What kind of thing are you trying to do? There's a COinS plugin for Wordpress that does a little bit of this, but it's not proper structured blogging. You might be better off using Drupal and defining your own CCK content types.

Structure Blogging Plugin

dalloliogm — Mon, 16 Apr 2007 10:43:15 -0400

Hi,
I was trying out this plugin, but it seems that it doesn't work anymore, it is not compatible with wordpress 2.1 and the developers aren't going to continue it (http://mail.structuredblogging.org/mailman/private/structuredblogging-di...).

Moreover, the page referred for the other plugin Notepress, is not avaible any more http://projects.hubmed.org/track/notepress :(

---

http://genome.imim.es/~giovanni

LingPipe's MEDLINE XML parser + DB + search demo

Bob Carpenter — Mon, 09 Oct 2006 18:18:09 -0400

LingPipe (which can be downloaded with source from our site -- see links below) contains a parser for the 2006 MEDLINE distribution. It works through a document-object model, creating Java objects for MEDLINE citations. It covers the entire MEDLINE DTD. We'll update it as soon as the 2007 data replaces the 2006 data; the format changes slightly every year.

We also provide a program that will automatically download MEDLINE from NLM's FTP site. It checks the checksums and retries if necessary. It can be run as a scheduled job to get the daily updates, too; it only downloads what doesn't already match the checksums.

We have a tutorial on using the parser to create a Lucene (search engine) index from the data, and another tutorial on using the parser to create a MySQL DB. The latter includes complete from-scratch MySQL installation instructions.

In our tutorial on MEDLINE, we show how to load data from MEDLINE into MySQL using our parser. It's easily extendible, but if you're looking for something that off-the-shelf that puts all of MEDLINE data into a search engine or DB, this only cherry-picks the most useful information.

The DB tutorial also shows how to run some of our natural language processing tools to do things like break the text into sentences and find references to entities of various kinds (e.g. protein subdomains, malignancies, variations such as SNPs or mutations, organisms, cell lines, etc.)

Here are the links:

LingPipe Home Page (Description and download links)

LingPipe Tutorial: MEDLINE Parsing and Indexing (Download MEDLINE, parse MEDLINE and index MEDLINE in Lucene)

LingPipe DB Text Mining Tutorial (Parse MEDLINE, load MEDLINE into MySQL and run natural language text mining tools over MEDLINE)

This is quite late reply

smarko — Fri, 14 Jul 2006 08:31:07 -0400

This is quite late reply from me, but I wanted to add that I wasn't referring to Sedna with that "Usual weaknessies" -list, but to some others yes. I've never had out of memory problems with Sedna, for example.

XML database performance

Maxim — Mon, 19 Jun 2006 07:37:33 -0400

Hi, I'm a Sedna developer.

Marko, thank you for reporting on Sedna!

Theoretically, Sedna can store data sets of unlimited size. Practically, we have tested it successfully with 150 Gb of XML data as a single document (!) and as collections of documents.

As concerns speed of query execution, it should be ok for 50+ Gb if you configure indexes properly. Sedna is not too fast for data loading but it usually works very good for querying. It would be interesting to find out why Marko found Sedna awfully slow for indexing and querying.

Errors like "out of memory" might happen but they are bugs (not fundamental Sedna limitations) which we will fix if you contact us. We are always avaliable for support, questions and bugfixes.

XML database performance is

FiReaNG3L — Fri, 16 Jun 2006 14:13:31 -0400

XML database performance is very meh I think, and I need split second fast queries. Right now I'm having fun with PHP::SimpleXML. It loads XML as an object in memory and then you can do fun things like foreach ($xml->MedlineCitation as $citation) { do stuff here }. Works wonderfully. 30000 records inserts in mySQL in 25 seconds... so it should take 5-10 hours loading the entire thing in. And then maybe a day to index it?

Sedna, a native XML database

smarko — Thu, 15 Jun 2006 17:07:01 -0400

I guess I could add my experiences with Medline database here, too. My website's (ebounce.info) PubMedTool -tool is currently using E-utilities offered by NCBI for querying and fetching data, but offline I've been making tests with native XML databases.

Most of the native XML databases practically choke, because of the huge amount of data. Sedna is one of those, which I found to handle things nicely. I mean, if one doesn't require immediate results and have time to wait for few minutes.

Usual weaknessies, which become clear in first 5 minutes:
- out of memory errors
- not possible to join databases or query multiple databases at the same time
- (awful) slowness of indexing of xml-files and actual querying
- ..

So far, I've mostly been using Sedna for playing with XQueries. However, Sedna doesn't support all the XQuery-functions yet, but so far it hasn't been a big problem. Also, while learning more about Sedna, I've started to miss different kind of indexing methods, XSLT support and few other things. It does have a API for few languages (C, Java, Python..).

Sedna is a good choice, if you don't need to query Medline in full, but maybe just last 10 years or so. Otherwise you'd get bored of waiting. And for querying smaller databases like MeSH (240Mb), it is a really good choice.

I've heard that Berkeley DB XML might also be worth a try, so I'll try it next. Again. Earlier I had some problems with it (I've already forgot, what it was).

PyLucene

alf — Sun, 11 Jun 2006 20:21:32 -0400

I have a Lucene index of the MEDLINE data, created with PyLucene - it's what HubMed uses for relevance-ranked searches. If you still want some help with the indexing let me know.

XML looks easy

FiReaNG3L — Fri, 09 Jun 2006 22:38:06 -0400

It's for my next web project (as biologynews.net is kinda finished, for now). I'm building something useful (I hope) for the biologists community. Still in the preliminary stages, but big databases don't scare me :) It could take time to build as I'm still busy with my PhD (wetlab and all, microarrays and HIV, tons of fun) and can only work by night / weekends on it. My goal is to have something functional by the end of the year.

Mapping XML2RDB

Neil — Fri, 09 Jun 2006 08:32:56 -0400

There are a few Perl tools around for parsing XML and mapping it to a RDB. XML.com is a good place to start.

Are you sure that you really want to do this? 50 GB of Medline XML sounds like a living hell to me. I assume this is some sort of text mining effort?

about the file size

lindenb — Wed, 07 Jun 2006 11:08:39 -0400

Oh, Sorry, I thought the post was about chunks of pubmed articles (such as the one you can fetch via EUtils).

I remember one day I inserted some data from dbSNP in a database. This time I used a lex/yacc (the grammar was trivial) to parse the XML and to quickly insert the records.

Pierre