nodalpoint.org - semantic web - Comments

"... are not part of the

Greg — Wed, 08 Aug 2007 02:18:26 -0400

"... are not part of the semantic web because you cannot HTTP GET them." Not quite, because you can't HTTP GET metadata about them, or URLs for that matter. As I said, I'm thinking from the perspective of a semantic web crawler/agent. I mean, what is the semantic web, if not the web as we know it but for machines ? In other words, imagine what an autonomous agent would need to navigate and browser the web rather than a human... let's keep going then shall we.

My requirements are:

A distinction between data and metadata.
A universally accepted, scallable, protcol for agents to access both data and metadata.
Data and metadata must be identifiable.

Two is probably inclusive of one, but we can refine these as we go.

At the moment the world wide web provides the identifier system (HTTP URIs, which includes naming, resolution and locating), and the protocol for accessing resources identified by URIs: HTTP. What we don't have is a universally accepted, scalable solution to accessing metadata about resources on the web. HTTP does not address this problem, as far as I know.

Of course you can layer any number of solutions on top of HTTP that will achieve roughly that goal (i.e. metadata about a resource). For example, standardize on an HTML linking scheme to point agents to metadata about the resource, transform HTML to RDF (GRDDL), or embed RDF/XML directly in the HTML. However, these solutions do not satisfy the scalability requirement. An agent will need to download each resource it wants to know about *before* it can know about it and it will have to know about each of these mechanisms. Second it does not satisfy the universally accepted requirement, it is likely that different organizations will adopt different standards for linking and/or embedding metadata in a webpage.

Alternatively you can just invent a whole new system for identification, location and resolution that includes a getMetaData facility. This would be LSIDs. Again, LSIDs fail the universally accepted requirement. It is now that I can say that I don't have any real problems with LSIDs themselves or any of the solutions I mentioned previously, it is just that I think the likelihood of adoption is low. This is also true of my preferred solution MGET (or URIQA). Without the W3C getting behind at least one of them and that solution also integrating easily into the existing web infrastructure, then yes, the whole proposition is a non-starter. Sure there will be isolated pockets of activity, but it will never take-off in the same way the HTTP web did.

At this point content negotiation, or embedding the metadata URL in a HEAD request are often touted as a solutions. Content negotiation doesn't deal with the metadata/data issue, only different representations of the data and HEAD requests are not scalable. For example, using content negotiation I may be able to have my agent accept application/rdf+xml, but it is doing this blindly. There is no way for it to know what it will get, other than its format will be RDF-XML. And even if both the server and the agent implement content negotiation, we are still back to figuring out what the resource is based on downloading the resource, rather than downloading authoritative metadata about the resource. Furthermore, some resources just don't have a logical RDF representations e.g. images, concepts etc. However metadata about those concepts are logically represented in RDF using descriptive ontologies.

To get around the "how do you know about a resource before you download it" problem, you can posit centralized google-like resources that can be queried (using SPARQL I guess) by agents for information about resources. It might work, but again, scalability, universally accepted. Also, how does a centralized database help me if I am the authority (owner of the domain identifying that resource) ? I want to provide my own metadata, not what google thinks it is about.

Thus, while HTTP is not the only game in town (think gopher, or ftp) when it comes to building a web of interlinked resources, it is the one most universally accepted. HTTP URIs are also not the only game in town for naming and locating resources (think LSIDs), but they are universally accepted. So HTTP is the web, HTTP URLs are the web, and if this is true, then HTTP need to be updated to facilitate access to metadata about resources by agents if it is to also be the semantic web.

I just can't GET enough

Duncan — Thu, 26 Jul 2007 05:25:06 -0400

OK, let's recap. If I've understood your arguments correctly, they are something like this:

On the semantic web, you think all URIs should be URLs that you can HTTP GET. Names, like URNs and ISBNs, in your opinion, are NOT part of the semantic web because you can't HTTP GET them.
Without an extra GetMetadata method added to HTTP, this whole idea of the semantic web is a complete non-starter.

The second proposition is interesting, which is why I just can't get enough, long after everyone else has lost interest in this thread :) Why can't the semantic web work with HTTP as it is? The web works OK with HTTP as it is so why can't the semantic web do the same? The current web isn't perfect, but it does work. People can still build and share ontologies on the web without changing HTTP. Reasoners (or "agents" if you prefer), are a key part of the semantic web, and they work perfectly well with HTTP as it is. So why do you think this shortcoming of HTTP is a show-stopper for the whole semantic web? Sorry if I'm banging on and on, but I find this discussion intriguing! I can see why GetMetadata is desirable, but I can't see why you think it's an absolute pre-requisite.

Is this ontology NOT part of

Greg — Wed, 25 Jul 2007 08:24:40 -0400

Is this ontology NOT part of the web because we can't GET all the URIs it uses?

As you suggest, there are two issues here: the resources themselves being gettable and metadata about the resources being gettable. In the case of resources that have no logical representation on the web (i.e. the Danube river, or p53), then no, they don't need to be gettable (but the metadata describing them does). In the case of an ontology description in OWL, yes they do. The side debate of hash vs. slash is important but not the main issue here.

A semantic web crawler needs to have a mechanism available to it, as a fundamental part of the web infrastructure, to get metdata about a resource, whether it be informational or non-informational (i.e. a concept with no logical representation or an HTML page). Since this does not exist, there is very little hope that the vision of the semantic web as being distributed data integration on the web will never be realized.

Slap me if you've heard this one before

Duncan — Wed, 25 Jul 2007 05:16:52 -0400

My prize should probably be a slap, for bastardising great works of english literature and music :)

Brief aside...

stewc — Tue, 24 Jul 2007 15:48:28 -0400

Duncan deserves some sort of punning prize, I feel.

Either than or a slap. Haven't decided. :)

You can't always GET what you want

Duncan — Tue, 24 Jul 2007 07:30:07 -0400

Lets use another example besides iSBN. The pizza ontology [1] uses several URIs, some of them you can GET, like the one which has some description and versioning information at http://www.co-ode.org/ontologies/pizza/2007/02/12/ and another identifying the OWL serialisation of the ontology at http://www.co-ode.org/ontologies/pizza/2007/02/12/pizza.owl

But it also uses these URIs which you can't GET http://www.co-ode.org/ontologies/pizza/pizza.owl, http://www.co-ode.org/ontologies/pizza/pizza.owl#SpicyPizza and http://www.co-ode.org/ontologies/pizza/pizza.owl#hasSpicines

Why should all these URIs be http gettable? Is this ontology NOT part of the web because we can't GET all the URIs it uses? I don't think so, because if you try sometimes you might find you GET all the URIs you need [2]. As for getting the metadata associated with these URIs, thats a seperate issue, I agree with you Greg. It'd be nice to have, but it might take a long time.

Alan Rector, Nick Drummond, Matthew Horridge, Jeremy Rogers, Holger Knublauch, Robert Stevens, Hai Wang and Chris Wroe (2004) OWL Pizzas: Practical Experience of Teaching OWL-DL: Common Errors and Common Patterns EKAW
Mick Jagger and Keith Richards (1968) You can't always (HTTP) GET what you want

What is bothering for this

Greg — Mon, 23 Jul 2007 19:25:20 -0400

What is bothering for this kind of application, however, is when people generate their own URIs for third-party resources, even if they already have a perfectly good, official URI (unfortunately opinions differ on what is "perfectly good", it appears).

I can't see a distributed system not having this property i.e. two people identifying the same resource (concept) with two different URLs. Inverse functional properties, owl:sameAs go some way to solving this, but the problem won't go away and is something the semantic web needs to live with. In communities that have better organization (i.e. the life sciences community) there may be some hope of reaching agreements on the, but on the web at large it will be hard to stop, I agree.

The idea of link rel="alternate" doesn't quite work for me, also content negotiation. I take both to mean getting an alternate representation of the resource being identified. What you need is a mechanism to get metadata about a resource before you download the resource itself. Otherwise you have a solution that just does not scale. If a semantic web crawler has to download every resource it is interested in and then figure out what it is, a lot of redundant fetches will be made. If there is a mechanism in place for getting metadata about a resource, then resolving non-informational resources will never occur. As the semantic web agent will first request metadata about the resource, learn that it doesn't have a representation on the web, and move on.

There are times when you'll

Greg — Mon, 23 Jul 2007 19:05:02 -0400

There are times when you'll want to HTTP GET them and times when you don't.

Unless we are talking about something other than the semantic web then no, you always want to use HTTP GET, because if you're not using HTTP GET then we are not talking about the web. And we don't want to just GET the resources, we want to get metadata about the resources. A semantic web crawler cannot function without this mechanism in place. Now Eric Jain is correct, if you're only using URNs in the context of a custom data integration system independent of the web then go-for-your-life, invent as many naming and resolving schemes as you like. But in my opinion, the semantic web, if it ever does exist, will be part of the web. That is, the web as we all naturally understand it: the HTTP gettable web. And therefore we need a mechanism baked into the web for getting metadata about a resource.

The problem with URNs having their own schemes for turning them into a location is that, as you suggest, there are a million ways to do it. I am not interested in coding a semantic web crawler that needs to know specific conventions for turning every URN it encounters into a location and then figuring out a separate mechnism to find out what exactly is being identified.

The semantic web is all about the web. Thus the web must have metadata a first class citizen, end of story. Yes, you are completely correct, implementing a standard getMetadata method across the web will be "challenging", but this is the mission the W3C is there to pursue.

I mean, if it is not the web, it is just not cricket !

To GET or not to GET. That's NOT the question...

Duncan — Mon, 23 Jul 2007 12:35:51 -0400

If I've understood this correctly, the Dubya3C have deliberately used the more vague URI rather than URL in their specifications, because the web uses both URLs and URNs. There are times when you'll want to HTTP GET them and times when you don't.

If you just want to uniquely identify something, URNs are fine. As a colleague of mine once put it, "its just a f**king string".

For example the Uniform Resource Name URN:ISBN:0805080430 uniquely identifies a book by David Weinberger without any need for HTTP at all thank you very much.

If we want to identify a thing describing David's book the we can transform the URN into a URI http://www.amazon.co.uk/exec/obidos/ASIN/0805080430 or in a millon other ways. Why should a scheme for uniquely identifying things be so tightly coupled to HTTP (or anything else for that matter)?

Where your URIs are URLs that you can GET, I agree, it'd be handy to have a standard getMetadata method. But adding extra methods to HTTP seems like adding extra letters to the alphabet, it might have its uses, but its going to be "challenging" to implement successfully :)

Don't know about which is

ejain — Mon, 23 Jul 2007 09:40:51 -0400

Don't know about which is the "right one", but "gettable" URLs seem to be the more simple and practical solution, for now.

HTTP gettable (or mgettable

Greg — Mon, 23 Jul 2007 09:39:23 -0400

HTTP gettable (or mgettable rather) is what they should be. But HTTP does not feature in that stack, and this is what annoys me about the whole issue. The semantic web mandarins have been intentionally vague about the implementation details. I get the feeling that they have gone for an abstract description of the stack on purpose, hoping that the correct implementation would follow i.e. URLs vs. URNs. Even if URLs win, which they seem to be doing, there is still the issue of accessing metadata describing the resource being identified by the URL. This is an implementation detail left out of the stack. So maybe your right, it will end in tiers, just more of them :)

Being able to resolve

ejain — Mon, 23 Jul 2007 09:30:06 -0400

Being able to resolve identifiers may be important for some applications (e.g. a Semantic Web crawler), but for others (e.g. a data integration system) I think it's often irrelevant. What is bothering for this kind of application, however, is when people generate their own URIs for third-party resources, even if they already have a perfectly good, official URI (unfortunately opinions differ on what is "perfectly good", it appears). Would be less of an issue if mappings (e.g. in the form of owl:sameAs statements) where provided. In fact, the ability to do so is what can make the Semantic Web work even when people can't agree on much (to be expected).

Note that in principal there already is a "standard" way to link to different representations: Just return a Web page by default, and include some link rel="alternate" type="..." elements in the header. And then there is also content negotiation... Something more easily and efficiently machine readable would be welcome, but shouldn't hold things back!

The semantic web: will it all end in tiers?

Duncan — Mon, 23 Jul 2007 05:39:02 -0400

You'll notice most semantic web software stacks (like the one below) have URI in them (not URL or URN)

The likes of LSID and URIQA don't feature in these stacks at the moment, so we've got two interpretations of the semantic web: One where every URI is an HTTP gettable URL another where URIs are URNs. Which is the "right" one?

Freebase invitations

PedroBeltrao — Sun, 01 Jul 2007 14:51:40 -0400

I also have some invitations to send way (pedrobeltrao /in/ gmail).