I realised that I wasn’t motivated enough to keep writing here, partly because not enough people are watching… which in turn is partly because the writing is all too dry and professional. So I started another blog, http://barefootliam.blogspot.ca/, where I feel more comfortable writing about a wider range of topics.
I spent some time recently thinking about RDF and the Semantic Web, and about ontologies, and about XML and topic maps and XML schema languages.
The Semantic Web people are doing something (or trying to do something) more global than I recall from the original RDF meetings; they are trying to make a global, shared metadata space. A space where, it sometimes seems, there is neither privacy nor attribution, where anyone can say anything and no-one knows who to believe. That’s more than a little inaccurate, but it seems that way.
A consequence of trying to make a global space is that there are certain kinds of thing you can’t say. Just as in the real, physical world I can’t say that there are no green-headed aliens, or that there are no people with feet instead of hands, since no exhaustive search is possible, so there are things one cannot say in this open-ended semantic world.
Unfortunately, they often seem to be things I want to say, albeit with smaller scope. I don’t want to say there are no colour photographs of Stokesay castle anywhere in the world, but I know that I don’t have any on my own Web site. I can say, I have some black-and-white pictures of the castle. (Digital purists want me to say that the images are monochrome, but people were talking about black-and-white photography long before computers were invented, and the words continue to work very nicely, thank you.)
In the XML world, we generally have somewhat lower sights. Rather than trying to make single vocabularies that work for everyone, everywhere, always, we give people the tools to build their own vocabularies. There are advantages and disadvantages to this. It is expensive, but the result, like a good suit, is a good fit and looks it. It isn’t an off-the-peg one-size-fits-all suit. (don’t forget to buy some classy socks to go with it).
In other words, we in the XML world tend to live on islands, linked by bridges. Or possibly linked by ferries, or sometimes you just swim across.
The biggest peril of this civilisation of heterogeneous annotation is that it is hard for anyone to search across multiple vocabularies. One group of people distinguishes between socks and tights, and another just records items of footwear. Another group records the fabric and size of each item, but not the colour. People record the information they value, or, sometimes, the information that they can imagine other people valuing as well as themselves.
But in an island of metadata, an isolated community, I can certainly make the claim, there are no colour pictures of Stokesay Castle. That’s useful to me, and I hereby make it.
Someone who searches my metadata, one might imagine, has the sense to understand that if I say there are no colour pictures of Stokesay Castle, I do not mean that there are no such things in the entire universe (or metaverse), but that I don’t have any such pictures. You might think that I could say, N is a whole number, and N is the number of such pictures I have, and N has the value zero. On my own island I can indeed say that, because I have is implied by the shores of my island. The implicit qualification on the Web is anyone might have, or have had, or will have, or can imagine.
So, in the XML world, we are an archipelago of weakly interconnected islands. This is not a bad thing, especially if you can swim. We need to make sure we have powerful tools for translating between our various vocabularies when that makes sense, and for searching across them. These are our bridges, our ferries, our ‘planes and water-paddles.
We build our bridges with XML interchange, and we paint and repair them with documentation and training, with education and outreach. We cross the bridges—that is, we translate between vocabularies, and search—with XSLT and XQuery, with SQL/X and other interchange apecifications. Those bridges must, in most cases, be built by hand. If your ontology has fortified buildings but does not distinguish between forts and castles, my search for a castle (a fortified habitation) that is not a fort (a fortified garrison) will be meaningless. But guess what? No-one else has solved that problem either.
So I live on my island in my archipelago, along with many other people, and sometimes we visit other islands for a time. And the world seems good. Yes, it’s hard to cross to those other islands, but no harder for us than anyone else. Easier, sometimes, because we’ve got pretty good bridges.
I was recently asked, how widespread is the use of XML? I had to stop and think. It’s almost as if someone asked me how widely used is air, or perhaps more fairly how many socks there are in the world.
So here are some places where XML is used, and I’d like to invite people to leave comments suggesting other places where XML is used. You have to register to post a comment, to keep out spam (and they are moderated), but I promise not to do anything bad with your email address.
So, where is XML used? The first use case we had was for technical documentation, for aircraft manuals, telephone switch manuals, car manuals and a host of other documentation that was already using SGML. The manual in the glove compartment of your new GM car was produced using SGML or XML.
Speaking of cars, chances are high (50% or more) in North America that if you fill your car with petroleum, the messages from the pump to the cash register in the filling station’s office are sent using XML, validated with XML Schema too. And the diagnostic computer on board your car probably talks in XML to the garage mechanic. I have been unable to verify a rumour that XML messages are passed around between components when you put your foot on the brake pedal, but maybe the danger of sharp pointy brackets falling out of the pedals would discourage barefoot driving?
When you get home perhaps there is a bank statement waiting for you, and perhaps, like millions of others, it was made using XML and XSL-FO; the Financial industry is a heavy XML user, as is the Publishing industry.
Your mobile telephone may well be sending and receiving messages in XML more often than you realise; this is one of the motivations for the W3C Efficient XML Interchange work.
I’m told that some people browse the World Wide Web using mobile telephones; I have yet to see that being convenient or useful, or for tha matter significantly cheaper than buying a desktop PC, looking at the Web page, and then discarding the PC, but maybe someone will convince me. At any rate, Web services are of course being used widely in the Enterprise, and other spacecraft, and mobile phones can also view SVG graphics (another reason for wanting EXI!).
All of the major commercial relational databases have some form of XML support today, and of course it’s part of ISO SQL; there’s also a standard way to call XML Query (XQuery) from Java. Most of the open source relational databases appear to have fallen behind, although MonetDB does have XQuery support.
If you connect the financial industry the database industry and Web publishing, you get people like eBay, who are big XML users: when you view pages on www.ebay.com, you are looking at the result of some of their 4,000,000+ lines of XSLT and XQuery. Of course, a lot of Web sites are using XML on a back end somewhere; my own uses XML and XQuery for the image search, and it’s really not that unusual.
Did you know that Microsoft Windows components communicate with each other using XML? Try double-clicking on a .zxqp file that the OS doesn’t understand, and notice it asks if you want to use a Web service to contact Microsoft and try to identify the file.
But it’s not only Microsoft that’s using XML. OpenSolaris, OS X, BSD Unix and Linux, to name a few, all ship with XML support. For that matter many operating environments are using XML to store configuration information, ranging from the shape of the windows and text of dialogue buttons in applications through to preferences.
I’m sorry if I have not mentioned your favourite XML application or industry sector. Perhaps you can help me out by leaving a comment.
I’d say that almost all computers in use today have XML going on inside them, along with a significant proportion of mobile devices. And automobiles. Probably somewhere there are XML-enabled shoes, although since I’m usually barefoot I wouldn’t notice.
Why did XML succeed? I’ll leave that for another blog entry I think. For now let’s just say that it’s widely used, and if that’s a measure of success, we did well.
I’m sometimes asked when you should use attributes and when you should use elements when designing a new XML vocabulary. There are in fact no hard and fast answers for all situations, but there are some constraints and guidelines that may help.
The attributes of any particular XML element are an unordered set of name-value pairs. There can be no duplicated names (hence a set), but duplicated values are allowed in XML. The values cannot contain element markup, although they can contain entity references and character references. These constraints all follow directly from the XML specification, which also allows you, via a DTD, to place some simple constraints on the values of attributes: this one is a list of name tokens, this one has a space-separated list of name tokens that must each appear elsewhere in the document as the value of an attribute declared in the DTD to be of type id, and so on. In practice the constraints on values that a DTD can impose are not very widely applicable, and people wanting to constrin content are more likely to be using W3C XML Schema to do so.
Conventions of usage outside the XML specification itself have led to some other observations, some of which were learned by the SGML folks before XML was even started.
Most of the time when you have running text that might be presented to a human user, you will run across the need for element markup in that text. Even book titles sometimes have fragments of mathematics in them, and in Japan even names can have markup (called Ruby). As a result, the first principle for deciding when to use an attribute is this:
Attributes are for computers and elements are for people.
If you have come to XML from HTML, you might be thinking of the img element and its alt attribute, or of the title attribute on a div or anchor. These are simply bad design. There is no reason why a link title should not contain an image, and no reason why replacement text for an image can’t contain formatting (consider a poem, for example!).
If you have come to XML from RDF, you might be thinking that all properties are strings and that elements in RDF serialisations denote a change of focus, the alternation between subject and relation. That’s because RDF serializations all suck, and is why markup must be escaped inside an RDFLiteral in RSS.
Wake up and smell the coffee, guys. Don’t put human-readable
text in attributes. Let elements nest naturally.
If attributes are for machine-readable content, what sort of tings are suitable? One approach is to say that attributes on an element are properties of that element. If you have a boat element, say, to describe a boat (Ballasted Open Aquatic Transportation device, for you military types), the attributes on that boat element would describe properties of the element, not properties of the boat. You might, for example, record the date on which the element’s contents were last modified.
Restricting attributes to be element properties quickly gets tedious, and in practice people intermix the model and the markup, and say, give their boat a length attribute that is intended to be the length of the boat in nautical miles (or in metres, or whatever). This rarely causes problems in practice, as long as it is clear to everyone involved which is which.
People coming to XML from the relational database world want to make serializations of database tables or views, and in those cases they typically do have what XML people think of as unstructured data: that is, plain strings without interior markup. Furthermore the columns in a relational database table have unique names and are not ordered, so it is entirely reasonable to have a row element with an attribute for each field name. I more often see sub-elements used here, and I am not entirely sure why, but probably because the database people think of the content of the individual cells as being data and want the data in content rather than in attributes, which they think of as representing properties. But if each table row represent an object, the values in the cells in the row are the defining properties of that object, so you could easily argue either position.
It then comes down to tools. Attributes tend to be second-class citizens in XML APIs. You can’t duplicate them with XInclude; you need to read and parse them all before you process any of them, in case there are namespace pseudo-attributes intertwingled amongst them; in a DTD you can’t define a content model for which attributes must or must not appear in the way that you can for elements.
So in practice people use attributes to mark up properties that modify the way we think of an element, and elements for human-readable content, and for other things people choose based on the tools they are using.
And then there are people who pervert attributes into element names, so instead of saying <part-number> they say <person name=”part-number“> or something like that. This practice has been ennobled recently with the name microformats, but really it’s a way for people to be able to extend their markup in an ad hoc way without having to think carefully about the consequences. If you are in control of the vocabulary you should of course not do this. If you are not in control of the vocabulary, you should consider using namespaces to introduce new elements instead of corrupting existing ones. Microformats are a last resort for people learning about markup in a stifling environment.
OK, that’s enough ranting for today. If I left you a little confused about when to use attributes, let me know, but remember that outside the firm rules that come from the XML specification you’re on your own: the XML specification doesn’t say any more because implementation experience with SGML showed us that it’s not always clear when to use elements and when to use attributes, so we left it up to people using XML to decide what made most sense for them.
Oh dear, this blogging thing doesn’t happen all by itself, you have to remember it.
I had originally wanted somewhere I could write about some of the philosophy and design issues of XML, but then I got intimidated, because people could see what I wrote. What if they read things into my writing that I didn’t intend?
I suppose that’s a bit like an XML document: people can read all sorts of meaning into my markup, and it’s just fine as long as they don’t go round asserting that I meant whatever it is they have inferred.
People sometimes talk about URIs as being opaque. Just because a URI ends in /liam/pictures/me/ doesn’t mean that you can assume it’s a resource associated with pictures of Liam. If it’s an http resource, you can assume that you can take /pictures/ off the end, although you don’t know anything about what you’ll get. You are simply resolving a URI of ../ against the given base URI, and that’s perfectly well defined.
So there are some things you know and some things you can guess, but the things you can guess are only ever guesses unless you have information from some other source. For example, you might dereference a URI and get an HTML document (/pictures/liam/me) that says this page is pictures of Milldeton Englebert, M.E. and then you know that me isn’t about Liam at all. Or not.
In the same way, XML namespaces are opaque. You can handle the syntax: a qualified name (qname) has a URI binding and a local name, and the URI binding is referenced via a prefix: xyzzy:room. The prefix itself (xyzzy) is defined to be meaningless except as a binding. This shows the syntax is poorly defined, but we were in a hurry at the time. It would have been better to have taken longer, in retrospect, but we did what we did. XML tools can and do rewrite prefixes. The local name part (room) is opaque just like a component of a hierarchical URI is opaque: you know only what the owner chooses to document, and no more.
Amongst other things this means you might think that it’s wrong to hang behaviour such as rendering or scripting off local element names. In fact you can use XML documents in any way you like, and indeed it’s common to use a single piece of markup in multiple and radically different ways. So go ahead and use any XML document you find in any way you like, subject to rules of copyright, libel, slander and aesthetic taste. But don’t go around saying you know the only way to process that XML: even its owner doesn’t know that. You can say you are the owner, creator, designer and mother of the data, and that you intended a particular usage, and that may be very helpful.
Your sister in XML,
I wanted somewhere I could post without sounding too authoritative; my Advogato account isn’t really focussed on XML and my personal page isn’t bloggish.
What is XML?
I’ve recently held a number of Future of XML sessions at various conferences. I posted about it on my Advogato blog too. I got quite a lot of feedback, but one thing emerged clearly: XML is what you make of it. There is no one answer. There is no One True Data Model. There is no One True Meaning, and no One True Processing Model.
One person confidently told me (and maybe 150 pther people in the room), “The 98% use-case for XML today is for Web services, and all features not needed for Web services should be removed.” Another (in the same session) said, more or less, “it seems to me that the main purpose of XML is to serialise RDF, and that RDF can do everything XML can, so let’s get rid of all the features of XML that RDF doesn’t need.” I do not mean to mock these two people: their viewpoints are, in their respective worlds, perfectly valid. It is clear, however, that neither viewpoint should be allowed to dominate to the exclusion of the other.
Last week I was at Extreme Markup, my favourite of the XML conferences. It’s where a whole bunch of markup geeks can talk about the philosphy of markup. Those conversations are important. At Tommie Usdin (conference co-chair) said, the edge cases are where the change comes from.
A strong theme at Extreme Markup this year (and last year) was the need to handle overlapping regions of text when one is using markup to describe existing non-XML documents. For example, consider the question, “How many times in Luke’s Gospel (a Biblical text) does Jesus speak?” It turns out that sometimes a quote can start in the middle of one verse and continue over two verses and end in the middle of another verse. So the verse structure and the speech structures overlap. But neither is intrisically more important than the other: they both exist. One can use empty elements to mark up verse boundaries, and then one can easily answer the question about speech. But then it’s really hard to fetch the contents of any particular verse.
If this all sounds really abstruse and irrelevent, think again. The very idea of SGML and generic vendor and application neutral markup was once considered equally weird and edge-case, and yet today HTML and XML are mainstream. It just took a while.
So, back to the question, what is XML? Is it a serialisation of computer data? A remote procedure call format? A vendor-neutral way to represent data? Or an application-independent way to represent text and its structure? It’s all of these, of course. The neat part is that a tool that handles any one of these might well handle them all, and yet specialisations (e.g. a graphics editor using SVG) can also exist alongside the more generic tools without any contradiction.
XML is many different things to many people. A former employer of mine, the late Yuri Rubinsky, used to tell the story of a group of blind people who encounter an elephant and try to describe it. One says it’s a long swinging thing (the trunk, get your mind out of the gutter!), another that it’s a long curved tusk, and another that it’s huge and flat (the middle part I suppose). In the same way, SGML (and today, XML) is many things to many people, and that is its strength.
This all means that on the one hand we must try to welcome new communities, and on the other hand we must keep the communities we already have: a difficult balance. But a worth-while one.