Planet Web Standards

W3C's MASE search engine got a nice new year present, in the shape of a new server. The additional computing muscle will be well-used to index and search among the hundreds of thousands of messages on the W3C's mailing-list forums.

Taunting two knights with one stone, we also took the opportunity to give the search interface a fresh layer of CSS paint, and, among other updates, a Period search feature which can be used to limit search results to a given Month and/or Year.

All this is possible thanks to the great team developing the Namazu engine as an open source project. Namazu is not supposed to be used on such a massive set of data as the W3C lists, but it works: our hacking was, and is, mostly limited to interfacing the engine with our lists system, and make indexing of individual lists real-time.

Update: for some gritty details on how the mail search works, see this companion entry in the W3C Systems Team blog.

Any bug you'd like to report on the search engine? Features that we really should be adding? The comment form is all yours!

Data portability and open data are hot topics these months. The multiplication of social network sites has increased the aggregation of these data in silos. The Semantic Web activity encourages to open your data with a new series of cute logo. Geographical data such as the Open Street Map initiative, government budget such as USA are the trend. But when it comes to our personal data, we need more granularity.

Danny Ayers on Talis Web site talked about Data Licenses for Social Network Services. He frames the discussion with a basic rule:

I own my data. This would correspond more or less to the default copyright on documents - even if you don't say anything explicit on something you write, you have the copyright.

In the past (before the Web), when I shared my data with a friend or someone else, phone number, specific email address or local postal address, in some cases, I would make a point of saying: « Do not share these with someone else. » The trust relationship I have with the person made it obvious, without relying on a signed contract. The person would put it in her/his paper or computer address book. It was local data, viewed only by one person.

Then the Web arrived with many online services. You sign usually a contract between the online service and the usage of your data. It is your choice to decide what you accept from these services. But there is an additional issue. In your data, there are data which belong to other people. When you send the data to an online service, you might break a trust relationship. This person doesn't necessary want you to share your data with this online service.

The big change is that we all became provider and consumer of data with a responsibility in sharing them. All of us are a social network service with responsibilities.

How do we manage that? Recently Danny Weitzner (W3C) has posted an article on Reciprocal Privacy for the Social Web (a.k.a. FOAF). He mentionned in this article a document is working on: Reciprocal Privacy (ReP) for the Social Web. Read it, share it and comment it.

This is one of the big challenges for the years to come.

RDFa is a way to enrich your Web pages with local data. The clear benefit is that your data are in context and then easier to manage. Yesterday, on the RDFa mailing list, Dan Brickley asked how we could use RDFa to extract the information of an HTML imagemap.

Dan says:

HTML has a somewhat neglected notation for describing regions of images, and associating them with links.

[snip]

BTW I have no idea what the current XHTML 2 and WhatWG/HTML5 folks have planned for these elements. But I think this kind of markup has a lot of potential for making images on the Web more automation friendly.

For now, HTML 5 map element doesn't do better than HTML 4.01 or maybe I have missed something.

Max Froumentin had design an XSLT which transforms the imagemap code and then convert it to an SVG ouput with the background tone down to make the imagemap zone more visible.

Dan continue:

I'd like a way to say, "this area of the image depicts the person who is the primaryTopic of http://danbri.org/".

A lot of work had been done already in the past on image descriptions with some practical examples, but Niklas Lindström started to give it a stab.

<map name="da8bb51_b" id="da8bb51_b"
    about="[_:the_area]" 
    property="ex:imageMapAreaElement">
    <area  shape="POLY"
        coords="..."
        href="http://danbri.org/" />
</map>

<img  about="[_:the_area]" 
    rel="ex:sourceImage"
    src="2169955372_503da8bb51_b.jpg"  
    width="1024" height="770"
    usemap="#da8bb51_b" />
<div about="[_:danbri]" instanceof="foaf:Person">
    <span property="foaf:name">Dan Brickley</span>
    <span rev="foaf:depicts" resource="[_:the_area]"></span>
    <a rev="foaf:primaryTopic" href="http://danbri.org/">(website)</a>
</div>

• Someone has a better suggestion in RDFa?
• Would it be better done in HTML only without breaking previous implementations of HTML 4 imagemap?

I see a lot of possibilities in games, identification, education, cartography. fun. fun.

C. M. Sperberg-McQueen has opened a blog recently, Message in a bottle. It is really cool. When I had questions about XML, I sometimes asked him on the W3C internal mailing-lists. Very often, I thought that it would be great if his answers were public. Not only, he knows XML, but his replies are very clear.

Two days ago in a message, Sandboxes and test cases about XML Schemas tests, he's telling about the process of creating test cases and trying in some implementations.

It's always interesting to construct test cases to illustrate some question or other that arises, whether from a comment on the spec or an inquiry by email. And I have directories with scores of small test cases I have constructed over the last few years. But until I started this more systematic construction of a schema validation sandbox, I contented myself with checking those test cases with one or two processors.

But also explains that playing with more tools is more fun.

But it turns out that having more processors to test is just a lot more fun. Here is a test case. OK, what does libxml say about it? MSV? Saxon? Xerces C? Xerces J?

Identifying issues in software helps people to improve implementations and specifications. But if you do so, please be kind. It is always better and more encouraging to receive an email saying "I may have identified an issue" more than "Dude. I broke your implementation. Take that!". It might sound silly but courtesy helps a lot to get things fixed.

Back to the article of Michael, he has been publishing a tiny collection of XML Schema test cases. And as he said:

I plan to put every schema example I generate this year there, instead of hiding it on my hard disk. At least the interesting ones.

Well done Michael and happy new year too.

The Video On The Web Workshop is finished. A report is being prepared with the outcomes of the workshop. In the meantime, three video interviews have been published by videolectures.net:

• Interview with Adobe Chief Software Architect
• Interview with the W3C Chief Executive Officer
• Interview with W3C Compound Document Specialist

Update: Another video this time explaining relationships between videos, metadata and Semantic Web technologies by Philippe Le Hégaret.

To make the distance to home when I travel a little shorter, for my birthday I got one of these digital picture frames. With a little fiddling, I got the picture and music features working, but I'm stumped on video. When I searched for support, I found I wasn't the only one:

I converted my file to various formats but unfortunately none of them would play on my frame.

Hindry 2007-07-31

When I picked up co-chairing the HTML Working group with Chris Wilson in March this year, little did I know how much video codecs would impact that part of my life too.

The scope of the HTML 5 draft is considerably larger than the scope of the HTML 4 Recommendation, and the Working Group is looking at the major differences in a couple ways:

• a HTML 5 differences from HTML 4 draft
• a list of requirements issues

Among those requirements issues is ISSUE-6, videoaudio. It has been discussed informally as far back as March/April (Opera, Apple and Microsoft, etc.) and it was added to the issue tracker during a discussion of media elements on the first day of our our November meeting in Cambridge.

A few minutes later, we added a design issue, ISSUE-7 video-codecs, regarding which codecs, should/must/may implementations of HTML 5 support. David Singer of Apple wrote an excellent summary on 9 November:

... interoperability at the markup level does not ensure interoperability for the user, unless there are commonly supported formats for the video and audio encodings, and the file format wrapper. For images there is no mandated format, but the widely deployed solutions (PNG, JPEG/JFIF, GIF) mean that interoperability is, in fact, achieved.

Licensing:

The problem is complicated by the IPR situation around audio and video coding, combined with the W3C patent policy. "W3C seeks to issue Recommendations that can be implemented on a Royalty-Free (RF) basis." Note that much of the rest of the policy may not apply (as it concerns the specifications developed at the W3C, not those that are normatively referenced). However, it's clear that at least RF-decode is needed.

So when Rudd-O's article says, "Nokia and Apple have privately pushed to give Ogg the noose treatment," that's just sensationalism. The truth is that Apple and Nokia are participating in an open discussion of the issue.

Many of us share the goal of an open, interoperable video codec for the web. But the state-of-the-art as of Nov 2007 is that there is no video codec that meets the community requirements. In that context, the 30 Nov message from Mikko Honkala of Nokia that said, "we support publication of HTML5 WD, including the canvas, audio, and video elements, but not the SHOULD clause for the baseline codec until proper patent assessment has been made" is not a "noose treatment"; it's just working group members trying to prevent the disappointment of authors coding to the spec and then finding out that it doesn't work.

The 11 December response from the editor, Ian Hickson, was to acknowledge the issue in the draft:

It would be helpful for interoperability if all browsers could support the same codecs. However, there are no known codecs that satisfy all the current players: we need a codec

• that is known to not require per-unit or per-distributor licensing,
• that is compatible with the open source development model,
• that is of sufficient quality as to be usable, and
• that is not an additional submarine patent risk for large companies.

This is an ongoing issue and this section will be updated once more information is available.

3.14.7.1. Video and audio codecs for video elements the HTML 5 draft, formatted for emphasis/clarity

Worse than not acknowledging the openness of the discussion, Rudd-O continues, "This destroyed all hope of having free (as in freedom) media embedded in HTML5 in an interoperable way." To give up hope at this point is the most counter-productive thing to do.

The response to the W3C Video on the Web Workshop Call for Participation was an outpouring of resources to tackle this issue. The presentations were inspiring.

I was also inspired by Håkon's demo in the video panel in the W3C Technical Plenary; he took the wikipedia octopus article and replace the photo with a video. It was easy to think back to when I was in third grade and imagine how much more of an impact it would have on me; then I remembered I have a third-grader at home; see One laptop per Kyle for more of that story, including a little video production of my own.

Putting that video of Kyle together was pretty easy with current consumer technology: I shot it with a digital video camera that we bought years ago, edited it with Apple iMovie, and an online service took the iMovie output and converted it to whatever everybody is using these days. But as Eric Hyche of RealNetworks pointed out last week, somewhere in the process I had to execute some terms and conditions with that online service; it's my video; what business is it of theirs? When I take a picture and put it up on the Web, I can use flickr if I want their value-add services, but I can also just stick the JPG file on an HTTP server of my own; the corresponding option for video is beyond the reach of the typical Web user.

Though I'd rather somebody more qualified chaired the W3C standardization effort to choose between video codecs, I have been studying digital media enough to be confident I could produce Ogg/Theora or Dirac; but I'm not sure my audience can consume it. After the workshop discussions, I'm particularly sympathetic to comments such as this one about the rich network of support for H.264:

... pushing Ogg/Theora might make you proud to have voted, but it will only distract from the industry's coalition to unitedly back H.264 from mobile devices to HD. There's far more FOSS support for MPEG-4 and H.264 than for Ogg/Theora ... Having wide support behind one good, open portfolio of standards will make it easier for FOSS to compete with and participate in the desktop computing world.

DECS (891519) on Sunday December 09, @05:33PM

H.264 has a lot going for it; the two main issues I see are:

1. the H.264 patent licensing situation
2. competition with VC-1 in Windows Media

Microsoft didn't send anyone to the workshop last week, so I'll leave the second issue for another day.

I heard a lot of creative ideas about the patent licensing situation and support for free software, though. Everything from stop-gap measures involving binary blobs to using the full force of W3C to address the patent situation head-on as we did in the P3P and Eolas cases. If you want to join the W3C members who have expressed an interest to help, please contact me and Philippe Le Hégaret and Chris Lilley, preferably with a copy in a public archive.

Today (21 December 2007) I am attending the Tokyo 10th XML Developers day. This is an annual event, held in Japanese, with latest news from the Japanese XML developers community. The event is organized by Murata-san. The presentations are of a great variety, providing both technical and rather "political" aspects of XML trends. Below is a summary of the talks, focusing on the technical aspects.

The World of Genji Monogatari - a new Edition

(slides) Miyawaki-san presents a project with the aim to relate additional information to the main text of Genji Monogatari: notes from himself or others, images, sound of a reading of the novel etc. In a previous version he used an approach of enriching an HTML file (the main text) with that additional information. Later he switched to an XML-based format. That allows a user to easily add new information, and multiple users to add content without conflicts. Miyawaki-san encounters some problems like XSLT-debugging, the need to process JavaScript (necessary for the HTML output) in XSLT, or error handling in XSLT. The latter even motivated him to rewrite some XSLT code in JavaScript ...

An interesting aspect of a generated HTML output is that it is presented in Internet Explorer in vertical layout (see this screenshot). For the Japanese audience who is used to read novels vertically, this is an important feature, though the layout still needs some improvement.

Parallel Narratology

Kobayashi-san from Justsystems presents so-called parallel narratology, realized with XML. He mainly uses the famous Japanese murderer story of Rashomon as an example. In that story, the same event is narrated several times from different perspectives. In an XML representation of the text, users can create links between the different narrations of the same scene (e.g. a murderer scene). Yamaguchi-san from Justsystems demonstrates the XFY editing and development tool which has been used for the creation of the links, and the presentation of the parallel texts.

The linkage allows readers to take a new perspective on the relations between narrations. Not only the author, but everybody can add such links: the ordinary "story readers" become "story writers". A problem mentioned during Q/A is: how to express overlap, i.e. non-hierarchical structures of markup. Such structures may be necessary e.g. to enable different users to mark up overlapping passages of the same text.

Character Encoding (again)

Kobayashi-san presents again, mainly about the "gaiji" problem. There are many ideographic characters which are so-called glyph variants of characters from the Unicode character set. However, these variants are not represented as distinct characters in the character set. Kobayashi-san introduces two solutions to this issue: shortly the Japanese standard JIS X 4166, and mainly Unicode variation selectors. The latter are used for the registration of such variations in the Unicode Ideographic Variation Database.

Yamamoto-san demonstrates how Adobe is using the variation selector approach. In Adobe fonts, there are glyphs available for much more characters than the Unicode character set encompasses. Adobe uses the variation selectors approach to create stable identifiers for these characters. The Unicode Ideographic Variation Database then serves as a registry for these identifiers.

Input Methods in the Age of XML

Masu-san from Justsystems presents on input methods. These are tools for making the input of scripts with a large set of characters (like in Japanese) easier. Masu-san then demonstrates enhanced input methods. They provide functionality which is not only useful for large character set input. For example, while entering a date, the input method recognizes the date and creates appropriate markup, hyperlinks to a calendar etc. Using such enhanced input methods, users not knowledgeable of XML are producing XML-code (e.g. a marked-up date) without noticing.

Kumaya-san from Tokyo University presents on how an enhanced Input method framework can be used. In the presented implementation, currently, schedule data is represented via Microformats, and location information is taken from Google maps. A question is how to describe the meaning of related, similar information, e.g. HCalender versus Google calendar. The answer is: humans have to make the relation explicit, there is no automatic way to achieve this.

A Prediction of the OOXML Vote Result and the Ballot Resolution Meeting

Murata-san provides insights in the rather political aspects of the development of the OOXML and the ODF office formats. The Ecma TC 46 committee has already approved OOXML. On 25 February 2008, there will be a Ballot Resolution Meeting (BRM) by JTC1 about OOXML. Murata-san predicts that at the BRM, there will be no discussion of real criticism on the format, but only discussion of minor technical changes. The reason is that contentious issues are outside the scope of the BRM.

He presents also work on ODF undertaken in ISO SC34 and a proposal for standardization of a round-trip conversion between ODF and OOXML. The discussion about the round trip conversion happens in DIN (Germany,) not in SC34 (yet). The DIN committee might want to submit the result to JTC1 using the fast-track procedure.

Murata-san admits that in the past, he thought the standardization of an office format would be impossible. Nowadays there are two on their way, which seems to be better than zero ...

Outline of AtomPub and Interoperability Testing Results

Asakura-san from NTT Communications presents the AtomPub format (RFC 5023). This is a protocol for publishing and editing Web Resources like Atom feeds via HTTP and XML 1.0. AtomPub defines the notions of collections (like Atom "feeds") and members (like Atom "entries"). Using the HTTP methods GET, POST, PUT and DELETE, collections and members can be retrieved, created, changed or deleted.

Asakura-san also provides insights into the experiences he had during the IETF standardization process, and reports from the the AtomPub interoperability event in Tokyo. Six companies participated successfully in that event.

Schemas for validating Atom and its Extensions

Murata-san again talks about Atom. Atom uses extensions from many namespaces, e.g. Gdata, Google Calendar, Atom Bidi, GeoRSS, Dublin Core etc. Murata-san gives mainly examples from Google calendar, which has a basic set of extensions. Others can be added.

Unfortunately, there are often no schemas for the validation of the extended Atom data available. To solve this issue, first Murata-san created an RELAX NG schema. Unfortunately, with more than 3-4 extension schemas, the task became really hard even for RELAX NG experts. He then switched to a different approach: NVDL, which is used for namespace-based validation of compound documents. This makes it easier to create schemas for Atom+extension validation. A remaining issue is that NVDL does not support validation of ID/IDREF-based types.

Using NVDL for Document Analysis and Synthesis

Miyashita-san from IBM also talks about NVDL. NVDL relies on analyzing, i.e. separating parts of a document for the purpose of validation. Miyashita-san's aim is to provide a mechanism for applying various other processing (e.g. editing) to the separated document parts, and put them then together again. He gives an example of Atom which contains XHTML. First the input document is separated in the Atom and the XHTML parts. Then format specific processing is applied, and the parts are put together again.

Miyashita-san demonstrates some testing with the Google Calendar data provided by Murata-san. One of his next goals for this framework is to use the XProc XML Pipeline Language as a means to define processing pipelines for the separated document parts.

On the HTML WG mailing list, Ben Boyle:

An interesting point, but I think this is highly dependent on whether you are publishing information primarily for browsers. Personally I like a combination of atom and html, both generated from a single source of data (for which I prefer XML). HTML is for the browsers, atom for feed readers and other (potential) consumers/aggregators. I use XHTML within Atom (personal preference to avoid CDATA where I can).

The browsers offer one rendered view of information on the Web among many possibilities. JSON, RDF, Atom, plain text, xhtml, html are parts of the choices to represent an information resource.

The Web exists because people wanted to connect to each others and share. They got involved. The first Web site was a kind of blog written by Tim Berners-Lee. People were experimenting, implementing, writing manual and tutorials. Tim was announcing the new servers that you could count each month on your fingers. You too can be part of it.

Just One Story Among Others

I have always known the W3C as a place to create and share. My academic background is not computer engineering. In 1991, I discovered the Web by chance on the computers of Physics Department of Montreal University and created my first two Web pages locally, page A and page B with just two links going from one to the other and back. I was really excited. In 1995, I was working in a Web agency in the Web design team: a biologist, a salesman, an english teacher, a lawyer, a journalist, a computer science student. We had the specifications such as Wilbur, HTML 3.2 codename, on our tables. Nobody was a specialist but everyone was helping each other. Some of us were translating in French the W3C specifications, others participating in forums, sharing quick tips.

On Going Discussions

Pointing fingers is sometimes necessary, but doesn't necessary achieve what we wanted. It often creates walls when we wanted to open fields. There are a lot of discussions about W3C, HTML and CSS these days, negative and positive comments. Jeremy Keith and Jeffrey Zeldman framed the discussions

If designers and developers are more aware of the problems than of the fact that the W3C is working to solve them, it's because the W3C is not great at outreach.

[...]

You want instant gratification, buy an iPod. You want standards that work, help. Or at least stop shouting.

Let me hear your standards body talk, Jeffrey Zeldman

In the comment Josh Stodola said What can I do to help , Jeffrey?

Get Involved!

I have written a few posts on this topic already and I will continue. Two days ago, I gave a link to the source code of an open source testing harness platform in PHP

• How to to contribute to W3C work? Tutorials
• How to to contribute to W3C work? Quick Tips

fantasai will publish on the CSS Working Group blog, in the next few weeks, a series of articles on how you can help W3C on the CSS front. In the mean time, a short list of tips to define your actions.

• Pick up one thing you can do for W3C. You have a job, a life, etc. Do not do too much but select one item that you have time to finish.
• Help with your own competences. If you are not a programmer, do not try to implement a specification. If you are a graphic designer, help with drawings.
• Listen and share. It is a community, a team work, it means a lot of patience, accepting to not have everything you want, but collectively to move forward.

You can be part of it too.

When I'm curious about tomorrow's weather (or need a quick look at a calendar), then I'll often invoke my laptop's Dashboard: It's a platform for Widgets, small applications written in HTML, CSS, and JavaScript -- Web technology, moved to the local host. These widgets provide front-ends to all kinds of information sources on the Web; several thousand are available for download. Obviously, the use of Web technologies has been a huge success here, enabling people to adapt their programming experience from the Web to their local platform. Apple's Dashboard is, of course, not the only platform: Google, Yahoo, Microsoft, and Opera have widget (or gadget) engines of their own; W3C's Web Application Formats Working Group has catalogued the properties of the various platforms in its widgets requirements document, and is working on a standard packaging format for widgets.

With the Web programming platform, though, come the Web's programming practices and security issues, sometimes with more serious consequences than before.

Let's begin with a classical piece of bad programming: Parsing data by evaluating it as code. JSON stands for JavaScript Object Notation; it means using JavaScript constant expressions as a way to transport structured data over the network. The easiest way to parse that data format consists in evaluating it as code. JavaScript has an eval() primitive that comes in usefully: It takes a string and interprets it. Now, one can argue that that's just the same thing as including another script, so it's not really a big deal when done in the client-side part of a Web Application. The effect is, after all, limited to one application, and that application's server-side code will have to deal with hostlie clients anyway.

However, while Widgets might feel a lot like the usual Web environment to the programmer, they really aren't: The limitations on what JavaScript code in a Widget can do are (sometimes vastly) different. For instance, the Dashboard has a widget.system method available; that method takes a string and executes it as a Unix shell script. Effectively, JavaScript has acquired an interface to the operating system. The question what scripts (or widgets) one executes has become the same question as the one what programs one installs. A widget that evaluates JSON data is now a way for an attacker to turn an attack against, say, a service like Twitter, or against your local network, into an attack that takes over an entire machine. That problem isn't just academic: Unsafe JSON parsing is a regrettably common programming practice among widget authors. And your web browsing behavior is intercepted whenever you're asked to pay at a commercial wireless hotspot. Exploiting these attacks is easy.

eval() isn't the only way to create this kind of attack surface: Using callback based JSON programming techniques by inserting <script> tags into a widget's document object model creates exactly the same kind of vulnerability.

But there's more: You might have heard of cross-site-scripting vulnerabilities as, for instance, the source of the recent Orkut worm. The problem here is that a Web application takes input from some untrusted source and writes it into the HTML that it generates, without paying proper attention to escaping tags that an attacker might have placed in that input. The attacker can then inject scripts into the Web application's output, and these scripts can confuse clients' behavior. Now, what has that to do with widgets? Consider that widgets use an HTML document as their user interface. Any data that are presented to the user are written to the that document by JavaScript code.

That JavaScript code might quite well use techniques like the non-standard innerHTML DOM property to add a large fragment of HTML to the user interface. If the data that are written to that property come from an untrusted source (e.g., the network), and aren't properly sanitized, then an attacker can once again get scripts executed.

The Google Gmail widget was an example for that kind of code, until quite recently. Here, HTML tags in an e-mail's subject header were written into the widget's UI without checking. The effect was stunning: One e-mail with the right JavaScript in the subject header was enough to take over the recipient's computer, if that recipient was running the Gmail widget. Google has now fixed the problem in this widget.

But Google isn't alone: There are more widgets out there with this kind of vulnerability. Triggering the vulnerability might not take an e-mail, but involve a network attack, or, maybe, a malicious blog post, a photo description, or some interaction on a social networking site. Or the widget itself might come from a malicious source. The point here is that we're dealing with a shift in what Web-style programming can achieve -- and that that shift does not come without risk, and responsibility. A shift, incidentally, that extends beyond just Widgets, as Web technology is also becoming the programming platform of choice for mobile devices, and as the capabilities of Web applications that run in the classical browser are extended.

Addressing this risk and responsibility will require work both inside the "Web industry" proper (and in the W3C Working Groups where it meets, including the Web Application Formats and Web API WGs), and with the broader community: This is a problem that extends beyond just the platform.

PS: I hope explore this space more at this year's Chaos Communication Congress in Berlin, and maybe at next year's Web Conference.

The Architecture of the World Wide Web includes a section on extensibility and versioning of languages and data formats. Quoting from the architecture document:

<recipe name="Tuna Salad" recipeLanguageVersion="1.0">
  <ingredients>
    <ingredient name="Tuna Fish"  amount="1 can"/>
    <ingredient name="Mayonnaise" amount="3 tablespoons"/>
    <ingredient name="Capers" amount="a few"/>
  </ingredients>
  <steps>
    <step>Open can</step>
    <step>Drain liquid from can.  Put fish in bowl.</step>
    <step>Add mayonnaise.  Stir well.</step>
    <step>Add capers.  Stir gently.</step>
  </steps>
</recipe>

<recipe name="Tuna Salad" recipeLanguageVersion="2.0">
  <ingredients>
    <ingredient name="Tuna Fish"  amount="1 can"
                picture="./CanPicture.jpg">>
    <ingredient name="Mayonnaise" amount="3 tablespoons"/>
    <ingredient name="Capers" amount="a few"/>
  </ingredients>
  <steps>
    <step>Open can</step>
    <step picture="./DrainCanPicture.jpg">
           Drain liquid from can.  Put fish in bowl.</step>
    <step>Add mayonnaise.  Stir well.</step>
    <step>Add capers.  Stir gently.</step>
  </steps>
</recipe>

<recipe name="ice cubes" recipeLanguageVersion="??">
  <ingredients>
    <ingredient name="water"  amount="1.5 cups"
  </ingredients>
  <steps>
    <step>Put water into ice cube tray.</step>
    <step>Freeze.</step>
  </steps>
</recipe>

Indeed, just these complexities have proven troublesome for the deployment of languages like XML 1.1. XML 1.1 is similar to XML 1.0, but it enables the use of some new Unicode characters (just as recipe language V2 allows for use of new image tags.) The XML 1.1 Recommendation suggests that:

XML Programs which generate XML SHOULD generate XML 1.0, unless one of the specific features of XML 1.1 is required.

In fact, it has often proven difficult to write software that generates documents labeled as XML 1.1 only when necessary: it's much easier for XML 1.1-compatible software to label all output as <xml version="1.1">, resulting in documents that are unusable with widely deployed XML 1.0 software. Perhaps for reasons like this, adoption of XML 1.1 has been slow.

Returning to the recipe example, maybe the version attribute should take a list of versions, and I should put in both 1.0 and 2.0? That could be helpful to consuming software, but it still means that I (or my software) must be familiar with all the previous versions of the specification.

So, we need to ask, is the version identifier used to convey:

• The earliest version of the language with which the document is compatible (1.0 in the recipe example)?
• The version of the specification I used as a guide when writing the document (2.0)?
• A list of versions with which the document is compatible?
• Something else?

The best answer is probably different depending on the language, how often it's revised, whether revisions tend to maintain backwards compatibility, etc.

Is having some sort of version identifier always a good idea?

That Good Practice Note quoted above says "provide a version indicator", but we've just shown that we're not always quite clear on what that would do anyway. Is it still good advice to suggest that surely each language should provide for something in the instance? If so, should its use be required or optional?

As shown above, it's common for the same instance document to be legal in many versions of a language. As long as such documents are likely to have the same or sufficiently compatible meanings per the different versions, then it may be better to omit any indication of version in the instance, and leave it to the receiving software to decide whether the document can be processed. After all, with the second recipe above, the receiver will soon enough discover that it can or can't process picture attributes, and if not, it either will or won't know that they can be safely ignored. Version attributes can be helpful in giving early warning of incompatibilities, or as a crosscheck for catching errors, but they're usually not essential to correct operation.

One important exception is in the case where the language is likely to change in incompatible ways. If the same document means different things in different versions of a language, then it's very important to indicate which version the author had in mind when creating the document. Putting that version indicator into the document itself is one good way to do it. So maybe the right advice is:

If version identifiers aren't always a good bet, what about namespaces? Many modern languages allow the creation of globally unique names, identifiers, tags, etc. In XML this is done through use of Namespaces. In RDF, it's done by using URIs as identifiers, etc..

Sometimes it's appropriate to use new identifiers for each version of a language, and mechanisms like namespaces can make that easier:

	<r:step xmlns:r="http://example.org/recipeLanguage1">

vs.

	<r2:step picture="./food.jpg"
                 xmlns:r2="http://example.org/recipeLanguage2">

In this example, the element with expanded name {http://example.org/recipeLanguage2, step} allows a picture attribute, but {http://example.org/recipeLanguage1, step} does not.

A full discussions of the pros and cons of using namespaces this way is beyond the scope of this note. One important advantage of using namespaces is that they can be easily applied not just to the root element for the language as a whole, but to mixtures of compound document markup, in which each sublanguage evolves with its own namespaces. Also, because namespace names are URIs, you can use the Web itself to get information about them.

Namespaces do have drawbacks. Imagine if there were 50 different namespaces for a language just because 50 separate bugs had been fixed in different errata. Would you republish all the markup in 50 namespaces? Would each document have lots of namespaces, with each element named with the last namespace in which it had been revised? Namespaces can be very useful for designating language versions, but there's no one idiom that's right for all languages. We note that most widely deployed tag-based languages for the Web (HTML, XML Schema, XSLT) have chosen either to use the same namespace(s) across multiple versions, or in the case of some flavors of HTML, not to use namespaces at all.

Conclusions

So, the TAG is having second thoughts about the suggestion that all data formats SHOULD provide for version identification. Sometimes it's a good thing to do, but sometimes not. Perhaps the right advice will be what's proposed in the revised Good Practice Note above. In any case, the TAG has been working for several years on a finding that will explore in detail many issues relating to versioning, and version attributes are likely to be among the topics covered. In the meantime, we thought we'd take the opportunity to signal that we're not so sure that the advice in the Architecture Document is as good as we thought.

By the way, TAG member David Orchard has covered some of the same topics as well as many others relating to versioning in his personal blog. Links to a few of his postings follow my signature below. Dave is also the principle author of the TAG's draft finding on versioning. Working drafts covering Terminology, Strategies, and Versioning of XML Languages are available for review. New drafts come out every few months, and we're hoping to have something more or less complete, well, real soon now.

Noah Mendelsohn

Note: unless otherwise indicated, opinions expressed in the TAG's blog are those of the individual authors, and do not necessarily represent consensus of the TAG as a whole.

Links to Dave Orchard's Blog Postings on Versioning

• http://www.pacificspirit.com/blog/2007/04/19/what_do_version_identifiers_identify
• http://www.pacificspirit.com/blog/2007/04/19/forwards_compatibility_with_version_s_requires_version_mapping
• http://www.pacificspirit.com/blog/2007/04/20/how_to_have_multiple_namespaces_per_name
• http://www.pacificspirit.com/blog/2007/08/09/guide_to_versioning_xml_languages_using_new_xml_schema_11_features_published
• http://www.pacificspirit.com/blog/2007/09/13/when_can_language_components_be_removed_and_maintain_backwards_or_forwards_compatibility
• http://www.pacificspirit.com/blog/2007/12/12/validation_by_projection_introduction

I have to admit something, sometimes I'm a bum. It's why I like tools which makes my life easier. I had written in the past that RDF is for the lazy person. I like also the LogValidator because it helps me to check the quality of my Web sites step by step without a big burde