The rather obvious change that HTML 5 brings to the table is the HTMLMediaElement DOM Interface, and in particular the currentTime property, which at any time reflects the part of the media content that is played.

This means that it allows to synchronize any part of your HTML page with the video, as well as navigate through the video by setting the property to the section of the video you want to play!

And since I had already gathered a lot of timing information in the transcript of the video, extracting meaningful timings of the various sequences of the video was again only an XSLT style sheet away, provided I added relevant metadata in the transcription: typically, identifying subsections as <div> in the timedtext transcript, with a ttm:title set (which I achieved directly through my transcribing tool, Transcriber, that has all the needed interfaces to set these metadata).

And so I wrote that XSLT, added some further out-of-band metadata linking to slides and additional notes that I wanted to include in my presentation viewer (more details on the process involved are available).

The fact that I couldn’t embed these additional data in TimedText is actually quite disappointing – that a Web format should be developed without any way to add hyperlinks seems quite wrong! Generally speaking, it’s not clear to me that timedtext should be anything else than a set of additional timing attributes on top of XHTML – but I can’t claim that I have explored that space sufficiently to give much credit to that assertion.

Given that these metadata were not stored in the TimedText file, I ended up having them embedded in the resulting HTML page; it occured to me that the best combination to store them there was to use the extremely experimental media fragment syntax within an RDFa description of the table of content, e.g.:

<ul class="toc">
            <li about="http://media.w3.org/2007/11/parisweb-dom.ogv#t=00:00:44.209,00:01:28.432">
               <a target="slides" rel="foaf:depiction"  
                    property="dc:title" 
                    href="http://www.w3.org/2007/Talks/11-parisweb/slide-1.html">
                    Introduction
                </a>
            </li>
</ul>

This essentially annotates a given section of the video (#t=00:00:44.209,00:01:28.432 meaning between 44.209 seconds after the start of the video and 1 minute 28.432 second after the start) with a title and an illustration (in this case, the accompanying slide) – I chose foaf:depiction as a property, but it probably isn’t the best match – I’m hoping thet Media Annotations Working Group will come up with a useful ontology that could be used in these types of contexts.

These annotations are then parsed by a small Javascript layer (built on top of JQuery) which reproduces most of what the TimedText javascript player does, but in a much less verbose way… – another incitation for hoping that timedtext was really just XHTML.

The resulting presentation viewer allows to navigate through the video, with synchronized slides, notes and subtitles, provided your browser supports the HTMLMediaElement interface, as Firefox 3.1 does:

(also available as Ogg/Theora video with a Timed Text transcript.)

It also carries a set of RDF annotations to the video itself.

(I discovered only today that apparently Ian Hickson made a very similar demonstration a few months ago)

I must confess that I’m not quite sure that the accessibility of the resulting page is great – it uses the <object> element to load external pages (the slides and notes), while it should probably include their content automatically through AJAX, with a pinch of WAI ARIA to alert of pages updates.

The AJAX inclusion of content would be much facilitated by scoped style sheets.

My conclusion from this exploration is that clearly the new HTMLMediaInterface DOM interface is of great importance to really bring video (and similarly audio) into the Web browser; I can see how it could be improved to make it much easier to create synchronization effects:

• using some sort of timer callback interface with a begin and end period – currently, you have to use the generic setInterval function that polls every tenth of second to check whether you are in a period of the video where something should happen; ideally, you would just say video.setTimer(callback_function,begin,end), and callback_function would be called each time the video enters the period of time between begin and end;Err… it seems that’s exactly what addCueRange is about.… I guess what I really need is having it implemented :)
• ensuring that the HTMLMediaInterface is applied to any element where a time-based animation is used: being able to use it on SVG animations, Flash animations, and maybe even animated GIF (!) sounds as useful as on videos and audios; maybe this “just” means that the <video> element implementations should support image/svg+xml and image/gif as acceptable media types?
• it seems really backward that any JavaScript layer be required at all to run synchronized subtitles, and the <video;gt; element should clearly support linking media content and their transcript in a uniform way;
• it would be really neat if the media fragment URIs could be used again directly to go to a particular section of a video included in the page, without the Javascript layer.

To celebrate the progress of GRDDL towards its final stage, and to replace the aging and somewhat unreliable XSLT-based GRDDL demonstrator, I’ve just released a brand new W3C GRDDL service.

It simply takes a Web page and extracts the RDF statements it can find in there using GRDDL.

I have made its source code available on W3C Public CVS server, but the gist the work is done by the underlying library, python-librdf, the Python binding for Redland. Particular thanks to Dave Beckett who provided amazing user support to help setting this up!

Kudos to the GRDDL Working Group for the amazing job they did of formalizing, cleaning and working out the hard issues that existed in the original specification!

My last WWW Conference was three years ago and didn’t leave me with great memories, but this year’s edition was certainly much better than any other WWW Conference I’ve been to, and getting close to be as inspiring as XTech was last year – I haven’t been to XTech this year, so I don’t know how good it was, but read some positive reports on the 2006 edition as well.

Clearly, one of the major improvements for me was the extension of the previous editions’ developers day into a developers track that ran during the whole conference, rather than being concentrated at the very end; I thought it made a rather nice balance to the more theoretical and academic side of the other tracks.

My own contribution to the conference was the set up and animation of a session in the W3C track on the Mobile Web, and esp. on the prospects of what the future of the Mobile Web might be, entitled somewhat provocatively What the Mobile Web 2.0 will be?; this was the occasion to present updates on the W3C Mobile Web Initiative, but more proeminently to listen at the views of six panelists on what might make the Mobile Web the next big thing for the Web, much like “Web 2.0″ is the currently used moniker to refer to exciting stuff on the Web.

While the opinions of the panelists were quite diverse, some trends emerged out of the discussions: access to the Web on mobile devices can be of a huge importance in granting Web access to a much larger variety of people (esp. in developing countries, where access to a connected computer remains a luxury), should allow users creations and interactions in a much broader context; Rolan Geisler from Nokia also alluded to the development of a core Open Source platform for mobile devices as a way to help the industry grow the Mobile Web into a scalable development platform (matching their announcement of the open source release of the S60 browser).

In any case, it was really nice to see all the energy there was around the Mobile Web, during this session and elsewhere; hopefully our ongoing work in the Mobile Web Initiative will allow us to build on this energy.

Well actually, the first thing I did was to review the existing bots; I’m OK with writing a bot to prove my attachment to the Semantic Web, but not if that requires some actual work. And the good news is: there is plenty of existing code to work from. A few notes on the ones I quickly looked at:

• Emeka that I didn’t really test, since it relies on a recent API addition to 4Suite which wasn’t available on my Debian testing
• FOAFBot had a lot of the features I was looking for my bot, but seemed a bit too complex for the kind of work I wanted to put in the project — I was really looking for a toy, nothing more
• swBot sounded interesting, but was obviously unmaintained since its code didn’t change for the past 22 months; in particular, it wouldn’t run with a recent version of cwm

The final winner of this harsh selection was Julie also known as Redlandbot; it almost failed my tests since it was so hard to find the actual code — Christopher, if you read this, Julie’s homepage links a non-existent SVN repository.

But I eventually found it, and downloaded the four modules needed: redlandbot.py, flowcontrol.py, datauri.py, and wrapper.py. And I quickly understood that redlandbot was a perfect match: it relies on Redland, and so provides seemlessly access to a complete and compliant RDF parser, to a reasonably complete SPARQL engine (thus allowing to query RDF statements); furthermore, the code was in Python, easy enough to read and understand for me, and seemed fairly easy to extend.

And indeed, since one of the features I wanted for my bot was the ability to read iCalendar resources, it took me only a few minutes and exactly 11 lines to add support for that by modifying the modules managing Web access; of course, parsing an iCalendar resource doesn’t take 11 lines of code, but again, it was easy to re-use existing code, and in this particular case, the iCalendar to RDF converter that was developed in the Semantic Web Interest Group.

So, what’s my new bot about anyway? Well, it doesn’t really do much more than Julie – to the point I should have probably imported Julie’s code instead of modifying it. But I didn’t know that when I started. Anyway, the few additional features it has:

• it reads iCalendars as mentioned
• it uses a slightly more user-friendly interaction language; instead of addressing it with commands starting with a carret (^), you ask “BigBrother, do this” which I find easier to read, and also allows more than one bot to be on a channel without all reacting to the said input
• instead of stored commands, BigBrother stores questions; as demonstrated below, you can ask it “BigBrother, where is dom?”
• its default set of questions is pretty cool; you can ask where someone is on a given date, what’s someone’s phone number, etc; see below for more details
• it doesn’t rely on a database connection, but on a in-memory storage, making it slightly easier to run anywhere – but probably slower too

So, here come a few examples of what the bot can do; I’ll start with a couple of fake URIs and modified responses, since the data on which the bot was working when this was captured are not publicly available.

So, let’s say that we have an HTML phonebook available at http://www.example.org/team-phonelist, and that this HTML has been annotated as FOAF data using GRDDL, for instance with one of my GRDDL transformations. We can ask the bot to load that page directly:

<dom> BigBrother, load http://www.example.org/team-phonelist
<BigBrother>  Model size increased by 571 to 11386 by adding GRDDL data.

And then ask it specific information from what it just read:

<dom> BigBrother, dom's phones?
<BigBrother>  tel:+33.4.55.55.55.55 Hazaël-Massieux
<BigBrother>  tel:+1.555.123.456 Hazaël-Massieux

Now, let’s ask it to look at an iCalendar file at http://www.example.org/team-calendar which holds information on who is where and when:

<dom> BigBrother,  load http://www.example.org/team-calendar
<BigBrother>  Model size increased by 1046 to 12432 by adding ICS data.
<dom> BigBrother, where is jo?
<BigBrother>  Lambda, Joe: Foo meeting, Edinburgh
<dom> BigBrother, where will be jo tomorrow?
<BigBrother>  Lambda, Joe: National Holiday
<dom> BigBrother, where will be jo on Dec 25?
<BigBrother>  Lambda, Joe: Vacation

As you can see above, the bot is capable of parsing date names in various string formats, thanks partly to mx.DateTime; unfortunately, python doesnt’ seem to have a function or a module nearly as powerful as PHP’s strtotime.

Now, let’s start to see other questions, this time with publicly available information. The list of W3C Team members is available in HTML GRDDL-annotated as FOAF; which allows to ask the following:

<dom> BigBrother,  load http://www.w3.org/People/all
<BigBrother>  Model size increased by 482 to 12914 by adding GRDDL data.
<dom> BigBrother, what does steven look like?
<BigBrother>  http://www.cwi.nl/~steven/steven-london.jpg Pemberton

Of course, we can also load directly RDF/XML:

<dom> BigBrother,  load http://www.w3.org/2002/01/tr-automation/tr.rdf
<BigBrother>  Model size increased by 2938 to 15852 by adding RDF/XML data.
<dom> BigBrother, what specifications did Dom edit?
<BigBrother>  Dominique Hazaël-Massieux Gleaning Resource Descriptions from Dialects of Languages (GRDDL)
<BigBrother>  Dominique Hazaël-Massieux QA Framework: Specification Guidelines
<BigBrother>  Dominique Hazaël-Massieux Variability in Specifications

And more generally, feed the bot with any SPARQL query (the line returns were added for sake of readability):

<dom> BigBrother, query PREFIX rec:<http://www.w3.org/2001/02pd/rec54#> 
            SELECT ?title WHERE {
           ?tr a rec:REC ?tr rec:patentRules <http://www.w3.org/Consortium/Patent-Policy-20040205/>
           ?tr dc:title ?title } 
<BigBrother>  xml:id Version 1.0
<BigBrother>  QA Framework: Specification Guidelines

All in all, I had a lot of fun playing with this, especially to see how powerful SPARQL is, and how useful RDF can be as a data agregation format; I’m not sure I’ll put any more effort in the code at this point, unless someone else work with me on it; there is already a few improvements needed to make it robust enough for a real use.

And it shows again how much power one can gain from the open source: had it not been for Dave’s, Christopher’s, DanC’s and others existing tools, I wouldn’t have been able even to start thinking to develop such a tool.

First and foremost, the conference was great! There were only very few talks that I attended that were not direct interest to me, and in any case, it was a great opportunity to meet and chat with some of the people you’ve met and chat with before, but only or mostly electronically…

The big highlight of the conference for me was the Open Data track where I spent most of my time and where I’ve seen some of the most interesting ideas developed. It was very much invaded by people from the BBC, but only because they have so much cool projects being developed, e.g. the BBC Backstage. Too bad some of their content licenses are UK-restricted, but I guess I can understand the reasons; I only wish France Televisions and Radio France had this type of combination of open-mindness and technology-awareness. I also particularly liked the two presentations that show how Web technologies could be directly used as tools for civic usage, with the annotated European constitution project and the directory of British parliamentarians, both showing how getting access to political data and giving access to these data can make a difference.

More techy but really awesome was Jon Towbridge’s presentation of Beagle; not only Jon’s presentation style was really engaging, but also the topic of the talk was of real interest to me: I’ve tried and installed Beagle on my computer a few times already, although it never completely worked, the first glimpses of it were really promising — I should probably try to install the latest release. In any case, Beagle sounds plenty of prospectives; I guess if Beagle and Piggy-Bank could be interfaced somehow, a whole new type of interactions between the desktop applications and the Web could start to appear. Of course, GRDDL should be part of the equation…

I also attended a few talks on the Browser Technology track, whose mere existence is a sure sign that browsers are fashionable again…. I mainly saw demonstrations of the technologies at work in Mozilla (XUL, XBL, and the very promising XTF). It also was an opportunity for me eventually to see what the dreaded <canvas> tag was about; I guess I still fail to understand how this new non-standard element can bring anything new to the Web (since it seems to be pretty much less useful than SVG tiny), while it’s fairly easy to see what it can remove from it (accessibility, machine processing, etc.). It looks like an example of technology for technology-sake, as far as I can tell.

In this same track, I saw Steven Pemberton’s presentation of XHTML 2 (which meanwhile was recently updated) and found that he made a pretty convincing demonstration of where this new version of XHTML fits. Although I’m not a big fan of languages wars, the claims that XHTML 2 would be more accessible, and more semantics-rich than the currently deployed versions of HTML are, certainly seemed grounded to me.

I couldn’t attend the WHATWG vision of the future of HTML since that was right at the time where I was presenting my own talk on GRDDL — while most of the conference participants were precisely watching Ian’s talk on HTML, I still got some 40-ish attendants to my talk, which I think went OK. It’s hard to me to assess how much the audience as a whole understood the details of GRDDL, but I got at least some interesting questions and manifestations of interest in the technology afterwards. Now that the conference is over, the paper that accompanies my proposal is available as part of the proceedings, and also in a more up-to-date version on the W3C site. I try to develop there what role GRDDL can play in terms of bridging the various communities using and creating semantics on the Web.

I guess I need to start thinking what I could present next year if I don’t want to miss the conference then!

• the document is now a Team Submission rather than a Coordination Group Note; this was the shortest way to update the specification, given that no Working Group has clearly manifested an interest in maintaining the document at this time (see the related discussion) , but this is not necessarily the final status of the specification
• using GRDDL as a way to describe a specific HTML profile has been officially added to the specification

This last change, while it had floated in a few implementations for quite some time (e.g. in the XSLT-based demonstrators), is probably one of the most interesting capabilities for the HTML community, and especially for the HTML Microformats. Basically, it means that a GRDDL processor when encountering an HTML document with a profile attribute set to a given URI (as for instance in XFN) will get to that URI, tries to GRDDL it (so that the profile document can be in any XML language), and if it finds the appropriate description, it will apply the XSLT transformation found to the initial document.

To take a practical example with the XFN microformat, since this microformat uses a well-defined profile, if the community is interested to make all the XFN content incorporated in the Semantic Web, they just need to update the said profile to point to a relevant XSLT transformation (DanC drafted one a while ago) and any GRDDL-enabled processor would be able to parse the data.

I believe this is a pretty powerful mechanism, and I’m hoping this will indeed happen at some point; the expected benefits are not only for the RDF/XML community that would get more data to churn: it is also a very good way for the microformats community to get a common inftrastructure to parse all the microformats, without having to write specific parsers for each of these formats.

And, as an aside, it offers a common way to describe HTML profiles, and using any XML language that one sees fit…

(I’m noting it now because I’ve been working the last few days at preparing the accompanying paper; I would link to it, but I’m not sure this would be fair to do it before the conference…)

Given the relatively high number of said modules (24 at this day), and knowing that there are interdependencies between them, it becomes quickly hard to follow what’s going on and in which order.

To help get a clearer view of the CSS landscape, I decided to fire up my XSLT gun armed with a bit of RDF…

1. a spidering XSLT style sheet gets from the RDF list of W3C Technical Reports those that were produced by the CSS Working Group
2. for each of these documents (after some heuristics to filter the CSS2 and CSS1 documents), it looks for a section titled “Dependencies” and extracts the list of CSS modules listed in it
3. the resulting RDF list, after some hand massaging, can then be converted in a visual graph, using the RDF Validator

The resulting PNG image, while far from being perfect, does give a pretty nice overview of what’s going in CSS3 land… That was the easy part; now is time to interpret it!

(see also Ryan Lee’s TR references tracker)

The test suite is composed of

• a series of input/expected output documents – for the time being, only XHTML input has been integrated in the test suite
• an RDF list of these test cases, binding input and output documents, and defining the purpose of the test case – much as described in the QA Wiki on test metadata; the RDF vocabulary I’m using to that end is the one that was developed for the RDF Core test suite, mainly because it existed already – I’m not sure yet if this will prove to be benefitial in the end in terms of tools
• a small test harness in python that runs the implementation under test – GRDDL processors in this case – on the input documents, and compares it the expected output; since the comparison involves RDF graphs, I’m using SWAP’s graph comparitor; although the said code is very simple, I expect it could be re-used in a variety of similar test suites, that is test suites that feed input documents to a processor and compares it to a well-defined output document; obviously, in most cases you would need to alter the comparison mechanism depending on the output format

The first results of the test suite shows that my XSLT-based implementation of GRDDL gives similar results as Dan Connolly’s in python, in cases where there is no dereferencing errors; I’ve tried and run it on the RAP implementation – that I eventually installed along with PHP5 debian packages – with only mixed success. Hopefully I’ll have the time to get a closer look at that, and maybe send a patch.

• get Evolution to be able to subscribe to an HTML page to import contacts from it; this would suppose: getting Evolution to understand an RDF contacts vocabulary (e.g. SWAD contact or FOAF), and then to get it to understand GRDDL – either with the full URI dereferencing/XSLT processing, or simply with a few well-known HTML profiles to encode it- maybe Chandler would be a better application to target?
• get Beagle to get new properties from Web pages when used with the Firefox extension
• get a feedreader to read HTML pages and extract RSS feed using GRDDL
• get a music player to read a Musicbrainz playlist from a Web page

With more and more projects starting to use Semantic Web or Semantic Web-like technologies for the desktop, getting GRDDL in there would simply allow a wider Web to carry the semantics.

The only restriction is that it relies on having the said schema served at the namespace URI of the given vocabulary; in other words, it wouldn’t work through xsi:schemaLocation, but that’s probably a very acceptable restriction, especially since the XML Namespace solution scales much better, in that you don’t have to duplicate the information again and again. This very much relates to the namespaceDocument issue the TAG has in its queue…

Anyway, this works both in DanC’s python implementation of GRDDL and in the on-line XSLT-based demonstrator, when applied for instance to the Purcharse Order example inspired from the XML Schema primer.

Of course, that means conversely that we should start thinking more carefully on the update policy for the GRDDL namespace…

The idea is rather simple: instead of relying on a specific profile (http://www.w3.org/2003/g/data-view) completed with a bunch of <link rel="transformation"> elements to point to the GRDDL-izers for a specific Web page, it makes it possible to bind a set of GRDDL-izers to any profiles; this means basically packaging GRDDL-izers in re-usable sets, so that you don’t have to duplicate the same information over and over.

For instance, imagine a Weblog profile, identified by the URI http://example.net/weblogProfile; the goal of the profile is to enable one to incorporate FOAF-data, RSS 1.0, and GeoURL directly in the visible XHTML code of the a Weblog, using a set of well-defined markup conventions (as for instance XFN does for what is roughly FOAF data).

Right now, this is only useful if the software you’re using has been programmatically told to recognize this specific profile, and to interpret it following these specific markup conventions.

Using profile-based GRDDL, a software that wouldn’t know what the profile means could on a best effort basis simply try and dereference the profile URI, see whether it is bound to any GRDDL transformations, and if it is, use these to add meaning to the original web page.

This completed with tools like the Gnome Dashboard could yield some really funky applications, applied on the clickable Web as we know it today.

(This works well for the browsing tools, probably less so for authoring tools, esp. WYSIWYG ones, since you need to define a method for the authoring tool to learn about these conventions, for which I don’t think XSLT is descriptive enough; maybe a Schematron-based solution would work better?)

To see this applied in the field, you can now for instance give the W3C Home Page in the GRDDL demonstrator and get back the included RSS data (a bit buggy, because the underlying GRDDL-izer isn’t (yet) fully GRDDL compliant), or do the same with the QA Working Group Home Page; the nice thing is that these pages were using these profiles before GRDDL even existed, and I only needed to add the relevant GRDDL data to their HTML profile documents; on this topic, note that GRDDL is applied recursively in these cases, since for instance, the W3C home page profile is itself simply completed with RDDL (not to be confused with GRDDL) markup, itself interpreted through another GRDDL-izer… It’s OK if your head aches :)