Hitting the on-line DTDs on every invokation of Saxon unnecessarily burdens the W3C Web site. I had already found guidance on how to use the Apache XML Catalog resolver to avoid that, but it wouldn’t work with the default XML catalog list provided by Ubuntu in /etc/xml/catalog for the XHTML DTDs.

After some investigation, it appeared that the use of a bogus URL as a SystemID in the intermediaries XHTML catalog files prevents the proper parsing of these catalogs, and thus make the local DTDs undiscoverable.

With the patch provided in my bug report, I can now happily use /etc/xml/catalog with saxon and never hit the network when transforming XHTML files:

java -cp /usr/share/java/xml-commons-resolver-1.1.jar:path-to-saxon8/saxon8.jar -Dxml.catalog.files=/etc/xml/catalog -Dxml.catalog.verbosity=1 net.sf.saxon.Transform -novw -r org.apache.xml.resolver.tools.CatalogResolver         -x org.apache.xml.resolver.tools.ResolvingXMLReader         -y org.apache.xml.resolver.tools.ResolvingXMLReader XMLFile XSLTFile

Transcriber, the tool I used to generate the captions of the video, saves the transcription into its own XML format:

<Episode program="ParisWeb 2007 - Les Bonnes Pratiques du Web Mobile" 
  air_date="2007-11-16">
<Trans scribe="Dominique Hazael-Massieux" 
  audio_filename="parisweb" version="5" version_date="090210" xml:lang="fr">
  <Speakers>
    <Speaker id="spk1" name="Stéphane Deschamps" 
      check="no" type="male" dialect="native" accent="French" scope="local"/>
    </Speaker>
  </Speakers>
  <Section type="report" startTime="0" endTime="44.209">
    <Turn startTime="0" endTime="19.933" speaker="spk1" mode="planned">
      <Sync time="0"/>
      Y'a quelque chose auquel on croit beaucoup à ParisWeb,
      <Sync time="3.458"/>
      c'est "les standards, c'est bon, mangez-en",
      <Sync time="6.553"/>
      c'est pour ça que cette association existe
    </Turn>
  </Section>
</Trans>

It offers the possibility to export it in a variety of other formats (including HTML), but for sake of exploring one of the technologies in development in W3C for that precise use-case, Timed Text DFXP, I started to look into transforming their XML format into Timed Text.

Another motivation for that was that Subtitle Editor, the other tool I had looked at, is able to import and export timed text data; this also meant that very same tool would allow me to quickly visualize the subtitles superimposed to the video, one of the advantages that it had over Transcriber.

It turned out (unsurprisingly, I suppose) that the conversion between the two formats was really quite easy through an XSLT style sheet; the main structural difference between the two formats is that Transcriber notes break points as XML elements (<Sync> in the example above), while TimedText wraps the transcripted content into elements (<span> or <p>).

So, now that I had a Timed Text version of my transcription, how did that help me putting the transcripted video on the Web?

Looking quickly on the Web, it seems that some Video hosting services, including dotSub and dailymotion but not (I think) Youtube, allows publishers to upload subtitles with their videos; as I have verified since, dotSub even supports importing and exporting subtitles in TimedText format.

But I was curious to know how to include these subtitles in a self-hosted video situation; I had little hope to find subtitles support through the classical <object> tag in HTML, but I was hoping that the new <video> element in HTML 5 would help solve that problem.

Unfortunately, it doesn’t out of the box as of the draft dated of February 12 :

[…] authors are expected to provide alternative media streams and/or to embed accessibility aids (such as caption or subtitle tracks) into their media streams.

That certainly seemed extremely suboptimal to me – having to download a whole video to access its transcript doesn’t sound like a good use of anyone bandwidth. Discussions on fixing that current state of the HTML 5 spec have apparently started, and brought to my attention the work that my colleague Philippe had started to implement a JavaScript-based TimedText player for HTML 5.

This was exactly what I needed, and I thus started to play with that code to embed subtitles of my video in an HTML page.

And this is what got me started to look into why the new <video> element in HTML 5 is actually a game changer, rather than just a nice wrapper around the existing functionalities in <object> – which is what my next blog post will look into.

See how it looks on my personal site:

I have preferred generating a separate Javascript file, since having embedded HTML in a javascript embedded in HTML does not play nicely with validation; also, it makes it easier to use in different pages.

The embedding HTML page needs to have the following elements:

• in the <head> element:

  
  <script src="http://maps.google.com/maps?file=api&v=1&key=GoogleMapAPIKey"
  type="text/javascript"></script>
  

  where GoogleMapAPIKey is to be replaced by your own API Key

• in the <body> element, at the place where the map must be generated:

  
  <div id="map"
    style="width: 900px; height: 700px">
  </div>
  <script type="text/javascript"
     src="map.js"&;gt
  </script>
  

  where map.js is to be replaced by the location of the script generated by the XSLT.

If you’re already marking up your pages with GeoURL and if your pages are in XHTML, a whole web site can be mapped very quickly on a Google Map using this system.

A few notes:

• the current style sheet deliberately doesn’t crawl URIs over the network; so it will only work on a local copy of the site to be mapped
• it doesn’t work on pages served as application/xhtml+xml; I don’t know if Google Map as a whole does – I know that Google AdSense doesn’t work by default with this MIME type
• ideally, there would be a way to generate a non-javascript version of the map for accessibility reasons; but there isn’t

Update (Jan 19): I’ve updated the XSLT to make it use proper DOM nodes instead of document.write; also, it now proposes a list of links for a given location, instead of the last link that was added for the said location.

]]> http://people.w3.org/~dom/archives/2006/01/geographical-site-map/feed/ 2 http://people.w3.org/~dom/archives/2005/04/links-annotater/ http://people.w3.org/~dom/archives/2005/04/links-annotater/#comments Thu, 21 Apr 2005 16:23:52 +0000 Dom http://people.w3.org/~dom/archives/2005/04/links-annotater/ The Web is a formidable tool to host documentation; nothing new about that.

But documentation, be it on the Web or not, tend to rot when not maintained. Nothing new about that either.

While documentation maintenance is probably better addressed at the social-engineering level, there are tools that can help manage it; namely, a few weeks ago, W3C Systems Team went through the process of cleaning up our internal documentation on processes, tools, services, configurations, etc. that sits on our Team-only Web site, but is too rarely kept up to date with the latest developments.

To help filling the gap, I wrote a small XSLT style sheet to annotate links with information on how recently linked pages were updated. The goal was to run it on our main documentation pages, and find pages that hadn’t been updated recently through a color code.

The idea behind this tool, as illustrated when applied to the QA Activity home page is to show the date of last modification of all pages linked from a given page. To make it easier to find the most outdated pages, it also uses a color code, from pale yellow (most recent) to red (oldest), to denote how recently the linked page was updated.

I think the tool is useful as is, although it could use some user interface polishing. It could also provide some ideas for new features in the W3C link checker, or the linkchecker Firefox extension.

Since it reveals several interesting aspects of developing with HTTP and XSLT, I’m going to give a bit more details on how it actually works.

Getting the information from HTTP

The links annotations rely on the Last-Modified HTTP header. This header, when set, indicates the date and time at which the origin server believes the [page] was last modified.

In other words, the annotations only give interesting results on servers that are properly configured to send this header; for a statically-served site on Apache (as most of W3C Web site), it is just a matter of enabling ExpiresActive On in the configuration file (see for other servers); it’s a bit more tricky to get it right for dynamically-generated pages, but still very much doable, and directly worth it in terms of getting a better caching behavior from browsers and proxies. In any case, Mark Nottingham’s caching tutorial is one of the best references on this topic.

Using XSLT to get and show the results

So, now that we know where the information is, how are we going to access it? Getting access to HTTP headers is usually pretty easy in most HTTP-aware programming languages, but this is not the case by default with XSLT. (Why using XSLT then? because it so damn easy to use for parsing XML!)

To circumvent this limitation, I’m using on the most powerful way to extend XSLT in my opinion: the document() function combined with HTTP GET.

Indeed, XSLT allows to import and process the content from other documents than the one being processed with the document() function that takes the URI of these other documents as parameter. As this URI can be created at runtime (i.e. can be the result of another expression), you can actually get the content of resources based on the main document being processed.

In this case, we’re going to use this flexibility to get the HTTP headers from a completely separate tool: as this tool uses HTTP GET to pass its parameter url and outputs the results in XHTML, one can parse in XSLT the results for any HTTP URI by concatenating the said URI to http://cgi.w3.org/cgi-bin/headers?url=, namely with document(concat('http://cgi.w3.org/cgi-bin/headers?url=',$uri)).

Since this capacity to access HTTP headers has been useful to me more than once, I created a while back a small XSLT interface that defines a set of named templates that I can import in new style sheets with <xsl:import>. Over the years, I have gathered a few of these interfaces that allow to considerably reduce the development of small or complex tools.

In practice, the code of the main XSLT does the following processing:

1. by default, we keep everything as is through the identity transformation:

   <!-- default: Identity Transformation -->
   <xsl:template match="*|@*|comment()|text()">
      <xsl:copy>
        <xsl:apply-templates select="*|@*|comment()|text()"/>
      </xsl:copy>
    </xsl:template>

   (there are simpler ways to specify the identity transformation, but it would get caught on one of the bugs of the above mentioned XSLT servlet)

2. since we’re interested on visible links to HTTP resources, we’re going to process each of these through:

   <xsl:template match="html:a[@href and (starts-with(@href,'http:') or not(contains(@href,':')))]">

   (i.e. applies what follows to all the a elements that have an href attribute which either starts with http: or doesn’t contain a URI scheme (and thus, are relative HTTP URIs)

3. then, we transform the URI of the link into an absolute URI (using another imported XSLT)
4. we check its HTTP status code to detect broken links, and for valid links, we actually extract the Last-Modification header
5. the rest of the code only deals with displaying only the interesting information of this header (namely, the date, since the day of the week and the time are less likely to be useful) and associating it with an HTML class, so as to facilitate the color coding through CSS

Et voilà for the processing!

The last bit is to provide an HTML interface to it; to that end, I’m using a CSS trick on the XSLT style sheet itself (trick that doesn’t work in Internet Explorer, last time I heard). The XHTML interface is directly embedded inside the XSLT, as a child of the root element, where XSLT processors are not required to complain about foreign elements. Then, with a CSS style sheet, I make sure the content of the templates is not displayed, and that way by default, browsers will only display the embedded XHTML.

This wouldn’t prevent from doing a separate HTML interface as I have come to do for other tools; but I have always liked the idea (stolen from DanC as far as I can remember) of embedding HTML in my XSLT if only as a way to document what they are for, and how to use them.

]]> http://people.w3.org/~dom/archives/2005/04/links-annotater/feed/ 1