dretblog

Comparing SMS URIs

it was a bit early to declare that the SMS URI scheme was finished. another iteration was required, this time an update to version draft-wilde-sms-uri-20. the main reason was that the draft did not contain any information on how to compare SMS URIs. Section 2.4 now explains how to compare SMS URIs (mostly based on the definition of how to compare to tel: URIs), and (once again...) hopefully that was the last change required before the draft can be published as an RFC.

this is a really long process, but on the other hand it is really good to see that people care enough about this to take a close look and comment on the draft. thanks for all the comments!

Links in HTML

looking at how HTML exactly works from a RESTful perspective requires to look at all the different links that HTML defines. these are actually quite a bit more than just <A> links, and based on the HTML 4.01 transitional DTD, here is the complete list of all links that can be specified in HTML. one interesting observation is that some of these links are not supported in today's browsers, such as the cite attribute on <Q> and <BLOCKQUOTE>.

Element	Attribute	Method	Response Resource	Semantics
BODY	background	GET	image/*	texture tile for document background
A	href	GET	*/*	URI for linked resource
AREA	href	GET	*/*	URI for linked resource
LINK	href	GET	*/* (based on rel and/or type, more specific types may be expected)	URI for linked resource
IMG	src	GET	image/*	URI of image to embed
	longdesc	GET	*/*	link to long description (complements alt)
	usemap	GET	text/html (fragment must be MAP element)	use client-side image map (often local fragment identifier)
OBJECT	classid	GET (relative to codebase)	application/*	identifies an implementation
	codebase	-	-	base URI for classid, data, archive
	archive	GET (multiple URIs, relative to codebase)	*/*	space-separated list of URIs
	data	GET (relative to codebase)	*/*	reference to object's data
	usemap	GET	text/html (fragment must be MAP element)	use client-side image map (often local fragment identifier)
APPLET	codebase	-	-	base URI for classid, data, archive
	archive	GET (relative to codebase)	*/*	comma-separated archive list
	code	GET (relative to codebase)	*/*	applet class file
	object	GET (relative to codebase)	*/*	serialized applet file
Q, BLOCKQUOTE	cite	GET	*/*	URI for source document or msg
INS, DEL	cite	GET	*/*	info on reason for change
FORM	action	GET/POST (depending on method)	POST Request: application/x-www-form-urlencoded or multipart/form-data (depending on enctype) Response: */*	server-side form handler
INPUT	src	GET	image/*	for fields with images
	usemap	GET	text/html (fragment must be MAP element)	use client-side image map (often local fragment identifier)
FRAME, IFRAME	longdesc	GET	*/*	link to long description (complements title)
	src	GET	*/*	source of frame content
HEAD	profile	GET (often treated as a name only)	*/*	named dictionary of meta info
BASE	href	-	-	URI that acts as base URI
SCRIPT	src	GET	application/*	URI for an external script
	for	-	-	reserved for possible future use

Good Lists are Hard

being a web plumber, one of pages i visit almost daily is the technical reports (TR) page of the W3C at http://www.w3.org/TR/. when this page was updated last week (as part of the launch of the new W3C site), it was amazing to see how many things you can break at the same time, and it also was an interesting lesson in how simple and smart design can be much more usable and useful than just adding functionality.

the old TR was a good web page in a variety of ways. it was lightweight, had a completely noise-free design, had all the recent updates on top as shortcuts to the complete listings further down, and in short was a well-designed and self-contained representation of the TR work done by the W3C. you could go there, quickly see up the latest additions with your browser clearly highlighting the pages you had not yet visited, and do everything else with the browser's find function. it was one of the pages i always showed to people when explaining why simple and smart is often better than just adding fluff without properly considering usability and utility.

the new TR page managed to break almost everything that was useful, and it does so in a variety of ways. it removed the previously smart quick/complete combined view, it added a lot of visual noise, and it breaks the previously correct usage of the W3C's spec URIs. here is where the new design fails:

• quick view plus complete info. one of the useful things of the old page was the fact that it had a short recent specs section at the beginning, and complete listings further done. it was a very elegant way to both have a quick view of what's new, and a complete view of everything, all available via simple browser-based search. now there are more options, but none of those is as useful as the old view, which is not available anymore.
• useless layout noise. the layout of the old page was simple yet effective, putting as much information in one browser window as possible. the new layout contains a lot of useless layout noise such as horizontal lines between each line, plus a lot of spacing surrounding the text and the lines. it probably doubles the space required per item, while also listing less information. using a table costs even more space without adding any value to the presentation.
• proper URI handling. W3C specs have versioned as well as canonical URIs, and the old TR page linked to the versioned URIs, which is the right thing to do, because the TR page contains a date-based of specs, and not a list of canonical URIs of the latest versions of all specs. the new page links to the canonical URIs, essentially linking to something other than the resource it is actually listing on the TR list. this even breaks basic principles of REST, where the pages users are GETting don't properly link to the resources they are referring to; instead they are referring to a canonical URI that does not represent the actual document that has been published. the very real consequence of this particular breakage is that the browser cannot reliably color the links based on whether the URIs of the particular versions are in the history or not.

in summary, it would be interesting to know whether the new TR has undergone any internal usability and utility testing before being released. it is broken in a variety of ways, which is kind of interesting, but from the perspective of a regular user, it really is sad to see that a page as useful and well-designed as the old TR page is being replaced by a much more noisy and much less useful version. i will certainly miss the old TR, and i guess so will others who were closely following the W3C's TR work.

if you think the old TR was better, please leave a brief +1 comment here, and maybe it will at least come back as a classic view for those who like web pages that are simple and effective.

REST and Link Computation

the idea of defining a Formal Hypermedia Context definitely has some appeal, taking the rather basic HATEOAS principle one step further. thinking about Media Types and Resource Types basically moves in the exact same direction: figuring out what it takes to describe and then use a RESTful service. deciding on a transition in a RESTful service not only depends on the links found in representations; it also critically depends on understanding (a) where to find those links in the first place (link syntax), (b) what those links mean in the context of the RESTful service (link semantics), and (c) which link to actually follow to achieve the application goals (link pragmatics).

we are currently working on a description language for RESTful services which is trying to make as much explicit about a RESTful service as possible. it is a bit similar to the model recently presented on InfoQ, but extends it in a crucial way. the InfoQ model has link relation types and XML schemas, but since there is no existing XML schema language that explicitly supports a link type, there is no way how a link can be found in any given XML. we change this by adding selectors that allow to select links in XML using XPath. XPaths may select URIs, but they can also just select IDs which can then be transformed into URIs by applying link-specific rules, such as adding a prefix.

this idea of computing links based on the hypermedia context could be as simple as prefix/postfix, more sophisticated such as using URI templates, even more sophisticated in the case of a even more expressive language for computing URIs. taken to the extreme, such a of link computation could even include variables carried over from previous navigational actions, in which case we would lose some of the nice context-free properties of one-step-at-a-time hypermedia links. we are still experimenting with how expressive these languages should be (so that more real-world services can be described, even if their RESTfulness is less than perfect), and how constrained they should be so that they're reasonably easy to understand and use. for now, we might just go the simple prefix/postfix route, but exploring the idea of a more formal model certainly would be an interesting exercise, and also help to understand at which point a more expressive model starts to compromise some core principles of REST. my guess would be that local link computations are acceptable, but that starting to carry over state from previous link traversals would too clearly collide with the HATEOAS principle.

Web Services for Recovery.gov

together with my colleagues eric kansa and raymond yee, we have just published a new paper about Web Services for Recovery.gov as UCB ISchool Report 2009-035. the main goal of this report is to highlight a side of the open and transparent data debate that is often overlooked. some people look a lot at web site design in terms of HTML/CSS design and usability/accessibility, and other people look a lot at how to make data available, focusing on what essentially boils down to building download portals. our main contribution is to highlight the fact that download-oriented data access is not sufficient to provide timely access to evolving data sources, for two main reasons:

• data is never complete. data is always connected to more data, and download formats often disrupt this connection, making the data decontextualized and making it harder to understand how it is embedded into the world. once data is decontextualized, it can be very hard or almost impossible to contextualize it again, re-establishing all the connections to related data that existed where the data originated.
• data is never static (unless it's purely historical data). downloads not only are decontextualized in terms of connectivity with other data, they also are decontextualized with regard to time. any data source that is frequently updated must have simple ways for synchronizing data, so that more efficient ways of data access than periodical downloads can be established.

starting from our earlier report on how ARRA reporting should use lightweight and web-based technologies, we are again focusing on technologies which are powerful enough to implement dynamic and machine-friendly services, but on the other hand also simple enough so that citizen developers can easily understand and use them. refining our earlier report, we are now focusing on widely established and supported technologies such as XML and feeds. we make concrete recommendations how to build web services for recovery.gov that are both more sophisticated than download approaches, while still being easy to implement both on the provider as well as on the consumer side. here's our report's abstract:

One of the main goals of the Recovery.gov Web site is to provide information about how funds for the American Recovery and Reinvestment Act (ARRA) of 2009 are allocated and spent. In this report, we propose a reporting architecture that would focus on the reporting services rather than the Web site and page design, and that uses these Web services to build the user-facing part of ARRA reporting. Our proposed architecture is based on simple and well-established Web technologies, and the main goal of this architecture is to provide citizens and watchdog groups simple and easy access to machine-readable data. Our architecture uses a more sophisticated framework than simple downloads of data files. Our proposed architecture is based on the principles of Representational State Transfer (REST) and uses established and widely supported Web technologies such as feeds and XML. We argue that such an architecture is easy to design and implement, easy to understand for users, and easy to work with for those who want to access ARRA reporting data in a machine-readable way.

you can find the Web Services for Recovery.gov report in the iSchool's eScholarship repository. any feedback about the report is very welcome, and please make sure to also visit our new Architecting Transparency web site, which has been specifically built to highlight and demonstrate some of the issues described in the report.

Media Types and Resource Types

one of the central points of REST is that it is resource-centric, and that learning about new resources should be done by following hyperlinks found in previous resources. how to discover these links of course depends on the types of resources a client is dealing with, which means that the ability to understand resources plays a central role in REST.

what we've recently been running up against is the fact that understanding actually happens on multiple levels, and based on these levels, more or less sophisticated ways to use a service are possible. some examples:

• site navigation links on web sites often would allow very sophisticated understanding of the major resources the site provides, but since HTML has no good way of exposing that information, it is not possible to extract the full richness of the links contained in the typical navigation menu. the best case in this scenario would be to have something akin to sitemap in machine-readable sitemap format, but there is no such thing.
• for specific types of pages (let's consider product pages on some major online retailer) it would be possible to get more specific links regarding a product if the layout (i.e., the type of the page) would be taken into account. this of course breaks as soon as the layout is changed, but given that only few web sites are carefully managing their @class and @id, this at least works well as long as the layout remains stable withing these limits. in this case, the layout of the page provides important semantic hints.
• looking at machine-readable formats, it is interesting to look at Atom. because it is based on XML, it could be served as application/xml, but because it is well-known, it should better be served as application/atom+xml, so that clients understand it's Atom and not just XML, and so that clients can specifically request Atom. however, if it contains additional information such as GeoRSS or feed paging, this cannot be signaled by the media type, but instead it needs to be found out by inspection on the client side. and since there is no media type for these formats, there is no RESTful way to request a GeoRSS or a paged feed.

all of this means that while the theory of only looking at media types and requesting media types looks well in theory, in practice (in particular when embedding something into an existing set of resources and media types instead of starting from scratch) it often is necessary to make assumptions about resource types based on media types and additional hints. these additional hints can be resource inspection, or they can be URI pattern matching. in particular the latter makes many REST folks see red, but i am really wondering whether this isn't something to consider as well when working with existing services or simply describing them as good as possible.

in the end, we want to build an application that starts with a URI, interacts with the URI through a given protocol, processes the resource received as a result of that interaction, continues by discovering links in that resource, chooses the appropriate link type and link instance for the next interaction, and eventually continues this cycle. if the problem of better understanding a resource's semantics can be improved by inspecting the resource itself or its URI, while the primary method of designing interactions still is based on finding links and following them, then things still should be pretty RESTful, even though not in the most ideal way imaginable.

i guess the main issue here is when to promote a pattern or a property of a class of resources to mediatypeness. if some class of resources exposed by a service has useful links in them, and they are not exposed as a specific media type, then there has to be additional information that can be used to identify a resource as a member of that class, so that you can build richer interactions with it. why this may stray from the purist path of REST, for our purpose of describing RESTful services (or almost RESTful services), this is the way that seems to make most sense, pragmatically speaking.

SMS URI Scheme Finished

technically speaking, this is just another update of the SMS URI scheme draft (this time to version draft-wilde-sms-uri-19 after submitting versions 18 and 17 earlier this month), but it's going into IETF last call now and it seems that all comments were successfully resolved. there is one lingering issue around whether the optional body text in an SMS URI should have Unicode normalization rules defined for it, but i think it shouldn't. it is up to the software composing the actual SMS message to make sure that any normalization rules are being followed (and i am still a bit unclear if there are any; it seems that there aren't). what happens before that can be any kind of Unicode as long as it's valid Unicode, at least that is what i think makes most sense.

i hope that the optimistic title is not much too optimistic, but it definitely would be great to see this finally being finished. i just checked and realized that version 00 of that draft was published in december 2001, so this really has been under construction for quite a while. the delay was partly caused by some confusion around the IETF URI scheme registration rules and procedures, and around the IETF specification for how to represent telephone numbers, but mostly, it was just me getting sidetracked by other things. i hope this time it's really sufficient to go to RFC, and then, i hope, straight into mobile browser implementations!

Web of Things 2010 (WoT 2010): Call for Papers

First International Workshop on the Web of Things (WoT 2010)

In conjunction with the 8th IEEE International Conference on Pervasive Computing and Communications (PerCom 2010), Mannheim, Germany

Web site: http://www.webofthings.com/wot/2010

Paper submission deadline: October 18, 2009 November 15, 2009 (Extended)
Notification of acceptance: December 21, 2009
Camera-ready papers due: January 29, 2010
Workshop Date: March 29 or April 2, 2010

The world of embedded devices has experienced radical changes over the past few years as real-world objects can now easily connect to the Internet. This convergence of physical computing devices (wireless sensor networks, mobile phones, embedded computers, etc.) and the Internet provides new design opportunities and challenges. The Internet of Things has mainly focused on establishing connectivity in a variety of challenging and constrained networking environments, and the next logical step in the evolution in pervasive computing builds on top of network connectivity by focusing on the application layer: how to develop ubiquitous computing applications on top of heterogeneous devices? The Web of Things is the vision that brings embedded devices into the Web by using Web standards as application protocol to interact with things. This workshop aims at bringing together the pervasive computing and Web communities to explore the possibilities for using the core principles and technologies of modern Web architecture (e.g., HTTP, REST, Atom, JSON) for seamless integration of things into the Web and developing applications on top of web-enabled devices (physical mashups). Research contributions are solicited in the following areas:

• Web-based interaction patters for embedded devices (streaming, eventing, etc.)
  
• Applications, deployments, and evaluation of Web of things systems
  
• Human-things interaction models and paradigms (mobile interfaces, etc.)
  
• User-oriented, context-aware discovery and dynamic search for the real world
  
• Web composition and macro-programming models (e.g. mashups)
  
• Semantic technologies for description of devices and services
  
• Optimization methods for embedded Web servers and applications
  
• Security, access control, and physical sharing of physical things on the Web

Since this workshop is the first in this area, the goal is to develop a community in this area and provide an interactive forum for researchers to learn and discuss about existing efforts to enable cross-fertilization. Hence, we expect all attendees to read all workshop papers in advance and to prepare questions for each. Presentations shall specifically highlight and address similarities and differences with other accepted papers. This will help to provide a forum to foster future collaboration, beyond the mere presentation of research results. In order to ensure a high-quality technical session, submissions must cover one of the topics above and should not exceed six (6) IEEE conference format pages. Research papers must be original prior unpublished work and not under review elsewhere as they will be published on the IEEE Digital Library. All submissions will be peer-reviewed and selected based on their originality, merit, and relevance to the workshop. Submission requires at least one author to fully register for PerCom 2010 and to present the paper on-site.

Organizers:

• Dominique Guinard, ETH Zurich and SAP Research Zurich
• Erik Wilde, School of Information, UC Berkeley
• Vlad Trifa, ETH Zurich and SAP Research Zurich

Program Committee:

• Gregor Broll, DOCOMO Communications, Germany
• Adam Dunkels, SICS, Sweden
• Martin Gaedke, University of Chemnitz, Germany
• Stamatis Karnouskos, SAP Research, Germany
• Gerd Kortuem, University of Lancaster, UK
• Marc Langheinrich, University of Lugano, Switzerland
• Thomas Luckenbach, Fraunhofer Institute FOKUS, Germany
• Friedemann Mattern, ETH Zurich, Switzerland
• Alexandros Marinos, University of Surrey, UK
• Rene Mayrhofer, University of Vienna, Austria
• Tatsuo Nakajima, Waseda University, Japan
• Benedikt Ostermaier, ETH Zurich, Switzerland
• Cesare Pautasso, University of Lugano, Switzerland
• Dave Raggett, W3C, USA
• Michael Smith, W3C, Japan
• Vlad Stirbu, NOKIA, Finland
• Inaki Vazquez, University of Deusto, Spain
• Agnes Voisard, Fraunhofer ISST, Germany

(( Update 10/8: Extended paper submission deadline from 10/18 to 11/15. ))

SMS URI Scheme Update

only three days after submitting my first SMS URI scheme update after almost a year, the next version (draft-wilde-sms-uri-18) is already available. i hope i am getting close to completion with that one, and the reviews definitely have become friendlier and easier to handle. the only substantial change for version 18 has been to introduce an extension model where future fields extending SMS URIs are already defined by the basic syntax, and have well-defined semantics (a simple mustIgnore; if you don't know an extension, just ignore it).

any comments about this draft are highly welcome! and by the way, here is the web-based I-D tracker for this draft, which means you can simply bookmark that page or even subscribe to its feed, and then you will see every official activity regarding the draft.

Distribution and the Semantic Web

RPC and SemWeb have the same implicit goal of hiding distribution that somehow goes against the grain of RESTful loose coupling was an observation i recently voiced on twitter, and that might need a bit of explanation. for now, i'll focus on the semantic web part of that, but it seems to me that this general approach of trying to hide distribution is amazingly similar in both the WS-* world and the semantic web. in both cases, this makes it hard to think of how to get them better aligned with basic web principles. to stay a bit more focused, i will concentrate on RDF for the purpose of this post.

RDF is great at making statements about URI-identified things, because this is what it is all about. build graphs of things that attach, by reference, to anything that can be identified by URI. how to identify RDF triples and graphs themselves is an entirely different issue, though. RDF's abstract model leaves them dangling in the void, and while SPARQL introduces the concept of named graphs (effectively turning triples into quadruples), these names are basically namespace URIs, which means they identify named graphs, but there is no standardized way of how to interact with them (they are never dereferenced). so this means that in a typical RDF store, triples are not identified at all, and even if they are (let's say somebody manages their RDF store in a way which guarantees that every triple is associated with exactly one named graph URI), there is no well-defined interaction with those triples.

this all starts with the basic approach of many semantic web projects: people want to have and use information about things on the web, or they start with the assumption that others have information about things on the web, and then they can harvest that information. this terminology is quite interesting, and if we want to get all metaphorical, then this is what harvesting is all about: you cut stuff, separating it from its roots, and you end up with a lot of useful harvested stuff, but it's sort of dead and you will never be able to reconnect it with where it came from. it may give you useful seeds for new things as well, but again, those things will only be very loosely connected to where they came from.

so, on the semantic web, it's all about getting many RDF triples, dropping them into one big silo (which is why you see all these statements about we have converted such-and-such into x billion triples!), and then working with that silo (maybe providing one tiny hole in that silo through which SPARQL queries can be funneled). this approach makes it easy to abstract from the fact that all the information started out from different sources (and probably still lives there and may be exposed via live services), but it also makes it hard to manage things such as provenance and authenticity (which are critically important for distributed and federated systems). as patrick murray-john pointed out in his comment on my recent post about Data, Models, Metamodels, Cosmologies, RDF's biggest strength is its excellent capability to mix data together. however, this seamless blending of data from different sources gets problematic when you look at REST, which is built around the general idea of coarse-grained interactions with resources identified by URI.

so when you are treating semantic web work as the process of harvesting data, and then using it locally in some form that allows read access only and loses track of the identity and the boundaries of the original data sources, then you're not working with a web of data, but you are just working with data you harvested from the web. this assumes that the harvested data is static, and that any updates to the harvested data will not have to be propagated to where it originally came from. essentially, such an approach treats the web, in an idealized world view, as one huge RDF store, assuming that the process of collecting all the data is taken care of, and that there is no channel of how to interact with the original data sources. and this idealized world view is reflected in many projects around the semantic web, where it is assumed that problems of distribution are taken care of, and can be regarded as solved. as a side note, i think this is why there recently has been such a huge uptake of semantic web work in the government data space, because the government can be conveniently viewed as one homogeneous data source, providing one big collection of RDF data that can be accessed through centralized services. many of the problems of federated data sources, conflicting data, changing data, data authenticity, data integrity, and heterogenous data sources and services can be ignored in such a centralized scenario.

the current work around SPARQL Update does not look all that different. it looks at how to allow the full spectrum of CRUD operations for SPARQL implementations (in the same way as XQuery progressed from a read-only language to one with updates), but it still follows the same approach of looking at RDF data as living in one big silo, and standardizing a language for how to interact with that silo. it's important that such a language exists and i think SPARQL and the Update extension to it are very important specs, but this language has as much to do with the web as SQL or XQuery: not all that much. it's back-end stuff, and for me as a client of a web service, all i want is a way of how i can interact with that service through HTTP so that i can use that service effectively; whether the back-end of that service uses SQL or XQuery or SPARQL or a column store does not really interest me all that much.

so what i am currently doing (collaborating with linked data expert michael hausenblas) is trying to figure out a way how to make SPARQL more REST-friendly, so that SPARQL can be easily used as a back-end to provide RESTful services on the web. i am looking forward to our work of making SPARQL more REST-friendly, but it is interesting to me how that turns into an exercise of ignoring the subject URIs of triples as actionable URIs, and instead using the named graph URIs to give triples an identity on the web, so that they can be exposed in a RESTful way. in such a RESTful SPARQL world, it will be possible to GET a bunch of triples from somewhere, and then to PUT the exact same bunch of triples to a URI that identifies a named graph that needs to be updated. it seems to me that this kind of RESTful approach will make it much easier to expose meaningful resources on the semantic web, and to allow RESTful interactions with them.