REST Programming Toolbox Requirements

REST is all the rage, and everybody claims they're doing REST. REST is a style of designing web services, and the question is how this style can be best supported by a programming environment. there are two possible perspectives of this problem. one is the more high-level view of how to best structure code that is built around the assumption of a uniform interface. a popular example for this is JAX-RS, released as JSR-311 last year, and implemented by the Restlet framework. this is not what i want to discuss here.

a different perspective is that of basic tools that might be useful for implementing RESTful services (in that case interpreting REST in its narrower sense of web architecture and its use of HTTP and URIs). in a recent blog post, marc hadley reports on Integrating Jersey and Abdera. the interesting thing is how his approach combines a high-level framework (JAX-RS), with more low-level tools for handling HTTP and content (in that case, feeds).

so looking at the low-level support one would like to see when programming RESTful services, what is required or useful? i came up with this list, which is probably incomplete, so any feedback about what else should go in here would be highly welcome!

• HTTP support: HTTP must be fully accessible, which means that all methods, headers, and status codes must be accessible to the application. if required, certain behavior might be enabled on request (such as following redirects), but this should be fully under the control of the application.
• HTTP encoding: since content in HTTP can be transmitted using chunked encoding or some transfer-encoding, it must be possible to either explicitly deal with these encodings, or to have this handled by the HTTP API.
• HTTP authentication: in many cases, HTTP authentication is required for gaining authenticated access to access-controlled resources. the HTTP API should support the predefined HTTP authentication methods (basic/digest), and should also have a plug-in mechanism so that other authentication methods can be used as well.
• caching: caching is one of the core features of how REST can be optimized. caching can be based on modification dates and/or on ETags. client-side caches can be shared or private caches. there must be built-in support for caching, API controls for handling shared and private caches, and there should be integration with a client-side shared cache for the client as a whole (imagine how useful it would be if all browsers on your computer were smart enough to share the same cache).
• URI parser: URI parsing is not trivial, and URIs are used everywhere. there should be URI prcessing support for parsing and resolving URIs. and depending on where URI templates are going (they are still under construction), support for them  might be very useful as well.
• language tag processing: since language tags are part of HTTP and can be used in HTTP mechanisms such as content negotiation, handling language tags is a integral part of HTTP. since language tags have a surprisingly complex logic behind them, support for parsing and matching languages is very useful.
• MIME type processing: similarly to language tags, MIME types (officially called media types) have a structure, are used in content negotiation, and often should be compared or matched. support for these operations should be part of a toolbox as well.
• XML tools: XML is the most widely used syntax for machine-readable data, so there should be a parser, validation support for maybe various schema dialects (DTD, XSD RELAX NG), selection via XPath, hopefully XPath 2.0 as the more powerful language, and finally more advanced XML processing might be supported by XML-centric languages such as XSLT or XQuery.
• JSON tools: many RESTful services support JSON, and sometimes only JSON. support for parsing JSON therefore also should be part of the toolbox.
• Atom tools: one of the most widely used formats for machine-readable data is Atom, and thus Atom support should be part of the toolbox, also supporting additional features which are part of the Atom landscape such as feed paging.
• AtomPub: AtomPub is one way of implementing a RESTful service providing write access to Atom-oriented services, and thus support for AtomPub (at least for the client side) might be useful as well.
• fault-tolerant parsing: many RESTful services access resources from a variety of sources. support for fault-tolerant parsing of HTML and feeds (in any of the RSS variants and in Atom) makes it possible to implement more robust applications.

while i am sure that this list is incomplete, i am really interested to better figure out what it really takes to provide a useful REST programming toolbox. currently, i am mostly interested to figure out what would be of general interest to most/many RESTful service developers.

Comments

[Scott Wheeler]

I'd add:

- Tools for systematic testing

- Tools for documenting the APIs

- Ideally WADL support (not seen much uptake yet, but I like the idea)

[Cesare Pautasso]

- What kind of WADL support? Not sure Erik would like code generation :-)

What about:

- High level exception handling (to manage HTTP status codes more declaratively*)
- Higher level conneg
- HTTPS out of the box (no special Java VM Configuration to make it work)
- URL Extraction (to find links embedded in arbitrary content)

*http://soundadvice.id.au/blog/2007/07/08/#httpResponseCodes

[dret]

@scott @cesare: it seems to me that WADL was a nice thought experiment but was largely rejected by the community. which of course doesn't mean it couldn't be supported, but it might not be something many people would use.

wrt tool: yes, tools are good. testing is something i'd see a bit outside of the actual language toolbox for development, but it definitely should be part of each developer's toolbox. but what kind of "APIs" would you like to document, scott? according to REST, they all have a uniform interface, so most of the "documentation" is in the media types and in the interactions. but yes, there should be some support for systematically documenting the grid of HTTP methods and "URI templates".

HTTPS: definitely something i should have mentioned, HTTPS should be supported as part of the toolbox, and it should support the largest possible set of encryption methods. authentication (which i did mention) is useful and often required, but without HTTPS, authentication often i only one part of security. another thing one might add to HTTPS is the support of client-side certificates, which are optional in HTTPS, but very useful in scenarios where all participants can agree on a set of certification authorities they are going to trust.

[mamund]

i've been working on a similar list lately. mine is for talks to the .NET crowd, but it also applies to other communities. i break things out as follows:

- a URI dispatcher (able to parse URIs and route based on rules)
- a resource handler (able to accept routing, map HTTP methods to executable code, able to understand HTTP Headers as needed)
- a transformation library (able to transform input and output data into appropriate representations [XHTML, JSON, Atom, SVG, PDF, etc.)
- a full-featured HTTP client library (able to handle requests/responses to other HTTP sources like remote servers, etc.)
- a caching library (able to understand HTTP Headers, honor support outbound caching details and honor caching of remote server responses)
- an authentication library (support for Basic/Digest as well as HTTPS details. other auth formats optional)

FWIW, i've been creating this kind of library/framework for the .NET space for the last year or so (http://exyus.com)