Extensibility is key to creating APIs that not only work initially, but that also can evolve in terms of being appropriated for different contexts, and/or evolving over time. Extensibility is not the only pattern that robust and evolvable APIs should follow, but a necessary and important one.
In the interest of focusing on patterns and best practices, this post describes general requirements for an API to be robustly extensible, and will not make assumptions about a particular metamodel that's being used. In the old days, XML would have been a popular choice, and schema languages such as XSD or RELAX NG might have been used to (partially) define the extensibility model. Nowadays, JSON is a more likely choice, and since (so far) no schema language has gained widespread popularity, API designers are more on their own. In RDF land, while there are "schemas", they are not good at capturing the things that matter for extensibility, but the new SHACL language (currently under development) may change that.
Why not absolutely necessary for robust extensibility, it is assumed that the API uses hypermedia (one of the core constraints of the REST style), which means that extensibility not only applies to extending individual representations, but navigational affordances as well, so that for example additional links can be added without disrupting the application flow of those clients that do not know those "extensions of the navigable hypermedia space".
While not strictly necessary, it can be helpful to make extensions discoverable. This is what RFC 6906 calls "profiles", and this post can be seen as an illustration of how to use profiles for API and representation design. It is not necessary to make extensions discoverable through the profile mechanism, but the constraints explained in the following subsection are the exact constraints that RFC 6906 profiles must follow, and making them discoverable can be useful.
1. Meaningful Core Semantics
It should be possible to build meaningful applications using just the core semantics. More importantly, the core semantics must always apply and remain unaffected by any extensions. Only this way it is possible to have independent evolution of publishers and consumers (Mark Nottingham's "HTTP API Complexity" provides a good overview of possible scenarios).
1.1 Meaningful Core Clients
It should be possible to build meaningful clients just using the core semantics. The reason for that is simply that the core semantics are the minimal fallback semantics (see following subsection), and if these are so minimal that it's hard to imagine how anything useful could be done with them, then it means that the overlap between the envisioned extensions may be too small, and that a more focused set of use cases should be tackled.
1.2 Reasonable Fallback Behavior
It always must be safe for either side of a conversation to fallback to default behavior, meaning that it always must be possible to safely ignore extensions. Clients not knowing extensions simply ignore them, which is why this strategy sometimes is referred to as a "must ignore" policy.
The opposite, a "must understand" policy, means that extensions either by default are or can be labeled as mandatory, meaning that essentially they create a rift in the ecosystem of implementations supporting them, and those that don't. For organically growing ecosystems of implementations, this kind of rift has a lot of negative side-effects, most importantly it is hurting interoperability, and crippling the network effect.
2. Well-Defined Extension Points
Extensibility can be achieved in different ways, and to a large degree, the underlying metamodel of a representation already makes some ways natural and easy-to-achieve, and others rather unintuitive and not a natural choice. If an API uses a schema language (which also is more natural for some metamodels than for others), then the facilities of that schema language very likely decide on how extension points for a model are designed.
Regardless of the specifics (which metamodel is being used and if/how schema languages are used), the most important aspect is that it is clearly documented in the API which extension points it has. A good way to do that is to clearly spell out where and how exactly extensions can appear, and to rule out any other ways of adding extensions. As described in the following section, it also is necessary to make it clear what to do when "extensions" do not follow these rules.
It is very helpful for any API to have a set of test cases that test the extension points, both in terms of adding test cases that leverage all possible extension points, as well as adding some that do not follow the extension rules.
The reason why extension points should be well-defined is that this makes it possible for implementations to know what to expect, so that they discover extensions when present, and may even report them to higher application layers. For example, when writing a library for an API, then that library ideally should not just handle the core API, but should also support all extension points and at the very least report when extensions are present.
Well-defined and well-documented extension points also have the benefit of giving extension designers clear guidelines how to extend the core format, creating a more consistent landscape than if no such guidance was available.
3. Well-Defined Processing Model
While extension points are necessary, it is equally necessary to define what implementations should do about them. This is what a processing model is all about: Telling implementers how their implementations have to behave when encountering extensions.
One part of that are the "must ignore" and "must understand" policies mentioned already. Those are two different models of how implementations are supposed to behave when encountering extensions. The "must ignore" policy is the better way to go in a loosely coupled ecosystem, because otherwise every extension tightly couples those that support it.
If "must ignore" is picked, then a logical conclusion is that it must be disallowed that extensions change any of the core semantics (as mentioned already). Apart from just requiring to ignore extensions, in some cases it may be possible to have processing models for them, such as the HTML rule that unknown elements must be ignored, but their contents must be rendered. However, such a rule may be harder to define for the machine-processing in APIs.
As mentioned above, the processing model is what tells implementers not just where to expect extensions, but also what to do about them. In generic implementations (such as libraries), this may just mean to make those discoverable and accessible to applications, which can then figure out if they know the extension, and if so, how to process it.
In summary, having meaningful core semantics, well-defined extension points, and a well-defined processing model makes it more likely that an API or a representation can serve as a robust foundation for an evolving ecosystem of extensions. In a follow-up post, I will discuss the different models how to manage those extensions, describing different models such as complete decentralization, registry-based models for easier discovery, and completely centralized models.
Comments