NetKernel News Volume 3 Issue 6

The book is going steadily forward, the latest build contains an almost complete (need to cut off the rough edges) Chapter 7 on XRL. Can you solve the exercise in that chapter without looking at the solution ?

The latest build is available from:

http://www.netkernelbook.org/serving/pdf/practical_netkernel_nk5.pdf.

As always, your feedback and input is highly appreciated. You can send it to tom(dot)geudens(at)hush(dot)ai.

Next to the book I also started a blog which you can find at:

http://practical-netkernel.blogspot.com/

It is meant as a counterweight for Tony's "Durable Scope" blog and Peter's "ROC of ages" explanations in this newsletter. The entry level is "beginner" although I dare say intermediate level ROC'ers will find something useful in there as well.

The blog will have a weekly entry (this week it is about templates) together with the regular newsletter. If you have a NetKernel subject you always wanted to know about but were afraid to ask, let me know at practical(dot)netkernel(at)gmail(dot)com.

Attentive readers will have noticed the book changed name (again). All this is obviously going to lead somewhere ... so watch this space !

Design Overview

Broadly speaking the RESTOverlay is an endpoint routing switch for HTTP REST services/applications.

For a given internal resource request from the HTTPBridge, it will select and invoke a target endpoint based upon a best match with the external HTTP client's expressed preferences and the set of internal target endpoints.

Target endpoints declare their capabilities by specifying user-supplied endpoint metadata. This is ROC metadata and can loosely be considered as a richer generalisation of Java annotations.

Features

The RESTOverlay supports the following capabilities...

• flexible simple grammars
• dynamic content negotiation
• dynamic language negotiation,
• dynamic compression negotiation
• HTTP method routing
• automatic Etag generation
• automatic 404/406 handlers
• automatic exception handling - global or fine-grained
• targeted pre-processes
• lazily evaluated functional wrapping of target endpoints

The RESTOverlay manages the ROC spacial scope so that the caching semantics of the responses of targeted endpoints are conserved.

Architecture

The RESTOverlay is a companion to the HTTPBridge and must be implemented downstream of the HTTPBridge since it interacts with the httpRequest:/ and httpResponse:/ resource sets.

A schematic diagram of where the RESTOverlay sits and how it routes requests to target endpoints is shown below...

Instantiation

The RESTOverlay is instantiated from the prototype found in the urn:org:netkernel:tpt:http space. The broad structure of an instance will look like this...

The space contained within the RESTOverlay is the location where the managed REST endpoints should be present.

Configuration

The RESTOverlay requires a <config> parameter with the following values

• <basepath> mandatory the basepath from which managed rest services will be offset. All paths must start with "/". If the root is to be the basepath then use "/" alone.
• <auto404> optional automatically handle 404 responses for any request into the basepath that is not matched. If the tag has a value this is user-specified target id for a custom 404 handler (see below).
• <auto406> optional automatically handle 406 responses for found resources but with insufficient acceptability for the client. The tag must contain a user-specified target id for a custom 406 handler (see below).
• <globalExceptionTarget> optional the target id of a catch all exception handler endpoint (see <exceptionTarget> below for interface requirements)
• <strict> optional when specified it indicates that the RESTOverlay must use strict mode. In strict mode matching of target endpoints must be exact. The default is to be tolerant and find a best match target endpoint.

Endpoint REST Metadata Declaration

To make an endpoint a potential routing target for the RESTOverlay it must declare suitable metadata. An endpoint declares metadata by providing a <meta> tag on its declaration...

The RESTOverlay searches for a <rest> metadata tag within the <meta> declaration. For example the following endpoint declares that it wishes to handle GET requests for a REST sub-path with a simple grammar of "hello"...

Grammars

A target REST endpoint must always provide a <simple> grammar definition. This is a full simple grammar and so it may use the pattern matching and named arguments capabilities of the simple grammar.

The value of the <simple> tag is prefixed with the value of the RESTOverlay <basepath> configuration parameter to construct a logical endpoint.

For example if the RESTOverlay has a basepath of /foo/ and a taret endpoint has a simple grammar of baa the RESTOverlay will automatically create a resolvable logical REST service with the simple grammar...

res:/foo/baa

If the declared simple grammar has pattern matching fields, these are passed through intact. For example if you have a basepath of /foo/ and a simple grammar of {country}/{state}/{city} the RESTOverlay will automatically create resolvable logical REST service with the simple grammar...

res:/foo/{country}/{state}/{city}

Multiple Targets

You may provide any number of target endpoints with the same <simple> grammar. At the level of the RESTOverlay these are logically combined into a single logical endpoint with the common constructed grammar.

When a request is resolved by the logical endpoint's grammar the RESTOverlay will then start its matching algorithm to determine which of the internal target endpoints that share the same <simple> grammar should receive the request.

Internal "True Grammar"

Whilst many target endpoints can share the same external <simple> grammar - declared in the /endpoint/meta/rest/simple declaration. Each target endpoint must have its own unique endpoint grammar - declared as /endpoint/grammar. To distinguish these, lets call this the "true grammar".

The "true grammar" must be unambiguous so that a potential external request can be uniquely routed to that endpoint.

The RESTOverlay will detect if any target endpoints have ambiguous "true grammars" and will log a warning. It will also show that it is incorrectly configured in the space explorer. If you see this warning, you must locate the endpoint with the ambiguous true grammar and make sure it is changed to be unique.

Argument Relaying

The external REST service logical endpoint may have a simple grammar containing argument fields. If you use arguments in your <simple> grammar you must declare the target endpoint's "true grammar" with the corresponding named arguments so that they can be relayed to the target endpoint.

For example this target endpoint implements the /country/state/city REST path <simple> grammar and its "true grammar" accepts the same named arguments.

The external simple grammar arguments (country, state, city) are relayed onto the request to the target endpoint. So for example a request

res:/foo/USA/California/LA

would be resolved by the RESTOverlay and then internally routed as a request for

	active:CityService
		+country@USA
		+state@California
		+city@LA

Note: In this case we declared the "true grammar" using an active grammar syntax. You don't have to make your internal endpoints use active grammars but we recommend it since they are normalized and ensure that you cannot get confused between external logical REST paths and normalized internal ROC resources.

Endpoint REST Declarations

We have seen that meta/rest/ tag must contain a <simple> tag. The following optional tags are also supported...

Routing Options

<method>

Purpose

Indicates which HTTP method or methods will be routed to this target endpoint.

Tag Value

A comma separated list of one or more GET, POST, PUT, PATCH, HEAD, DELETE etc.

<produces>

Purpose

The RESTOverlay can perform dynamic content negotiation in order to locate a target endpoint that satisfies the expressed content requirements of the HTTP client (via the incoming HTTP Accept header).

The value of the produces tag must be the mimetype which this endpoint produces.

An endpoint must only produce one mimetype. Declare multiple endpoints with different <produces> tags if you wish to support multiple resource types.

Tag Value

A single valued mimetype.

<consumes>

Purpose

The RESTOverlay will perform dynamic content delivery for entity bearing methods (POST, PUT, PATCH) and will use the mimetype of the body content to select a target endpoint. The <consumes> tag indicates which mimetypes of the body content the target endpoint is prepared to accept.

The values of the consumes tag must be the set of mimetypes which this endpoint is able to consume.

Tag Value

A comma separated list of one or more mimetypes.

<language>

Purpose

The RESTOverlay can perform dynamic language negotiation in order to locate a target endpoint that satisfies the expressed language requirements of the HTTP client (via the incoming HTTP Accept-Language header).

Tag Value

The value of the language tag must be an HTTP standard language code. eg "en" or "fr" etc

<mustBeSSL/>

Purpose

The RESTOverlay will ensure that the requested URL was on the https: SSL channel. If it was not it will automatically 302 redirect to the same URL with https: scheme.

Content Processing Options

<Etag>

Purpose

If specified the RESTOverlay will automatically perform content hashing and associate an Etag header with the resource.

Tag Value

The algorithm to use for the hash. Must be one of MD2, MD5, SHA1, SHA256, SHA384, SHA512

<compress/>

Purpose

If specified the RESTOverlay will automatically attempt to compress the representation using a negotiated compression algorithm that is acceptable to the HTTP client. Implemented formats include gzip and deflate compression.

The HTTP Header Content-Encoding is automatically set as appropriate.

Processing Options

<exceptionTarget>

Purpose

The RESTOverlay will catch any exception in the main target request and issue a request to the specified exception handling target.

The target endpoint must have a grammar that accepts request and exception arguments. For example...

The request argument will be the INKFRequest which was issued to the failed internal target endpoint. The exception argument will be an org.netkernel.http.rest.RESTOverlayWrappedException which wraps the thrown exception.

A local declaration of <exceptionTarget> will override any setting of the <globalExceptionTarget> config parameter.

<preTarget>

Purpose

Before the main request to the resolved target, the RESTOverlay will issue a request to the endpoint specified by the preTarget identifier.

The target endpoint must have a grammar that accepts a request argument. The request argument will be the INKFRequest which will be issued to the the target endpoint.

The preTarget endpoint can perform authentication, validation, logon etc. If necessary it can make requests to the httpRequest:/ and httpResponse:/ to do any necessary process.

Response

The preTarget endpoint must return a boolean indicate if the RESTOverlay should proceed with issueing the request to the target endpoint.

<wrapperTarget>

Purpose

The RESTOverlay will automatically wrap the target request with a request to the wrapperTarget endpoint.

A wrapper endpoint must accept an operand argument. The operand is a pass-by-request containing the request to the primary target. Only when the wrapper actually SOURCE's its operand argument is the primary target request triggered.

This pattern is functional lazy evaluation and looks like this: w(f(x))

The beauty of the pattern is that before sourcing the operand, the wrapper is able to perform a pre-processing phase. After sourcing the operand, it can move to a post-processing phase.

The RESTOverlay carefully constructs a spacial scope so that requests to the wrapper are cacheable and have a dependency expiry. The net effect is that GET endpoints that generate a cacheable resource can be combined into a composite in a wrapper handler and the composite resource is also cacheable.

Response

The response from the wrapper is returned as the final response from the overlay.

Examples

The restoverlay-test package can be installed with apposite and provides a comprehensive set of examples of the use of the RESTOverlay.

The installed module is located as modules/urn.test.org.netkernel.tpt.http.RESTOverlay-x.x.x/

The unit tests can be executed here

The demo-addressbook package has a simple CRUD address-book that presents both a web-application interface and a REST service interface using the RESTOverlay for content negotiation, a decorated wrapping template pattern, compression and Etag generation.