NetKernel News Volume 3 Issue 5

November 25th 2011

What's new this week?

• Repository Updates
• Kernel Updates: HEADER_EXCLUDE_DEPENDENCIES / Thread-Safe Deadlock Detection
• New: active:xpath, nCoDE - while/until endpoints
• RESTOverlay Available for Review

Catch up on last week's news here

Repository Updates

The following NetKernel 5.1.1 updates are available in the NKSE and NKEE repositories...

• kernel 1.22.1
  • Added support for new HEADER_EXCLUDE_DEPENDENCIES (see below)
  • Added detection of a self-deadlocked non-thread-safe endpoint (see below)
• lang-ncode 1.7.1
  • Added While and Until endpoints - kindly contributed by Tom Geudens.
• layer0 1.76.1
  • Updated to reflect kernel updates
• nkse-dev-tools 1.42.1
  • Fixed CSS so that Space Explorer and Visualizer now look consistent in Safari/Chrome on OSX
• nkse-doc-content 1.40.1
  • Added doc for HEADER_EXCLUDE_DEPENDENCIES
• xml-core 2.3.1
  • Added a new active:xpath1 endpoint - kindly contributed by Tom Geudens.

Kernel Updates: HEADER_EXCLUDE_DEPENDENCIES / Thread-Safe Deadlock Detection

Sometimes the requestor of a resource might want to impose their own decision about the cacheability of the resulting response. For example, suppose you make an HTTP request to a POST service - but rather than being always expired, you might like to hold on to the response for a period of time?

This scenario recently came up for Glenn Proctor (as discussed on the forum) and others also then described that this was a pattern they also experienced. There were various discussions of ways to implement an intermediate layer to decouple the mixing of the expired resource into the dependency model of a higher order resource.

This was all a bit fiddly. So this week we've added a simple update to the kernel to support the HEADER_EXCLUDE_DEPENDENCIES header. This header, when set on an issued request, causes the kernel to exclude any subsequent expiry determinants from the set of expiries being accumulated for the requesting endpoint.

INKFRequest request = context.createRequest(...);
request.setHeader(INKFRequest.HEADER_EXCLUDE_DEPENDENCIES, true);

The requesting endpoint may now issue its response and those requests which had excluded dependencies will no longer mix in to the expiration determinants of the response. The net effect is that the requestor can provide an expiration dependency "glass-ceiling" above which its semantics for expiry become definitive.

Thread-Safe Deadlock Detection

Back in September I explained how the thread-safe declaration of an endpoint can accidentally result in a recursive indirect deadlock.

With today's update, the kernel now implements a self-deadlock detection feature such that if an endpoint is currently executing a request and the same thread that was issued into that endpoint re-entrantly tries to request the same endpoint (an extrinsic recursion) then an exception is thrown, which should take care of accidental issues for undeclared threadsafe endpoints.

New: active:xpath, nCoDE - while/until endpoints

Tom Geudens has kindly contributed some useful tools that have been released into the repos this week.

active:xpath

Firstly, the xml-core library adds a new utility tool for slicing and dicing XML based upon XPath 1.0 expressions. It implements the same features as the xml-saxon active:xpath2 tool but for the original XPath 1.0 feature set.

Here's the documentation...

nCoDE while/until

Secondly, in lang-nCoDE there are two new built-in endpoints - providing support for While and Until loops. The components appear in the "Built-in" palette and look like this...

The condition is sourced as a Boolean for each do of the loop (either prior or post the do respectively). The tolerant argument indicates whether to tolerate exceptions in the do process.

These contributions nicely demonstrate what I discussed, last week - that DPML is a language with no functions - both of these "functions" are just endpoints which are now included in the lang-ncode module. If you wished you could use them as standalone services just by importing lang-ncode as a service library.

RESTOverlay Available for Review

As initially discussed last week, we've implemented a new RESTOverlay. It is a companion to the HTTPBridge and performs the role of a dynamic router, selecting a target endpoint based upon the HTTP-client's preferences and the declared REST capabilities of the targets, expressed as user-defined metadata.

An overview of the architecture is shown below...

The blue boxes show some example fields for the rest metadata. In this example, the endpoints are differentiated by the HTTP method, while both are candidates for the same logical resource (res:/basepath/foo) since they have the same simple grammar declaration <simple>foo</simple>.

The RESTOverlay implements the following features...

• <simple> Full simple grammars with pass-through arguments to targeted endpoint
• <method> HTTP method/verb routing
• <produces> dynamic content negotiation using client's Accept header
• <consumes> Body mimetype target selection on entity bearing methods.
• <language> dynamic language-based content negotiation
• <mustBeSSL/> Bounce to SSL if the endpoint says it must be trusted connection.
• <compress/> Dynamic negotiated compression based on client capabilities (gzip, deflate).

The RESTOverlay takes a <basepath> and an optional <strict/> tag as its config. By default the routing algorithm is tolerant and will always find a target endpoint even if the parameters are not strictly matched. Strict mode will fail with a 406 response code if an exact match to the client's specified requirement cannot be determined.

An <auto404/> configuration option causes the RESTOverlay to create a catch-all 404 handler endpoint as the endpoint of last resort which has a catch-all grammar rooted at the basepath. If it is resolved it sets the necessary 404 response code.

If the managed target endpoints have ambiguous grammars the RESTOverlay will incidate that it is not correctly configured. You will have to find the reported endpoint and ensure that it has a unique grammar so that the RESTOverlay is able to unambiguously relay a request to it.

The RESTOverlay is semi-transparent. If its' wrapped space contains endpoints which do not have metadata suitable for the RESTOverlay (ie no meta/rest/ declaration) then they are transparently relayed through the overlay as though it were not there. This allows you to freely mix managed-REST and non-REST endpoints in the wrapped space as required.

Download

We've posted a new version of tpt:http module incorporating a preview of the RESTOverlay here...

http://temp.1060research.com/2011/11/urn.org.netkernel.tpt.http-2.14.1-RESTOverlay.jar

the tests are here...

http://temp.1060research.com/2011/11/urn.test.org.netkernel.tpt.http-2.14.1-RESTOverlay.jar

Install

The download jar is a module and it directly replaces the existing tpt:http module that comes with NK. To try it out, in modules.xml, comment out the current one and add a reference to the new one. In lieu of detailed docs, the unit test module is self-contained and provides full coverage of the capabilities so far.

Performance

The adaptive routing algorithm that we've devised does a sorted-sift of the candidate target endpoints. It maintains priority order based upon the client-side weighting preferences (Accept, Accept-Language, Accept-Encoding etc).

The Overlay is dynamically configured and automatically reflects changes if you add endpoints, change metadata etc etc. Configuration change happens out-of-band and thereafter the operating set is semi-static. The net result is that the overlay is really light-weight. After repeatedly running the tests I'm seeing an average response time of just 0.12ms - so effectively the RESTOverlay has negligable cost.

All resources which can be cached inside the target space will be cached as normal. When an http request comes in then an eTag will be set and if re-requested and it's still cached a 304 response is issued. (This is just the usual HTTPBridge behaviour - all the RESTOverlay does is make sure the caching boundary of the app is consistent).

Feedback

The core features of the tool are now pretty much done and are solid. Give it a go - let us know how you get on.

Still To Come

• User supplied 404 and 406 endpoints to allow the user app to supply handlers for these error conditions.
• User supplied pre/post-process endpoint target - for arbitrary pre/post-process handling.
• User supplied exception handler endpoint target
• MD5/SHA1 eTag hashcode generation.