NetKernel News Volume 1 Issue 25

April 23rd 2010

What's new this week?

• Core infrastructure tweaks.
• Exceptions are resources too.
• XRL tutorial.
• Thoughts on linked data runtimes.

Repository Updates (NKSE / NKEE 4.1.0)

Prompted by the NetKernel protocol development work, some minor tweaks have been made to the core infrastructure. These don't affect the user-side but were needed to ensure NKP can cleanly map the NK abstraction.

• kernel: Optimized expiry accumulation.
• layer0: INKFRequestReadOnly now has getHeaders() method.

standard-module: StandardTransportImpl now does not expose an arbitrary logical endpoint.

Other updates

• xml-core: DOMXDA transreptor updated to make SAX parse errors cacheable (see discussion below).
• nkse-doc-content: Added links for xrl-tutorial installation.

Exceptions are resources too, Caching Exceptions

Development of NKP is getting close to a public release (hopefully next week). We've been working through the many different error handling scenarios and ensuring that when they happen the connection and ROC abstraction 'do the right thing'. One design intent is that the NKP connection should handle errors and also, when the error condition is put right, it should transparently reconnect.

One such error scenario is a badly formed configuration resource. ie an XML config that will not parse. So consider a request arriving at the NKP client, if it is not yet connected to the server it must obtain its configuration, establish a connection and issue the request to the downstream side.

But, as I covered last week, the client endpoint is locally responsible for resolution requests for the remote logical endpoints. In this error scenario, with no configuration, you can never connect. But our requirement is to have the client automatically connect when errors are fixed. You can see that there is the potential for (hundreds) of repeated attempts to parse the broken configuration resource - which would be a huge waste.

Now you might think, ok, put state in the client endpoint and make it back-off on config errors. But that introduces hideous and escalating state management/thread safety complexity in the endpoint. It also misses a very powerful trick...

We say it over and over but maybe its implications aren't easily seen: Everything in the NK/ROC abstraction is a resource.

For example, we see that requests for resources from accessors are obviously resources. But so too are Requests (think of the pluggable overlay, the mapper configuration, the declarative request, f(g(x)) withRequest), Spaces are resources too (look at dynamic import configuration). Even, as I discussed last week in the comparison of NKP with HTTP, the Metadata that provides the structure to the system is a resource. So the NK ROC abstraction, to first and second order, is self-consistent.

Given the title of this section, you'll not be shocked when I also reveal that Exceptions are resources too. It might not feel like it since the NKF API is designed to make it feel like exception throwing and catching is innate to the language. But did you ever wonder how it's possible that in NK we are able to throw exceptions in an endpoint written in one language and catch them in a requesting endpoint written in another?

The answer is that an Exception is not special. Its a resource representation. You should think of an Exception as being that representation you obtain when the contextual state that is available to the requested endpoint results in an error.

So failing to parse badly formed configuration yields an Exception resource representation. Until someone goes and actually changes the configuration (by editing it, or changing the code that dynamically generates it), you'll always get the same representation every time you request it.

As you're becoming familiar with ROC, you'll immediately see the following logical consequence...

NetKernel caches all requested resources representations. Exception are resource representations. NetKernel caches Exceptions. NetKernel has a dependency model. The Exception resource can come from cache for as long as its dependents are not changed.

Therefore the computational cost of computing a resource that results in an error can be paid once, for all time, until the problem is fixed. This means hard-core production time error conditions do not impact the overall system performance.

Now, everything I've just said is true. Only its been turned round to show you the potential when you think of Exceptions as resources.

The problem with errors is that they are usually very significant when they occur at development time. So caching exceptions during development would make debugging a first class ticket to insanity.

For this reason, we've made it the default in all endpoints that exception representations are expired (not cacheable). Therefore, out of the box, any endpoint you create will behave just like what you expect and are familiar with in your chosen language.

But, if you're writing an endpoint and you know that your error condition will recur (ie like classical memoization it has referential integrity and only depends on the input state). Then you can very simply change the default and make the exception cacheable.

There's a detailed guide to the NKF API's for dealing with Exception caching here...

http://docs.netkernel.org/book/view/book:guide:physicalreference/doc:physicalreference:exception

If you've followed this line of thought then this is something you can introduce to your endpoints as an additional runtime optimization. One thing to beware of though. Languages don't really have any classification model for exceptions. Therefore you must be very aware if an exception you are handling is generated by an "externality" you must not cache it. For example, an IOException should never be cached since it may be tat the external cause is transient, therefore for you have no choice but to retry (externalities are not referentially transparent).

(Of course even with externalities, you can always add time or other dependent expiry functions in order to implement back-off patterns.)

So now you know why we changed the DOMXDA parser. It turns out that with one line of code...

try
{ ...
}
catch (SAXParseException spe)
{   aContext.createResponseFrom(spe);
}

The transreptor now issues cacheable exceptions for badly formed XML resources. And now the whole stateful NKP client scenario disappears. Bad config is sourced from cache and our NKP client's configuration can be completely dynamic.

[Incidentally NKP lets you throw exceptions between systems too - they're resource representations so it happens for free.]

XRL Tutorial, Thoughts on linked data runtimes

Brian Sletten reminded me that in NK3 we had a tutorial that teaches XRL. This prompted me to translate it and freshen it up for NK4. Its now available as an interactive tutorial - just go to apposite, sync and look for 'tutorial-xrl'. Or search the docs for "XRL tutorial" after you've got all today's updates.

I've mentioned a few times recently that XRL (XML Recursion Language) is a pretty powerful way to create composite resources by recursively evaluating resource references - in essence it effectively constitutes a linked data runtime.

Now I'm sure some people will get hung up on whether XML constitutes linked data (I hear distant crys of "Its got to be RDF dummy!"). But the truth be told, I couldn't give a stuff what forms you choose for representations*.

To the ROC abstraction, whatever your resource model, implementing a linked data runtime boils down to identification and composition of information resources.

At least, if you come from a Web/REST perspective, those are the two eigen-vectors. For example, a web browser is a linked data runtime. When it receives an HTML page it dynamically sources the CSS, Image, Script etc etc resources and forms a composite representation which is rendered into the browser's display.

I know I'll get a reputation soon, but did I ever mention that REST is not a complete model? The Web/REST is not complete because it makes a presumption that there is a single uniform global context in which we issue our resource requests. The Web is a model in which a single address space provides (implicit) resolution of point identifiers to point resource representations (in the Maths of Set theory it is a Bijection http://en.wikipedia.org/wiki/Bijection ).

Seems to me that the linked data community is aiming for something similarly rich but for systems not requiring (quite so much) human input. Something like a simplified model of what we have when a human and a web browser combine in the form of a stateful contextual endpoint. Notably, it is the human element that provides the context.

Now here's something that Tony and I have been living with for the past 7 years. Its a bit weird and certainly hard for the followers of the doctrine of the absolute canonical *World Wide* Web to be comfortable with. It turns out that identification of resources is relative. Or put another way, the names(symbols) that we use for things are relative to context.

In your gut you know this already. Your cellphone probably has presets for your Wife/Husband/Mother/Father etc. It has an address book in which the short name is used instead of the full international telephone number. When you want to call someone you find their name and hit call. The local relative name is resolved to the telephone number, the request issued to the local cell network, to the regional distribution network, to the international trunk network, back up to the regional network, to the cell and ping the request is resolved to the connected call. This is a simplified picture, but it shows an example of a relative contextual resolution in an extended, non normalized address space.

I could go on (and on and on) with examples of relative addressing in our everyday experience (see some of the ROC papers on the site for some more http://www.1060research.com/netkernel/roc/ ). But the point I've been building to is that for linked data to work you have to recognise the role of context. Better yet, you have to be able to dynamically adapt that context as you accumulate the state of the resource data too!

You'll see in the first part of the XRL tutorial, that it doesn't actually talk at all about the mechanics of XRL recursive evaluation. Instead it discusses how you can provide a context in which the XRL evaluation yields the resources you require.

NetKernel generalizes the Web and REST. It does it by making context a a first order eigen-vector which you can control and dynamically adapt...

Gentlemen, start your runtimes.

[* To open a can of worms I know I'll regret, my gut instinct tells me that tree structures seem to have about the right "dimensionality" for humans to cope with their creation and their consumption. Libraries are crammed with tree structured representations: we call 'em books. A book with an index is a tree containing its own inversion map - clearly also a very efficient data structure for humans. Machine efficient data structures should always be transreptable to human efficient representations (we prefer to code in Java and Groovy etc and not assembler).]

Another long one to fall asleep over. Thanks to those of you that gave me feedback last week - its nice to get encouragement and know that I'm not just whistling in a vacuum.

Have a great weekend,