NetKernel News Volume 1 Issue 22

April 2nd 2010

What's new this week?

• XRL gets full declarative requests
• Search accuracy improvements
• Documentation review/update.
• Linked-data and ROC.
• Heads-up on NK protocol.

Newsletter switchover: This is the final newsletter using the old mailing list technology. So, if you want to keep receiving these newsletters and you haven't already done so, register on the portal which will renew your subscription...

https://cs.1060research.com/csp/

If you're already registered on the portal and don't want to get this weekly newsletter, just uncheck "NetKernel Newsletter" in your portal settings https://cs.1060research.com/csp/settings/

[Not sure if you're already receiving from the new system? Emails from the new system have a footer with unsubscribe details].

Repository Updates

lang-xrl: For consistency and to enable extra capabilities we switched active:xrl2 to use the layer0 declarative request and changed xrl:resolve to be consistent with other tags. Changes are backwards compatible.

layer0: There's an enhancement to declarative request, with a new argument method "from-string" (see docs for details).

nkse-search: Search now correctly uses the same analyser as the indexer.

nkse-docs: We have created a hybrid "porta-stemming" analyser that specifically deals with the mixed content that is present in the doc system. System documentation search is much much more accurate, we've also weighted important concepts so that the key reference content has most "page-rank".

The following packages have all had editorial work to improve documentation, searchability etc... nkse-doc-content, lang-dpml, layer1

apposite: Now completely removes a repoository if you have deleted all of its collection sets. Warnings are given when deleting a collection that contains installed packages.

XRL - Linked Data Runtime

There's a lot of talk these days about hypermedia, linked-data, HATEOS etc etc. You'll have noticed we're pretty c**p at marketing, but maybe we need to shout a bit louder to get over the noise to discuss this pattern in ROC. In fact ROC generalizes the pattern beyond the REST perspective and enables "linked-resources" that may be reified by evaluating resource references in the relative multi-valent spacial context.

For example, and to be concrete, the XRL runtime (active:xrl2) is a recursive "linked data" reification engine. Each <xrl:include> statement is a full declarative resource request. Each reified resource is then recursively evaluated and it's linked resources lazily reified. The runtime lazily computes the composite representation embodied by the "linked-data resource" references in the requestor's spacial context. Every composite part is identified and therefore cached and so the whole is also cached, with a full dependency relation ensuring consistency.

XRL finds a concrete place to hang its hat by requiring that all resources must be transreptable to XML. But this doesn't mean XML is special, just that it provides a convenient uniform structure to work with.

Linked data patterns (with both XML and custom resource models) are used extensively in the implementation of the NetKernel platform tools/system even within the embodiment of the abstraction. For example the dynamic import pattern exploits this. The module management system. The spacial address space boot-strap process. The mapper. The xunit test framework. The documentation and search system. The apposite repository system. In fact any code resource that uses createRequest() is effectively a hypermedia resource.

Linked data is intrinsic to NK. It is so intrinsic and basic we never realized it was anything to crow about. We need to get out more. (Thanks Brad for hammering on the window!).

Development Heads Up : NetKernel Protocol

Not unrelated to the last section, we've successfully concluded the first stage of development of a new NetKernel-to-NetKernel ROC Protocol. The design goal is to provide transparent contextual ROC requests between NetKernel's and to go beyond the limitations of REST / HTTP's push-state transfer to support generalized ROC distributed clusters (I hate to use the word "cloud").

The core functionality is completed and includes full request bridging between NetKernel's. It includes a bi-directional distributed-superstack with resolution and request evaluation from downstream NetKernel back up to the upstream NetKernel request context. It also offers transparent cache dependency propogation across a cluster.

As a cheap trick, here's a test case we have running that demonstrates one possible use case: you have a top tier that needs to source a computationally intensive resource, you can load-balance requests for the resource over to a second tier compute cluster which provide runtimes for the task. The actual computation code resource does not need to be known in advance in the compute tier, it can be sourced on demand from the top tier (up the contextual distributed-superstack).

Because the NKP protocol transparently bridges ROC address spaces, the top tier application does not need to be changed to switch it from local to offloaded "cluster processing". Just import the compute tier's space instead of the local runtime's space. And the compute farm doesn't have to know about any code its going to be asked run in advance. "Contextual distributed code state transfer".

We have a little meta-level stuff to add to the system, so that remote endpoints are transparently bridged to the client side with full metadata etc, but essentially the architecture and core functionality is done. We will get something out for you to start playing with very soon!