NetKernel News Volume 3 Issue 20

April 20th 2012

What's new this week?

• Repository Updates
• *New* NKP LoadBalancer
• *New* HDS Recursion Language
• Map Reductio Ad Absurdum
• Help Wanted

Catch up on last week's news here

Repository Updates

The following new tool is available in the NKEE repository...

• nkp-loadbalancer 1.1.1
  • New NKP Loadbalancer (see below)

The following updates are available in both the NKEE and NKSE repository...

• http-server 2.15.1
  • Fix to null response from httpRequest:/method
• lang-hrl 1.2.1
  • New HDS Language Runtime (see below)
• layer0 1.81.1
  • HDSBuilder now tracks and reports depth of cursor node.

*New* NKP LoadBalancer

We're very pleased to announce that the NKEE repository now contains the nkp-loadbalancer package.

If you've played with NKP you'll know that it supports two styles. One-shot requests can be issued with the active:NKPRequest endpoint to a given server - in a manner which can be loosely considered as the ROC generalisation of HTTP. It also supports a dedicated mount-style pattern in which an NKP client prototype is instantiated and effectively provides a seamless distributed import of a remote server's address space.

This latter is very powerful since it means that a client-side solution has no need to even know that its requests are being issued to a non-local address space. It also means that all of the remote space's metadata resources are distributed to the client side and so resolution, documentation, interface model etc etc are available on the client side all of the time.

The NKP-LoadBalancer endpoint generalises this latter mount pattern. Instead of creating a dedicated mount of one space it acts as a distribution switch to an array of mounted remote address spaces.

As with all NK tools - the design of the NKP loadbalancer is to be a Unix-like dedicated component which can then be readily combined with other tools to create architectural patterns. As the documentation (below) shows, it is therefore very simple to set up, with just two endpoints, a dedicated NKP "network loadbalancing server". But, as we'll show below it is also just as easy to use the NKP-Loadbalancer to implement map-reduce architectures etc.

As with any NKP connection - cache coherence is preserved across the cluster.

Here's the details of the endpoint (below that we discuss the choice of balancing algorithms that are offered)...

Load Balancing Algorithms

When building distributed systems it is very apparent that one-size does not fit all. The need for architectural flexibility is supported by the NKP-Loadbalancer in the form of a choice of prototypes, each embodying a particular LB algorithm.

Round Robin

The Round Robin Load Balancer evenly distributes requests amongst available nodes in the load balancing pool in a strict rotational basis.

Concurrency

The Concurrency Load Balancer evenly distributes requests amongst available nodes by ensuring that each node receives the same number of concurrent requests to process and if multiple nodes are handling the same number of requests the one that has processed the least so far will receive the request ensuring an even workload.

Session Affinity

The Session Affinity Load Balancer allocates requests to a host within the load balance pool based on the value of request header "NKP_HEADER_LB_AFFINITY". Any value may be used but the value should remain constant for the requests which you wish to be routed to the same host. Internally a hash constrained to the pool size is generated from the header value and this determines which host to choose. If host becomes unavailable an alternative host will be chosen for subsequent requests until the host is available again.

Failover

The Failover Load Balancer distributes all requests to the first available host in the pool. If that is not available the second is tried, then the third, etc...

Licensing

The NKP Loadbalancer is a value added tool for use with NKEE. It requires its own dedicated license. If you're running the new utility license infrastructure it will run out of the box and the meter-client will obtain a metered license for it just like any other NKEE component. If you're using manual license files then please get in touch or raise a support ticket if you'd like to get hold of developer evaluation licenses.

*New* HDS Recursion Language

We're very pleased to announce another innovation today. The lang-hrl package is immediately available in the repositories. This package provides the active:hrl HDS recursion language runtime.

HRL is a new member of the family of recursive resource request languages. Following XRL and TRL. In this case it recursively evaluates HDS data structures for resource requests.

Requests may be in two forms. Just as with XRL/TRL they may be declarative requests embedded in-band in the HDS structure. But, since HDS is also a simple structure for POJOs then active:hrl will also find and issue INKFRequests embedded in the data structure.

Operational Attributes

A simple set of attributes provide the control for fanning out the recursion asynchronously, optionally terminating recursion and tolerant behaviour in the event of exception responses. Also, since HDS supports arbitrary POJOs you may also optionally indicate that you'd like to embed the full response object in the resulting tree - rather than just the representation.

Declarative Example

Here's an example that will look familiar if you know XRL/TRL...

INKFRequest Example

Here's an example using the HDSBuilder to create and issue a set of requests...

import org.netkernel.layer0.representation.impl.*

b=new HDSBuilder()
req=context.createRequest("res:/test/stuff/hello.txt")
req.setRepresentationClass(java.lang.String);
b.addNode("request",req);
req=context.createRequest("res:/test/stuff/hello.txt")
req.setRepresentationClass(java.lang.String);
b.addNode("request",req);
b.pushNode("b");
//Add an async request
req=context.createRequest("res:/test/stuff/hello.txt")
req.setRepresentationClass(java.lang.String);
b.pushNode("request",req);
b.pushNode("@async",true);
b.popNode();

req=context.createRequest("active:hrl")
req.addArgumentByValue("operator", b.getRoot())
rep=context.issueRequest(req);

resp=context.createResponseFrom(rep)

Install / Tests

To start using active:hrl just install the latest updates and the lang-hrl package with Apposite. After install the documentation is provided here:

http://localhost:1060/book/view/book:lang:hrl:book/doc:lang:hrl:title

The unit test module provides a set of examples which you can get as a module here...

http://resources.1060research.com/packages/2012/04/urn.test.org.netkernel.lang.hrl-1.0.1.jar

Map Reductio Ad Absurdum

You will immediately spot that when active:hrl is combined with the NKP loadbalancer then it is very simple to create fault-tolerant scalable map-reduce architectures. #PowerOfComposition

Tom's Blog

Tom's been at it again...

---

Today I got sidetracked into talking about ... logging. No, not swamp logging as you can see it on National Geographic, but even worse ... IT logging !

http://practical-netkernel.blogspot.com/2012/04/logging.html

As always, your feedback and ideas for future posts are most welcome at practical<dot>netkernel<at>gmail<dot>com.

Help Wanted

I've been told that the notice I posted in the last couple of newsletters was too subtle. So lets be blunt:

We're hiring. Get in touch.

Put simply, working with us will be personally rewarding on every level. But looking bigger - you'll be doing something even more worthwhile... how often do you get the opportunity to change the world?