NetKernel News Volume 1 Issue 27

May 7th 2010

What's new this week?

• http-server, nkperf updates.
• International Domain Name support.
• HTML5 WebSockets.
• NKP status.

Repository Updates

• http-server: Sorted out internationalization in the Jetty transport code

such that inbound URL is correctly translated to unicode identifier on request into the NK ROC address space (see below).

• nkperf: Fixed broken link to the jquery library.

IDNs Go Live

The internet took a big step this week with the activation of IDN internationalized DNS. There's a basic summary article here...

http://news.bbc.co.uk/1/hi/technology/10100108.stm

With full specification here...

http://www.ietf.org/rfc/rfc3490.txt

So domain names/URLs may now be unicode. What does this mean for NetKernel? Short answer, absolutely nothing.

NetKernel already treats all identifiers as unicode, not just host/domain names. Therefore there is nothing to prevent you from using rich internationalized resource identifiers in your ROC solutions - both for the external URLs (needs the http-server update) *and* for the internal address spaces.

In layer0, the declarative request, NKFRequest and Grammars also all treat identifiers as unicode.

res:/hello/مصر

(This will surely get mangled in the mail system, but it said res:/hello/"Egypt" in mixed ascii/arabic.)

HTML5 WebSockets

You may be aware that the HTML5 standards process includes a proposal for WebSockets. This is an extension to HTTP to allow an HTTP client to establish a bidirectional asynchronous communications channel between client and server. This is client initiated by an HTTP request asking to "upgrade" the connection to an open websocket channel. Once established a client and server may send messages asynchronously and each needs to implement event handlers. The protocol does not prescribe the structure or form of the message.

Given the work on NKP we thought it would be a good idea to catch up on the current state of the art.

The client side "API" spec is published here...

http://dev.w3.org/html5/websockets/

The protocol level spec is an IETF draft here...

http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-46

Reading up, its clear that this still has some way to go and there are a number of open issues, not withstanding the actual conceptual model. For example, Greg Wilkins, of Jetty fame, makes some very enlightened comments and good suggestions for improvements to the mechanics of the protocol...

http://blogs.webtide.com/gregw/entry/how_to_improve_websocket

We'll certainly implement support for Websockets in the ongoing evolution of the HTTP transport and HTTP bridge. It will be relatively simple to provide coverage in the ROC domain since we would deal with message sending/receiving using the same pattern we use for the JMS transport. In terms of the HTTP handling, Greg's Jetty 7.0.x already includes a prototype implementation of the low-level HTTP upgrade to a socket.

If you read up on Websockets, you might be left with the the question - does this really justify its name "Web"Socket? Its just a bidirectional pipe with a prescribed client side event handling API. Resources and resource oriented thinking don't get a look in.

From our perspective, its been worth reviewing the state of thinking. 

But we didn't find anything that can't be incorporated into NK's existing HTTP client/server stacks. Most importantly Websockets has no resource oriented heritage and will be used for certain push style architectures, probably together with HTML5's client-side worker threads. Its clear that websockets will have considerable server side scaling implications.

My guess is that Websockets will have minimal impact on the use of regular AJAX client-side HTTP requests which will still be the dominant approach for resource oriented web applications (ie what everyone does now) and will be used for the bulk of UI (DOM updating) patterns.

As far as our consideration in respect of NKP, there's nothing really to take away. NKP offers a generalized resource oriented protocol that allows client-side initiated bidirectional channels that exist within the full ROC address space. All will become clear when we finally get NKP out to you (see below).

NKP Status

We're making progress. The second generation asynchronous design is implemented and passing the earlier generation's unit tests. We're now optimizing the scaling and throughput to ensure that defaults for thread pools and queues are well matched, and, where its possible, making the underlying infrastructure automatically scale-up to meet demand.

Given how mission critical NKP is likely to be, we're not going to rush out a release until we've been through some exhaustive testing. We're at the stage where we are doing low level network fault simulation to ensure that the ROC/abstraction level will always provide consistency.

Watch this space.

Network Latency Simulation

Incidentally if you ever need to simulate network latency/jitter, packet loss/corruption/reordering etc then its built into linux kernel 2.6 with the "netem" filter...

http://www.linuxfoundation.org/collaborate/workgroups/networking/netem

Say you want to set up artificial 250ms latency and 5% packet loss on the lo local network (lo will be used when your server and test client are on the same host ie using localhost 127.0.0.1) then run these commands as root...

tc qdisc add dev lo root netem delay 250ms  //start filter + add delay
tc qdisc change dev lo root netem loss 5%   //add packet loss

To reset it to normal again do this...

tc qdisc del dev lo root

Have a great weekend.