NetKernel News Volume 2 Issue 16

February 11th 2011

What's new this week?

• Repository Updates
• NetKernel 4.1.1 on Windows Java 5
• NKEE Apposite 10x Performance Boost
• Death Star 2.0
• Automatic Negotiated HTTP Cache Consistency

Catch up on last week's news here

Repository Updates

The following update is available in the NKEE repository...

• nkee-apposite 1.22.1
  • Workaround for Windows Java 5 inconsistency. Also 10x synchronization performance optimization.

The following update is available in the NKSE repository...

• apposite 1.24.1
  • Workaround for Windows Java 5 inconsistency.

NetKernel 4.1.1 on Windows Java 5

Our support for Java 5 on Windows (J5W) came perilously close to being deprecated this week.

stefan144 reported on the forum, that he was having problems synchronizing and updating with Apposite on his J5W system.

This was really weird since we test on Windows. We'd also not heard any other specific reports of problems.

The symptom of the problem was a JDBC error - but this was not the cause. The difficulty was that we didn't see the problem on any of our test systems.

Eventually we isolated it to being a difference between Java 5 and Java 6 on Windows. J5W was working differently to every other Java/OS combination we tried. For the record, Java 5 and Java 6 on *nix were equivalent and so was Java 6 on Windows.

After two days of tracing the needle in the haystack, we figured out that it was inconsistent serialization/parsing of whitespace in text nodes in XML on J5W.

We put in place a very simple workaround. The updates to Apposite that you'll see in the repos implement this to ensure J5W behaves consistently with other platform combos. We've also slipstreamed the workaround into the NK downloads so that new users will be able to update if they have J5W platforms.

I hesitate to get on my high horse again. I'll just refer you to my previous frustration at Java versions taking liberties with representation consistency.

With a solution in place, I can make the non-announcement that we continue to support NK on Java 5 on all common operating systems.

That said, my advice would be that if you are working on Windows, you should consider moving to Java 6. Who knows what other inconsistencies may have been fixed? Oracle/Sun publish an End-of-Life statement for all versions. Java 5 Standard Edition reached "End-of-Life" status in October 2009.

Finally and for the record, there are no issues running NetKernel on Java 5 on *nix and we run a couple of production systems on Java 5 as our "canaries".

You may also want to know that NetKernel 3.x.x does not have repository updates and so is not affected by this issue and remains fully supported on J5W.

NKEE Apposite 10x Performance Boost

Reviewing the Apposite client in order to implement the workaround did have a positive consequence. We found a couple of optimizations. On the server-side we added a brief window of HTTP-level caching to the repository metadata to eliminate six unnecessary http requests (see below for general discussion). Also, given the fix we made to the SSL proxy-chain last week, we removed the use of a bespoke connection factory. We also found a couple of places where we didn't need to make requests and could transrept some pre-existing state.

The net result is that today's update of the NKEE Apposite client is about 10x faster at synchronizing. Blink and you'll miss it.

Death Star 2.0

A long time ago, in a village not far away...

After six-years living with his original home automation system, Darth Butterfield has revealed "Death Star 2.0"...

http://durablescope.blogspot.com/2011/02/home-monitor-reloaded.html

Tony has something of a Death Star fixation. In fact this has been one of the driving motivations for NetKernel and ROC.

Back in the day, shortly after he'd joined the team at HP, he proclaimed "You know the problem with Star Wars science fiction is not the technology. Its the information. You could never build a Death Star because the software complexity would kill it."

So now you know why NetKernel is so powerful - its the work of an evil genius.

Automatic Negotiated HTTP Cache Consistency

Did you know that NetKernel's HTTP stack automatically negotiates client and server cache consistency?

When you issue a response from your endpoint, the HTTP bridge does the following:

1. Checks the representation is not expired (ie its cached and remains valid in the server-side NetKernel cache)
2. If its not expired it computes an ETAG hashcode for the representation.
3. It sets the ETAG header on the HTTP response (note: no time-based cache directives are set by default - see below).
4. It streams the HTTP response back to the remote client.

We're basically saying to the remote HTTP client: "Cache this representation on the client side. In the future we can decide together if its life is longer than this one time request."

Any well-behaved HTTP client, will, when re-requesting the resource for which it is holding an ETAGed local representation, return the ETAG hashcode in the IF-NONE-MATCH header of the client-request.

At this point NetKernel does the following:

1. it always issue a resource request into the ROC domain.
2. very often it will immediately hit the cache and nothing will need to be computed.
3. it computes the ETAG hashcode of the server-side representation.
4. it checks it against the IF-NONE-MATCH hash
5. if they match NetKernel sends a 304 Not Modified response (no state has to be streamed).

This is very efficient.

The implication of this is that applications on the NK server-side don't have to resort to artificial "time-based" cache windowing.

The NetKernel server-side automatically provides realtime state consistency, and performs negotiated cache consistency between client and server sides.

This happens for free - you just don't have to think about it.

Timed Caching Directives

Sometimes, as in my example of the Apposite client, your server-side application can do even better and will know that the resource will be valid for a certain duration.

If you're so inclined you can set the appropriate timed expiration HTTP headers with the httpResponse:.

But web clients are not consistent and you may not cover all the bases. So we provide a simplified cover-all option. You can set the NetKernel response header HTTP_CACHE_EXPIRES_DELTA_SECONDS. Don't get your headers confused, we're not talking HTTP protocol level headers, NetKernel supports internal response headers.

When the HTTPBridge sees the HTTP_CACHE_EXPIRES_DELTA_SECONDS header on the ROC response it automatically sets the HTTP procotol-level EXPIRES and CACHE_CONTROL headers with a computed offset time, in seconds.

So as not to get confused by who's headers are who's. Here's an example of what to do in your code if you want to tell the remote HTTP client that your representation should be cached for X seconds (and don't come back before then!)...

/*
...Your impl here...
*/
resp=context.createResponseFrom(yourRepresentation)
resp.setHeader("HTTP_CACHE_EXPIRES_DELTA_SECONDS", 86400)  //1 Day

Which is all very API-centric. What if you have a more resource oriented view? Why bother writing code to do this stuff?

The mapper lets us set up NK response headers over a set of resources. Here's an example of a set of static images with client-side caching of 1 day. And an example of a dynamic service cached on the client-side for 2 minutes...

But remember, even if you don't set this time based cache directive, NK will still negotiate cache consistency with a remote client with ETAG/304's to automatically optimize network transfer costs. In fact even if you do set the time-based cache directive, NetKernel will still look at the ETAGs and not run any code, even should the browser return because the timeout has passed!

One final point. Switching perspective to thinking about NK as your HTTP client.

NetKernel's HTTP client automatically examines remote server side HTTP cache directives and, whenever possible will cache the representation in the NK cache with the best possible expiration function. Therefore, as in the case of the Apposite optimization I mentioned, an NK http client request is only made if we don't already have a valid resource representation in the client-side NK system cache.

NetKernel: optimizes the hell out of everything.

NetKernel West 2011

You think that stuff above is neat? You ain't seen nothin' yet. The conference will blow your mind. Find out all about it here...

http://www.1060research.com/conference/NKWest2011/

Have a great weekend.