NetKernel News Volume 2 Issue 50

October 21st 2011

What's new this week?

• Repository Updates
• NetKernel 5.1.1 - Released
• <script> Tag Tip
• Spotted: A Really Steep Learning Curve

Catch up on last week's news here

Repository Updates

NKSE and NKEE repositories for NetKernel 5.1.1 are now available. New to these repositories are...

• demo-ncode-twitter 1.1.1
  • A twitter client written in nCoDE
• demo-ncode-dropbox 1.1.1
  • A dropbox client written in nCoDE
• lang-ruby 1.1.1
  • The Ruby language as an ROC language runtime.
• lang-ruby-lib 1.8.1
  • The Ruby 1.8 classlibrary as a mounted ROC resource module.
• lang-trl 1.1.1
  • Text recursion language runtime

NetKernel 5.1.1 - Released

We are very pleased to announce the immediate availability of 1060 NetKernel Standard Edition v5.1.1 and 1060 NetKernel Enterprise Edition v5.1.1.

Downloadable now from...

• NKSE: http://download.netkernel.org/nkse/
• NKEE: https://cs.1060research.com/csp/download/

Features

This release consists of a collection of refreshes and enhancements to various packages that comprise the distribution. It is a progressive evolution and at a fundamental level it is 100% compatible with the NK 4.1.1 system.

The key areas of enhancement of the core infrastructure include a new kernel build, new cache build and layer0 with the following significant enhancements:

• Optimization of the kernel cost and expiration aggregation data structures optimised for larger request trees
• Optimization of the kernel resolution algorithm
• Optimized cache keying
• Enhanced request resolution debug tracing
• New resolution trace infrastructure to support more consistent and detailed resolution traces
  • logical endpoint resolutions shown
  • reason for no match shown
• New Simple Grammar - for really simple pattern matching expressions.

As well as enabling some enhanced tooling, these core updates have yielded some quite significant performance benefits. We think you'll notice that the system just feels "even snappier" but it's also born out in the nkperf benchmark numbers. Here's the NKMark10 score for my laptop on NKEE 4.1.1 and 5.1.1 respectively...

Of course the NKmark is an artificial benchmark. However our experience is that the net effect of the changes is a very considerable performance gain on both NKSE and NKEE. The NetKernel Enterprise Edition 5.1.1 build features some additional optimizations and so will exhibit significant performance gains over NKEE 4.1.1 and over the basic NKSE 5.1.1.

System Enhancements

Above the core infrastructure several system tools and libraries have been updated:

• Brand new visualizer with many more tools and views
  • integration with space explorer
  • New "view callstack" view gives traditional point-in-time call stack for any request
  • New "view trace rooted here" view gives a way to manage big traces by just viewing a fragment
  • cachability traffic light indicator (with reason in request detail view)
  • request trace rendered with tree view with visible branches aiding navigation
  • faster performance with large traces
  • async/sync request icon
  • richer cross-linking with space explorer
  • convenience "clear cache" button
  • use of jQuery datatables for better layout
• Improved request trace tool using new resolution debugging
• Updates to the unit test framework
• xml-core drops legacy Java 5 support of XMLStream (see below) lang-dpml is updated due to this transition.
• HTTP transport update to use latest Jetty 7.5.3 with latest WebSockets support (coming soon - see note below).

*Jetty 7.5.3 - we have implemented the http-server with the latest Websocket support and it is passing all our tests internally. Unfortunately during live production deployment (we always switch our own servers to the latest NK before we release it!) we discovered what seems like a rare but consistent unclosed socket problem causing a cummulative file handle leak. We tried very hard to resolve this but in the end have decided that we would rather understand this properly than risk shipping an unreliable component. So for now the http-server uses the proven and ROC-solid Jetty 7.2.0 (with an older generation experimental websocket impl). A new http transport will be provided via the repositories once we have fixed Jetty.

Java 5 Officially End-of-Life: Please note this release marks the end of support for Java 5. Notably the XMLStream API has been removed from the xml-core library and while still supported the implementation relies on the built-in Java 6 APIs.

Documentation Refresh

This release also provides a general refresh and review of the documentation with the following updates:

• New readme and introductory materials
• Enhanced documentation covering caching
• New documentation on understanding and resolving memory leaks
• New ROC Patterns book

Migration from NK4 to NK5

Those familiar with NK's modular architecture will see that these packages could easily be deployed to an existing NK4 system to bring it up to NK5 spec. However, we have decided that whilst it would be possible to provide an in-situ migration tool to switch repositories and update an existing NK4 system, we think that a clean download and install is the better migration strategy and guarantees that you start with a clean and known system.

We strongly recommend you make the simple switch over to NK5 so that you will continue to receive the latest updates and features via the NK 5 Apposite repositories.

Enterprise License Migration

Please get in touch or raise a support ticket and we will be very happy to reissue NK5 licenses for your migration from NK4.

Library Compatibility

Your existing applications and libraries will run on NK5 with no changes. Indeed, the majority of the existing NK 4.1.1 user-level library packages are unchanged and are now provided in the NKSE 5.1.1 Preview repository.

The detailed list of specific packages that have changed between 4.1.1 and 5.1.1 is as follows:

• apposite-1.25.1.nkp.jar
• bootloader-1.18.1.nkp.jar
• cache-ee-1.5.1.nkp.jar
  • OPTIMIZE: representation cache (lower CPU use and better cache recall in some corner cases and cost threshold to discard cheap representations)
• database-relational-1.11.1.nkp.jar
  • FIX: relational database pool will incorrectly create duplicate pools under some situations
• http-server-2.10.1.nkp.jar
  • New RememberMe cookie implementation
• json-core-1.5.1.nkp.jar
• kernel-1.19.1.nkp.jar
  • OPTIMIZE: resolution failure exceptions only formatted on demand (over 100 RFE during out-of-box boot)
  • OPTIMIZE: kernel cost and expiration aggregation data structures optimised for larger request trees
• lang-dpml-1.16.1.nkp.jar
  • FIX: Non evaluated SOURCE of non existent arguments in DPML script would incorrectly throw exception
  • Now uses Java 6 compatible XMLStream parser
• lang-ncode-1.4.1.nkp.jar
• layer0-1.71.1.nkp.jar
  • Declarative request support for relaying requested representation by default
  • FIX: logging with null message would break log file formatting
  • FIX: mapper now relays requested representation unless overridden in declarative request
  • OPTIMIZE: validation cache (used by all system configurations with schematron validation) persisted for increased boot speed
  • NEW: Simple Grammar - enables really simple pattern matching grammars.
• layer1-1.33.1.nkp.jar
  • Layer1 fls (filesystem list) accessor to support returning empty directories
• module-standard-1.50.1.nkp.jar
  • New <literal> tag in standard module to support literal resource declaration
• nkse-control-panel-1.20.1.nkp.jar
• nkse-cron-1.11.1.nkp.jar
  • FIX: CRON management view pause/resume buttons not working
• nkse-dev-tools-1.36.1.nkp.jar
  • Updated Configure NetKernel tool with additional kernel.property values and defaulting of new values
  • Logging view now shows "debug" icon for debug level log messages
  • Asynchronous news feed with new styling on management console homepage
  • Deployment editor now supports direct adding of a filesystem module (no need to edit modules.xml)
  • FIX: Double newlines appearing in scripting playpen on windows
• nkse-doc-content-1.36.1.nkp.jar
  • Lots of updates - rewritten readme and getting started.
• nkse-doc-patterns-1.2.1.nkp.jar
• nkse-docs-1.16.1.nkp.jar
• nkse-http-fulcrum-backend-2.3.1.nkp.jar
• nkse-http-fulcrum-frontend-2.2.1.nkp.jar
• nkse-license-1.6.1.nkp.jar
• nkse-visualizer-1.13.1.nkp.jar
  • New complete refresh
• nkse-xunit-1.7.1.nkp.jar
  • Now clears cache before and after running a test-set but does not expire requests within a test set. This change allows for the testing of distributed caching patterns.
• photonk-0.5.1.nkp.jar
• security-core-1.4.1.nkp.jar
  • New GeneratePasswordHash/CheckPassword Hash accessors
• system-core-0.29.1.nkp.jar
• xml-core-2.1.1.nkp.jar
  • Removed legacy Java 5 impl of XMLStream API.

<script> Tag Tip

One thing I noticed with migrating our production applications to NK5 was that I had been getting away with HTML murder. Or rather dodging an HTML bullet. You may know that self-closing (coalesced) tags are not well handled by modern browsers. In particular most browsers fall over when they see a script tag like this...

Its a long story and very painful, but is explained in depth here...

http://stackoverflow.com/questions/69913/why-dont-self-closing-script-tags-work

But in short we server-side folk just have to live with it and not use coalesced tags. Now here's how I was dodging the bullet (and how it finally got me)...

The DOMSerializer transreptor will automatically use non-coalesced tags for <script> when the DOM representation has a mimetype of "text/html".

Since we use XRL to build our composite apps - then we work in the tree-structure domain and then do a final serialization.

To date, I had been lucky that the transreption was being triggered in the outer layer of my architecture when the mapper we use for the indirection of the service interfaces had already set the mimetype to text/html. But actually this delayed transreption was formally incorrect! In NK5 the transreption now correctly triggers before leaving the mapper layer and therefore before I had set a mimetype of "text/html". Which meant the transreption was coalescing my script tags. Which browsers absolutely refuse to touch.

Now, you could take inordinate care over the order of your composite processes and the setting of appropriate mimetypes between each layer. But there's a much simpler and fail-safe tip. Always declare your script tags like this...

(Notice the underscore keeping the tags separate) This works in either XML or text domain and will always work with browsers.

Spotted: A Really Steep Learning Curve

We've worked hard on NK5 to keep chipping away at the learning curve. As we repeatedly say - NK isn't difficult its just different. Just go with the flow and it will definitely click.

But as some encouragement if you're self-teaching, it could be worse - a lot worse. Take a look at this Crochet for Beginners pattern...

http://www.guardian.co.uk/lifeandstyle/2011/oct/18/make-crochet-apple-jacket

The first line of code is utterly baffling...

"Round 1: ch1, 9dc into central loop, sl st into 1st chain to close up the circle."

Incidentally what the ****** heck is an apple jacket? I thought it was for an iPod