NetKernel News Volume 4 Issue 3

January 25th 2013

What's new this week?

• Repository Updates
• New Second Generation Representation Cache
• Heap Tuning for Large NetKernel Instances
• Documentation Wormhole Tip

Catch up on last week's news here

Repository Updates

The following updates are available in both NKSE and NKEE repositories...

• lang-dpml-1.20.1
  • Enhancement to enable access to the Primary argument when writing scripts that receive SINK, NEW verbs.
• lang-ncode-1.8.1
  • Same enchancement as for DPML above.
• nkse-dev-tools-1.49.1
  • Changes to enable new cache views (see below).

The following updates are available in the NKEE repository...

• bootloader-1.20.1
  • Added a workaround to a low-level Java URL connection content type determination bug when a URL is handled by an alternate provider. Means that fileset resources from inside an sjar do not lose their mimetype.
• cache-ee-1.5.1
  • Second generation representation cache (see below)
• nkee-architecture-1.10.1
  • Updates to support 2nd gen cache tools.
• nkee-dev-tools-0.27.1
  • Adds new second generation cache visualization tools.

New Second Generation Representation Cache

We are very pleased to announce that we have released the all new NKEE second generation representation cache.

This release is a very significant enhancement both in terms of capability, performance and tooling. The new features include:

• Support for huge heap sizes
• Enables use of in-memory computing techniques with ROC solutions
• Automatic cache weighting based upon discovery of the natural caching layers.
• Rapid tracking and adaptation to modal changes in statistical distribution.
• Complementary cache culls to work with smooth asynchronous Java-level GCs
• Realtime "Cache Visualizer" tool suite including search and filter
• Cache layer hit rate reporting
• Cache efficiency metric (improves over naive hit-rate model)

Extensive details and the back story to this release are provided here.

Thanks to all the testers and users who have contributed to this release.

This isn't quite the final story - we've got one more extremely cool tool on the bench which we'll be releasing as a follow-on very soon...

Heap Tuning for Large NetKernel Instances

The new cache, like any engineering system, can be tuned to provide a balanced solution for your application and the load you are serving. Out of the box, it will default to a very lean and modest system - setting the max cache size to 10,000 items with a 15% heap headroom.

However, the new cache scales to tens of millions of items with extremely large (100GB+) heaps. However, when you go large its very important to also understand and take decisions about the tuning of the JVM garbage collection and heap management.

In his blog this week Tony has provided a very timely discussion of the Heap Tuning considerations...

http://durablescope.blogspot.co.uk/2013/01/heap-tuning-for-large-netkernel.html

Documentation Wormhole Tip

Here's a simple technique for accessing an image resource from within your own module and using it in your NK module documentation.

Make an entry for the image resource in your module's etc/system/Docs.xml file. Like this...

The <uri> is the resource's identifier within your module's address space. Note this does not have to be exported to the Front-End fulcrum - it only has to be resolvable within the same public <rootspace> as your other document resources.

This "document entry" will be aggregated into the set of all document id's and so be a known resource in the NK document system.

Doc Source Wormhole Service

The documentation system provides a useful public REST service /doc/source/[document identifier] which serves the raw binary stream of any specified document identifier in the doc system.

Its operational pattern is to lookup the full resource identifier for a given document id, it also discovers the resource's home address space. It then performs a wormhole request for the resource and serves it as a BinaryStream.

It follows that you can get direct wormhole access to images from deep inside your module.

So in a document, here's how we might embed "some-image.png" (as registered in the example <doc> entry above) in a document...

{image}/doc/source/doc:mymodule:some:image{/image}

It follows that you can use this service directly as the URL in an HTML image link too. Here's the same thing done with raw HTML...

<img src="/doc/source/doc:mymodule:some:image"/>

Content Type

This service works for any bitmap images. But of course these are just binary stream resources coming from your module - so you can use the same trick for any resource - for example an SVG image (or even source code, zip files, you name it - anything that's a resource in your module that can provide a binary stream).

The service will relay through the mimetype of the resource you serve - but if you're serving exotic stuff then you can help browsers along by including the file extension as the end of the registered <id>. In which case the /doc/source resource will be a public url that ends with the file extension and so will provide a clue for a dumb browser.

Internal SVG reference

If you use the {svg} macro to reference embedded SVG resources then, since this service is inside the ROC domain it can go direct to the internal address of the wormhole service res:/doc/source/xxxx. There's no need to go outside to HTTP. So for example you'd do this..

{svg}res:/doc/source/doc:mymodule:some:svg:image{/svg}

Where the SVG would have a <doc> entry like this...

Live Documents

All the examples above are for static files. But when you remember that in ROC everything is a resource - then there's nothing stops you from embedding "live resources" into your "docs" using this same technique...

When you embrace everything is a resource, you see that the doc system is actually just a resource composition runtime based on a wiki markup resource model.

You might be wondering why this discussion about documentation... watch this space - we've got something fun to share next week...

Video Links

If you like the video, you can spread the word by embedding it...

<iframe
	width="853"
	height="480"
	src="http://www.youtube.com/embed/9_LVMRRqCAQ?rel=0"
	frameborder="0"
    allowfullscreen="true"> </iframe>

or here's the direct link to YouTube.

Oh my - someone did get a new Mac, didn't they...