NetKernel News Volume 1 Issue 17

February 26th 2010

What's new this week?

• A series of enhancements to detect and prevent commission-time space circularities.
• Detailed analysis of the insides of the Portal application.

NKSE Repository

NetKernel uses the ROC abstraction to bootstrap the commissioning of address spaces. The dynamic import capability means that spaces can be commissioned and discovered in any order. In architectures with multiple tiers of dynamic import it is therefore possible to have very exotic discovery. To prevent recursive behaviour there is circularity detection at commission time. The following enhancements were made...

layer0: detection and stable handling of misconfigured modules containing circular dependencies. Fix to HDSUtils.inOrderTraversalNext()

standard-module: Fixes to startup stability during space commission of certain multiple tiered dynamic import spacial configurations.

layer1: Changed hds aggregate/simple dynamic import to avoid circular dependencies.

Other changes...

kernel: changed some interfaces to enable thread profiling

nkse-control-panel: added pds config so that it doesn't rely on finding another in fulcrum

http-server: Just when you think you've got Jetty's encoding sorted out you launch a portal and get someone registering with an umlaut in their name. Turns out, that last week we fixed the encoding when Jetty was guessing, but now find that when the client is well behaved (JQuery AJAX) and provides the encoding on the Content-Type header, then Jetty actually uses it correctly and mangling is not needed. So this week we trapped this corner case too and relaxed the mangling. (Thanks to Thomas).

lang-xrl: Documentation typos fixed.

NKEE Repo

All of the above NKSE updates are available in the NKEE 4.0.0 preview-3.1 repository along with a feature enhancement...

nkee-dev-tools: Enhanced health check, now provides a space integrity check and reporting tool.

Portal Architecture

We've had some great feedback from people trying out the portal - we think we've fixed all the issues that were reported to us - so we'll be going public next week. If you've not tried it you can get to it (and join) here...

https://cs.1060research.com/csp/

We thought it would be interesting to provide some deep and detailed statistics of this codebase now that its in production. A high-level architectural overview is available here...

http://resources.1060research.com/docs/2010/02/CSPPortalArch.pdf

It shows the outline of the portlet pattern we're using and shows how we actually have a double level architecture - there is a second "1060-in-house" portal that dynamically imports a large set of 1060 portlets (70% of the system is "dark-matter" in house stuff). The 1060 portal is itself deployed as a portlet into the main services portal! Not shown are the infrastructural components: virtual host space, async HTTP transport, throttle, sessions, double-gatekeepers, decorating XRL template wrapper.

We've had a number of people asking if we can talk through the design in detail. So we'll host a webmeeting session for a deep dive through the code, as well as tour of the dark matter side of the system. Send us a support ticket expressing your interest and we'll arrange a time that covers the most timezones.

Incidentally if you wondered how to send a support ticket - the option is only visible when you have a support plan in place. However, you get complementary support cover when you download the NKEE distro. If you've not downloaded it then you may not have a support plan in place yet.

Finally some statistics, the following spreadsheet provides a very detailed analysis of the code, static resources, database, spacial topologies and development time...

http://resources.1060research.com/docs/2010/02/CSP-analysis.ods http://resources.1060research.com/docs/2010/02/CSP-analysis.xls

A few extracts:

• There are 14 modules to the solution deployed from a custom application repository with hot-updates
• There are 33 rootspaces with a further 38 spaces contained within.
• There are 192 application channels (= distinct REST/URL interfaces) of which 64 are AJAX micro-services (we use JQuery).
• There are 145 custom accessors all of which are written in groovy with the mapper used to bind them to the address space grammar.
• There are 49 tables in the RDBMS
• There are 1182 unique points where requests are issued.
• The average number of requests per application channel is 6.
• The average NetKernel response time (transport request to transport response) per channel is 10ms (network and browser rendering latency dominate) (The NKEE visualizer provides high resolution/high granularity request timing data).
• It took 1 person (guess who) 288 hours to develop with a 50:50 split between data model development (working out just what we needed and developiong the DB schema) and actual physical "coding" (144hrs - spacial architecture, pattern implementation, implementing channels, cutting resource endpoints).
• The development rate translates to about 8 endpoints per hour.
• The system was written from scratch since the start of this year.
• This week in production it has taken an average 5 minutes end-to-end to fix a bug, cut an update, deploy to repository, sync and deploy to production.(NKEE has ARP a repo building tool which you can do the same with).

Advertorial Alert: We are available to solve your problems. No job too small (after all, with ROC most jobs are small). Great rates and access to the horses mouth (is that a benefit?).

ROC News

Randy Kahle will be speaking at the Denver Open Source group meeting on Tuesday next week:

http://denveropensource.org/upcoming

Still no time to blog - sorry.

Have a great weekend.