NetKernel News Volume 2 Issue 12

January 14th 2011

What's new this week?

• Repository Updates
• DZone REST RefCard
• NetKernel West 2011 - Registration is Open
• *New* NetKernel Portal Framework

Catch up on last week's news here

Repository Updates

There are no updates this week. Steady as you are.

DZone REST Refcard

Brian Sletten has been busy again. His REST Reference Card has been published by DZone...

http://library.dzone.com/sites/all/files/refcardz/rc129-010d-rest_1.pdf

It includes a brief section on NetKernel, "one of the more interesting RESTful systems".

The DZone card seems to have been quite popular and has resulted in quite a lot of new people looking at NetKernel. If you're one of them, welcome to the NK/ROC Newsletter.

If you're familiar with REST and want to see how ROC and REST are related, you might be interested in reading the series of articles I posted last month, which progresses from scripting language runtimes through to discussing the relationship between REST and ROC. You might want to read this whilst also playing with the ROC Basics tutorial that is shipped with NetKernel.

Thanks for the mention Brian!

NetKernel Refcard

Incidentally, the DZone reference card series was the inspiration for the NK/ROC quick reference which you can download from the portal...

https://cs.1060research.com/csp/download/quickref

NetKernel West 2011 - Registration is Open

In case you missed last week's announcement, the NetKernel conference details are now available and registration is open...

http://www.1060research.com/conference/NKWest2011/

(Again, if you're new to NK, a one-day bootcamp precedes the conference and provides a fast-track into NK and ROC).

Looking forward to seeing many of you face to face in Colorado.

NetKernel Portal Framework

As promised, I set aside a day earlier in the week to rewrite the portal framework we use for the NetKernel portal. The result is a self-contained framework which cleanly accepts dynamically discovered portlets.

Carl Conradie has helped test this new framework and confirms that its compatible with portlets he was developing on the original version. Given its short gestation, I'll hold off from releasing it through the repositories just yet - we'll give it a week or two in case there are any refinements or enhancements needed.

In the mean time you can get it now here...

http://resources.1060research.com/packages/2011/01/nkpf-1.0.3.nkp.jar

...this is a single package of modules and it can be installed using Apposite's "Upload Package" feature, or just expand the jar and extract the modules and add them to your etc/modules.xml.

Included in this package is the portal framework module itself plus implementations of an example security manager, static styling and templates plus three example portlet implementations.

Trying It

Once you've got it installed you can try it at...

http://localhost:8080/portal/

You can login with several test accounts. One of the use-cases that has been added is the ability to give a portlet a group access policy. The example security manager has a self-contained set of demo user accounts as follows:

Each of the three example portlets belongs to groups: public, group1 and group2 respectively. So as you log in with different users you'll see different sets of portlets correspondingly.

Portlet Development

Writing a portlet is very simple and essentially it comes down to providing two resources in your address space:

• res:/etc/PortletTab.xml - provides the tab id, title and other metadata, it also now provides the group name for all endpoints in this space.
• res:/etc/system/SimpleDynamicImportHook.xml - this is your usual dynamic import hook. It needs to have the value nkpf.portlet to have the portlet sucked into the portal layer...

Examine the example implementations for a template to start your own portlets.

Content Wrapping

A portlet endpoint can produce any resource representations it likes. However if you want to create HTML and have it appear wrapped inside the portal template (ie in the lower-left frame below the tabs), then you give the response the following header...

WrappedNKPFPanel

Where the header value should correspond with the portlet id specified in PortalTab.xml

For example here's a mapper declared static endpoint that specifies this header plus another header to set the page title...

Of course you can also set these headers with procedural code with INKFResponse.setHeader()

Also, be aware that when you use the response wrapping it causes it to be evaluated in the XRL runtime - so you can easily put xrl:includes into your generated HTML blocks to source other resources etc.

Take a look at the example portals, it should be fairly straightforward to evolve them.

Architecture

I will write a detailed set of embedded documentation with the portal when we ship it. However in the mean time I've written quite a lot of verbose inline comments which should help with local understanding of how its working.

However it'll also help to have a sense of the overall architecture. The following diagrams show the spacial structure - click on the images to get the high-definition versions, the coloured regions are plug replaceable through dynamic imports...

Portal Structure (Front End)
[Click to expand]

The diagram above shows the primary architecture of the rootspace of the portal framework (found in the first rootspace of the urn.org.netkernel.pf.portal module).

Static Content

The first point of interest is the import of the StaticImports space which provides all the images, templates and styling for the portal. The static content can be dynamically imported with import hook nkpf.static - in the diagram you can see (highlighted in yellow) the example static content module's space is dynamically imported.

Session Overlay

The next endpoint of interest is the HTTP Session overlay which comes after the static content and which wraps all requests with a session: address space. The session: space is used to hold the user's login state.

Wrapper Overlay

Inside the session overlay we see a pluggable-overlay which has pre and post processes.

The pre process is used to ensure that the inbound resource request's representation is denormalized - it clones the request and sets its representation class to Object - which in NK terms means "give me whatever you've got". What the hell am I playing at?

This makes sense when you recall that way up at the top of the stack is the HTTP transport which issues its requests for IReadableBinaryStream representations. This is fine, but at this point in our architecture we can keep the entropy low by letting the end-user's portlet return whatever representation it likes.

We'll see in the post process that by not having HTTP transport's needs potentially force serialization (transreption) in the portlet layer below, it means we can very efficiently compose the resulting portlet resources into a wrapped composite resource without serialization/deserialization overheads imposed by the HTTP Transport's requirements.

Apart from the entropy management trick, the pre process essentially just relays the outer request into the inner address space (to the portlets etc discussed next).

The post process is more interesting. This endpoint receives the resulting response from the inner request. It examines the headers associated with it and if it has the header WrappedNKPFPanel it uses a pipeline process to wrap the returned HTML representation in the portal styling.

First it generates a user-specific view of the tab-bar corresponding to the group access level of the portlets. It then injects the tabs into the main HTML page template using the recursive XRL runtime. At this point any other xrl:includes etc that you specify will also be evaluated so that the full portal resource is composed in one seamless lazy pull.

The post process will transparently relay any response that does not have the wrapper header.

Inside the pluggable overlay, you can see there are the GateKeeper, WrapperImpl and 404 spaces. The WrapperImpl space contains the actual implementation services for the pre/post processes and can be considered a black box. The 404 space is the last item in the resolution order and it provides a safe fallback to catch any unresolved requests matching res:/portal/.* - and returns a polite 404 message, 404 error code - and since it is inside the post wrapper it gets nicely formatted by using the WrappedNKPFPanel header to trigger wrapping.

GateKeeper

The main branch of the architecture is now through the GateKeeper. Again this uses a pluggable-overlay and this time it implements a pre process - to perform access control on the request. Click on the next diagram to see the Gatekeeper space and the architecture that it contains...

GateKeeper Space containing Security and Portlet Dynamic Imports
[Click to expand]

The GateKeeper pre process needs to check that the user making the request has the correct credentials and group privileges for the resource they have requested. It uses a nice feature of NK4, every resolved endpoint has a global unique id.

By the time the pre process is executed, NetKernel has performed the resolution phase and has determined which endpoint inside the gatekeeper overlay will execute the request. The pre process therefore examines this endpoint id and, using some relatively advanced techniques, walks the superstack to find the portlet in which that endpoint lives.

Given it now knows the portlet, it can find the portlet's group policy declared in the res:/etc/PortletTab.xml.

So the gatekeeper knows the user and the group of the requested endpoint. It now makes a request to active:nkpfUserInGroup to determine if the user has the required group membership.

active:nkpfUserInGroup is not implemented in the portal module, its provided by the dynamic import of a security manager - which you can see in the diagram as the blue zone. This pattern is exactly the same as for the static imports earlier - only now your security manager module can provide implementaions of the service interfaces that the gatekeeper layer uses.

For convenience the demo provides an example implementation which you can easily customize to authenticate/validate against your own database or a corporate single-sign on system.

Portlets

Finally, the actual focus of the portal is revealed. Inside the gatekeeper is the import of the dynamically discovered portlets (shown in green). This is where you can concentrate on creating your web-apps which can be dynamically deployed by adding your portlet space to the system (with the required import hook and PortalTab.xml).

Dynamic Equilibrium

If you run the visualizer while playing with the portlet examples you'll see that everything is dynamically linked but after the first request it is pseudo-static and so everything gets cached. The net result is that whilst this is a dynamically reconfigurable architecture, the runtime cost is nearly nothing since nearly all the discovered structural state is cached.

As an example, the gatekeeper group lookup for a given endpoint is only done once for any given endpoint and thereafter that code never has to run again.

So the net result is that the computational cost of "operating the framework" is one-time and is ammortized across the lifetime of the system's deployment. As Carl Conradie commented before, that's why this portal is "insanely fast".

Feedback

Please give it a try and let me know if you have any ideas or suggestions. Once we've let it bed-down for a while we'll cut a production build and release it as a library through the Apposite repositories.

Have a great weekend.