NetKernel News Volume 5 Issue 10

June 13th 2014

• Repository Updates
• Update - NKP Resolvable Filter
• KBO - Belgium Business Linked Open Data
• Tip: Dynamically convert Fontawesome 3.x to 4.x
• XSS - How NetKernel makes it hard to be as dumb as Tweetdeck
• NKP 3-Party Delegated Trust Demo

Repository Updates

We've released an enhancement to NKP in the repositories. It adds a new NKP Resolvable Filter to cover a scenario we encountered when creating the 3-way trust demo (below). We discovered that there was a potential access-control paradox caused by the symmetry of the NKP client/server relationship...

By default, NKP prevents any request resolving to an endpoint that is marked as private.

It is therefore relatively simple to prevent external NKP requests accessing complete sets of endpoints simply by importing them into your NKP server/client space and marking the import as private.

However the symmetrical nature of NKP means that sometimes it is essential to have a client-side endpoint that is public but which you do not wish to be accessible to a server-side initiated NKP callback.

In order to add an additional level of protection to your address spaces you may enable the <NKPResolvableFilter> on either the client and/or the server side of the NKP connection.

When the NKP Resolvable Filter is enabled it will only allow NKP requests access to public endpoints that have explicitly declared that they should be resolvable to NKP requests. Endpoints may mark themselves as NKP resolvable by supplying the following endpoint metadata...

The NKP Resolvable Filter is symmetrical and can be deployed on both server and client sides. It may be enabled on the server-side to add extra constraints for the resolution of external NKP requests. Equally, if the request scope is exposed, it may be enabled on the client side (both one-shot and mounted modes) to constrain resolution of server-side initiated NKP callback requests.

During development of distributed systems it is generally sufficient to simply use the default protection of the <private/> filter. But when allowing callbacks from untrusted parties it is advisable to enable the NKP Resolvable Filter to miminize and make auditable the exposed surface area of your client-side address space.

NetKernel has been the platform for the Dutch government's open linked data services for a while now. It's therefore exciting to report that NetKernel is now also powering a new linked data platform for Belgian Business data (KBO).

In his blog, Tom Geudens provides an introduction...

http://practical-netkernel.blogspot.co.uk/2014/06/presenting-kbodata.html

Great stuff Tom, Paul and Benjamin. And Winston Churchill would appreciate the KBO acronym.

If you did what I did, you may have liberally scattered fontawesome icons into your content (ie these newsletters) and all of a sudden, with fontawesome version 4, they've completely changed their namespace and so all my icons disappeared!

For example in 3.2.1 a class of ...

icon-star

has, in 4.1.0, become...

fa fa-star

I suppose I could just stick with 3.2.1 but that seems regressive. But there's no way I'm going to go back through every newsletter and change them. So here's what I did, I added this snippet of jquery javascript as a quick fix for legacy pages, and my new content will use the new fa classes. All sorted...

$(document).ready(function() {
  //Fix font-awesome 3->4 compatibility.
  $('[class^=icon]').each(function ()
  {	
  	v=this.getAttribute('class').match(/icon-.*/g);
  	v=new String(v);
  	v=v.substr(5);
  	$(this).addClass('fa');
  	$(this).addClass('fa-'+v);
  }
  );
});

The exposure this week of Tweetdeck's (really Twitter's) XSS vulnerability was a pretty big screw up.

So I thought it was timely to point out an article and set of demos from 2011 that discuss how, with ROC, it's simple to completely and provably eliminate XSS vectors...

• Part 1: Cross Site Scripting Attacks (XSS) a Resource Oriented Perspective
• Part 2: Cross Site Scripting (XSS) Vectors Demo/Discussion

The demo I refer to is still there in the Apposite repos and is still just as relevant. To get it install...

demo-xss-vectors

And after install hit this link for the detailed guide.

It's noteworthy that in the introduction doc for the demo I say...

"One of the innate weaknesses of defensive technologies, such as Web Application Firewalls, is an assumption that the only vector of attack will be from the client-side (browser). But with increasingly coupled services and/or data layers, XSS is just as much an issue when using state from the data-side (RDBMS, NoSQL store, RESTful data mashup services, etc etc)."

Twitter/Tweetdeck users were exposed to exactly this form of data-side attack!

Put simply: You have a choice of resource representations. Working in the serial domain is the fools choice.

Last time out, we announced the release of the One way trapdoor overlay. It seemed like a good idea to create a demo showing how it can be used, but also, to provide a clear example of some of the power that NKP offers to traditionally complex N-way distributed systems.

There's a new demo in the NKEE repository called demo-nkp-notary which you can install from Apposite.

This demo provides an example of a 3-party interaction. Traditionally this problem is complicated by the need to write code to correlate bi-directional messages whilst also managing the mediation of trust between the parties. It can be seen from the simple 3-party problem that as the number (N) of parties increases, then traditionally, the total complexity increases as N-squared.

It it is shown in this demo that when using ROC (distributed over NKP protocol), the complexity of N-party interactions only increases linearly. Whilst at the same time, acquiring very significant engineering qualities such as provably auditable trust boundaries, delegation of authority, linear scalability, very high network fault tolerance and distributed cache coherence. All of this, simply from the configuration of ROC address spaces to use the existing NKP infrastructure, and with effectively no code.

Demo

After installation the demo can be executed here.

The demo is very boring. There's really nothing to see - all that you get is a signed purchase order receipt. The best way to experiment with the demo is to run the visualizer to see the requests.

The demo is implemented in three modules. Each module corresponds to the the 3 party's discussed below. For convenience all modules are running on the same box (your NK instance) but the spaces only communicate via NKP connections running over the local network.

The following discussion explains what's actually going on....

Scenario: Delegation to a trusted Notary for non-Repudiation of Transaction

Consider a transaction between a Buyer (Party A) and a Seller (Party B). Party A wishes to issue a purchase order to Party B. Party A does not trust Party B and so requires irrefutable evidence that Party B received and accepted the order.

Party A chooses to appoint a Notary - an independent 3rd party witness to the transaction, such that if Party B fails to fulfil the order, Party A has a legally independent witness that Party B has violated the agreement.

Using traditional message passing code and request response communication (shown above), A would make a purchase order request to B. B might respond with a provisional (but not final) acceptance of the order. A would send back a response indicating that they had received and acknowledged the provisional order acceptance. B might now create a final order incorporating A's acknowledgement. A would receive this final order confirmation and, since they require irrefutable evidence of the order, would send the order to the notary to be signed. A would receive the notarisation and then send this to B - so that both parties have the same proof for the transaction. B will keep the proof and finally indicate to A that the transaction has completed to the satisfaction of all parties.

This is a very common scenario. It is also a simple 3-way interaction. It's easy to see how N-way interactions get more and more and more complex. Worse still if you approach these problems by starting with code - it becomes harder and harder to assert any confidence in the trust within this system. Its very likely that zero-day vulnerabilities will be left in the system.

Furthermore, the complexity of the problem is further increased by the need to deal with the possibility of failure during any stage in the process. Only by sending and accepting acknowledgement messages are the parties able to be confident that the overall system is progressing the distributed transaction. It is clear that this greatly increases the complexity and total time to perform a given transaction. The more the complexity and the more interactions - the higher the likelihood that the system will fail.

Contextual 3-Way Distribution with ROC

With ROC we have an abstraction that lives above the low-level code (and as will be shown, the network communication). It allows us to model the 3-party relationship simply as a configured architecture of three interacting address spaces.

This allows us to create trust relationships in which the untrusted parties can interact with trusted parties within a constrained and auditable spacial context. A coherent envelope of trust provided by logical requests ensure that the transaction either completes or fails but no acknowledgements are required to coordinate the transaction - massively reducing the complexity and increasing the confidence in the trustworthiness of the solution.

In addition, since ROC separates logical resource requests from physical execution, the system is extremely robust and tolerant of transient failures.

The diagram below shows how the 3-party problem can be solved in ROC.

We see that the 3 parties each have a resolvable ROC address space. Party A may issue purchase requests to Party B. The system is configured so that Party A also has trusted access to the Notary's address space (via an NKP import).

We see that when Party A makes the purchase request to Party B, the transient spacial scope for the request becomes distributed between the two parties.

Party B will receive the purchase, but it has no need to issue an intermediate response to A, since it has not yet completed the transaction to the satisfaction of A.

Instead B issues an additional request (starting in B's address space) for the notary service to notarise B's purchase receipt.

The distributed spacial scope means that the notarise request is not resolved in B's space. Instead it goes up the superstack to A's space, where it is resolved by the Notary service and the request is issued to the Notary service in the Notary's address space.

Just like with an import, NKP enables the Notary service to be resolvable as a logical endpoint in A's space!

The notary responds with the signed notarisation. The response is returned (via A) to B. B receives the fully signed notarised response, keeps a copy for its records and finally completes the transaction by responding to the original request (which has been pending throughout and therefore providing the contextual envelope for the distributed transaction).

The entire solution consists of just three endpoints and two requests (three requests if you distinguish B's superstack notarisation request and the NKP mediated delegation of the request to the notary endpoint - but this happens under the hood without any user-level code).

The complete log of the the requests are shown below (the requests are colour coded - so Buyer is black, Seller is blue and Notary is green).

The NKP requests have an index number in square brackets. Client-to-server requests are positive, while server-to-client superstack requests have a negative index.

Notice how the initial request (1) is also last to receive its response (-1). All the sub-requests are within the contextual envelope of this initial request.

Therefore what we see is that the traditional N-squared complexity request-response with correlation code is transformed in the ROC abstraction to a linear complexity, contextual request sequence: request1-request2-request3-response3-response2-response1 with no correlation code.

Delegation of Trust: Provable Security

Lets now review the security of this solution. The detailed implementation of the demo architecture is shown below...

The first thing to note is that the primary application (there must have been something to initiate the purchase), is completely isolated from the 3-party transaction spaces via the One-way-trapdoor overlay.

The initialising purchase request can go down to the purchase endpoint, but no requests can go up.

This is not simply a filter of requests - the trapdoor actually constructs a new spacial scope such that for requests that pass through, the only space in the inner request's scope is the trapdoor space. The core application is not just protected - it is completely decoupled from the 3-party spaces by the trapdoor.

It follows that, even if we screw up our implementation inside the trapdoor, we have 100% confidence that A's internal services cannot be reached by either B or the Notary.

Next let's consider the request from A to B. B provides an NKP server endpoint (you can think of this as just like an HTTP server). A issues a transient one-shot NKP request to B's TX endpoint (you can think of this just like an HTTP client request). The NKP connection is authenticated and encrypted with TLS.

The logical resource request from A is issued via NKP into the B address space. B has configured its NKP server to use the NKPResolvable Filter (discussed above). It follows that logically its space has a very small surface area - just the single TX endpoint to receive purchase orders.

B's TX endpoint receives the purchase and constructs a request to the notary service. Its not shown in the diagram - but in the demo implementation we have protected against fishing attacks from B on A, by providing the identity of the notary service as an argument in the initial request to B. It is therefore possible, if we are paranoid, to give B a unique logical notary service for every single request. However this is really overkill, since as we'll see in a moment, B cannot resolve anything other than the notary endpoint anyway.

B issues the notary request (we simply use the convention of requesting the notarize service and providing the resource to be notarised as the operand argument). The notary request fails to resolve in B's space (there are no other endpoints here as we established before). The request resolves back up the superstack into A's address space.

In the initial NKP request A configured its request to permit callback superstack requests (by setting "ExposeRequestScope" on it NKP request). A has therefore allowed access to its request space within the context of the original request. Just to be clear - A has not exposed any public services to wider access. The superstack request from B can only be made within the context of a request from A. A is providing limited transient access to B.

A has also implemented the NKPResolvable filter. Most of A's internals are marked private - so they cannot be resolved by external NKP requests. However the original purchase endpoint cannot be private - otherwise the core application would not be able to initiate the purchase. However the purchase endpoint does not declare that it is NKP resolvable it follows that it is impossible for an NKP request to call it. Therefore it is impossible for B to attempt to make false purchase orders from A back to itself!

We can examine A's space and discover that in fact there is only one NKP resolvable endpoint present. It is the Notary endpoint that is provided by the NKP import of the notary space. B does not know about the Notary space. B's request is logically resolved in A's space. B does not know that its request is issued over A's trusted connection to the Notary.

B cannot do anthing other than ask for notarisation. The Notary exposes nothing but its single notarisation endpoint. A exposes nothing of itself to either party and only exposes to B the logical notaristion endpoint of the Notary.

Whilst this is starting to sound a bit complicated - it is actually very simple to understand when creating the spaces. It is also very simple to assert that the spaces are only exposing these tiny surface areas.

In addition we see that what we have accomplished is to provide a very constrained context in which A is able to transiently delegate its trust to B in order for B to use the Notary service on behalf of A.

Delegated 3-party trust with measurable and assured limitation on liability. Not bad for no code and an architecture consisting of six declarative endpoints (when we count the NKP servers and import).

Engineering Properties

Finally lets consider the engineering properties of this solution. Firstly we notice that party B and party C are trivially simple spaces. They consist of just an NKP server and their respective functional service.

NKP is completely asynchronous. Requests issued on the client side do not block the requesting thread. Requests received on the server side are assigned a thread, while callback superstack requests are also non-blocking.

There is no blocking anywhere in this architecture. This means all 3-party's systems can continue doing other work even when they are engaged in logical requests within this 3-party architecture.

NKP is logically decoupled from the physical network. All of the requests in this solution are logical - unlike, for example HTTP, they do not break if the physical communication network fails. NKP client/server endpoints will re-establish their connections in the event of network failure. You can actually configure B to be a slow endpoint (taking 10 seconds or so), pull the plug on the network, reconnect the network and the logical requests will continue without failure.

NKP is very efficient. On my laptop, the total time for the complete transaction roundtrip including callback requests and delegation to the Notary is 20ms (the Notary signature itself takes 8ms!). NKP treats the connections as resources. It caches a connection as a representation.

Repeated use of the same connection does not require the re-establishment of the connection. Once the distributed architecture has been resolved, just like with local ROC resolution, there is almost no subsequent interaction latency. NKP is very very fast.

NKP is ROC. This discussion might have been detailed - but the most important point is that NKP embodies the ROC abstraction. Any pattern you can implement locally can be implemented in distributed architecture. Or vice versa, any potential distributed architecture can be prototyped locally before it is distributed. In fact the whole demo was created with imports instead of NKP and made to work first. We added the NKP distribution in a few minutes and it just worked.

NKP distributes the cache coherence. This demo doesn't need caching but because NKP is ROC. Then the same caching that you get locally happens between the parties with NKP. So repeated use of a cached resource that is provided from B to A will hit the cache in A without requiring repeat wire-level request. The dependency model of ROC is preserved too - so, for example, the golden thread pattern can be distributed.

NKP is measurable - you can run the visualizer and measure the performance of the solution. If you have hotspots you can put NKP loadbalancers in to scale out. If you have serious demand spikes you can call your cloud provider's API and bring up more servers and dynamically add them the the load balanced pool.

NKP is fail-safe - any NKP connection between two parties can be made fail safe by adding the NKP failover load balancer. Redundancy is not an option it's built-in.

NKP is simple. You can measure it, you can deterministically audit the exposed surface area. You can build highly complex distributed architectures that can still be reasoned about. You can establish and demonstrate provable trust with very high performance.