NetKernel News Volume 3 Issue 23

May 11th 2012

What's new this week?

• Repository Updates
• HTTP Transport Released
• WebSocket Demos
• On State and Reality

Catch up on last week's news here

Repository Updates

The following updates are available in the NKEE and NKSE repositories...

• demo-websockets 1.2.1
  • First repo release of the websockets demo
• http-server 2.18.1
  • Updated to use Jetty 7.6.3

HTTP Transport Released

As reported last week - we have updated the tpt-http transport to the latest 7.6.3 version of Jetty. We've done hardcore load testing and we're running the update in production, all without a blink. Therefore it has been released to the repositories and is ready to go.

WebSocket Demos

In the repositories you'll find the demo-websockets package. This contains a couple of simple example WebSocket-based demos. The helloWorld demo shows the basic flow of requests in establishing a socket and then sending messages from the server-side to the browser...

http://localhost:8080/demo/websocket/helloWorld

The second demo is a chat-room example with a psychotherapist Eliza who will talk to you about your problems...

http://localhost:8080/demo/websocket/groupTherapy

The source code should be pretty easy to raid if you want to play with your own Websocket apps. The details of the implementation model are detailed here...

http://docs.netkernel.org/book/view/book:tpt:http:book/doc:tpt:http:WSTransport

And of course you also need to keep an eye on the Browser's Javascript API, here's the Mozilla reference docs which are tracking the standard process pretty closely...

https://developer.mozilla.org/en/WebSockets/Writing_WebSocket_client_applications

On State and Reality

Before we start, there are papers, theses, books and probably an entire barely tapped academic discipline lurking here. Or put another way, I don't know what I'm talking about (but then I don't think anyone else does either). So don't expect a complete picture - but here are some thoughts and observations spoken aloud.

Last week the question of verbs arose again. In particular the ability to translate from REST methods to ROC verbs.

So why is this a thorny subject? As Tom observed, in some lights it seems pretty simple to just provide a mapping?

My answer is, because fundamentally any information system is a model comprising state in which we are trying to manage a "good enough" approximation to reality. Our tools are identity, context (space) and representations. But our model is in the domain of the abstract infinite world of resources.

The world of abstract resources is a hard thing to get your head around. My personal sense of it is informed from my background in quantum mechanics. One way to think about resources is as set(s) of potential information. When we make a request in ROC we are making a measurement and causing the collapse of the abstract "wavefunction" into a concrete eigen-state (representation). But when I think hard about this I recognize that fundamentally, just like in QM, there are paradoxical and apparently unknowable limits.

Leaving that can of worms to one side (ultimately I think we'll need an "ROC Copenhagen Interpretation" - essentially an agreement to agree to disagree, but which is sufficient to allow us to concentrate on the application), in the light of the previous couple of paragraphs, what are verbs?

Well I'd suggest a verb is just a flag to indicate a particular mode of state transition.

So the reason verbs are a thorny issue is that if we don't take care with how we consider transitions in our state then our approximation to reality can get out of whack.

Some people get the cart before the horse. They start to think that the model is reality and that the world outside ought to comply with it. Others are more realistic and know that the world is out there, but they would dearly like to imagine that their model is concrete and fixed and immutable; that we can create a set of concrete rules for which any relevant event in the world will be correctly translated to the state of the model.

But remember, there is a fundamental limit; any state must always be an approximation.

This is true at a high level, since it is fundamentally impossible to transcribe measurements of the physical world in sufficient fidelity to add to our model's state. We approximate all the time.

But, as Feynman discusses very eloquently in his lectures on computation, even the core elements of our computers - the representation of bits in silicon or as magnetic dipoles on disks - these bits when used as ones and zeros are actually approximations. Their states are in fact "one-enough" and "zero-enough". There is a finite probability that they might not be what you thought they were. A computer is a constrained context in which the probability of them not measuring the same state each time becomes vanishingly small. But always remember it is still only a good-enough approximation.

But I believe there is something even more fundamental here. I know as fact that the identity of resources are contextually relative. Any given abstract resource can have any number of identifiers. (If you think hard about this then you start to realise that there is no place to hang your hat - here's one of the most paradoxical unknowables). And, get this, when you request the state for the given identifier, the representation you receive depends upon identifier and the context of where you request it. In short in ROC's generalisation from REST, its not just what you ask for, but also where you ask for it.

The truth of this is seen in how NK caches information. It doesn't just use the identifier as the key, but the whole ROC spatial scope. [The brilliant part here is that Tony came up with the high-water-mark algorithm that allows the cache to figure out how much of the spatial scope is actually relevant, in order that re-using existing state doesn't deviate from being a good approximation to reality. (In classical software this is an amazing result, but in everyday reality its dead obvious - we do this all the time in order to interact with the world).]

Where the hell am I going with this wandering ramble? Back to verbs.

Now, given the background perspective, and maybe if you've seen even the briefest glimpse into the convergent contextual spacial nature of the abstract resource oriented world, my instinct tells me that verbs are just a short-hand convenience - a "pattern" if you will. I sense that a verb is actually an additional part to the identifier. And that if the verb is concerned with an action to modify the state of the model (ie NEW, SINK, DELETE) then that state is itself just another resource in a context.

In short, we can apply action on our model's state with nothing more than an "EVENT", an identifier and a sufficient context. We don't need verbs - or rather we just need to ensure our space has resolvable within it the set of resources for those identifiers that constitute "the state of a change of state". (Bear with me - its tough I know, but its resources all the way down).

Furthermore, as is shown with pass-by-value arguments - it is not necessary to even think of state as being transferred. Weirdly perhaps, we can follow the rabbit to finer and finer granularity, by narrowing identity and broadening context, until we can think of the individual bits of state in a vast network of context. At which point we discover something very like Zeno's paradox, state transfer cannot happen. That is if we choose to, we can look at the system in a way in which its state is never transferred, there are ultimately only EVENTs, identity and context. State transfer is an illusion due to evolving context.

Stepping back up to the scale of our "normal" developer view, it is for the endpoint that responds to an EVENT to determine how it will choose to interact with the identifier and the context (and so the extrinsic state).

For example, if the identifier correlates with DELETE then it may choose to change the state of the model such that a subsequent request will receive a null representation. Or it might implement itself such that a subsequent request for the identifier of the resource might fail to resolve at all.

EXISTence

Here we get into something very interesting indeed. What does existence mean? After all in the ROC verbs there is the EXISTS verb.

My interpretation (note "interpretation", you are free to choose your own) of existence is that if an identifier is requested with an EXIST verb - the space (context) will resolve to a true or false representation indicating that a request for that resource would be resolvable and some state would be forthcoming. Existence is basically saying a resource is "measurable". Conversely, non-existence means there is no state for that identifier.

But then you realise that there must be another kind of existence. For what if you have a space that contains no resources. The empty space. By definition no resource state EXISTs there. But more fundamentally, there is not just no state, there are no abstract resources at all!

We might call this concept reifiability and it has fundamental and profound implications, not least in terms of how we can offer provable assertions about the trust (security) of our model.

I told you this was a deep can of worms.

Am I saying that verbs are wrong? No not at all. What I'm saying is that they are a high-level convenience but they are not fundamental. I hope you can also see that there is no right or wrong about verbs. Since they are ultimately just parts of the identifier of some resource - perhaps "meta resources", but nevertheless still resources.

What I do worry about though is "automatic verb translation". Since, the fact is, you are responsible for deciding what a "good enough approximation" to reality is for your solution. Therefore since verbs are flags to indicate that the model's state could be transformed then its important you take responsibility for that change in a way that remains consistent.

Hopefully you can now see why I didn't allow the standard module's mapper endpoint to provide anything other than identity transforms (mappings). It very deliberately does not offer verb mappings. Why? Because just consider this, what if you changed a SOURCE to a SINK? What part (argument) of the request's identifier should itself be isolated in order to provide the primary argument on the SINK request? What if it doesn't exist? What if there are multiple arguments and the state change is the converged set of all of these?...

The point is that the mapper is meant to be a simple and consistent tool and it will never lead to your solution's "approximation to reality" getting out of whack.

If you need to take more control then you are always free to do that. If you want a concrete, rigid set of rules about what the meaning and interpretation of verbs is - then you are also perfectly free to define that too. But hopefully, if this discussion has shown anything, it is that these are, and must, only be conventions. They are not axiomatic.

When I say we have not scratched the surface of ROC I mean it. There is a world of reifiable truth out there and such beautiful patterns as will take our breath away.

But don't think that we can't use what have now - we already have a high-level set of building blocks that even if you don't care about the underpinnings, work damned well and allow systems to scale and evolve like nothing else. Its just you know I can't resist digging and refuse to close the door on the bigger picture...