|
NetKernel News Volume 2 Issue 49
October 14th 2011
What's new this week?
- Repository Updates
- NetKernel 5 - Status Check
- Simple Grammar
- Ruby on ROC
- Unix, the Web and ROC: How they are related - A Tribute to Dennis Ritchie
Catch up on last week's news here
Repository Updates
Nothing to report this week - lots to come next week...
NetKernel 5 - Status Check
Well that title sort of gives it away doesn't it. We've decided to take one more week before we release NK5. We're very close but we want to take our time to do one more round of QA testing and to add another coat of polish and give it chance to dry.
Simple Grammar
With the window closing for features, we took the time this week to implement what I hope will be a very valuable usability tweak. We've added a new "Simple Grammar" language. This is something that has been on the cards for a while and was first proposed by Brian Sletten.
Here's the reference documentation...
Simple Grammars
The Simple Grammar is a very quick and simple path matching grammar. It is best suited for services with external interfaces such as simple REST services. It is not recommended for use with internal services or libraries since it is not a normalized form and cannot be nested (see active below).
Simple grammars are declared like this...
<simple>res:/{country}/{state}/{city}</simple>
</grammar>
Where the static string text must match and the {argname} delimiters identify the named argument within the identifier. The grammar above would match
res:/USA/CA/San-Francisco
Regex Patterns
Any argument delimiter may also implement an additional regex pattern after a colon (:) seperator between the argument name and the regex. For example...
<simple>res:/{country:(UK|USA|France|Belgium)}/{state}/{city}</simple>
</grammar>
This grammar would match
res:/USA/CA/San-Francisco
but not
res:/Scotland/Lothian/Edinburgh
Escaping { and }
You may escape curly braces with \{ and \} (backslash-prefix on the bracket). These escaped characters can then form part of the static structure or (more likely) be used within regex patterns to mark quantity delimiters.
We'd very much like to get your feedback on this. We have it implemented in NK5 and have a small window of opportunity to tweak this.
You can play with simple grammars today on NK4 by installing this module, which has an impl of the grammar factory and a set of unit tests which request a sort of self-hosting "Simple Grammars Kitchen" built over the grammar factory...
http://temp.1060research.com/2011/10/urn.test.org.netkernel.simple.grammar-trunk.jar
This is just a standard module and will happily run on NK4. To install it add an entry in your modules.xml, the unit tests are then located here...
http://localhost:1060/test/exec/html/test:urn:simplepath:grammar:dev
The tests themselves show lots of example grammars and can be viewed and played with in the module's file /resources/test/testlist.xml. Note you can only play with the grammars by requesting the active:simpleGrammarPlaypen - the simple grammars will not be recognized in the NK4 standard module.
Our hope is that this Simple Grammar will provide a low-learning curve on-ramp to start exploring the binding of resources to endpoints. It is not meant to be comprehensive - it is far less expressive than the normalized and nestable Active Grammar or the complete and universal Standard Grammar, but hopefully it strikes the right balance of convenience and conceptual simplicity to learn.
Please let us know what you think? Does this help lower the learning curve?
Ruby on ROC
In case you didn't see this week's conversation on the forum (and I actually tweeted about it too - which is a once in a blue moon event). We just crossed the Rubycon...
After an on-off (but mostly off) relationship for the past four years - we finally knuckled down and worked out a high-performance and elegant integration of JRuby as a runtime (and prototype).
To see the story, or scroll to the end of the second page for the gripping conclusion, look here...
http://www.netkernel.org/nk4um/topic/820/
My last post provides links to 3 modules containing the runtime, ruby libs and unit tests. If you're Rubista - or if you just fancy another alternative - please give it a try and let us know how you get on.
When we've gathered feedback and made any tweaks, we'll post this as a supported language in the repositories.
The Challenge
One of the nice proof of concepts of the completeness of the Python integration was that I was able to boot Django on NK. I am a jack of all trades when it comes to languages - but my mastery of Ruby is low. Therefore my challenge to you is to tell us how you get on running existing Ruby systems as class library modules on NK.
You can take the Ruby 1.8 library integration as an example of how to put Ruby classes into the ROC address space (err, its as simple as one <fileset> mount). You can see the unit test module as a way to set up the res:/etc/RUBYPATH resource to provide the class path to active:ruby.
Now go to it and tell us when you reach the edge (if you reach the edge ;-)...
Incidentally - although to some it would make no sense to run something like Rails on NK - since it would mean running its own web-server etc. But imagine, even if you never make more than one "boot request" to active:ruby on NK, NetKernel's modular hot-deployable infrastructure, makes it possibly the worlds smoothest Ruby application server. Think about it.
My thanks go Charles Oliver Nutter and "mml" (who he? - we still don't know) for their assistance and for providing the spur to make this happen.
Unix, the Web and ROC: How they are related - A Tribute to Dennis Ritchie
The father of Unix, Dennis Ritchie, died this week. He must have been a very satisfied man.
In the beginning Unix was a specialist high-end world. Progressively the power and elegance of the abstraction it embodies have come to dominate computing platforms.
- The majority of web-servers run on Unix (aka Linux)
- Switches and most routers and wifi hotspots run on Unix (Linux again)
- Increasingly large numbers of PCs run Unix (Apple OSX and Linux)
- Nearly all sales of smart-phones are now Unix (Apple, Android)
- Nearly all Tablets are Unix (Apple, Android).
- Your iPod runs Unix (Apple and Linux).
- Most aircraft seat-back entertainment systems are Unix (Linux)
- Most vending machines, etc etc etc
Looking at "computer" in the broadest terms, we can therefore say that the majority of computers sold today use the "Unix Abstraction". It just goes to show that elegance and beauty will out.
For the longest time we have said on our web site: "ROC combines the core ideas of Unix and REST into a new and general model for creating powerful software solutions." It seems only proper that I should justify this statement and, in doing so, pay tribute to Dennis Ritchie...
| Unix | Web | ROC |
| Everything is a File | Everything is a Resource | Everything is a Resource |
| A file can have more than one name (aliases, symlinks etc) | Implicit (but not Explicit) | A resource can have one or more identifiers (grammars, mappings, rewrites) |
| A small set of simple verbs determine operation on the state of files (wrx) | A small set of simple verbs determine operation on the state of resources (GET, POST, PUT, DELETE, HEAD) | A small set of simple verbs determine operation on the state of resources (SOURCE, SINK, DELETE, EXISTS, NEW) |
Build complexity by composing small well crafted tools.
curl http://bbc.in/pBRhWU |grep Dennis |wc |
Initially dominated by monolithic server-stacks. But progressively RESTful composite architectures are winning (aka Mashups - spit) |
Build complexity by composing small well crafted tools.
active:wc+operand@
active:grep
+operand@http://bbc.in/pBRhWU
+operator@Dennis
This highlights one evolutionary step taken by ROC. Unix is a predominantly "push state transfer" architecture. The Web is a predominantly "push state transfer" architecture. NetKernel and ROC is a predominantly "pull state transfer". For example NK eliminates the need for POST - its just SOURCE with a different spacial context (see below). Pull means everything is done Just-in-Time which is cheaper in the long run (because frequently it turns out you didn't need or couldn't use the state anyway - branches are a case in point but also when a system is dynamically adapting to externalities). |
| One representation model: the binary stream. All state is transferred in this lowest common denominator form. All tools are written to the same underlying representation model - the highest order abstraction is to consider the stream as line-delimited text resources. | One representation model: the binary stream. All state is transferred in this lowest common denominator form. All tools are written to the same underlying representation model - mimetypes provide a higher order abstraction that allows tools to more efficiently consume HTML, XML, Images, Javascript. (Line delimited text has a mimetype text/plain). |
**Conceptual Leap**
Unlimited representation models. State is embodied in object representations. As much as is possible, state does not move, what we conceive of as "state transfer" is in fact a change of spacial context. Not moving state is very efficient. NetKernel has transreption, we can isomorphically transform state into higher and higher order forms, when combined with caching this means we can constantly eliminate disorder (entropy) in the state of the system. All tools can be written to different heterogeneous representation models (APIs) - impedance matching transreptor chains are automatically discovered. (But if you want to, you can do everything with lowest common denominator binary streams. You can also use mimetype or object class instance types to determine state handling) |
| The filesystem is the primary resource space - all resources are loaded from this space. Logical tree structure abstracts away from underlying physical instances. |
The "World Wide Web" is the primary resource space - all resources are accessed through this space.
At first (due to Tim Berners-Lee being at CERN - which runs on Unix!?) the address space was naturally partitioned as a logical tree structure (path based URLs). Over time and with CGI the web community has come to realise there is no fixed structure to logical addresses (identifiers). Although for historical and human organizational reasons - hierarchical paths are still the most common. Load-balancers and Web-servers abstract away underlying physical instances. |
**Big Conceptual Leap**
Any number of logical address spaces. Spaces can have tree structured addressing models. But can model arbitrary sets of logical resources. Grammars allow identifiers to be mapped to structured set membership relations. Endpoints abstract away from physical level. Spaces can be composed together to create architectures. Just as the Unix and ROC philosophy is for composable tools - so ROC takes this philosophy to the general conclusion: we can compose address spaces too. Create simple well formed spaces containing well crafted simple tools and resources - compose them together to reveal emergently complex architectures. |
| Concept of filesystem PATH is used to provide a simple relative file resolution mechanism. File resources are contextually relative (to current PATH, working directory). | DNS provides a "centralized" resolution mechanism. Resources are absolute to the global single DNS/web address space. |
**Big Conceptual Leap**
Resolution is decentralized and delegated to address spaces (plural). Address spaces further delegate resolution to individual endpoints. Resources are always relative. Web's absolute model can be replicated with one monolithic address space as the relative context. The Unix PATH search pattern can be used - but it is just one of many patterns. |
| All actions happen through uniform executable object code. | All actions happen through one consistent component type "the HTTP web-server". | All actions happen through one consistent component type "the endpoint". |
| Nobody cares what language tools are written in. | Nobody cares what language web solutions are written in. | Nobody cares what language endpoints are written in. |
| All executable code is stateless and each execution starts with the same initial state. | Most Web-servers are "predominantly stateless" with each execution starting with the same initial state. | All endpoints are stateless and each invocation starts with the same initial state. |
| An executable can obtain its state with push state transfer (switches, argv arguments) or pull state transfer from filesystem paths (eg looking in ~.toolconfig or /etc/tool/config) | A web-application obtains its state with push state transfer (headers, request body). Web-servers tend to be off-ramps from the resource address space - though increasingly it is realised that they can be symmetric server-clients. |
An endpoint can obtain its state with supplied arguments or by sourcing configuration resources in the address space(eg res:/etc/RDBMSConfix.ml). But all state transfer is pull.
**Conceptual Leap** |
| Caches all filesystem resources. "Write-through" expiry model. | Multiple points of caching. At client-side, network intermediaries, load-balancing front-end. Time-based expiry model - with some limited hashcode (ETAG) consistency correlation to synchronize caches. |
**Conceptual Leap**
Cache everything, everywhere. Arbitrary and extensible expiry models. Chained dependency model accumulation. Distributed state coherence. Virtual dependencies (golden thread pattern). In ROC there is more than one address space. NetKernel knows this and ensures consistency of caching in multiple-dimensions. Finds and tracks the systemic (multi-dimensional) thermodynamic Energy and Entropy minima. |
| Unix has a scheduler which manages physical CPU allocation. | Load balancers manage physical server allocation. | NetKernel has a scheduler which manages physical Thread allocation. Load balances requests over CPU cores *and* across cloud instances. |
When we first showed people NK back in the day, one of the ways I described it was as a "software operating system" - unfortunately back then I got responses like "you don't know what an operating system is - its an abstraction over physical hardware".
Today, having gathered a growing band of supporters to give us courage, we beg to differ: an operating system is a system that presents a self-consistent, uniform set of logical resources. Resources are in constant flux and an operating system must provide a highly efficient, consistent and sufficient approximation of their state.
I know Unix is an operating system. I hope this comparison goes some way to showing that NetKernel is itself a new type of operating system - one that stands on the shoulders of giants and extrapolates on from where Dennis Ritchie et al began 40 years ago.
Thanks Dennis.
Incidentally
A concrete example that ROC extends and is a superset of Unix: the new Ruby runtime works entirely the same as it does on Unix - it can load its class library from the ROC address space just as it would load from the logical filesytem on unix. It has no idea that NetKernel is not Unix. (PS And so too does Python - another Unix heritage language).
NetKernel Europe Bootcamp - Brussels, Belgium, Thursday 27th October 2011
 |
| Sponsored by Steria Benelux |
http://www.1060research.com/conference/NKEU2011/
The event takes place on Thursday 27th October. If you've registered already, thank you! We will soon be getting in touch with the details so you can plan your travel.
If you want to take advantage of face-to-face opportunity we will also be around on both Wednesday 26th and Friday 28th for meetings etc.
We're looking forward to seeing you in Brussels.
Have a great weekend,
Comments
Please feel free to comment on the NetKernel Forum
Follow on Twitter:
@pjr1060 for day-to-day NK/ROC updates
@netkernel for announcements
@tab1060 for the hard-core stuff
To subscribe for news and alerts
Join the NetKernel Portal to get news, announcements and extra features.
NetKernel will ROC your world
 |