ESP Rules

I was briefed by yet another ESP vendor recently. Right now I am not sure whether the technology was better or worse than other ESP technology I've looked at, but it was certainly respectable. I doubt if it's going anywhere though, as it's marketing message was terribly confused. Then I was approached by yet another company (Alerilabs) claiming to be in this market. I'll report on them when I'm briefed.

As I think I've mentioned before, this is a sexy software market and quite a few companies, which may not really have viable technology, are claiming to be ESP products. StreamBase (one of the earliest players) published a paper under the title of "8 Rules of Real-Time Stream Processing" in an attempt to draw a line in the sand. (If you can satisfy these rules you're an ESP player, if you can't you're not). There needs to be something like this, or the market is likely to get thoroughly confused. You see, ESP is sexy.

The problem with StreamBase's 8 rules is primarily that they came from a vendor. Such origins always imply some kind of partiality. For that reason I've decided to compile my own set of rules, using the StreamBase paper as a prototype. So starting this week I'll describe them. Let me make it clear what this is. It is a bit like Ted Codd's Relational Database Rules. The more you obey, the more right you have to claim to be an ESP product.

Rule 1. ESP involves the Time Critical processing of a data stream

The point here is that most software processes streams of records or event information in some way. By using the term "time critical" we are demanding that the software is optimized to process the stream as fast as it is able. Products that do very little to optimize the processing of a stream can't claim to be ESP products. Particularly, ESP products must include memory management capabilities for instruction code efficiency in respect of processing the stream. We have phrased this as a demand for a "real-time" capability and we would have, but for the fact that the term "real-time" is a little too closely associated with process control systems. ESP applications are structured differently to most real-time applications.

Rule 2. ESP requires a Scalable Architecture

An ESP product needs to be able to process high volume streams. We are not looking here for a specific figure—20,000 events per second, or anything like that. We're looking for technical features that support high volume processing, such as parallelization of processing and a distributed architecture. We note here that most current ESP applications are not run on high volume streams, but we expect ESP applications involving high volume streams to become quite common in time.

Rule 3. ESP requires that Scalability and Time Criticality are not be mutually exclusive

This means what it says. An ESP product must be able to obey rules 1 and 2 simultaneously without reconfiguration. There are performance techniques that don't scale well. An ESP product must have some that do.

Rule 4. ESP requires high availability

Rule 1 implies this demand, but it makes sense for it to be explicit. A true non-stop architecture is preferred for ESP, but high availability—which means being able to failover an application, with a degradation of service measured in seconds—is acceptable.

As I was writing these rules I noticed that these first 4 rules are about technical performance and resilience. It's curious that Ted Codd's relational database rules didn't really care about such things. They were all about relational "purity". Luckily there is no conceptual purity in ESP (at least not yet).

Next week I'll add some more rules.

Mashups-the threading of Web Services

In terms of software development, the big move within the enterprise at the moment is from Web Services towards SOA. Which means, among other things, the creation of composite applications. So if that's what's happening in the big shops, what's happening with composite applications out there on the web? The answer is "mashups". The idea is simple. You build web-based applications by writing some of the capability yourself and linking to already available Web Services out there in cyberspace.

The "mashup" action is being driven by Ajax (if you don't know what it is, go look it up now, there will be a test in future blogs) and to some extent by the ecosystem that is growing up around Google (particularly Google Maps). The Internet is gradually turning into a wild Service Oriented domain where mashups can reproduce like rabbits.

Notice I didn't use the word architecture above, for the obvious reason that architecture has nothing to do with this. Mashups don't attempt to provide a guaranteed service, although if you build one and get traction I'm sure you'll quickly move to a Software As A Service business model and then you'll be adding the architecture.

The term mashup, by the way, is being widely used to describe other phenomena—such as merging several photos together or making video footage by combining multiple clips. Some observers are suggesting that we now live in a mashup world, where everyone builds stuff from "bits-lying-around". The first mashup I remember was John Lennon's Revolution 9 on the Beatles White Album. To be honest I preferred the stuff he built from scratch (all of it)

We're Done Here

This has nothing to do with IT, it's just me using this blog to do some bitching. I'm getting tired of hearing the words "we're done here". It started with all the cop programs on US TV. The scene is ‘the accused' and his/her lawyer (for he/she is all-lawyered-up) and the series cop is asking him/her questions in an interview room. At some point the lawyer will intervene and say something like "This interview is over, we're done here".

This pernicious phrase seems to inhabit every goddam cop show on TV and is now infecting other shows; comedy shows, hospital shows, etc. and worse, people's normal conversation. I was in Starbucks last week and some guy in front of me asks for a "Grande, non-fat, no foam, latte" and the server asks "will that be all?" and he says "Yeah, we're done here".

"And fall thy edgeless sword; despair, and die." [As I murmured to myself and, also, the ghost of Clarence said to Richard III.]

"We're done here" is a memelet. Memelet derives from meme which is; meme n. A unit of cultural information, such as a cultural practice or idea, that is transmitted verbally or by repeated action from one mind to another. (according to Answers.com). A memelet is; memelet n. A truly irritating phrase which lost its meaning long ago and yet is repeated mindlessly and incessantly by everyone, even the French.

"We're done here" reminds me of the 1970s when everyone kept using the term "no way" to no good purpose. As in:
"Would you care for a drink?"
"No way".
Not "No thanks" or "I'd rather not", but "No way'. It wasn't said for meaning, but only in order to use the mindless memelet.

OK I've made my point.
Nous sommes fait, ici.