There has been some delay in this, the third and final article on untangling events. And just as well. I initially asked the question as to whether the log and event management vendors were more likely to expand into the sensor management and complex event processing (CEP) spaces or vice versa. In the second article I took LogRhythm as an exemplar of the former (but it could have been LogLogic, Sensage, Prism, Arcsight or AN other) and I was going to do something similar with a CEP vendor in this article. I was also pretty much convinced that it was (and is) the CEP vendors that are more likely to be under threat from the log and event management guys moving into their space rather than the other way around.

However, as I said, it's just as well that I got delayed because I've found a company that potentially does all of these things, and more, already. This vendor is SQLstream.

SQLstream, which is also the name of the product, is a SQL-based product that processes streams of data. It has the ability to process both CEP style events and log events (it is simply a question of building the right adapter, if it does not already exist) as well as what the company describes as continuous ETL (extract, transform and load).

There are a number of differences between SQLstream and traditional (if that is the right word) CEP vendors. The first is that it operates in the database layer and not in the application server tier. I regard this as a good thing because it should improve performance. Secondly, SQLstream does not use an event processing language based on SQL but uses SQL itself. That is not to say that SQLstream has not extended SQL but that it has used SQL wherever it can so that extensions are minimal. If you really need to use anything else then the product supports UDFs and UDXs (user defined functions and transforms) so that you can build specialised functions (for example, aggregating log events). It is also worth noting that SQLstream co-founded the open source Eigenbase project, which is an advanced reference implementation of the SQL 2003 standard. SQLstream intends to make further contributions to Eigenbase.

SQLstream is initially targeting three markets: real-time ETL/ELT, operational BI, and CEP style applications such as fraud prevention (which includes monitoring things like failed log-ins). This is somewhat disparate and difficult for a relatively unknown company to manage so the major target will be real-time ETL/ELT in the first instance.

I had better explain why SQLstream is suitable for this market because it is not something that I have previously encountered as a CEP market. The rationale is this: ELT (extract, load and transform) has become increasingly popular because you can use SQL in your data warehouse to transform the data, making use of the power of the warehouse platform and eliminating the need for a powerful intermediate processor. Also, using SQL has the advantage that it is commonplace and therefore relatively inexpensive in terms of talent and resources. However, on the first point, running the transforms in the warehouse still requires a significant amount of processing power that is taken away from query processes and/or costs extra. In addition, it is still essentially a batch rather than a real-time process.

SQLstream fixes these issues by transforming data in real-time, in a streaming fashion, but still uses SQL. Neat!