By: David Norfolk, Practice Leader - Development, Bloor Research Posted: 22nd April 2013 Copyright Bloor Research © 2013

In the first part of this piece, I raised some issues around data protection legislation and Big Data. Now I want to get some expert advice - as anyone starting a Big Data initiative should.

I asked Philip Howard of Bloor whether he'd encountered such issues in his Big Data practice and he noted that Facebook data was being made available for mining by third parties (see here) and he wondered whether Facebook or its prospective customers had considered the compliance and reputation risk associated with the latest EU data directive proposals (you can find a overview guide to these from Robert Bond, Partner; Alexia Zuber, Solicitor; Dominika Kupczyk, Data Protection Executive; from Speechly Bircham's IP, Technology and Commercial Group, here; registration details needed).

I guess that the answer, for Facebook, is "yes" - and that it didn't much like what it found - because it suggests changing the law here. As for its prospective customers, who knows?

I've been raising questions in these pieces so far and avoiding giving answers. If these questions might concern you, you should seek your own answers, in the context of your specific circumstances. This doesn't mean finding someone on the web who endorses whatever your current practice is or promises you a "get out of gaol free card" (gaol? let's not go there); nor does it involve asking your local database support technician about data protection. Neither "what the blogoshere knows" nor the opinions of DBAs or web techies on compliance law rate very highly in the courts. You need to actually read the directives and so on yourself, talk to your compliance people and get input from your legal advisers.

In that spirit, I talked to Bloor's compliance specialist, Peter Howes of Rite-Choice Ltd. He warns that the Article 29 working group issued a report earlier this month which definitely references big data - see "Annex 2: Big data and open data" on page 45 of the report here. He also suggests taking proper legal advice but says, "my take on this (as another person who is not legally qualified) is that, whilst there is an issue when you get to the Business Analytics and Intelligence, it is also a problem when the data is initially captured; even if the full scope of the details of analysis and intelligence inference are not fully defined at the time of capture". And, he confirms that, "as you point out, this is not a widely understood problem yet".

Peter also expands on the Facebook issue which arises, in part, from the fact that, when the Data Subject (i.e. the Facebook customer) is in Europe, Facebook (et al) will in future have to comply with the European legislation wherever their data centres are located. "The main reason why the American companies are so worried is because of the provisions in the expected replacement legislation that are broadly referred to as the "Right to be Forgotten"", he says.

"This 'Right to be Forgotten'," he says, "will have a major bearing on "Big Data" inside EU as well as for US based service providers with the replacement Data Protection legislation and should be considered now and accommodated in the solution design (unless the organisation deploying a "Big Data" scheme including personal data intends that the solution to only have a 2 or 3 year life)".

Ah - another exemption! So that's all right then? Well, no, as I implied before, relying on EU data protection exemptions is not really recommended - it is often cheaper and easier (and safer) to just comply anyway. This one is particularly risky: how do you prove that personal data will only be kept for 2-3 years and do you have tested policies and procedures to ensure that its retention isn't extended, that none of it is copied to other systems with different policies, that none of it is kept on local hard-drives or that "useful reports" on paper aren't kept past the limit? And what about backups and data retained for long periods for compliance purposes? As Peter points out: "no way would anyone today plan for that short a life or reasonably expect such a quick termination. And, if they did, they should expect the content to be somewhere in the organisation afterwards (not just the unauthorised retention, but also the normally retained backup)".

Luckily, I know some lawyers too. Robert Bond is a Partner and Notary Public for Speechly Bircham LLP and a noted data protection expert. He says that "data protection laws apply to personal data in storage as well as in other uses such as analysis, research, sharing and transfer. An individual may impliedly consent to the use of personal data for purposes for which they reasonably anticipated use but not for unanticipated uses particularly profiling. In any event, if any personal data contains sensitive information such as health then consent needs to be more expressed than implied."

So, what that all means is that if you are starting a Big Data project you need to think about what data you are collecting, what uses you might want to put it to; and, if it is collected from or about people, whether you need their consent (either implicit or complied) before storing it, let alone using it. You need to become familiar with data protection legislation (both as it is now and as it is expected to evolve) and, probably, pay for legal advice on whether it impacts you, and how. Think about the cost of not complying, if you are caught - not only fines, but reputation risk; and the regulators may see you as a likely "useful example" when the law next changes or a different regulation becomes high-profile. Then you have to estimate the cost of complying with data protection law (not complying shouldn't be considered as an option) - including the cost of finding out about it - and make sure this is included in the ROI estimates for your Big Data project.

But look on the bright side. If you are thinking of collecting and storing vast amounts of data, using new and comparatively untested technology, with no very clear idea of what you'll use it for, or when, or for how long - isn't that looking like a very risky project? A bit of due diligence now, spurred on by some of the data protection issues discussed in my two papers, may focus you on why you are jumping on the big data bandwagon at all, what resources it is using and what the business outcomes are, that might justify the Big Data adventure. And that must be a good thing.

By: David Norfolk, Practice Leader - Development, Bloor Research Posted: 17th April 2013 Copyright Bloor Research © 2013

I met someone at a recent CMSG meeting who suggested that one aspect of the Big Data opportunity was about to hit major problems because it was collecting data for targeted marketing to individuals and no-one is paying attention to the EU data protection implications of this. Philip Howard, however, points out that this isn't really a Big Data issue but something that needs to be considered by the Business Intelligence and analytics applications further on down the line.

OK, that makes sense; but reading what lawyers are saying about emerging EU data protection legislation makes me wonder whether that is the whole story, from a governance rather than from an IT point of view.

Looking at the current UK Data Protection Act (DPA) guidance, it seems likely that whatever Big Data is, if it is being "recorded with the intention that it should be processed by means of such [computer] equipment" then if it contains "personal data" it is subject to the DPA provisions even if it isn't actually being processed yet. And why would you go to the expense of collecting and managing this data it if you don't intend to use it for analysis and decision support?

So according to an article in Out-law.com, a Pinsent Masons online legal news service, the "individuals' consent is 'almost always' required by firms when using personal data in big data projects centred on profiling" and "such consent should be required, for example, for tracking and profiling for purposes of direct marketing, behavioural advertisement, data-brokering, location-based advertising or tracking-based digital market research." OutLaw.com is reporting on an EU working party report that is available in full here.

Now, is implied consent from signing up to a loyalty card, say, or signing a 28 page document no-one reads, sufficient? That's a question for the courts, perhaps, rather than for an IT specialist. And since 'data subjects' can request access to their personal data and even have it changed if necessary, have Big Data projects ensured that such requests can be actioned efficiently and effectively - or is this an unanticipated expense that will bite once the project goes live?

One risk is that disgruntled customers use data protection legislation as a weapon against a company, partly because the regulations have increasingly serious teeth anyway: there's a whole range of new penalties and sanctions with fines starting at 0.5% of annual company turnover worldwide for minor breaches and rising to 2% of annual worldwide turnover for intentional or negligent breaches. So, having to satisfy requests (malicious or not) from data subjects may be a significant cost, as may being prosecuted - to say nothing of the reputation risk involved.

Is collecting large amounts of data and personal IDs (potentially allowing the identification of individuals and their buying patterns), even in advance of actually having any BI or analytics systems processing it, a DPA risk? Well, ask a lawyer, don't ask me - or even (probably) your IT group - but it might be good to find out in advance of doing it. And the answer might affect the anticipated ROI for the project.

I don't want to scare-monger; and there are exclusions "where businesses engage in big data projects that involve trying to "detect trends and correlations" from personal information, they may not require individuals' consent to process their data for that purpose providing they put in place certain safeguards". However, do people in your organisation know what these safeguards are and have they been costed in? They probably include ensuring that the information is kept confidential and secure and that "all necessary technical and organisational measures" have been taken to ensure that "this 'functional separation' of the data [thus preventing identification of individuals] is maintained" - how would you prove in court that you've done this?

If you are embarking on a Big Data project (whatever that means - and that could be part of the problem), even a 'proof of concept' using live data, shouldn't you be looking at the governance implications, especially around EU Data Protection directives, as a matter of urgency?

In part 2 of this paper, I talk to some experts in the field about these and associated issues; and locate some more resources.

By: Clive Longbottom, Head of Research, Quocirca Posted: 28th February 2013 Copyright Quocirca © 2013

Since the original Basel Accord was agreed and signed in 1988, central governments, driven by the EU, have been trying to ensure that financial institutions were managed in such a way as to provide a solid platform to the global economy. Starting with Basel I, increasing levels of central oversight have been put in place to try and maintain a good view on what could be happening within the markets. Through the Capital Requirements Directive (CRD), first instituted in 2007, certain levels of capital are required to be held by the banks and insurance companies so that they are able to weather any economic storms that come the way of the markets.

CRD IV is the latest version, and it nominally came into effect on January 1st, 2013. "Nominally" will be covered later...

At the highest level, the basis for CRD IV is covered under the Basel II and Basel III Accords for the banks and under Solvency II for insurance companies, which increase the amounts of common equity and Tier 1 Capital that the institutions are required to hold. Basel II also covers how the banks will need to provide centralised prudential reporting—and this mandates the use of the extended business reporting language, XBRL.

In October 2012, Quocirca carried out research across the UK, Germany, France, Italy and Spain for EMC to gauge the preparedness of financial institutions for the use of XBRL as well as their understanding of the whole CRD IV process.

The research provided some interesting findings—just under half of respondents felt that adopting XBRL would be a major impact on the business, with 65% saying that integrating existing systems into an XBRL system would be of major concern. Unfortunately, only 25% of respondents had even chosen an XBRL solution for something that was to be mandated as of January 1st (at the time, only 3 months away), leaving the notion of the financial markets being ready to meet the implementation date as being a bit far-fetched.

But, back to the "nominally". As the financial markets collapsed, the EU went into prevarication mode. There was always a transition period built in to CRD IV and Basel III, but this was meant to be for a move along a maturity model with everyone essentially staying in step along a defined set of processes. Although the nominal dates for CRD IV and Basel III remained as 1st January, the EU started to change the goalposts, saying that banks must hold more liquid assets and so lower their risk if facing another meltdown.

Country financial bodies, such as the Financial Services Authority (FSA) in the UK, had to move to more of an advisory mode—without agreement from the centre, little in the way of solid process guidance could be provided by them.

So, although few banks and insurance companies were ready for the requirements of CRD IV and Basel III on 1st January, it makes little difference, as the central bodies concerned were still fiddling while the economy burned.

However, this is not an adequate excuse for the financial institutions concerned to be so far away from being able to meet the technical requirements of CRD IV. The need for centralised prudential reporting is still there—and the failure to plan to implement XBRL systems means that these institutions are incapable of meeting this need.

At some stage, the Powers That Be will get their act together and CRD IV will become law with the necessary Directives in place. Financial institutions would do well to ensure that they are implementing the right systems now to meet their reporting needs—without them, they will fall foul of legal requirements, which could cost dear in fines.

Quocirca's report on the subject can be downloaded for free here.

By: Dana Gardner, Principal Analyst, Interarbor Solutions Posted: 13th December 2012 Copyright Interarbor Solutions © 2012

Open-source provider Talend has received a favorable advisory ruling from the U.S. Customs and Border Protection (CBP) agency concerning the government's ability to purchase open-source software, opening the way for all software vendors to increase their share of business with US federal agencies.

The CBP has determined that software products comply with the Trade Agreement Act (TAA) when that software is manufactured in what is known as a "designated country," even if the majority of its source code was created in a non-designated country. [Disclosure: Talend is a sponsor of BriefingsDirect podcasts.]

The US TAA says that government agencies may acquire only products or services produced in certain countries—known as designated countries. This has sometimes hampered the agencies from acquiring open-source software if some of the code was developed outside of those countries, even when the majority of production took place inside designated countries.

“Country of origin” issues sometimes have been used as a pretext to make a case against the procurement of open-source software. Talend conducts the vast majority of its software production in the U.S., France or Germany but, like many manufacturers, it also seeks talent in countries that can fall outside those considered designated countries

"With this finding, any other company that meets the same criteria can get the same approval," said Yves de Montcheuil, Vice President of Marketing at Talend. "And then government buying can meet the trade agreement status. The process can now be easily repeated."

While governments around the world have been moving to embrace open source for a long time, adoption has been slow and inconsistent in the U.S., though it is steadily growing as more federal agencies revise their guidelines and regulations, and some states pass laws requiring the consideration of open-source options.

Useful guidance
"The Talend Ruling is significant because government users now have useful guidance specifically addressing open source software that is developed and substantially transformed in a designated country, but also includes, or is based upon, source code from a non-designated country," said Fern Lavallee, DLA Piper LLP, counsel to Talend. "The timing of this ruling is right given the Department of Defense’s well publicized attention and commitment to Better Buying Power and DoD’s recent Open Systems Architecture initiative."

"This is great news for everyone in the software industry," said Bertrand Diard, co-founder and CEO of Talend. "While the news is significant for Talend and offers an opportunity for us to address needs in the federal space, our belief is that many software vendors—whether they are open source based or not—will benefit from the ruling."

A copy of the advisory ruling can be obtained by emailing press@talend.com.

The U.S. Department of Defense (DoD) is currently and significantly revising the December 2011 draft of the “DoD Open Systems Architecture, Contract Guidebook for Program Managers.” The guidance document, expect by the end of 2012, helps DoD program managers use Open System Architecture principles for National Security Systems.

By: Clive Longbottom, Head of Research, Quocirca Posted: 6th September 2012 Copyright Quocirca © 2012

Many organisations look to the cloud to provide some level of contingency against their own systems going down, be it off-site data backup, failover servers for business applications, or the use of high-availability servers and software. The level of disaster recovery (DR) and business continuity (BC) a given organisation chooses to put in place will vary according to its own risk appetite and budget.

The degree to which cloud services are suitable for providing a safety blanket will vary from one case to another. So which one is right for your organisation?

The following use case scenarios provide some guidance, starting with the most basic level of data backup and moving to full business continuity.

1. Simple data backup – the cloud can act as an external storage system where files can be stored so that if there is a problem with on-premise storage, individual files can be recovered, or images of specific machines can be restored to a device. This can be very cost effective – but as with similar on-premise solutions, there will be a level of down-time while the data is identified and restored to the live environment. Also, large amounts of data will take a long time to be recovered over the internet – which is why Quocirca recommends that data be recovered from the cloud to a local physical device which is then couriered to the customer’s site and then recovered to the target storage system at local area network (LAN) speeds. However, the service provider may be able to offer additional archiving services that could work well for compliance needs (as Quocirca points out in a previous blog post)
2. Secondary data storage. The cloud can be used as the place where a mirror of existing data is kept. Then, when there is a failure in an on-premise data storage device, systems can failover to use the data being stored in the cloud. Although this may look as if it provides good levels of business continuity, organisations must bear in mind that providing data to on-premise applications from outside the data centre may lead to latency issues, and that the synchronisation of live data may not be as easy as first thought.
3. Primary data storage – no data is stored on-premise, instead being held directly in the cloud. Although this should provide better data availability due to how the cloud provider architects its storage platform, the latency from the on-premise application to the data will generally make this a non-viable option. However, data backup and restore is now being carried out at LAN speed.
4.  Applications and data are held in the cloud, with data back-up and restore being integrated. This moves the application and data closer to each other so that direct latency is no longer an issue. As long as the application supports web-based access effectively, the user experience should by good. Should the prime data storage be impacted, restores can be carried out at LAN speed so recovery time objective (RTO) is shortened. However, this only provides data continuity – if the application goes down, the organisation will still be unable to carry out its business.
5. Applications being used as virtual machines with data being mirrored. This is getting closer to real business continuity. By using applications that have been packaged as a virtual machine, the failure of a single instance of the application can be rapidly fixed through just spinning up a new instance. Data needs to be covered as well, and should be mirrored to a different storage environment so that there is a high level of data availability in place. Such an approach can lead to recovery times measured in a few minutes, and will be enough for many organisations. This is also known as a “cold standby”, as standby virtual machines are not running all the time.
6. Stand-by business continuity. Here, the stand-by application virtual machine is permanently “spinning” (i.e. provisioned), but is not part of the live environment. On the failure of the live image, pointers can be moved over to the stand-by image in a matter of seconds, using existing or mirrored data storage. Also known as “hot standby”, as the virtual machines are ready to take over as soon as a failure occurs.
7. Full business continuity. Here, everything is provisioned to at least an “N+1” level. Multiple data storage silos are mirrored on a live basis and multiple live application virtual machines are maintained. Workloads are balanced between the virtual machines, and two-level commit is used on data to ensure that any problem with the data itself is not mirrored across all the data stores at the same time. This is the approach used by large organisations that have to have the capability to continue working through a systems failure – but is outside of the cost capabilities of the majority of other organisations. Cloud computing can bring such a capability into the reach of more organisations through the economies of scale.

Obviously, there are cost issues as the amount of cover increases through the table. This is why any organisation must first understand its corporate risk profile, building up a picture of exactly what business risks it cannot afford to carry and that which it is capable of carrying. Once a risk profile has been created, the right level of technical 'insurance' can be found from a cloud or hosting provider. The cloud makes the costs less of an issue, as each level can be offset through the number of organisations that are sharing the infrastructure. Therefore, an organisation that has previously regarded business continuity out of its reach and has settled for disaster recovery can now look to the cloud to create a more business-capable platform.

Originally posted at Lunacloud Compute & Storage Blog

By: Clive Longbottom, Head of Research, Quocirca Posted: 2nd August 2012 Copyright Quocirca © 2012

As is the way with IT, as soon as one bandwagon begins to be understood by the general public, another one has to be rolled out. In this case, as cloud computing starts to become more of a reality, big data is rearing its head as—depending on the commentator—the next greatest opportunity or threat to the organisation. 

As there was with cloud, there’s a lot of confusion out about big data. Many of the database vendors tried to play big data as purely having a lot of data in one or more databases. But that is not big data, it’s large data—a problem that can be handled with database federation, standard business intelligence and analytics. 

Next, it was said to be a mix of data held in the organisation that needed to be brought together so decision makers could see everything the organisation held around a specific topic to make better informed decisions—but only through whatever information the organisation was already aware of. So if the organisation wasn’t already aware of something, that was to be excluded from the results—see the problem here?

Many technology companies—aided by the PR organisations employed to monitor their brands—pushed the idea that big data was moving towards the field of social networking. They said big data was all about using the wisdom of the crowd and identifying the sentiment of the masses.

But social networking has not usurped much that went before, so any solution still has to include all the information feeds such as e-mail, call recordings, customer relationship management (CRM) records, scanned documents and so on. 

All the approaches cover some aspect of big data, but they all miss the point as well. The best, simple definition of big data comes down to volume, velocity and variety.

The volume aspect of big data is actually the one that is the least important. Big data is not about petabytes of data—it can be down to relatively small volumes that need to be dealt with in a manner that requires a big-data approach. 

However, for most organisations, big data will involve bringing together many different data and information sources which, by their nature, will tend to result in the overall amount of data under consideration being big. Therefore, volume is not something that is under the direct control of the organisation—what has to be considered is how the volume of data that ends up being analysed is minimised, (more on this later). 

Again, the velocity aspect of big data may well be a moot point—everyone wants results against their analysis of available data in as short a period as possible. However, everything is relative—for example, every millisecond added to providing results to a financial market trader can cost millions of pounds, whereas someone tracking variations in the global movement of tectonic plates may not be that worried if results take a few seconds to come through. 

The one aspect that really matters is the variety of the information. Big data is all about the mix of data and where it is held at any time. Here, formal databases under the organisation’s direct control are only a very small component of the overall mix. There are all the office documents held as files across the organisation and you may need to include voice and video files as well. 

Then there’s the information held in the value chain of suppliers and customers—information that is critical to the process or service being provided, yet isn’t under the organisation’s direct control. Then, there may well be a requirement to include information from the various social networks out there—and whatever approach is taken has to be inclusive.

Inclusivity of data sources
For example, it is pointless constructing something that is Facebook-specific, if most comments are appearing as hashtags in Twitter.

Further, it’s a waste of time writing multiple connectors to cover all of today’s social networks—remember MySpace, Bebo and Second Life? They were all the darlings of their day, but have faded to a withered existence, or almost non-existence, as newer players have taken over. 

Sites such as Pinterest are showing signs of major interest—yet this was also the case with Google+, which more resembles a Western desert than a viable, active social network, after just a short time. 

Any social network solution has to be able to embrace new platforms at minimal cost, so new networks that are just 'spikes' on the continuum do not use up lots of money in creating connectors specifically for them. 

Even the largest organisations will have little control over anything beyond a small percentage of the total available data. The two-edged sword of the internet raises its ugly head in that it does provide massive extra information resources—but then again, it also includes a massive amount of dross that doesn’t add anything to the sum knowledge of an organisation. 

So how are we to deal with this real big data challenge, without running into Dilbert’s pointy-haired boss’s dictat, “Just run me off a copy of the internet”? 

Storage and structure
Storage needs must be fully considered. EMC, NetApp and Dell are now talking about object, block and file storage, rather than focusing purely on high-performance database object storage to cover the various types of big data that needs to be controlled. 

Other storage vendors, such as Nutanix, Coraid, Amplidata and FusionIO provide systems that focus on one aspect of big data, partnering where necessary to cover others. 

The need for structure around semi- or unstructured data is leading to an explosion in interest in noSQL-based databases, such as Apache Cassandra, 10gen MongoDB, CouchDB and so on. Systems such as Apache’s Hadoop, (which enables a massively scaled-out platform for providing distributed processing for large amounts of data), can use MapReduce (the use of 'chunking' data analysis into packets of work that can be dealt with in a parallel manner across a large resource pool) approaches to minimise the amount of information that needs to be dealt with. 

What is being aimed for here is to take the seemingly infinite amount of available data and filter it down into manageable chunks. Standard internet searches can feed into a Hadoop-based system, which can then act as a feed into either standard SQL-based database or into a noSQL-based one, depending on the type of information being dealt with. 

Extra information can be added automatically via rules engines or manually, as required, as metadata that adds to the value of the information stored. Once the information is held in a recognised form, it is then down to being able to apply the right form of data analysis against it to provide suitable feeds to the decision maker. 

This is where the main problems still reside, but much work is being carried out. Unsurprisingly, a lot of this is coming from the incumbent business intelligence suppliers, such as SAS Institute, QlikTech and JasperSoft as well as those who have gained entry to the market through acquisition such as IBM (Cognos, SPSS), SAP (Business Objects) and Oracle (Hyperion, Endeca). 

The storage suppliers are also making plays in the space—EMC acquired GreenPlum and Dell continues to acquire companies that will help it create a more cohesive and complete big data approach. 

Buyer dos and don’ts
The key for buyers is to treat big data as a journey. Set short- and medium-term targets of what is required and then put in place solutions that help to move towards these targets. 

Don’t put in place anything that could result in a need for major fork-lift upgrades at a later date—embrace open standards, look for suppliers who espouse heterogeneity in storage systems and in tooling, as well as an approach that covers a hybrid mix of private and public clouds. 

Don’t fall for any supplier who says that the world is moving to or from “standard” SQL-based databases—the move is to a mixed environment of a Hadoop-style system paired with SQL and noSQL-based systems. Look for business analytics packages that enable links to be made to data sources of any kind that reside anywhere on the internet, and that can link into semi-structured systems such as social networking sites in a meaningful manner.

Big data may appear to be just another bandwagon at this stage—but it is important, and needs to be addressed carefully and sensibly, rather than in a bull-in-a-china-shop manner that seems to be pushed by many suppliers. The journey can be carried out at a measured pace, leveraging existing systems in conjunction with new systems. It just needs a strategic plan built from careful planning—and an eye to the long-term future. 

By: Clive Longbottom, Head of Research, Quocirca Posted: 8th May 2012 Copyright Quocirca © 2012

If an organisation is sitting on top of 10 databases, each of which is 100TB in size, it has a big data issue, right?

Not necessarily – it certainly has a problem in that it has a lot of data to deal with, but federating databases and applying data cleansing, master data management (MDM) and business analytics can provide a pretty decent solution to this. Big data introduces a lot of different problems – ones that require a bit of different thinking which may take many outside of their comfort zone.

Let’s begin by taking a simple view of information within an organisation. In the dim, dark past when I got into the ITC world, a rule of thumb approach was that around 20% of an organisation’s information was in electronic format, the rest on paper. Of the electronic stuff, about 80% was held within formal databases. Roll the clock forward by a couple of decades and this has essentially flipped – around 80% of an organisation’s information is now in electronic format, and only around 20% of that will be held in a formal database. The rest of the electronic stuff will be held in various file formats dotted around on file servers, personal devices and so on.

Any “big data” approach that just deals with the data held within databases is therefore only using 16% of the available information – not a good way to reach mission critical decisions.

This is further complicated by how information usage has changed. Back at that earlier time, an organisation’s data assets were pretty easy to define – the data was in that database that was on that server in that data centre. Now, the organisation’s information assets have to include shared information across the value chain of customers and suppliers – and then beyond that into the information held in the internet itself and across social networking sites.

All of a sudden, the “big data” approach of federating information across those large databases that the organisation controls is looking a little measly. Even if it is assumed that those databases are large – say a total of 10 petabytes (PB), or close to 1,000 times the amount of information held in the American Library of Congress – the total size pales into insignificance against the volume of information held on the internet, where other information that could be useful could be found in semi-structured or unstructured formats. The current information volume of the internet is estimated to be around 2 zettabytes (ZT) – or 2 million PB. Bringing that into the equation brings that 16% of available information that you may have thought you were acting against down to a very small fraction of a single per cent.

Sure, a lot of the available information out there on the internet is either complete dross or is not germane to the problem you are dealing with. The problem is that some of it is – the views of customers being propagated through the social networks; the performance and activities of competitors; the dynamics of the markets in which you are operating, whether these are vertical or geographic. You need the tools to identify that useful stuff, and then the means to bring it into an environment where it can be analysed and reported against in a manner that allows intelligence to be gleaned from a broader set of sources – in other words, a true big data approach.

A term that is being used around big data sums it all up nicely – it is about volume, velocity and variety. The volume side is the one everyone accepts, but is also the one that vendors have latched on to and focused on. The velocity side is where the big battles seem to be being played out – how fast can one vendor provide insights against this large volume of data that is under focus?

But variety is often glossed over – and yet it is the most important. Less structured information held in documents and spreadsheets, along with information that can be gleaned from less traditional sources such as voice and video and those internet sources alluded to earlier are all potentially relevant. Those who can use the right technologies in order to bring this variety of information sources together such that volume and velocity needs are also met will be the outright winners in world of true big data – those who just look at it as a problem with volumes of structured data under their direct control will face major problems.

For a bit more on this subject, see Quocirca’s argument on why “Big data” should be re-termed as “unbounded information”, here.

By: Bob Tarzey, Service Director, Quocirca Posted: 20th September 2011 Copyright Quocirca © 2011

There has been plenty written, not least by Quocirca, on the danger of data loss and how to prevent it. Less has been said about how to clear up afterwards; when the measures taken to protect a business from such losses have failed or were not present in the first place. In particular the responsibilities an organisation has when it comes to disclosing that such an incident has occurred.

One of the reasons for this is that legal situation is a bit vague, so there is a temptation to think that the problem can be brushed under the carpet.  Organisations that do this may find themselves in hot water if details emerge at a later date, or at least hotter water than they would have been had the leak been reported in the first place.

For any UK based business, the first stop is the Data Protection Act (DPA) enforced by the Information Commissioners Office (ICO). The specific advice on the ICO web site with regard to disclosure is as follows:

“Although there is no legal obligation in the DPA for data controllers to report breaches of security which result in loss, release or corruption of personal data, the information Commissioner believes serious breaches should be brought to the attention of his Office. The nature of the breach or loss can then be considered together with whether the data controller is properly meeting his responsibilities under the DPA”

So that’s alright then, keeping hush-hush is OK? Not really, just because the “data controller” (that is the person in any given business charged with the security of personal data) is not required to report a leak, it does not mean that the leak has not occurred. If the problem comes to light at a later date, and this is when the ICO finds out, then he is likely to take a dimmer view than if the leak had been reported up front. And remember, if personal data is involved, “data subjects” (that is you and me, in our roles as private citizens) may the first to find out and their privacy is enshrined in the Europe Human rights Act (article 8).

Furthermore, the pressure to disclose was increased on May 26th 2011, at least for certain organisations. The “Privacy and Electronic Communications (EC Directive) (Amendment) Regulations 2011” (PECR), specifically requires service providers to notify the ICO, and in some cases individuals themselves, of personal data security breaches. PECR was introduced mainly to target the use of cookies that internet service providers can use to gather personal data to personalise web services.

Beyond the DPA and ICO there are other pressures to disclose. For example, the Financial Services Authority (FSA) arguably obliges the firms it regulates to notify data breaches as part of their general reporting duties. Another standard that requires disclosure and already affects many businesses is the Payment Card Industry Data Security Standard it (PCI-DSS).

PCI-DSS compliance is required for any business that accepts payment cards – even if the quantity of transactions is just one. It is enforced via the major card brands (VISA, MasterCard, AMEX, Discover and JCB) and the obligation to disclose is in their contracts. For example VISA advises the following steps be taken:

• Contact law enforcement
• Contact bank
• Contact VISA fraud control
• Preserve logs
• Make notes of all these actions

VISA also advises:

“Make sure you have a written policy with an incident response plan and make sure all employees are aware of it”.

VISAs advice is pretty good for handling any data loss, getting control of the situation at early stage and informing effect parties makes sense for any data leak.

Beyond payment card data, there is plenty of other advice available.  Field, Fisher and Waterhouse, a law firm specialising in data protection law has a 10 point plan for handling the theft of a laptop. One point it makes is to have a media strategy, not just to get the media on side ASAP, but it may also be the most effective way of informing data subjects. This will depend on the nature of the data loss and if a criminal investigation is likely to ensue.

The trend towards an obligation to disclose data leaks is clearly happening on a number of fronts. However, even if you think a given circumstance you can get away without disclosing a leak, you would almost certainly be wrong to do so. A leak is a leak, whether you disclose it or not, it needs pro-active management from the moment it has occurred and your organisation needs to be prepared for the seemingly inevitable.

Quocirca will be presenting at the UK Infosecurity Virtual Conference on Sept 27th 2011 on the topic of “Responsible Data Braech Disclosure”, for more information go here.

By: Peter Abrahams, Practice Leader - Accessibility and Usability, Bloor Research Posted: 21st June 2011 Copyright Bloor Research © 2011

Apple has announced the next version of OS X, the operating system for Macs, called Lion. It has 250+ new features, including 11 specific accessibility features and several more that could have accessibility benefits.

OS X ships with a built-in screen-reader, VoiceOver, which has been extended to:

In previous versions you have been able to increase the size of the cursor arrow but when you did this the arrow became pixelated and the edges were rough; a small improvement in Lion is that the larger cursors remain crisp and sharp. I have my cursor at a medium size, it makes it easier to find on a large iMac screen and I look forward to this small improvement.

Another feature I use quite frequently is screen zoom. If there is something on the screen that is small, some text or often an image, I zoom the whole of the screen so I can see the relevant section blown up. The problem is that I lose the rest of the screen. Lion will offer a function to have a section of the screen in a separate window and to zoom on that. This is the best of both worlds with magnification of the bit of the screen of interest whilst still being able to see the context of the rest of the screen.

Lion improves Braille support with support for more languages and more control of the verbosity.

A significant usability feature is that for existing OS X users Lion will be downloadable from the Mac App Store. The advantage being that there will be no distribution of CD and installation from CDs. For people with disabilities this should be a welcome improvement, just a couple of clicks to download (see my article Usability and Accessibility of Apple Mac App Store) then a few more to install.

FaceTime, the video calling facility built-in to Lion, provides high-definition video which should make it possible for deaf people to use sign-language when communicating remotely. Lion improves and extends the support for full-screen apps. Full screen applications are beneficial to people with vision impairments as the content can be bigger and also there are no distractions. Full-screen should also help people with dyslexia, and some cognitive limitations. With Lion you can have multiple applications open in full-screen mode and you can navigate from one to another using a gesture.

Preview is the tool for looking at images and PDF documents. Lion provides a magnify feature to enlarge specific text or images.

Safari, the built-in browser, has some new features that will benefit people with disabilities.

The Screen Sharing feature enables one Mac to observe or takeover control of another Mac. This provides an excellent remote user support facility. Many users with disabilities will find this useful as it means that small issues can be diagnosed and resolved quickly and effectively by a remote friend.

And finally you can resize a window from any side or corner.

Lion will ship in July and is great value at £20.99 in the UK ($29.99 in the US). I plan to upgrade as soon as it ships as the accessibility benefits are significant as well as many other of the 250 new features which will improve my usability and general user experience.

By: Natalie Newman, Senior Analyst, Bloor Research Posted: 25th January 2011 Copyright Bloor Research © 2011

I am not referring to Cloud Computing but rather the cloud of confusion prevailing over geographic information amongst the general public. The confusion over this type of information; the confusion over the many terms used for information that can be linked to the earth's surface; and the confusion over maps.

Watching a TV program the other evening called, ‘The Beauty of Maps' highlighted the subjectivity of maps. The map maker has cartographic licence to create a map display which projects his interpretation of the subject; whether it is to visualise the topography correctly and read the labels easily, or to project an image that might not be true. This program described William Morgan's 1682 Map of London. He created a map of a city after it was destroyed by The Great Fire. His map illustrated the city he envisaged London would become. St Paul's Cathedral was well illustrated on the map even though it was totally destroyed and had yet to be rebuilt. Maps project what the creator intends.

There is a book written by Allan and Barbara Pease called ‘Why men don't listen and women can't read maps'.The theory goes that "due to their different roles in evolution, men had to hunt and stalk their prey, so became skilled at navigation, while women foraged for food and so became good at spotting fruits and nuts close by" [The Telegraph website]. I am not sure that explains it and, if one can generalise quite so simply, women should then be the bigger enthusiast about SatNavs. Maybe the ‘don't listen' bit prevents men from asking for or listening to directions :)

Returning to the subject—there is a great lack of understanding amongst laymen about location and geographic information systems (GIS)—as my previous article described the need to increase aWhereness. Location information—or whatever we want to call it—is simply the position on the earth's surface to the accuracy that is possible, and/or the accuracy that is required.

Initially Google Maps and Google Earth provided much needed publicity for geographic information. Google Maps, or similar, is used by most people I know to find their destination and obtain directions to reach it. Google Earth stirred an interest in places we might not visit but can view. So much good has emanated from those two applications to raise the profile of location.

The downside is that there is still not enough understanding or appreciation of the implications of geographic information and the systems. The associated costs are now even harder to sell as ‘Google is free'.

The Google application, Latitude, enables a mobile phone user to allow certain people to view their current location. I assume that these locations include both the longitude and latitude measurement; just the distance from the equator would not really help anyone.

Another term to increase the confusion, or is Google taking latitude with Latitude?

In addition, according to the latest Apollo survey table measuring the media coverage per technology company, Google came 1st in Europe and in USA, and 3rd in UK!  With that much media exposure, we should not underestimate the influence of Google!

We will have to tell a convincing story about the necessary investment to add location to your business systems. We will have to ensure that the longitude accompanies the latitude and makes good sense.

That means we, geographic professionals will have to work that much harder to tell—and sell—our story.

By: Peter Abrahams, Practice Leader - Accessibility and Usability, Bloor Research Posted: 11th January 2011 Copyright Bloor Research © 2011

In December 2010 the British standards Institute (BSi) published "Web accessibility - Code of practice (BS 8878:2010)" http://shop.bsigroup.com/en/ProductDetail/?pid=000000000030180388; this document is based on, and replaces, "PAS 78: Guide to good practices in commissioning accessible websites". It extends, updates and improves on its predecessor and is therefore essential reading for anyone intending to create or update a web product.

This new document, like its predecessor, concentrates on the processes, procedures and practices required to create an accessible web product; it does not discuss coding or technical issues but does provide references to relevant standards, guidelines and practices; so there is no conflict between this standard and the guidelines produced by the W3C Web Accessibility Initiative (WAI).

Jonathan Hassell, from the BBC, who lead the development of the standard says "Most web product managers know accessibility is important, but need a guide to the decisions they make during product development which can impact disabled and elderly users of the types of multi-platform, interaction-rich products they are creating. BS8878 is that guide, and encompasses the best advice and experience from many experts from all round the world on how to make products that include these people.".

Firstly it describes the policies and structures that an organisation needs to have in place to support accessibility.

Secondly it describes a series of steps required to create an accessible web product. The steps are summarised in the document as follows:

• Research and understand the requirements for the web product;
• Make strategic choices based on that research;
• Decide whether to create or procure the web product in-house or contract out externally;
• Produce the web product;
• Evaluate the web product;
• Launch the new product;
• Post-launch maintenance.

The document describes the specific accessibility issues that should be considered at each step. At first sight this may look like a lot of new work but in reality nearly all of the steps are considered good practice for any web product development.

This is followed by an introduction to the existing guidelines for developing accessible web products as well as discussion of accessibility of non-browser interfaces and special consideration when developing for older users.

Finally there is a detailed section on "Assuring Accessibility throughout the web product's lifecycle", which identifies and discusses the various methods of accessibility validation.

Graeme Whippy, of Lloyds Banking Group, one of the authors of the standard, said "Lloyds Banking Group is committed to best practice in accessibility and sees significant business benefits in making our websites as accessible as possible".

The standard is about 90 pages long and the second half is made up of fifteen extremely useful annexes. These cover areas such as definitions, laws, standards, responsibilities, challenges, examples of web accessibility policies and statements, guides to testing and a comprehensive bibliography.

I have read the standard and found the information in it clear, concise, insightful and pragmatic. It is laid out in such a way that it can be read in small chunks as required by different audiences and steps of a project. It provides all the parties involved in the creation of web products the information they need to understand the issues, decide how to proceed towards an accessible product and, importantly, how to deal with real world conflicts between ultimate accessibility and other market forces.

It provides a single source for accessibility best practice and information on the law and standards regarding accessibility.

The only criticism I have is that it does not discuss in sufficient detail the importance of ensuring that new content added to the web product after launch is accessible. It hints and implies that this is essential but does not highlight the issue.

Having seen the document, Gail Bradbrook of Fix the Web, an organisation set up to help people with disabilities report web accessibility issues and get them fixed, said "if every web product used the standard then we would not be needed and could close down; unfortunately that is not the case yet and we are very busy and need more volunteers (see http://www.fixtheweb.net )."

To ensure the maximum benefit is obtained from the standard there is a need for a community to be built up around the standard that can add to and refine the standard based on new experiences, technologies and opportunities and I expect some organisation will step up provide the platform for this community.

The standard is an essential purchase for anyone creating web products, as it provides:

• Pre-digested research into accessibility and best practice;
• A roadmap showing how to ensure accessibility is built into web products;
• A template for recording the decisions made about accessibility which will help to show good intentions if complaints are made.

Its cost should be recouped within a few days of starting any significant web product development and it will continue paying dividends throughout the whole life-cycle. It should be used by all commissioners and developers of web products.

By: Nigel Stanley, Practice Leader - IT Security, Bloor Research Posted: 21st December 2010 Copyright Bloor Research © 2010

I recently presented at a webinar alongside LogLogic on the issues of compliance for IT professionals. Here is an edited transcript of my talk.

Until fairly recently, information security people were buried away in server rooms configuring firewalls and patching servers. With the sudden surge of compliance and regulatory requirements being placed onto a business, IT security people are now required to understand and help implement compliance solutions.

But how can security teams help join the dots between their security work and compliance issues? How can compliance requirements be met without placing undue strain on the organisation causing paralysis by analysis? How can information security people add value to a business following a compliance agenda?

The pressure to deliver a secure IT infrastructure against a background of constantly changing compliance and regulatory demands is tough, and not helped by a reduction in budgets to achieve this ever-changing goal. The first part of this process is to get an understanding of exactly what compliance requirements you need to be worried about and, more importantly, those that can be put to the background. Not only do we need to consider state laws, federal laws and international laws, there are industry-specific regulations that further complicate the picture. Those organisations trading across international boundaries face even more challenges as they get to grips with different legal structures and cultural demands. During this webinar you will have a chance to learn about the realities of achieving an acceptable level of compliance for your organisation, and hopefully get some help for your work down in the trenches.

I would imagine that everyone knows only too well the demands on us as information security professionals. I think it could be argued that we have one of the most difficult jobs in the IT business as we need to be seen to add value whilst at the same time often saying no—often a contradictory position.

As the current financial situation rolls on we are faced with doing more with less, and organisations are increasingly worried about reputational risk more than ever before as any damage to the business will have an affect on often slim profits. This work needs to be balanced with the relentless slog of dealing with malware and other unexpected gotchas waiting in the wings to pounce.

Some of us are lucky enough to enjoy a lot of support from the executive team downwards. Unfortunately other boards may see the information security role as nothing but a pain and something they wish they could make go away. If this is your position you have my sympathies!

Data security is now getting a lot of attention as it is subject to legal and regulatory compliance requirements. Failing to adhere to appropriate laws and regulations can result in legal actions, fines, reputational risk and maybe, in extreme circumstances, imprisonment.

The benfits of compliance
Achieving  compliance, in the broadest sense of the word, can be a good thing as it often instils good practices and procedures.

On the other hand over-compliance can be detrimental as the business can be bogged down in achieving a goal that delivers little direct business benefit. Many medium-sized businesses are struggling with compliance requirements as they are big enough to be caught by various requirements but too small to have resources to cope. Of course failing a compliance audit can result in lots of difficult questions from the board of directors, shareholders and partners.

The only thing we can promise is that there will be more compliance and regulatory requirements coming down the line to affect data handling and security. The demands of a business culture that is becoming more and more compliance oriented can be major. The problem is that this change in culture leads to some strange ideas.

One objection to additional security spend I hear from businesses is that they are fully compliant, as proved by external auditors, and therefore don’t need much or any more investment in their IT security systems. Some business managers are then astonished when they realise that security has been breached, especially after they had spent considerable sums on establishing this compliant business environment. Indeed, the fact that the business is compliant, whatever that means, has induced a level of complacency in some as regards information security.

IT security managers have a need to help educate business managers in the differences between compliance and security. That way a business can make investment decisions based on accurate information rather than assumptions.

I feel for medium-sized businesses that are captured by the compliance net but have little or no resources to meet what can be seen as an onerous requirement. Fortunately some compliance and regulatory demands have planned for this and offer suitable break points so that small and medium sized business don’t fall foul of regulations whilst being able to run their day to day business.

The cost of poor compliance
So what about the real cost of poor compliance and bad information security? In March 2010 Zurich Insurance announced that it was going to improve its information security after losing personal financial information on 46,000 British clients through careless handling of unencrypted back-up tapes.

The back-up tape, which also contained personal details of 1,800 third party insurance claimants from the UK, was lost by Zurich's South African sister company during what was described as a routine transfer to a data storage facility in South Africa in August 2008.

In total, 51,000 British records were on the tape, along with a much larger number of details about Zurich customers in South Africa (550,000) and Botswana (40,000). Zurich's UK arm wasn't informed about the problem until a year later.

They were fined the equivalent of $5m by the Financial Services Authority, the highest fine levied in the UK on a single firm for data security failings. This is the cost of non-compliance.

US compliance
In many respects, the United States has led when it comes to data security laws that mandate stricter requirements and harsher penalties if data is compromised.

The implementation of state-level data breach notification laws in California in 2002 was seen as a prime example of addressing individuals' concerns about their data privacy. In this case, if personally identifiable data has been lost then those individuals possibly affected must be notified and steps taken to help them manage any ongoing consequences. 44 of the US states now have similar laws in place but, of course, if data has been demonstrably encrypted, then there would be no obligation to disclose its loss.

Since 2002, many US states have introduced even more draconian laws. The state of Massachusetts has introduced regulation 201 that is designed to protect personal data, for which encryption plays a big part. The compliance date was set for January 2010 and violators face penalties of $5,000 per infringement.

Other US laws encompass data security and imply that data encryption is required, even if it is not explicatively stated in the legislation.

The Health Insurance Portability and Accountability Act of 1996 gives powers to the Department of Health and Human Services to watch over and enforce rules applicable to the safe and secure handling of patient data, including that which contains personally-identifiable health information. It is applicable to all entities that use such data, including healthcare providers, insurance companies and public health authorities. There are three safeguards that need to be implemented covering administration, physical and technical areas of data management. The technical safeguards require that patient health information is not improperly modified and any deliberate misuse could result in a prison term.

The Sarbanes-Oxley Act of 2002 was intended to improve the regulation and accountability of publicly owned companies following the spectacular corporate failures that occurred in the early part of that decade. Under Section 404: Management Assessment of Internal Controls of the Sarbanes-Oxley Act, there is a need to prove the integrity and confidentiality of financial information.

The U.S. Congress passed the Gramm-Leach-Bliley Act (GLBA), also known as the Financial Services Modernization Act, in 1999 to assist in the growth of the US financial services industry. One part of the Act (Sec. 501b) addresses the safeguarding of customer information including the integrity and confidentiality of non-public personal information and customer records.

EU compliance
The EU has a very different make up to the United States. The European Union currently comprises 27 member states. It was established following the Maastricht treaty in 1993, which renewed the union originally called the European Economic Community, or EEC. The EU generates approximately 30% of worldwide GDP and has around 500 million citizens.

The EU has developed a system of laws that apply to the movement of goods and people and the creation of a single trading entity. Each member state is subject to both EU and their own locally created national laws. There are countries that form part of Europe geographically but do not have membership of the EU, for example Switzerland. These countries are therefore not subject to EU-based laws.

As part of its remit, the EU has created business-related compliance and regulatory requirements, including laws that cover the safe keeping and management of data in computer systems. Failure to comply with these laws can result in criminal proceedings and prosecutions, so any organisation operating in the EU needs to take such laws as seriously as those developed by individual nation states.

When considering EU law it is important to understand the structure of the EU and how laws are enacted.

The EU Council represents national governments and is a council of ministers run by a 6-month rotating presidency. National ministers attend meetings as appropriate to their portfolio. The European Parliament is elected every five years by citizens of the member states. Members of the European Parliament have geographically-based constituencies that are generally larger than those for members of a national parliament.

The European Commission acts as a civil service and drafts new laws, which are passed to the European Parliament for discussion and enactment. The EU is based on a rule of law, which is laid down in a series of treaties and directives. These then become a collective legislative act of the EU, which is then enacted in member state laws. If a member state fails to enact a suitable law then action can be taken against that state in the European Courts of Justice, which is the judicial institution of the Community.

The compliance and regulatory framework in EMEA is never far from the spotlight, more so as the current worldwide financial situation is forcing regulators to review their oversight and regulatory activities in an attempt to prevent a similar crisis happening again. This is against a backdrop of relentless data loss incidents across both the private and public sector.

So let’s look at some key requirements in detail. The UK Data Protection Act is a useful example of a data privacy law and the PCI DSS is an interesting example of an international requirement put in place by a non-state organisation.

Data Protection Act
The UK Data Protection Act imposes legal obligations on anyone processing personal data to ensure there is good practice and management of that data. In part 1 of the Act there are 8 enforceable principles of good personal information handling. Data must be:

• Accurate and up to date.
• Fairly and lawfully processed.
• Secured.
• Not allowed to leave the UK unless the destination countries have similar legislation.
• Processed in line with a person’s rights.
• Only kept for as long as necessary.
• Processed for limited purposes.
• Adequate, relevant and not excessive.

Part 2 of the act gives individuals rights to find out what personal information is held about them on computers and most paper records. The UK Information Commissioner’s Office (ICO) has legal powers to ensure that organisations comply with the requirements of the Data Protection Act. A data controller who persistently breaches the Act and has been served with an enforcement notice can be prosecuted for failing to comply with a notice. From April 2010 the ICO can impose penalties not exceeding £500,000 for serious breaches of this act. We are still waiting for the “big one” to hit, but I understand there are some ongoing investigations that may result in the maximum fine. Certainly if the loss of 25 million records, as happened a couple of years ago by the UK’s HM Revenue and Customs happened today then the ICO has publicly stated that it would have levied the maximum fine. Then, of course, we have discussions about public money travelling from one place to another but that is beyond the scope of this presentation.

In Germany the Bundesdatenschutzgesetz (BDSG), adheres to the seven basic principles of EU Directive 95/46/EC in the protection of data relating to individuals or data that allows an individual to be identified. The 16 Länder have their own data protection regulations that cover local public bodies. These local regulations are similar in spirit to the Federal Data Protection Act. In July 2009, German legislature passed a number of amendments to the act to strengthen its powers. Most notably there was a new requirement introduced to provide notification of data breaches in a similar way to the United States. These were effective as from 1st September 2009.

PCI DSS
This is probably one of these regulations that appears to have achieved a good compliance vs. effort balance as organisations that I work with are generally satisfied that they can achieve their required level of PCI DSS compliance without it breaking their businesses. If you take a look at the 12 requirements of PCI DSS no one could argue against the sanity of putting in place these measures:

• Build and maintain a secure network including installing and maintaining a firewall configuration to protect cardholder data and not using default passwords.
• Protect cardholder data and encrypt transmission of cardholder data across open, public networks.
• Maintain a vulnerability management program and use regularly updated anti-virus software. Develop and maintain secure systems and applications.
• Implement strong access control measures and restrict access to cardholder data on a need-to-know basis. Assign a unique ID to each person with computer access and restrict physical access to cardholder data.
• Regularly monitor and test networks and track and monitor all access to network resources and cardholder data.
• Maintain a policy that addresses information security.

I don’t see how any information security professional could argue against implementing these requirements as they all go to make up a commonsense set of security structures. Having recently had my credit card details stolen I am as keen as anyone to see merchants achieve a better level of security and compliance.

Contrast the relative clarity of PCI DSS with the Sarbanes-Oxley requirements in the US. This imposes rather mystical requirements on information security. For example section 404 of Sarbanes-Oxley requires organisations to, “provide internal controls and report on their effectiveness” and section 802 says that organisations must, “ensure the integrity and availability of records”. This is a charter for auditors to make a lot of money!

As we have seen, compliance is now a big requirement for many businesses and I think most people would agree that the depth and breadth of compliance requirements is only going to deepen. As organisations switch on to the world of compliance they realise that it is far more cost-effective to run compliant systems 24/7 rather than hastily scrabble to clean up prior to an audit. Those days should be long gone and organisations should ideally be “audit ready” at all times, or at least strive to be. Any investments in systems that assist in gathering data and then produce compliance documentation will inevitably be proven to be a wise one, if even in the short term there is some practical and fiscal pain in purchasing and implementing the system.

This is where knowing the unknowns can pay dividends. I worked with a very large organisation recently that was feeling under pressure to come up to scratch from a compliance viewpoint. The IT infrastructure was (and indeed is) huge, and quite frankly systems, servers, networks and deployments ran away with themselves for a number of years. The IT management was feeling overwhelmed and needed to try and get a grip. To that end they installed and configured some automatic discovery tools to try and scan the network to see how it matched with their “official” documentation. The scale of additional network segments, hidden wireless access points, secret departmental databases and a wealth of other unauthorised IT was frightening.

This shook up the management and lead to a far more structured planning and network management process. Luckily they managed to get most of these issues addressed prior to a looming audit.

Compliance adding value
We, as information security professionals, need to be adding value to the business. Instead of being seen as the people that say no, we should be a conduit to ease the implementation of compliance systems. By understanding not only the technical challenges of compliance requirements but also the business context we can be seen to add value from the off. The good news is that, as we have seen, investing in compliance can also help us deliver a secure working environment. That said, it is beholden on us to ensure the business really understands the difference between compliance and security but at the same time sees the improved business case of delivering appropriate security projects on the back of a compliance requirement. Information technology can be notoriously complex and we often see business managers chased away from involvement in decisions related to technology. Whilst this may be appropriate in very narrow technical decisions it is important that business understands IT and how it is benefiting the business.

From a compliance perspective it is very easy for the business to be frightened by talk of liabilities, whilst technicians appear to spend budgets with limited care for the overall business benefit. When considering IT compliance, it is imperative that a strategic approach is taken based on clear, rational thinking. Many businesses have rushed into a technical solution that was sold as solving compliance issues only for them to quickly realise the limitations of the product.

IT security professionals have a responsibility not only to define an effective technical solution but to ensure that the solution is developed and deployed to mitigate fully the exposure and risks facing the business. Businesses must recognise that IT security is not only an important aspect of today’s business requirements but a permanent feature, the importance of which will only grow as the rights of the individual are ever more politicised and enshrined in EU and national law.

Data is either static or on the move. In both cases businesses must be able to secure it and to demonstrate to all parties that it is doing so. In our industry nothing stays still for long.

A word of caution now needs to be sounded about cloud-based systems and compliance. The race to the cloud has seen a number of organisations fall foul of data protection regulations and issues such as data privacy. Of course the cloud delivers some interesting business benefits but these must be balanced against the associated security and regulatory issues—joining the dots between security and compliance initiatives when talking about cloud computing can be very tricky.

The good news is that aligning information security and compliance, although a challenge, is probably getting easier now than it was up until a couple of years ago. The availability of tools to help in this process should reduce the compliance headache and help us get some value out of the compliance process.

New compliance requirements
We have seen new compliance and legislative requirements continually emerge in response to political initiatives, market dynamics and the need to manage new technologies.

Although many of these were not directly aimed at IT systems it is inevitable that such systems will be used to transport, store and manage data that will be subject to audit and control. There will therefore be a need for data to be held and moved demonstrably in a safe and secure way such that integrity is retained.

Examples include the UK’s smart metering initiative, where household energy meters will be upgraded to devices connected to a network and data transferred automatically to central billing facilities. Requests for data privacy comments have been made by OFGEM, the energy regulator. Although a lot of existing regulations and laws such as the Data Protection Act will be applicable it would not be surprising if tailored requirements emerge.

Effective governance that protects all constituents and demonstrates compliance and clear corporate responsibility will become an increasingly key component of data-related business solutions. Increasing awareness of the consequences of non-compliance will drive requirements for transparency and complete end-to-end visibility of data movements within the enterprise and, ultimately, throughout the value and supply chain.

Does compliance = MOT?
I will leave you with one last thought. Here in the UK, after the second world war, lots of people were driving cars that were in pretty bad repair—brakes were poor, lights were damaged and steering was often ropey. This lead to accidents and injuries that could have been prevented. In 1960 the Ministry of Transport introduced a compulsory test, now commonly called the MOT,  on all vehicles over 10 years old in an effort to ban the most dangerous cars from the road. Over time the age of annual tests reduced to its current of 3 years and the breadth and depth of the MOT has now expanded to incorporate new technologies such as catalytic converters.

Is the growth in IT related regulations and compliance requirements following a similar trajectory to the evolution of the MOT test?

All in all we now see far fewer old bangers or clunkers on the road than at anytime in the past and I wonder whether we will benefit in seeing fewer data breaches and security lapses as computer systems are put through regular audits or MOTs.

Of course the mistake many people make when buying a car is to assume that a current MOT certificate is proof that a vehicle is roadworthy. Of course it isn’t—all it means is that at the time of testing the car was able to pass the MOT test.

In a similar way a computer system may pass an audit but very rapidly collapse into a state of non-compliance due to mismanagement. Constant attention to audit and compliance is the only sensible way to manage these needs.

Who knows, with the development of decent compliance and regulations we may see less dangerous IT systems and fewer data loss accidents and mishaps!

By: Nigel Stanley, Practice Leader - IT Security, Bloor Research Posted: 17th December 2010 Copyright Bloor Research © 2010

There are a variety of ways in which websites and public-facing computer systems can be attacked by hacktivists, and attacks on websites continues to be a popular form of political demonstration.

In December 2010, around 36 Pakistani government websites were hacked by an online hacker group called the Indian Cyber Army. All hosted on the same server, the sites that were hacked included the Pakistan Army, the Ministry of Foreign Affairs, Ministry of Education and the Ministry of Finance. The attacks consisted of messages and graphics inserted into the web pages with political messages, some of which related to the attacks in Mumbai.

Also in December 2010 a number of financial payment websites were subject to denial of service attacks by hacktivists disgruntled at these companies no longer processing payments to the WikiLeaks website.

For commercial websites that trade across the internet, this can be catastrophic and is the equivalent of having all their real-life stores closed down in one go. Denial of service attacks can range in their level of sophistication from destruction of physical internet connection points through to the flooding of websites with extraneous data that overwhelms web servers, forcing them to close down. This is similar to blocking the switchboard of a business with lots of phone calls that are terminated as soon as they are picked up, but uses the TCP/IP protocol that runs the internet to flood servers with bogus messages. These attacks can be coordinated using hijacked networks of computers, called botnets, which, in turn, are forced to send high levels of spurious data to target websites. There are steps that designers can take to mitigate such attacks but, in reality, a significant attack can be difficult to manage, and often the best course of action is to take down the servers and hope the attackers go away.

More sinister is a malware threat that emerged in 2010 called Stuxnet. Researchers had been aware of this malware for many months, but it hit the media headlines when reports emerged of Stuxnet finding its way into Iranian nuclear plants. Excellent investigation by Symantec [1] has enabled us to see inside this malware and understand how it works.

The malware was apparently written to target industrial control systems such as those used in manufacturing and processing plants. Its ultimate aim is to reprogram control systems by modifying computer code on programmable logic controllers, or PLCs, in such a way that plant operators would never suspect anything was wrong. In contrast to a denial of service attack that is extremely noisy, Stuxnet is a very clever and covert attack. Bundled with the Stuxnet malware is a whole arsenal of additional components designed to assist in this control system attack, including zero-day exploits, antivirus evasion and a Windows rootkit, an advanced form of malware.

So why bother to mess with PLCs?

In fact Stuxnet only affects specific PLCs controlling electric motors that run at special high speeds and frequencies. These are only available from two specified companies and the attack will only be initiated if there are at least 33 of these devices present. The majority of Stuxnet infections were found in Iran and these devices are regulated for export by the United States Nuclear Regulatory Commission as they can be used in centrifuges used for uranium enrichment.

Yes, the implication is that Stuxnet is a powerful piece of malware created to disrupt the enrichment of uranium by the Iranian government.

Clearly this advanced malware has not been developed by a back-bedroom hacker, as it needed very specific insight into the workings of complex industrial control systems. This is a high watermark in terms of malware, and evidence is starting to emerge that conventional cybercriminals are adapting Stuxnet for more conventional criminal activities.

We have not seen the end of Stuxnet yet.

Is your organisation a target?
It could be argued that, in the great scheme of things, most businesses and organisations will never appear on a cyberterrorist’s radar, as the type of work they do is not one that attracts attention from such people. On the other hand it could be argued that every person and organisation is a target for cybercriminals, so a reasoned, objective risk assessment should always be undertaken to gauge a likely risk profile. This must include all aspects of a business, including the supply chain, employee travel, executive profiles, nature of the business and, of course, the ever-changing worldwide geopolitical situation.

This risk assessment needs to be continuous and fully integrated into the decision-making process of the leadership team. Informing this risk assessment must be intelligence gained and shared with colleagues, industry communities and the authorities ensuring a two way flow of up to date, actionable and relevant information.

Polices and procedures need to be built that encompass this risk assessment and it is vital that a converged approach is taken, such that information security experts work with physical security experts to develop plans and skills to manage a cyberterrorist attack. These attacks will rarely come from nowhere and the sharing of skills and information is vital.

Employees are often in the front line against cyberterrorists, as their day-to-day activities are often subject to reconnaissance and investigation from potential attackers. Phishing emails, social engineering phone calls and strange conversations are just some of the indicators that an organisation is being scoped for attack. These users must be educated about the importance of both physical and information security, supporting a converged approach, in their day-to-day jobs and have a means to raise their concerns in an open way that supports these reports and avoids any embarrassment if a genuine report is false.

Finally, organisations and businesses need to be doing their job, focusing on delivering value, products and services to their clients and shareholders. In support of this it makes complete sense to work with expert third parties that can take on a lot of the risk management work, freeing up the business to do what it does best.

Summary
Over these 3 articles we have seen that the internet is awash with threats to organisations and individuals, but it is also an amazing force for good in the world, supporting commerce and the freer flow of information. Inevitably criminals, rogue states and terrorists will see the internet as an ideal tool in their armoury but, by taking some reasonable precautionary steps, many of these threats can be significantly reduced.

References

[1] Symantec. Stuxnet: A Breakthrough. Available at  http://www.symantec.com/connect/blogs/stuxnet-breakthrough Last accessed 9th December 2010

By: Dana Gardner, Principal Analyst, Interarbor Solutions Posted: 23rd November 2010 Copyright Interarbor Solutions © 2010

Three megatrends are shaping the next generation of successful businesses and governments. We're talking about pervasive mobile applications, highly responsive cloud-computing models, and knowledge-adept social collaboration.

Indeed, by the year 2020, The Economist newspaper predicts there will be two trillion devices connected to the Internet. And taking a look at where we are right now, McKinsey Quarterly reported in August that in 2010 some four billion people have cell phones, and 450 million have access to a full web experience.

Moreover, Jupiter Research reports that by 2014 there will be 130 million enterprise users involved with mobile cloud activities. Not only is access pervasive, but the amount of information available is also exploding. The Economist again reports that in 2005 mankind created 150 exabytes of digital data … and in 2010 we will create fully eight times more data.

These changes are at a pace they’ve never seen before as they address them and try to drive these into their business or government environments.

As these trends literally rearrange business ecosystems, a gap will surely emerge between the companies that master change -- and exploit enabling technologies -- and those that fall ever further behind.

For those that do step up to the challenge -- expect a relentless emphasis on rapidly recurring innovation to meet dynamic customer and citizen demands.

Our latest BriefingsDirect podcast therefore focuses on how these trends -- and rapidly evolving customer, citizen, and user expectations -- are newly impacting the enterprise. We also examine how technology advancements are making it possible to drive innovation to meet these new demands for instant gratification.

Please join HP executive Dave Shirk, Senior Vice President of Worldwide Marketing at HP Enterprise Business, as we explore how HP is working to make headway, so that the next few years bring about a generational opportunity -- and not a downward complexity spiral. The discussion is moderated by BriefingsDirect's Dana Gardner, Principal Analyst at Interarbor Solutions.

Here are some excerpts:

Shirk: We're seeing a lot of shift going on in the marketplace right now. When we look at where consumers are driving business or where citizens are driving government, it's fundamentally changing the way they operate. We've seen three core things come out.

The business models are all starting to change the way in which people approach markets across the globe. That's having to really rethink the ways in which they've approached them versus traditional methods.

The second thing we see is this whole shift in mobile computing meeting cloud computing and the enterprise trying to figure out exactly how to take best advantage of that to create this competitive advantage. Then, the overall demographic piece weighs into that.

We've seen the rise of the millennials, as they're being referred to. All of these things are forcing business and government to stop and say, "You know what, if we're going to grow or we're going to create a service differentiation, we're really going to need to do things differently and we're going to have to do it way faster than we've ever done it before."

According to the Society for Engineers, you now have over 800,000 graduates in China, over 300,000 graduates in India, 100,000 some in Japan, etc. It's over the last 10 to 12 years that each of those graduation rates has occurred. They are part of the workforce now.

When they went through that process, they were always connected and they always were involved in a social network-based environment. They have a level of their lifestyle that is all tied to this always-connected environment. When you think about the ubiquitous computing that that has brought to them, as they enter the workforce, they are looking at things a lot differently than ever before.

They bring new ideas. They bring new ways to that. They're looking for businesses that will support that kind of methodology and structure. ... So, when we think about that Gen X group that's out there, we see them driving an enormous part of this change.

The last statistic I saw was that they are now over 50 percent of the workforce. The analogy that's always used is that, to them, being connected and always involved in some type of networking-based collaboration or information sharing of some sort is about the same as it is for you and me to pick up our remote controls and turn on our television sets. That's already having a very profound effect on how business and government are changing and the expectations that are out there in the marketplace.

It's this [demand for] immediate or instant gratification: "If I can't get what I want in the following way, I’ll find the business or government environment where I can." While the government piece maybe a bit harder to change, the business piece isn't, and so the competitive pressure to serve this audience, both as the consumer and also as employees, is a big part of that shift.

We see technology as the cornerstone to being able to solve some of these trends and some of these challenges.

We call that the "Now Problem." They want this, they want it done now, and they want it to work a certain way. We see technology as the cornerstone to being able to solve some of these trends and some of these challenges.

These changes are at a pace they’ve never seen before as they address them and try to drive these into their business or government environments.

This is probably best represented in the words of Professor Gary Hamel, who is the foremost business visionary person out there in the marketplace. In his book, Future of Management, he described it as "whiplash change."

That's very much the case when I speak with our clients both on the business side and the government side. That's exactly what they're sitting there and thinking and working through right now.

Role of technology

We look at the technology piece of [the change] and say that you really can't [react] any other way -- the pace of it, the speed of it, and some of the complexity associated with it. For a long time, business has tried to use labor as an arbitrage to try to work their way through this and just throw bodies at it. That's quickly dissipating. The speed and the connectedness that we see, and the confidence level that all of these types of services require make it no longer possible to go through that.

What we see is IT completely embedded in the business. Over the next couple of years, that's going to continue to be the trend and the strategy that will play out in the way in which business and government work this. Ultimately, that's going to be the differentiator that drives an ability not only to serve these constituencies but to out-serve them, and that's going to be the name of the game.

[The solution] starts with a desire to change and to drive innovation in a different way. We sit and we think about the fundamental change in this. We talked for years that the business was focused on business processes and business process reengineering. While that’s still very important, it isn't going to go away any time soon.

It's becoming obvious that the bigger driver and the more significant trend is the information process, understanding the segments of business or government that need to be addressed. What their needs are, what they want, what they want to talk about, the ways in which they want to interact is all part of this change that’s taking place.

Closing the gap

So, as we start to pull back and step back from this, we look at that and we look at this vision that we have for the Instant-On Enterprise and how we’re enabling end-users to become a part of that, how we’re enabling businesses and governments to provide that type of capability. It really is about closing the gap between what IT can provide and what the business needs to be able to serve each of those audiences.

What we’ve launched with this vision is to put the foundations in place to make that possible and take a journey with our clients both from the business side and government side and help them move down that particular path, find ways to navigate these challenges and these trends, and to out-serve and to over-serve all the audiences that they need to meet the needs of.

[This change] is inevitable. Different businesses and governments will have, at different times, one of these four elements be more important or more significant to them at different points. All of them share the innovation requirement. We see that in all things.

Our view is that the innovation has to take place throughout that information process. It doesn’t matter whether it happens back at the data center or at every touch point. Innovation has to take place throughout for the business to meet the needs of those segments I’ve referred to earlier -- how it services it, how it conducts itself, and ultimately how it meets our needs or exceeds the needs of the audiences.

Agility really is about instant expectations, and can we turn things on and off, instead of just setting them up for a rainy day and hoping that they will be used.

Agility, optimization, and risk all vary in and out with innovation in terms of their need and their level of importance.

Agility really is about instant expectations, and can we turn things on and off, instead of just setting them up for a rainy day and hoping that they will be used. A big part of technology’s trouble in the past was that we created all of these things and we never had a plan for ending their lifecycle or turning them down slightly, so that we could turn up other activities or other possibilities in an instant-on environment and an instant-on enterprise. A core part of the vision that we see is being able to drive that agility to meet those changing business needs.

When HP looks at the Instant-On Enterprise, the enablement of that is really a journey, and we’ve got to figure out what pieces make the most sense. There are some things that are much easier to focus on first and then, over time, to gain more and more of an Instant-On nature.

Critical success factors

Flexibility, security, speed, automation, and insight, those absolutely are attributes that we look for. We see them as the critical success factors in the way in which every part of the environment that IT leverages, drives, and embeds in the business has to come forward.

And yet, everybody is stuck in this mode of an enormous legacy that they have to deal with, and that gets in the way of being able to provide some of these new capabilities.

We’ve spent a lot of time and gotten a lot of expertise over the years trying to figure out the best ways to address these albatrosses that are keeping IT from being able to deal with the needs of the business. In the Instant-On Enterprise journey, that's a big part of the set of steps that we have to work through and work with our clients to make sure that they understand where to prioritize.

In the first few months that I have been here, one of the things that I've learned is that HP, as a company, has this incredible breath and depth of portfolio.

Our view is that we work with our clients and figure out ways that they can, as we say, shift that equation. How do you shift from 70 percent of that equation being focused on operational management, and 30 percent, if you are lucky, being spent on new and innovation-based capabilities to help or assist the business and its growth versus shifting it the other way? How do you get to 30 percent operational mode, and move forward with 70 percent focused on the business?

Changing business models

When I spend time with clients and listen to them, a big part of what they're asking for is, "We’ve got these pressures. We're seeing the business models change and we're experimenting with some things. We're seeing the mobile and the cloud computing pieces coming at us like a freight train. At the same time, we're seeing the demographic shift both on the end-user consumer side and on our employee side. We need strategic partners to help us with this. How do we navigate this? What is the way in which we should do that? HP, do you have a point of view?"

We're in a unique position, because we're the only company in the marketplace that has a full suite of consumer products, and yet we stretch all the way back through to the data center. All the capability, all the offerings, that are in between, all the services that are necessary to address each of those pieces, are contained inside the portfolio capability that HP has of hardware, software, and services.

We looked at this and said, "How do we take the best combination of that breadth of portfolio and bring those together in a set of solutions to best address what we are hearing over-and-over from some of the research that we’ve done and listening that we’ve done with our clients?"

They need to figure out how to modernize their applications. We want to make sure that we are there and we’ve got a set of solutions for that. They’ve got huge data-center issues in terms of how they're going to transform their data centers and deal with more virtualization-based techniques and capabilities and bring networking and storage and compute power together in some fashion.

They’ve got this issue of enterprise security. They need to figure out how to secure the enterprise. I don’t mean desktops, but all points, all touch points of the enterprise -- how they build applications, how this information is accessed inside and outside of the organization, and then fundamentally optimizing that information, the ways in which you store it, the way in which you deliver it, the way in which you print it for that matter, all those pieces.

Hybrid delivery for us is our answer to the multiple ways in which a customer or client has to go through the process of building or delivering on these various technology services to their enterprise or their government.

Then, they need to underpin that by the best way to figure out how to deliver it. Do we do it for them? Do they build it themselves with our architecture, and our capability set, and our consulting expertise? What combination of ways makes the most sense to set that up?

... We help our clients work their way through that with a series of workshops that we do to get in and investigate. We ask a series of questions, do a series of exploratory-based activities that help prioritize where we think the quickest return on investment is, because all these require some level of return to feed the next one and then the next one.

Hybrid delivery for us is our answer to the multiple ways in which a customer or client has to go through the process of building or delivering on these various technology services to their enterprise or their government.

There’s an enormous amount of talk about cloud in the marketplace today. HP has been at the forefront of that, but we have a little different position. We think it’s unique and we think we're the only ones out there that are really positioned to do this, which is the concept of hybrid IT, where you’ve got a mix. You’ve got a mix of traditional on-premises-based capabilities, but then you figure out what private cloud or public cloud-based capabilities best serve your business on a global basis.

HP comes in and, unlike other companies that try to force you into a one-size-fits-all structure, we sit down with the client. Our unique IP in this area is that we have an incredible depth of intellectual capital in this particular area, which is helping the clients figure out the best balance or mix of the delivery methods.

We can help them build it. They can host it or we can host it for them. We can provide those services from our public cloud-based capabilities or from our private cloud based capabilities. We really don’t care, if that blend changes over time. That’s the beauty to the journey to this Instant-On Enterprise.

Starting small

Our data says that most customers still start with a small private cloud implementation to really understand the value of the cloud and demystify it. We’ve said that there is going to be something after cloud. We don’t know what that level or that style of computing is going to be, but our architecture is built such that we’ll be ready for that. For our clients, we’ll help navigate them through each of these pieces, and that’s the important thing for us.

We have our new HP Hybrid Delivery Strategy Service, which is a place for a client to start, get a basic orientation, sit down and understand kind of where we think they might consider beginning that journey. So that, along with a number of other capabilities that we have to help them through these various workshops, I think is really the best place for them to start.

There are a whole series of workshops globally that our teams are set up to do, everything from a small couple-of-hour based interaction to a full suite of in-depth analysis and consulting engagements to work with a client. ... We ask a series of questions, do a series of exploratory-based activities that help prioritize where we think the quickest return on investment is, because all these require some level of return to feed the next one and then the next one.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a full transcript or download a copy. Learn more. Sponsor: HP.

You may also be interested in:

• Shoemaker on how HP CSA Aids Total Visibility in Services Management Lifecycle for Cloud Computing
• HP Business Service Automation portfolio gives IT the tools it needs to compete with clouds
• HP eyes automated apps deployment, 'standardized' private cloud creation with integrated CloudStart package
• HP adds new consulting services to smooth the enterprise path to cloud adoption

By: Dana Gardner, Principal Analyst, Interarbor Solutions Posted: 16th November 2010 Copyright Interarbor Solutions © 2010

The lives of IT admins in Windows environments should get a little easier with the launch of rPath's rBuilder 5.8 for "push-button" deployment of Windows Server instances.

The Raleigh, N.C. company's rBuilder 5.8 introduces release automation to the world of Windows Server applications. With the new software, rBuilder 5.8 earns bragging rights as a first commercial solution to address deployment automation for Windows instances and apps. [Disclosure: rPath is a sponsor of BriefingsDirect podcasts.]

The deployment challenge

For most IT organizations, deploying Windows apps into production is complex, cumbersome, and time-consuming. That complexity can lead to long delays in full deployments that leave a dark cloud hanging over service levels and business agility.

The rise of public cloud services such as Amazon EC2 has further motivated IT to become more responsive to business lines.

With its automation approach, rBuilder 5.8 is wrestling that challenge to the ground with what it calls “push-button deployment” of Windows apps. This software helps to automatically resolve dependencies to virtually eliminate deployment-time failures, automatically generate standard MSI packages that are ready to deploy, apply version control to all packaged elements, and eliminate drift between dev, test, and production release stages, says rPath.

rBuilder 5.8 also generates image output on demand for rapid deployment or retargeting between physical, virtual, and cloud environments, makes way for targeted changes for low-overhead, conflict-free maintenance, and provides a single enterprise solution for automated deployment of any application, running any platform, deployed to any execution environment -- physical, virtual, or cloud, said rPath.

There are some more resources available on the capabilities and new release: Attend a free, live webinar Nov. 16; watch a short video; read a whitepaper, and learn more.

The need for deployment speed

Deployment dysfunction is a primary source of delay in delivering IT services in response to business demand. The rPath solution also works to complement Microsoft development and operating environments, including Team Foundation Server and System Center Configuration Manager.

With some 70 to 80 percent of IT spending due to operating expenses, nearly half is attributable to deployment-related tasks. This is particularly true for Microsoft Windows environments, which constitute 74 percent of the data-center server market. If rBuilder 5.8 lives up to its promises, it could find a home in many Windows-based IT departments. And it lends a hand in migration and hybrid deployments, too.

rPath has also joined the Microsoft System Center Alliance, a partner community in support of the System Center ecosystem. The System Center Alliance provides an online community that aims to help partners collaborate on the creation of solutions for the System Center and deliver an information resource about these new solutions for customers and sales channel partners.

BriefingsDirect contributor Jennifer LeClaire provided editorial assistance and research on this post. She can be reached at http://www.linkedin.com/in/jleclaire and http://www.jenniferleclaire.com.

You may also be interested in:

• rPath brings data center automation to Windows environments
  
• Trio of cloud companies collaborate on new private cloud platform offerings
  
• rPath offers free management tool for applications aspiring to the cloud

By: Dana Gardner, Principal Analyst, Interarbor Solutions Posted: 11th November 2010 Copyright Interarbor Solutions © 2010

How do IT architectures at software-as-a-service (SaaS) providers provide significant advantages over traditional enterprise IT architectures?

We answer that "Architecture is Destiny" question by looking at how one human resources management (HRM), financial management and payroll SaaS provider, Workday, has from the very beginning moved beyond relational databases and distributed architectures that date to the mid-1990s.

Instead, Workday has designed its architecture to provide secure transactions, wider integrations, and deep analysis off of the same optimized data source—all to better serve business needs. The advantages of these modern services-based architecture can be passed on to the end users—and across the ecosystem of business process partners—at significantly lower cost than conventional IT.

Joining us here is a technology executive from Workday, Petros Dermetzis, Vice President of Development there, to explore how architecting properly provides the means to adapt and extend how businesses need to operate, and not be limited by how IT has to operate. The discussion is moderated by BriefingsDirect's Dana Gardner, Principal Analyst at Interarbor Solutions.

Here are some excerpts:

Dermetzis: We have a unique opportunity to stand back and see what history and evolution provided over the past 20 years and say, "Okay, how can we provide one technology stack that starts addressing all those individual problems that started appearing over time?"

If you think of the majority of the systems out there, the way we describe them is that they were built from the ground up as islands. It was really very data-centric. The whole idea was that the enterprise resource planning (ERP) system gave all the solutions, which in reality isn't true.

What we tried to do at Workday was start from a completely white sheet of paper. The reality around ERP systems is actually making all this work together. You want your transactions, you want your validations, you want to secure your data, and at the same time you want access to that data and to be able to analyze it. So, that’s the problem we set out to do.

What drove our technology architecture was first, we have a very simple mentality. You have a central system that stores transactions, and you make sure that it's safe, secure, encrypted, and all these great words. At the same time, we appreciate that systems, as well as humans, interact with this central transactional system. So we treat them not as an afterthought, but as equal citizens.

If you go back in time to when mainframes started appearing, it was about transactions, capturing transactions, and safeguarding those transactions. IT was the center of the universe and they called the shots. As it evolved over time, IT began to realize that departments wanted their own solutions. They try to extract the data and take them into areas, such as spreadsheets and what have you, for further analysis.

ERP solutions evolved over time and started adding technology solutions as problems occurred. They started with a need to report data and very quickly realized it was like climbing a ladder of hierarchic needs. When you get your basic reporting right, you need to start analyzing data.

The technologies at the time, around the relational models, don’t actually address that very well. Then, you find other industries, like business intelligence (BI) vendors, appeared who tried to solve those problems.

The way things evolved, you started with an application, and integrations were an afterthought; they got bolted on. ... They kept on adding more and more and more layers of vendors, and the more the poor enterprise IT customers are trying to peel it, the more they start crying—crying in terms of maintenance and maintenance dollars.

Old approach won't scale
Right now, the state of the art is hard-wiring most of these central solutions to these third-party solutions, and that basically doesn't scale. That’s where technology kicks in and you have to adopt new open standard and web services standards.

What we try to do at Workday is understand holistically what the current problems are today, and say, "This is a golden opportunity." This is opposed to finding all existing technologies, cobbling them all together, and trying to solve the problems exactly the same way.

If you're managing any system with HRM systems, you need to communicate with other systems, be it for background checks, for providing information to benefit providers, connecting to third-party payrolls, or what have you.

Obviously, [traditional ERP vendors] were solving the problem incrementally, as they were going along. What we tried to do was address it all in the same place. Where we are right now is what I would describe as very business transaction-centric in what I define as legacy applications. Then, we want to take it more to an area which is business interactions, and interactions can happen from humans or machines.

We're creating a revolution in the ERP industry. As always, you have early adopters. At the other end of the bell-shaped curve, you've got the laggards. When you're talking to forward thinking, modern thinking, profit-oriented, innovative companies, they very quickly appreciate that the way to go is SaaS.

Now, they've got a bunch of questions, and most of the questions are around security—"Is my data safe?" We have a huge variety of ways of assuring our customers that these are actually probably safer in our environment than on-premise.

Some customers wait, and some will just jump in the pool with everyone else. We are in our fifth year of existence, and it’s very interesting to see how our customers are scaling from the small, lower end, to huge companies and corporations that are running on Workday.

A blast from the past
Applications are built on top of relational databases today, and then they are being designed thinking about the end-user, sitting in front of a browser, interacting with the system. But, really they were designed around capturing the transaction and being able to report straight-off that transaction.

The idea of integrating with third parties was an afterthought. Being an afterthought, what happened was that you find this new industry emerging, which is around extract, transform and load (ETL) tools and integration tools. It was a realization that we have to coexist within the many systems.

What happened was that they bolted on these integration third-party systems straight onto the database. That sounds very good. However, all the business logic, all the security, and the whole data structure that hangs together is known by the application—and not by the database. When you bolt-on an integration technology on the side, you lose all that. You have to recreate it in the third-party technology.

Similarly, when it comes to reporting, relational technology does a phenomenal job with the use of SQL and producing reports, which I will define as two-dimensional reports, for producing lists, matrix reports, and summary reports. But, eventually, as business evolves, you need to analyze data and you have to create this idea of dimensionality. Well, yet another industry was created—and it was bolted back onto the database level, which is the [BI] analytics, and this created cubes.

In fact, what they used were object-oriented technologies and in-memory solutions for reasons of performance to be able to analyze data. This is currently the state of the art.

The same treatment
Conversely, any request that comes into our system, be it from a UI or from a third-party system by integrations, we treat exactly the same way. They go through exactly the same functional application security. It knows exactly what the structure of your object model is. It gets evaluated exactly the same way and then it serves back the answer. So that fundamental principle solves most of our integration problems.

On the integration side, we just work off open standards. The only way that you can talk with a third-party system with Workday is through web services, and those services are contracts that we spec to the outside world. We may change things internally, but that’s our problem.

That’s the point where we have a technology around our enterprise service plus our integration server that actually talks the language that we do, standards web service based. At the same time, it's able to transform any bit of that information to whatever the receiving component wants, whether it’s banking, the various formats, or whatever is out there.

We put the technology into the hands of our customers to be able to ratchet down the latest technology to whatever other file structures that they currently have. We provide that to our customers, so they can connect them to the card-scanning systems, security systems, badging systems, or even their own financial systems that they may have in house.

We're a SaaS vendor, and we do modify things and we add things, but those external contracts, which are the Web services talking to third-party systems, we respect and we don’t change. So, in effect, we do not break the integrations.

Best way to access data
The next architectural benefit is about analyzing data. As I said, there are a lot of technologies out there that do a very good job at lists and matrix reporting. Eventually, most of these things end up in spreadsheets, where people do further analysis.

But the dream that we are aiming for continuously is: When you are looking at a screen, you see a number. That number could be an accumulation of counts that you'd be really interested in clicking on and finding out what those counts are—name of applicants, name of positions, number of assets that you have. Or, it's an accumulation. You look at the balance sheet. You look at the big number. You want to click and figure out what comprises that number.

To do that, you have to have that analytical component and your transactional component all in the same place. You can't afford what I call I/Os. It's a huge penalty to go back and forth through a relational database on a disk. So, that forces you to bring everything into memory, because people expect to click something and within earth time get a response.

The technology solutions that we opted for was this totally in-memory object model that allows us to do the basic embedded analytics, taking action on everything you see on the screen.When you are traversing, you come to a number in a balance sheet, and as you're drilling around, what you are really doing in effect is traversing an object model underneath, and you should be able to get that for nothing.

So the persistence layer is really forced by the analytical components. When you're analyzing information, it has to perform extremely fast. You only have one option, and that is memory. So, you have to bring everything up in-memory.

We do use a relational component, but not as a relational database. We use a relational database, which is really good at securing your data, encrypting your data, backing up your data, restoring it, replicating it, and all these great utilities the database gives you, but we don’t use a relational model. We use an object model, which is all in-memory.

But, you need to store things somewhere. In fact, we have a belief at Workday that the disk, which is more the relational component, is the future tape. What you used to use in legacy systems was putting things on tape for safety and archiving reasons. We use disk, and we actually believe, if you look at the future, that nearly everything will be done exclusively in-memory.

Make way for metadata
And, there is another bit of technology that you add to that. We're a totally metadata-driven technology stack. Right now, we put out what we describe as updates three times a year. You put new applications, new features, and new innovations into the hands of your customers, and being in only one central place, we get immediate feedback on the usage, which we can enhance. And, we just keep on going on and keep on adding and adding more and more and more.

This is something that was an absolute luxury in your legacy stack, to take a complete release. You have to live through all the breakages that we mentioned before around integrations and the analytical component.

As soon as you can have the luxury of maintaining one system, let's call it one code line, and you're hanging our customers, our tenants, off that one single code line, it allows you to do very, very frequent upgrades or updates or new releases, if you wish, to that central code line, because you only have to maintain one thing.

Multi-tenancy is also one of the core ingredients, if you want to become a SaaS vendor. Now, I'm not an advocate of saying multi-tenancy A is better than multi-tenancy B. There are different ways you can solve the multi-tenancy problems. You can do it at the database level, the application level, or the hardware level. There’s no right or wrong one. The main difference is, what does it cost?

All we're looking at is one single code line that we have to maintain and secure continuously. We believe in one single code line, and multiple tenants are sharing that single code line. That reduces all our efforts around revving it and updating it. That does result in cost savings for the vendor, in other words, ourselves.

And as far back as I can remember, when humans realized that you take time and material, package that for a profit, and send it to your end-market, as soon as you can reduce your cost of the time or the material, you can either pocket the difference, or move that cost saving onto your customers.

We believe that multi-tenancy is one of the key ingredients of reducing the cost of maintenance that we have internally. At the same time, it allows us to rev new innovative applications out to the market very quickly, get feedback for it, and pass that cost savings on to our customers, which then they can take that and invest in whatever they do—making carpets, yogurt, or electric motors.

Listen to the podcast. Find it on iTunes/iPod. Read a full transcript or download a copy.

By: Dana Gardner, Principal Analyst, Interarbor Solutions Posted: 10th November 2010 Copyright Interarbor Solutions © 2010

WSO2 recently announced the debut of WSO2 Carbon Studio, an Eclipse-based integrated developer environment (IDE) for WSO2 Carbon.

The new offering allows users to build service-oriented architecture (SOA) and composite applications based on WSO2 Carbon. [Disclaimer: WSO2 is a sponsor of BriefingsDirect podcasts.]

Highlights of WSO2 Carbon Studio include the ability to:

• Organize artifacts that span the multiple runtimes common to composite applications into a single project—a Carbon Application (CApp).
• Develop applications using tools designed for WSO2 Carbon-based products including the WSO2 ESB, WSO2 Web Services Application Server (WSO2 WSAS), WSO2 Business Process Server (BPS), WSO2 Governance Registry, and more.
• Test and debug WSO2 Carbon-based applications directly within the IDE.
• Export Carbon Applications in the new Carbon Archive format.

“We have found that many of our customers are developing sophisticated applications that span the WSO2 Carbon product family, and they are taking advantage of the unique strengths of our platform when used as a whole,” said Dr. Sanjiva Weerawarana, founder and CEO of WSO2. “We’re now revving up our tooling support with WSO2 Carbon Studio—helping developers to organize, develop, test, and deploy these composite applications with greater ease than ever before.”

Middleware platform
The WSO2 Carbon Studio IDE is designed to take advantage of the open source WSO2 Carbon middleware platform. The Eclipse-based offering includes graphical editors for XML configuration files, an enhanced Eclipse BPEL editor, and easy integration of Carbon-based applications with the WSO2 Governance Registry. Additionally, Carbon Studio offers a rich set of third-party Eclipse plug-ins, including Maven and the OpenSocial Gadget Editor.

Carbon Studio supports SOA projects that often combine multiple application types into a single composite application or service. Developers also have single-click function for testing Java-based applications and services—without leaving the IDE. Debugging tools support Axis2-based services, Apache Synapse mediators, registry handlers, and data validators.

Tools to support SOA development include Apache Axis2 and JAX-WS, Data Service, BPEL, ESB, and ESB Tooling, as well as a gadget editor.

WSO2 Carbon Studio, available now as a set of Eclipse plug-ins, is a fully open-source solution released under Eclipse and Apache Licenses and does not carry any licensing fees. WSO2 offers a range of service and support options for Carbon Studio, including development support and production support.

By: Nigel Stanley, Practice Leader - IT Security, Bloor Research Posted: 1st November 2010 Copyright Bloor Research © 2010

Published recently the third paradox report, based on research sponsored by McAfee and written by Nigel Stanley at Bloor Research, highlights some interesting security statistics from across the world. Here are some highlights - further details and the full report available here.

Key findings worldwide

• 54% of mid-sized organisations have seen an increase in IT security risks facing their company from 2009 to 2010, up 2% on last year.
• 40% of mid-sized organisations have had data breaches in the past year, an increase of 13% from last year.
• 75% of mid-sized organisations said that there is a chance that a serious data breach could force them out of business, up from 70% in last year's survey.
• 30% of mid-sized organisations had to manage multiple network security incidents, of which 55% took up to 5 hours to investigate and remediate.
• 58% of worldwide respondents spend less than 3 hours per week working on, evaluating and researching IT security. Last year it was 65%.
• 5% of mid-sized organisations reported that they had suffered a data loss that had cost them more than $25,000. Of these 25% were from China, 14% from France and 11% from India.
• 47% of all reported intellectual property losses were from EMEA-based mid-sized organisations.
• 88% of mid-sized organisations said they were concerned or very concerned about non-malicious/inadvertent security incidents.
• 60% of worldwide mid-sized organisations admitted to knowing less than 75% of the pertinent regulatory and compliance requirements pertinent to their organisation.

By: Dana Gardner, Principal Analyst, Interarbor Solutions Posted: 28th October 2010 Copyright Interarbor Solutions © 2010

Businesses are looking to cloud-computing models to foster agility and improve time-to-market for new services. Yet attaining cloud benefits can founder without higher levels of unified server, data, network, storage, and applications management.

These typically disparate forms of management must now come together in new ways to mutually support a variety of different cloud approaches -- public, private, and hybrid. Without adoption of such Business Service Automation (BSA) capabilities, those deploying applications on private and hybrid clouds will almost certainly encounter increased complexity, higher risk, and stubborn cost structures.

This latest BriefingsDirect discussion therefore focuses on finding low-risk, high-reward paths to cloud computing by using increased automation and proven reference models for cloud management—and by breaking down traditional IT management silos. In doing so, the progression toward cloud benefits will come more quickly, at lower total cost, and with an ability to rapidly scale to even more applications and data.

We're here with two executives from HP Software & Solutions to learn more about what BSA is and why it's proving essential to managed and productive cloud computing adoption: Mark Shoemaker, Executive Program Manager for Cloud Computing in the Software & Solutions Group at HP, and Venkat Devraj, Chief Technology Officer for Application Automation, also in HP’s Software & Solutions Group. The discussion is moderated by BriefingsDirect's Dana Gardner, Principal Analyst at Interarbor Solutions.

Here are some excerpts:

Shoemaker: There is hardly a place we go that we don’t end up talking to our customers about cloud. Most of the enterprise customers we talk to are looking at private cloud, the internal cloud solution that they own, that they then provide to their business partners, whether that’s the development teams or other elements in their business. Most of them are looking to build on the virtualization work that they've already done.

They want to improve their productivity, definitely get better utilization out of what they have already got. They want IT to be your better partner in the business. What that means is to shorten the time that the business has to wait for the services.

Devraj: There is also an interesting micro trend that’s occurring. A lot of the application teams, end-user business teams, are getting increasingly sophisticated. They're learning about private cloud implementations. Consequently, they're demanding levels of service from IT that are difficult to provide without a private cloud.

For example, because of things like agile development methodologies, application teams are doing a lot more application deployments and code releases than ever before. It's not uncommon to see dozens of application releases for different applications happening during the same day.

IT operations are just bombarded with these requirements and requests, and they are just unable to keep up based on yesterday’s processes, which are relatively static. These application teams and business unit teams are quite influential.

They're even willing to fund specific initiatives to allow their teams to work in self-service mode, and IT ops are finding themselves in reactive mode. They have to support them, make their internal processes more fluid and dynamic, and leveraging technology that allows that kind of dynamism.

... The third-party companies, the cloud providers, the pure-play server enablers, have an unfair advantage. Because they were started relatively recently, in the last few years, they have the advantage of standardized platforms and delivery units.

They can say, "Okay, I'm going to deliver only Linux-based platforms, Windows-based platforms, or certain applications." When you look at the typical enterprise today, however, IT has a lot more to deliver.

There is a lot of prevailing heterogeneity in terms of multiple software platforms and versions. There is a lack of standardization. It's very difficult to talk about cloud and delivery within the enterprise in the same breath, when you look at these kinds of technical challenges.

As a result, IT is undergoing a lot of pressure—but they have to deliver given the kind of challenges that they face. That’s going to require a lot of education and access to the right kind of technology, training, and guidance.

Shoemaker: Just to add to Venkat’s comment, we're seeing the business driving IT and demanding that agility and that flexibility. We talk to a lot of our customers, where their own coworkers have taken corporate credit cards and gone out into the public cloud, procured space, and have begun developing outside of them. IT really has to get in front of this. They have to manage all this.

... The one thing that’s different about cloud is that it really is a supply chain. It’s the supply chain of IT technology that the business consumes. If you think about what a supply chain is, it’s something that’s got to be repeatable. It has to be governed, and it provides a baseline or foundation and building blocks to build those services that you can then customize on top of the business.

So, the farther up that you can go with your standard building blocks, the less difficult it is to manage and focus on the custom business-facing functions on the front-end.

To do this, cloud has helped us out in a lot of ways. One of the challenges IT has always had is to get the business to consume standards. Because of a lot of hype in the market, the business absolutely is convinced that they get it, and they want the business benefits that cloud offers.

Even if the business decides to go to a public cloud, they still have to consume those elements in a standard fashion. There's no way out of that.

Devraj: And yet, the software used by these enterprises tends to be disparate, heterogeneous, and requires a lot of domain knowledge to be able to manage, resulting in significant delays and bottlenecks associated with service delivery. Those processes just don’t scale in the cloud.

At Stratavia we had built a patented technology to manage and control varied software stacks, such as databases, web servers, application servers, and even well-known packaged applications, including Microsoft Exchange, Oracle E-Business Suite, and SAP.

The content that I talk about becomes an abstraction layer, where the customer, the end user, the people who consume the services, see a very easy to understand service catalog. They can click on it. They can choose some menu options, some values from a drop-down box, and then specify exactly what they need, and have the response come back in minutes and in hours, rather than days and weeks, as is traditionally the case.

For example, just at the database layer, within the enterprise, it's very common to see four or five different platforms in use, such as DB2, SQL Server, Oracle, and so on. By automating the operations management lifecycle around these layers, Stratavia has made it possible for the enterprise to deliver and manage these assets as a service within the context of the cloud.

As more and more of HP’s and Stratavia’s joint customers started seeing value in that capability, HP brought Stratavia into its BSA/Business Technology Optimization umbrella.

There's a big gap in IT today, which is IT/Ops Engineering or IT/Ops Architecture. That’s a big missing silo within IT/Ops. And a lot of the operators today that rely on scripts, command-line stuff, and point-and-click tools need to evolve themselves to more of an architect approach. They need more of taking stock of the big picture, and taking the tribal knowledge that they have in their heads and looking at the out-of-the-box content that HP provides and selecting the right content that corresponds to their tribal knowledge.

When they go into the cloud, the underlying management, things like compliance and governance, are not out of whack. They're able to successfully take that knowledge, put it in there, and then, in their new role as architects or engineering folks, they're able to watch, measure, and make modifications as appropriate.

So, the role that people play, that key subject matter experts play, is very crucial as part of walking before running with automation.

Gardner: Now that you have mentioned Stratavia, and for the benefit of our listeners and readers, HP has acquired Stratavia, and there was also quite a bit of related product and service news on Sept. 15 around BSA as the acquisition was unveiled.

Shoemaker: Obviously, the Stratavia acquisition was a huge, huge win for us, and puts us in a great position to help our customers transform their infrastructure. ... And several other things have happened in the last 60 days. We had VMworld, and we presented a cohesive strategy for infrastructure and even PaaS built on the BladeSystem Matrix hardware platform that we have, Converged Infrastructure. We've combined that with two other pieces and a piece of Cloud Service Automation (CSA) software.

CloudStart is a consulting and a professional services-led engagement capability where we come in and work with the customer to get that transformation process nailed, so we can quickly get them moving into the cloud benefits.

On the back end of that, there is another piece that we announced called Cloud Maps, which is really more knowledge, but in a different capacity, in that it offers downloadable templates, preconfigured applications, and best practices for sizing.

We see the Stratavia acquisition fueling this fire, because in the end, cloud is a solution, and a solution needs content, and content wins. Content is what the customer is able to consume and use day one, when the solution is in. So it's important. And we've done a lot there.

We now have a best-in-class content provider in Stratavia that’s come on board to help round out the capabilities and add more into what the customer can get out of our solutions in very quick order.

All that sits on a recently refreshed BSA portfolio, with significant enhancements and new capabilities across network, automations, servers, and storage, that really makes all this happen.

... Let's face it, a lot of the CIOs are looking at a data center that’s packed full of applications that they probably don’t feel as if they have got a good handle on. Now, cloud is coming into the picture, and they've got two things to do here.

Number one, they need to start applying those new business methodologies to IT around providing cloud and the things that go with that, but also they have got a transformation piece to go along. And that can be very daunting.

What we've done is looked at the experience of helping previous customers do that work and we have applied that into the CloudStart and Cloud Maps, CloudStart being the planning and the upfront work that you need to get done.

So, we're right there with you. You don’t have to read chapter one of the book.

Then, as we put the infrastructure in with CSA for Matrix in the frame, we're embedding some of the CSA software inside of the Blade Matrix frame. So you have a way to build infrastructure as a service (IaaS) and manage it through the platform throughout the lifecycle.

Then, on the back end of that, we have the preconfigured application templates. If I need a SQL Server image to put into the system, I can pull that from Cloud Maps, build it into a framework and offer that very quickly. I don’t have to go and figure out how to size for this piece or what golden template looks like for this application.

It's really about obtaining a running start into the cloud, and one that’s not going to leave you wanting in a year or two. You have to be careful. Cloud is a great enablement technology and a lot of people are looking at IaaS, but that’s the starting point for it, and then you have to manage everything that you put inside of that as well.

Listen to the podcast. Find it on iTunes/iPod. Read a full transcript or download a copy.

By: Rob Bamforth, Principal Analyst, Quocirca Posted: 15th October 2010 Copyright Quocirca © 2010

The term ‘unified communications’ conjures up many meanings, but is most often used by those with software or network assets to sell. Whether it is routers, switches, hubs, directories, phones or high definition video conferencing equipment, the thrust is often the same—we have the hardware to remove complexity from your network and software to unify those different modes of communication that your users ‘enjoy’. Basically it’s the IP dividend of voice over IP (VoIP) mixed with video over IP plus anything else over IP with a bit of contextual status thrown in via ‘presence’.

Sounds good to those managing a complex mix of networks, or those paying for separate forms of connection when they can see what looks like a great big free (or perceived to be free) fat internet pipe that will take all IP traffic. Unify the packets over IP and you’ve unified communications, right?

The problems come when trying to see how users fit into the deal and it does not always end in a fully cross functional, matrix managed, dispersed workforce collaborating all the way across the extended enterprise. The technology is fine, the commercial aspect works, but the social side just does not deliver, because it depends on acceptance, initiative and commitment from the workforce, and generating that takes more work than installing a CD or network appliance.

So how about taking a different approach?

There is much talk about the influx of consumer technology into the workplace, and an interesting area to look at here is social networking. However this time it is not about the use of social networking tools to connect with customers, reinvigorate marketing budgets or make the business look cool. Nor is it about the fears of employees spending so much of their time glued to their social networks that they forget to work, or how to interact with real people; although these issues do merit some attention from organisations.

An aspect of social networking that might catalyse and support the broader adoption of unified communications is the current trend towards ‘social dashboards’. These are coming about partly in recognition that most people like and use a multiplicity of social communications tools—YouTube, Facebook, Twitter, LinkedIn, instant messaging, email etc—to hook up with their friends and contacts, yet would like to avoid the complexity of using these as separate applications. A single live ‘portal’ embracing the other tools would be ideal, but who would be the master site/supplier?

It may be too early to narrow down as there have been false dawns and social networking failures, but current players are positioning themselves as ‘accommodating’ as the market evolves. Recent innovations and updates from Microsoft around Live Essentials and the new look Twitter are examples of the trend towards this.

So what is a ‘social dashboard’ and what are the characteristics that have merit for consumers, which might turn out to be a valuable in a business context? There are several recurring themes:

• Feeds – these are live updates, tickers, messages, blogged and tweeted lifestreams or even streaming audio and videos. Ever present, constantly updated without the need for the recipient to make requests.
• Finds – uploaded responses or comment using scraps of information, interesting webpages, uploaded photos and videos can be simply and easily fed in and propagated to all contacts, ‘inline’ and without the need to open new windows or be diverted by separate applications.
• Feedback – instant opinion and comment on feeds and finds from all those in the network, a loose collaboration, trending and sometimes herd-like behaviour in the crowd. Voting and recommendation engines might seem too democratic for business decisions that need top down command and control, but with suitable moderation there may be wisdom in the crowd.
• Filters – the key to making sense of a cacophony of information. Filtering by areas of interest, favouritism dependant on the contact type (e.g. messages from the boss, or the activities of a key customer), current activities or status (do not disturb, busy working, on holiday so friends only etc). Organisations may also be able to push down centralised policies to provide automated filtering and implement security measures to block malware, filter inappropriate content and mitigate risky behaviour or data leakage, as well as permit more personal policies to improve productivity by adapting to ensure information is relevant to the context of the place, time and person.

Finally there is also the underlying ability to grow the network by finding contacts, or suggesting potential friends. When applied with business intelligence, this mechanism of seeking out the right person to contact would be extremely useful in many organisations where the traditional ‘org charts’ are always out of date or the sheer volume of external relationships make the divisions of ‘employee’ and ‘contractor’ meaningless.

Buddy lists and presence directories are already part of many unified communications solutions, but they could go a lot further to envelop the groups, commonalities and relationships that people really build their personal communications networks on. Simply having a directory with phone number, contact details and current status or presence is not enough, and the social network element provides some provenance, knowledge of, or social value of the contact. Social networks have meaning attached to the link as well as the point of the connection.

Many unified communications vendors have overly focused on the networking technology and forgotten the key part of communications; it is about people. Perhaps they could learn something relevant for businesses from social and consumer oriented tools?

By: Dana Gardner, Principal Analyst, Interarbor Solutions Posted: 6th October 2010 Copyright Interarbor Solutions © 2010

The trend toward converged infrastructure—a whole greater than sum of the traditional IT hardware, software, networking and storage parts—is going both downstream and upstream.

HP today announced how combining and simplifying the parts of IT infrastructure makes the solution value far higher on either end of the applications distribution equation: At branch offices and the next-generation of compact and mobile all-in-one data center containers.

Called the HP Branch Office Networking Solution, the idea is that engineering the fuller IT and communications infrastructure solution, rather then leaving the IT staff and—even worse—the branch office managers to do the integrating, not only saves money, it allows the business to focus just on the applications and processes. This focus, by the way, on applications and processes—not the systems integration, VOIP, updates and maintenance—is driving the broad interest in cloud computing, SaaS and outsourcing. [Disclosure: HP is a sponsor of BriefingsDirect podcasts.]

HP's announcements today in Barcelona are also marked by an emphasis on an ecosystem of partners approach, especially the branch office solution, which packages 14 brand-name apps, appliances and networking elements to make smaller sub-organizations an integrated part of the larger enterprise IT effort. The partner applications include WAN acceleration, security, unified communications and service delivery management.

Appliances need integration too
You could think of it as a kitchen counter approach to appliances, which work well alone but don't exactly bake the whole cake. Organizing, attaching and managing the appliances—with an emphasis on security and centralized control for the whole set-up—has clearly been missing in branch offices. The E5400 series switch accomplishes the convergence of the discrete network appliances. The HP E5400 switch with new HP Advanced Services ZL module is available worldwide today with pricing starting at $8,294.

Today's HP news also follows a slew of product announcements last month that targeted the SMB market, and the "parts is parts" side of building out IT solutions.

To automate the branch office IT needs, HP is bringing together elements of the branch IT equation from the likes of Citrix, Avaya, Microsoft, and Riverbed. They match these up with routers, switches and management of the appliances into a solution. Security and access control across the branches and the integrated systems are being addressed via HP TippingPoint security services. These provide granular control of application access, with the ability to block access to entire websites—or features—across the enterprise and its branches.

Worried about too much Twitter usage at those branches? The new HP Application Digital Vaccine (AppDV) service delivers specifically-designed filters to the HP TippingPoint Intrusion Prevention System (IPS), which easily control access to, or dictate usage of, non-business applications.

The branch automation approach also support a variety of network types, which opens the branch offices to be able to exploit more types of applications delivery: from terminal serving apps, to desktop virtualization, to wireless and mobile. The all-WiFi office might soon only need a single, remotely and centrally managed locked-down rack in a lights-out closet, with untethered smartphones, tablets and notebooks as the worker nodes. Neat.

When you think of it, the new optimized branch office (say 25 seats and up) should be the leader in cloud adoption, not a laggard. The HP Branch Office Networking Solution—with these market-leading technology partners—might just allow the branches to demonstrate a few productivity tricks to the rest of the enterprise.

Indeed, we might just think of many more "branch offices" as myriad nodes within and across the global enterprises, where geography becomes essentially irrelevant. Moreover, the branch office is the SMB, supported by any number and types of service providers, internal and external, public and private, SaaS and cloud.

Data centers get legs
Which brings us to the other end of the HP spectrum for today's news. The same "service providers" that must support these automated branch offices—in all their flavors and across the org chart vagaries and far-flung global locations—must also re-engineer their data centers for the new kinds of workloads, wavy demand curves, and energy- and cost-stingy operational requirements.

So HP has built a sprawling complex in Houston—the POD Works—to build an adaptable family of modular data centers—the HP Performance Optimized Datacenter (POD)—in the shape of 20- and 40-foot tractor-trailer-like containers. As we've seen from some other vendors, these mobile data centers in a box demand only that you drive the things up, lock the brake and hook up electricity, water and a high-speed network. I suppose you also drop them on the roof with a helicopter, but you get the point.

But in today's economy, the efficiency data rules the roost. The HP PODs deliver 37 percent more efficiency and cost 45 percent less than a traditional brick-and-mortar data centers, says HP.

Inside, the custom-designed container is stuffed with highly engineered racks and the cooling, optimized networks and storage, as well as the server horsepower—in this case HP ProLiant SL6500 Scalable Systems, from 1 to 1,000 nodes. While HP is targeting these at the high performance computing and service provider needs—those that are delivering high-scale and/or high transactional power—the adaptability and data center-level design may well become more the norm than the exception.

The PODs are flexible at supporting the converged infrastructure engines for energy efficiency, flexibility and serviceability, said HP. And the management is converged too, via Integrated Lights-Out Advanced (ILO 3), part of HP Insight Control.

The POD parts to be managed are essentially as many as eight servers, or up to four servers with 12 graphic processing units (GPU), in single four-rack unit enclosures. The solution further includes the HP ProLiant s6500 chassis, the HP ProLiant SL390s G7 server and the HP ProLiant SL170s G6 servers. These guts can be flexibly upped to accommodate flexible POD designs, for a wide variety and scale of data-center-level performance and applications support requirements.

Built-in energy consciousness
You may not want to paint the containers green, but you might as well. The first release features optimized energy efficiency with HP ProLiant SL Advanced Power Manager and HP Intelligent Power Discovery to improve power management, as well as power supplies designed with 94 percent greater energy efficiently, said HP.

Start saving energy with delivering more than a teraFLOP per unit of rack space to increase compute power for scientific rendering and modeling applications. Other uses may well make themselves apparent.

Have data center POD, will travel? At least the wait for a POD is more reasonable. With HP POD-Works, PODs can be assembled, tested and shipped in as little as six weeks, compared with one year or longer, to build a traditional brick-and-mortar data center, said HP.

Hey, come to think of it, for those not blocking it with the TippingPoint IPS, I wish Twitter had a few of these on those PODs on the bird strings instead of that fail whale. Twitter should also know that multiple PODs or a POD farm can support large hosting operations and web-based or compute-intensive applications, in case they want to buy Google or Facebook.

Indeed, as cloud computing grains traction, data centers may be located (and co-located) based on more than whale tails. Compliance to local laws, for business continuity and to best serve all those thousands of automated branch offices might also spur demand for flexible and efficient mobile data centers.

Converged infrastructure may have found a converged IT market, even one that spans the globe.

By: Simon Holloway, Practice Leader - Process Management & RFID, Bloor Research Posted: 4th October 2010 Copyright Bloor Research © 2010

We all know that Manufacturing is all about products and that you have to keep reinventing your product portfolio to keep ahead in today’s market. Perhaps what it is not so well known is that the majority of R&D products don’t even make the market and of those that do only 1 or 2 really make a worthwhile profit. Therefore product development is a risky business, but one we can’t avoid. So how can we limit the risks and get better control of the process of controlling the life of our products?

Andy Michuda, Chief Executive Officer of Sopheon told me, “Product life cycle management (PLM) is the most vital business process in manufacturing today. A right decision on which product ideas to develop and produce can transform a company’s future. A wrong decision can bring a company to its knees. In the race for growth and profitability, the capacity to understand and act on PLM’s power will separate the winners from the losers”. But what exactly is PLM? There seem to be no standard definitions of PLM—everyone has something slightly different to say. Even the site http://www.product-lifecycle-management.info has a number of different definitions!

Let me give you my condensed definition of PLM. “It is the business process of managing the entire lifecycle of a product from its conception, through design and manufacture, to service and disposal. It integrates people, data, processes and business systems and provides a product information backbone for companies as well as their partners, suppliers and customers.” PLM is first and foremost a business discipline, whose goal is to eliminate waste and improve efficiency, and is considered to be an integral part of the lean production model. However, because of the business complexity and rate of change that requires organizations execute as rapidly as possible, application software is becoming more and more crucial to the success of PLM. It is one of the four cornerstones of a corporation's information technology structure. Shoenhair of Ping, a PTC Customer, supports this view: “PLM can be difficult to measure, but it is absolutely critical to leaning out processes, and critical to improving information flow and control.”

Where do ERP and PLM fit? Most manufacturing companies distinguish two main process chains: the operational process chain and the technical process chain. ERP systems largely address the operational process chain, whereas PLM systems automate and enable predominantly the technical process chain.

Figure 1: ERP and PLM (Source: PLM Technology Guide)

Johan Malmström, PLM Business Development Manager, SAP, emphasised the collaborative nature of PLM, “PLM makes sure that everyone works towards one version of the truth, with clearly defined tasks and responsibilities. It manages the product structure and related information, the usage of this data across the product lifecycle as well as the process of creating this data. Process support includes workflow capabilities, program and project management, resource management etc. to make sure that the correct resources are working on the correct tasks in order to deliver the right products to the market in the right time.”

Michuda explained that PLM is implemented in practice on three different levels, each of which is supported by a different tool set.

• Transactional Processes: Enterprise resource planning (ERP) applications manage transactional processes. They are designed to unify materials planning, purchasing, financial transactions, accounting and reporting into streamlined transactional processes. Supply chain management (SCM) and customer relationship management (CRM) applications also address process needs at this level.
• Technical Data: Computer-aided design (CAD) applications, as well as those related to formula, recipe, or product data management (PDM), are primarily focused on managing the masterfile of descriptive data within the product lifecycle. These PLM systems streamline and continuously improve the processes of defining, designing and producing products, while potentially also supporting aspects of product innovation. They offer collaboration capabilities that enable enterprise-wide sharing of product designs, reducing the chance of design and manufacturing errors.
• Business Information: The business level of PLM deals with business issues around critical business-related decisions within the product lifecycle. At the business level of PLM, the emphasis is on solutions that handle innovation governance issues such as process management, decision support, idea management, product portfolio management, expertise management, and intelligence around markets, competitors and technologies. Regulatory compliance and sustainability that important not only during product innovation but also to effective management of the supply chain are also included within the business level.

So what tools are used in a PLM solution? The PLM Technology Guide shows the core technology of a PLM system and some of the many solutions that can rest on the basic technology. The orange line outlines Product Data Management (PDM), which is typically used for basic CAD file and Data Management.

Figure 2 PLM Functionality Source:  PLM Technology Guide

Who are the main players? The major players in PLM space can be grouped under 3 broad categories:

• PLM product vendors such as Dassault Systemes , PTC ,Siemens, Sopheon, Aras
• The ERP vendors such as Oracle Agile, SAP PLM, Infor PLM, Epicor, IFS
• Consulting & implementation companies such as Accenture, Atos Origin, Capgemini, ITC Infotech, IBM, Infosys, KSA, Wipro and HCL Technologies.

What is coming? Dassault Systemes, on their web site, describe PLM v2 – “PLM 2.0 is a major redefinition of the PLM markets targeting all users creating, consuming and remixing IP. PLM 2.0 is to PLM what Web 2.0 is to the Web, harnessing collective intelligence from online communities. Any user can imagine, share and experience products in the universal language of 3D. PLM 2.0 brings knowledge, from idea to product experience (IP), to life. It merges the real and virtual in an immersive lifelike experience.” SAP’s Malmström sees the following three trends:

• Consumer-Driven Sustainable Innovation: with a focus on developing the right products at the right time in fast innovation cycles.
•  Global Price and Time Pressure: requires development efficiency, sharing of information in dynamic development networks.
• Increasing Product Compliance and Regulations: manage compliance, controls, documentation and visibility.

Mike Spragg, Infor's UK director for the process industries, sees the increase in environmental awareness and the incorporation of the ‘green’ agenda as an area of PLM expansion, “PLM has much to offer manufacturers.  PLM begins at the earliest possible stages of design, meaning these new green considerations are factored in long before products are manufactured and then enter the supply chain. This can save costs that would have to be borne were the products reworked at a later date.”

Deepankar Ghosh, Head – Manufacturing Practice, ITC Infotech, provided a clear idea of the importance of PLM, “PLM industry is comparatively a niche industry which is gaining more currency and acceptance as organizations are realizing the value that the PLM process brings to the table. With an ever increasing pressure on bottom line it is imperative that companies make IT investments where the ROI is not only high but faster. A more informed and demanding customer is seeking not only cheaper but innovative and trendy products more than ever before. For an organization to be ahead of its competition, collaboration across key roles and functions within the company and with its supply chain has become critical. The environment for the PLM practice to grow is just right and we will soon be witnessing an unprecedented interest in this area.”

So, if ERP manages your operations, PLM manages your product portfolio from creation to end of life. My experience of PLM solutions is that they really do provide value—you just need to find the one that best suits your pocket and needs. If that is the case then come along to PLM Connect and find the answer.

By: Simon Holloway, Practice Leader - Process Management & RFID, Bloor Research Posted: 29th September 2010 Copyright Bloor Research © 2010

If ERP is all about managing and controlling resources, Sales and Operations Planning is the brains behind the process and can be the difference between profit and loss.

 Andrew Kinder, Solutions Director at Infor told me, “As Europe moves out of recession, many business leaders are reflecting on the lessons they have learnt. Thankfully the phrase: ‘if only we had known how the credit crunch was going to hit us’ has been joined by ‘what can we do to make sure this never happens again?’. Businesses are now examining the systems and processes that offer not just growth but protection and resilience.”

At the top of this list of options is S&OP. In a recent Supply Chain Management survey for Infor conducted by AMR Research, 88% of respondents said they are already using or planning to deploy an S&OP solution in the next 12 months. The report also found that the number one area of S&OP businesses want more support in, is in its ability to provide “what-if” simulation capability—such simulation is a critical tool in dealing with the volatility present in today’s businesses. But has S&OP changed with the times and is it applicable in today’s global agile world? Does it apply to both large and small organisations? These and a number of other questions are key to success in today’s collaborative world.

So what is it?
S&OP is a business planning process that aligns the traditional demand/supply view of the world, with the financial and business goals of the organization. S&OP is a response to the accusation that the operational plan and the business plan are often seriously mis-aligned.

Supporting this cross-functional business process is information. And that means integrating a number of different pieces of planning data around sales, production, inventory, finance and HR to provide the executive with focus, alignment and synchronisation about the company. Plan frequency and planning horizon depend on the specifics of the industry. A properly implemented S&OP process routinely reviews customer demand and supply resources and “re-plans” quantitatively across an agreed rolling horizon. The re-planning process focuses on changes from the previously agreed sales and operations plan.

Figure 1: Putting S&OP into Context (Source: Hitachi Consulting[1])

As John Dougherty[2] said, “Its ultimate goal is to always keep the detailed sales, manufacturing, purchasing and capacity planning systems in synchronization with the latest high level plans of management (the business plan).” Or you might prefer Chuck Poirier’s view[3], “it’s about balancing supply and demand in a way that overcomes the deficiencies of weak forecasting and results in more optimum performance—from the initial suppliers to the satisfied customers.” Kinder explained that there are many different definitions that have evolved over time. At Infor, they defined S&OP for the purposes of driving their new product design as “enabling decision makers to achieve consensus on a single operating plan that profitably matches supply and demand.”

The Association for Operations Management (APICS) defines S&OP as the "function of setting the overall level of manufacturing output (production plan) and other activities to best satisfy the current planned levels of sales (sales plan and/or forecasts), while meeting general business objectives of profitability, productivity, competitive customer lead times, etc., as expressed in the overall business plan. One of its primary purposes is to establish production rates that will achieve management’s objective of maintaining, raising, or lowering inventories or backlogs, while usually attempting to keep the workforce relatively stable. It must extend through a planning horizon sufficient to plan the labor, equipment, facilities, material, and finances required to accomplish the production plan. As this plan affects many company functions, it is normally prepared with information from marketing, manufacturing, engineering, finance, materials, etc."

What is involved?
The S&OP process brings together many areas of the business to determine anticipated demand volume and how the company plans to supply product to meet that demand and best serve the customer within the financial goals of the company. The S&OP processes is characterized by:

• A top-down and bottoms up approach, linking the company’s business plan with the current demand and supply plans
• A cross-functional, collaborative process that focuses on improving business performance
• A structured, formal set of consensus business processes based on a set time period, usually a month

Figure 2: The Sales and Operational Planning Process (Source: Chuck Poirier, CSC)

The process starts with gathering the projected demand information and compiling it in a common format. From this information, a demand forecast is generated, typically beginning with the sales forecast originally used for planning purposes, but augmented with inputs from key customers and amended by knowledge of current operating and market conditions. The next step is to match the demand forecast against any known or anticipated manufacturing and logistics constraints. Any issues identified are then resolved; this often includes looking at alternative strategies. The final step is to monitor progress versus the altered demand and supply plans.

So what we have is different functions or business processes operating with different buckets of information granularity. Information flows both bottom up (sales, customer, VMI and co-managed programs, POS data, supply chain capacities) and top down (budget, business plan, category or customer plans, market share objectives, NPI plans). It is the reconciliation of these information flows to provide actionable planning that is the key to successful S&OP. The planning component and iterative feedback loops require common business language.

Figure 3: Sales and Operations Planning Benefits (Source: Hitachi Consulting0

So what had changed?
Hitachi Consulting [4] sees the following as the key changes that have occurred that affect S&OP:

• Globalization: Diverse and distant supply base for components and finished goods assembly increases complexity and supply lead times. There is a need for accurate, longer forecasting horizons and reduced near term flexibility.
• Contract Manufacturing (CM): While the approaches can vary from full turn key to consignment CM, the common requirement is cross organizational communication for lead time and supply commitment. This means there is a need for better planning to collaborate with CMs and longer forecast (5–9 months) horizons.
• Technology and Market Evolution: Changing consumer tastes and evolving technology mean there is a need for integrated, holistic decision-making to plan, adjust, and adapt while maintaining profitable operations.
• S&OP Supporting Technologies: Workflows can now model both decision making and optimization processes while integrating with disparate functional systems, breaking the demand planning, supply planning, BI technology silos. Therefore there is a requirement for the ability to reduce organizational effort and time needed to develop robust S&OP processes.
• Customer and Channel Focus: Conflict among direct, indirect and key customer channels means there is a need for coordinated channel and profitability management.
• Mergers: 43% of companies note M&A activity has resulted in need to connect merged operations and manage business plan impacts. S&OP therefore needs to support established, robust planning to assist in assimilation.
• Changing Operating Constraints and Costs: new product introductions, changing supply base, new customers, and fluctuating supply chain costs, all mean that there is a need for adaptive S&OP processes.
• Trial and Error: 15+ years of siloed S&OP attempts, Demand Planning and APS implementations, and ERP initiatives have led to “silo optimized” plans or led to domination by one functional group. Enterprise data is more available but not intelligently used for planning. So there is an increased desire for decision making transparency and cohesive planning.

What is happening next?
What we have is an evolution of what we expect from S&OP. Initially it was simply matching demand and supply; balancing supply with the best expectation of demand. This is the coordination of an inventory, production and procurement plan to meet demand, balancing supply with demand at the stock keeping unit (SKU) level. This remains an essential component of any planning process, but lacks a financial view of the plan. Does the plan meet with the financial goals of the business in terms of matching forecast to sales revenue expectations? Is the supply plan affordable in a way that delivers the expected margins of the business?

The next evolution was to allow the user to manipulate both demand and supply. It also included the ability to incorporate events such as new product introduction and product changes. This evolution is sometimes called scenario management. Kinder sees that this is where most organisations are, or strive to be, in their S&OP maturity curve. Planning is more strategic—12–24 months out—and operational plans are expressed in financial terms: revenue, costs and margins.

The latest evolution is to make the planning process even more agile and flexible as well as robust. Kinder explained that practitioners at this level sometimes prefer to use the term “Integrated Business Planning”—elevating the process to a higher level than “sales and operations”. The goal is an executive planning process that seeks to define the total strategic plan for the business and completely align strategy with execution. Kinder gave this example, “For example, a business may incorporate product portfolio planning into their S&OP processes, scrutinising when products are retired and when new ones are brought on-stream. Other considerations will include pricing options, channels to market, expansion and consolidation plans, mergers and acquisitions, and network design changes.”

Who is involved?
As S&OP is a major part of a manufacturing planning process, then, as you would expect, all the major ERP packages would provide modules in their ERP solution that support S&OP. Yet, this is not always the case and an Aberdeen survey revealed 85% of organizations resorting to spreadsheets to support their S&OP processes. However, the usual players such as SAP, Oracle, Sage, Infor, Microsoft Dynamics, Epicor, IFS claim to provide solutions.

The specialist supply chain management solutions such as I2 Technologies, ICON-SCM, Kinaxis, Logility and TXT e-solutions, similarly provide support but their solutions are very supply chain focused, as one would expect, and don’t support the complete picture that many organisations now need.

IBM position Cognos as a solution for S&OP. Cognos is well-known and well-used business intelligence product and therefore to use for S&OP one would need to configure the product to do the job. However IBM provide for their customers—free of charge—a set of frameworks called the IBM Cognos Performance Blueprints which provide a set of preconfigured solutions. However, the BI family of products do not provide detailed demand planning or constrained supply planning that is an important aspect of simulation within the S&OP process.

There are also a number of specialist niche players such as:

• Demand Solutions S&OP is fully integrated with Demand Solutions Forecast Management and Demand Solutions Requirements Planning and imports data through the Forecast Management database. The user-defined Import/Export utility within Demand Solutions products makes it easy to interface with other business systems.
• JDA’s Executive S&OP Workbench has been developed to take account of the Integrated Business Planning concepts I have described earlier. It utilizes key-metric graphs and charts to visually present the aggregated state of your business for informed decision making.
• Steelwedge’s Sales Planning & Performance Management (SPPM) suite leverages four modules (executive, sales, operations and collaborative) and S&OP platform that incorporate best-practice S&OP collaborative technologies with business workflows and performance management capabilities that help companies take enterprise-wide top-down (and bottom-up, middle-out) control over the revenue planning process.

Conclusions
So what does such a successful implementation of S&OP actually deliver to the business? According to research from Aberdeen Group, S&OP leaders report healthier financial results in terms of customer service levels, forecast accuracy, profitability and cash-to-cash cycle times—key measures for any business.

Simon Pollard, VP Manufacturing Operations and Execution for SAP EMEA, in a recent discussion with me gave me this scenario, “Most companies do S&OP on a weekly or monthly basis. Once the plan is done the ‘Real World’ takes over, destabilising the plan. If you join plant floors to ERP you can monitor those operations up from the shop floor to business goals. However the problem now is that currently the information at the lower levels can only be picked up quarterly and only key stakeholders are involved.”

In a recently published IDC report[5], it states, “Inaccurate forecasts can make planning and allocation of resources and servicing new projects very challenging and it can make adequately servicing customers difficult if orders come in all at once. With strained economic conditions in 2009 and 2010, planning was harder as previous year’s revenues provide little indication of future sales.” IDC concludes by recommending discrete manufacturers to adopt S&OP which synchronizes the demand forecasting process with production and customer fulfilment planning.

Kinder feels that, “S&OP has become an essential business process in de-risking the supply chain. The reality is that in any operational planning process there are multiple ways to meet customer demand. But which is the best plan? Best for customers? Best for the business? S&OP—and the modern technologies that support it—deliver confidence that a business has explored the alternatives and hit upon that elusive best plan.”

So it would seem that although S&OP is such a key element in manufacturing today, it means different things to different people depending on where they are on the evolutionary road. But if you want to move to the nirvana of Integrated Business Planning, then you are looking at joining on one side the shop floor data from Manufacturing Execution Systems (MES) with data from your supply chain partners (certainly for your Tier 1’s if not your Tier 2’s) with your HR data, Capacity data and Sales data to produce a plan which may now need to be refined more often than monthly. So we are talking about integration and collaboration not only at a technical level but also at a business process level.

[1] Trends in Sales and Operations Planning, Hitachi Consulting

[2] Getting Started With Sales & Operations Planning, John R. Dougherty

[3] Sales and Operations Planning – A Key Element of Supply Chain Success, Chuck Poirier, CSC

[4] Hitachi Consulting, AMR Research 2005 S&OP Study, Aberdeen Group 2006 Study

[5] Beating complexity, achieving operational excellence, IDC Manufacturing Insight, Pierfrancesco Manenti and Megan Dahlgren, July 2010

By: Dana Gardner, Principal Analyst, Interarbor Solutions Posted: 24th September 2010 Copyright Interarbor Solutions © 2010

An often-overlooked aspect of data center transformation (DCT) is what to do with the older assets as newer systems come online. Much of the retiring IT equipment can possess sensitive data, may be sources of significant economic return, or at least need to be recycled according to various regulations.

Improperly disposing of data and other IT assets can cause embarrassing security breaches, increase costs, and pose the risk of regulatory penalties. Indeed, many IT organizations are largely unaware of the hazards and risks of selling older systems into auction sites, secondary markets or via untested suppliers.

Compliance and recycling issues, as well as data security concerns and proper software disposition, should therefore be top of mind early in the DCT process, not as an after-thought.

In a recent podcast discussion, I tapped two HP executives on how to best manages productive transitions of data center assets—from security and environmental impact, to recycling and resale, and even to rental of transitional systems during a managed upgrade process. I spoke with Helen Tang, Worldwide Data Center Transformation Lead for HP Enterprise Business, and Jim O'Grady, Director of Global Life Cycle Asset Management Services with HP Financial Services.

Here are some excerpts:

Helen Tang: Today there are the new things coming about that everybody is really excited about, such as virtualization, and private cloud. ... This time around, enterprises don’t want to repeat past mistakes, in terms of buying just piles of stuff that are disconnected. Instead, they want a bigger strategy that is able to modernize their assets and tie into a strategic growth enablement asset for the entire business.

Yet throughout the entire DCT process, there's a lot to think about when you look at existing hardware and software assets that are probably aged, and won’t really meet today’s demands for supporting modern applications.

How to dispose of those assets? Most people don’t really think about it nor understand all of the risks involved. ... Even experienced IT professionals, who have been in the business for maybe 10, 20 years, don’t quite have the skills and understanding to grasp all of this.

We're starting to see this  sort of IT hybrid role called the IT controller, that typically reports to the CIO, but also dot-lines into the CFO, so that the two organizations can work together from the very beginning of a data center project to understand how best to optimize both the technology, as well as the financial aspects.

Jim O'Grady: We see that a lot of companies try to manage this themselves, and they don’t have the internal expertise to do it. Often, it’s done in a very disconnected way in the company. Because it’s disconnected and done in many different ways, it leads to more risks than people think.

You are putting your company’s brand at stake, through improper environmental recycling compliance, or exposing your clients, customers, or patients’ data to a security breach. This is definitely one of those areas you don’t want to read about in a newspaper to figure out what went wrong.

One of the most common areas where our clients are caught unaware of is the complexity of the data security, and the e-waste legislation requirements that are out there, and especially the pace of its change.

We suggest that they have a well thought-out plan for destroying or clearing data prior to the asset decommissioning and/or prior to the asset leaving the physical premise of the site. Use your outsource partner, if you have one, as a final validation for data security. So, do it on site, as well as do it off site.

Have a well-established plan and budget up-front, one that’s sponsored by a corporate officer, to handle all of the end-of-use assets well before the end-of-use period comes.

E-waste legislation resides at the state, local, national, and regional levels, and they all differ. There's some conflict, but some are in line with each other. So it's very difficult to understand what your legislative requirements are and how to comply. Your best bet is to deal with a highest standard and pick someone that knows and has experience in meeting these legislative requirements.

There are tremendous amounts of global complexities that customers are trying to overcome, especially when they try to do data center consolidation and transformation, throughout their enterprise across different geographies and country borders.

You're talking about a variety of regulatory practices and directives, especially in the EU, that are emerging and restrict how you move used and non-working product across borders. There are a variety of different data-security practices and environmental waste laws that you need to be aware of.

A lot of our clients choose to outsource this work to a partner. But they need to keep in mind that they are sharing risk with whomever they partner with. So they have to be very cautious and be extremely picky about who they select as a partner.

This may sound a bit self-serving, but I always suggest for enterprises to resist smaller local vendors. ... If you don’t kick the tires with your partner and you don’t find out that the partner consists of a man, a dog, and a pickup truck, you just may have a hard time defending yourself as to why you selected that partner.

Also, develop a very strong vendor audit qualification and ongoing inspection process. Visit that vendor prior to the selection and know where your waste stream is going to end up. Whatever they do with the waste stream, it’s your waste stream. You are a part of the chain of custody, so you are responsible for what happens to that waste stream, no matter what that vendor does with it.

You need to create rigorous documented end-to-end controls and audit processes to provide audit trails for any future legal issues. And finally, select a partner with a brand name and reputation for trust and integrity. Essentially, share the risk.

Enterprises should well consider how they retire and recover value for their entire end-of-use IT equipment, whether it's a PDA or supercomputer, HP or non-HP product. Most data center transformations and consolidations typically end with a lot of excess or end-of-use product.

We can help educate customers on the hidden risk and dispositioning that end-of-use equipment into the secondary market. This is a strength of HP Financial Services (HPFS).

Typically, what we find with companies trying to recover value for product is that they give it to their facilities guys or the local business units. These guys love to put it on eBay and try to advertise for the best price. But, that’s not always the best way to recover the best value for your data center equipment.

Your best bet is to work with a disposition provider that has a very, very strong re-marketing reach into the global markets, and especially a strong demonstrative recovery process.

We're now seeing it migrate into the procurement arm. These guys typically put it out for bid and select the highest bid from a lot of the open market brokers. A better strategy to recover value, but not the best.

Your best bet is to work with a disposition provider that has a very, very strong re-marketing reach into the global markets, and especially a strong demonstrative recovery process.

From a financial asset ownership model, HPFS has the ability to come in and work with a client, understand their asset management strategy, and help them to personalize the financial asset ownership model that makes sense for them.

For example, if you look at a leasing organization, when you lease a product, it's going to come back. A key strength in terms of managing your residual is to recover the value for the product as it comes back, and we do that on a worldwide basis.

We have the ability to reach emerging markets or find the market of highest recovery to be able to recover the value for that product. As we work with clients and they give us their equipment to remarket on their behalf, we bring it into the same process.

When you think about it, an asset recovery program is really the same thing as a lease return. It's really a lot of reverse logistics—bring it into a technical center, where it's audited, the data is wiped, the product is tested, there’s some level of refurbishment done, especially if we can enhance the market value. Then, we bring it into our global markets to recover value for that product.

We have skilled product traders within our product families who know how to hold product, and wait for the right time to release it into the secondary market. If you take a lot of product and sell it in one day, you increase the supply, and all of the recovery rates for the brokers drop overnight. So, you have to be pretty smart. You have to know when to release product in small lot sizes to maximize that recovery value for the client.

We're seeing a big uptake in the need to support legacy product, especially in DCT. We're able to provide highly customized pre-owned authentic legacy HP product solutions, sometimes going back 20 years or more. The need for temporary equipment just scaling out legacy data center hardware platform capacity that’s legacy locked is an increasing need that we see from our clients.

Clients also need to ensure their product is legally licensed and they do not encounter intellectual property right infringements. Lastly, they want to trust that the vendor has the right technical skills to deal with the legacy configuration and compatibility issues.

Our short-term rental program covers new or legacy products. Again, many customers need access to temporary product to prove out some concepts, or just to test some software application on compatibility issues. Or, if you're in the midst of a transformation, you may need access to temporary swing gear to enable the move.

We also help clients understand strategies to recover the best value for decommissioned assets, as well as how to evaluate and how to put in place a good data-security plan.

We help them understand whether data security should be done on-site versus off-site, or is it worth the cost to do it on-site and off-site. We also help them understand the complexities of data wiping enterprise product, versus just the plain PC.

The one thing we help customers understand, and it’s the real hidden complexity is how to set up an effective reverse logistic strategy.

Most of the local vendors and providers out there are skilled in wiping data for PCs, but when you get into enterprise products, it can get really complex. You need to make sure that you understand those complexities, so you can secure the data properly.

Lastly, the one thing we help customers understand, and it’s the real hidden complexity, is how to set up an effective reverse logistic strategy, especially on a global basis. How do you get the timing down for all the products coming back on a return basis?

Tang: We reach out to our customers in various interactions to talk them through the whole process from beginning to end.

One of the great starting points we recommend is something we called the Data Center Transformation Experience Workshop, where we actually bring together your financial side, your operations people, and your CIOs, so all the key stakeholders in the same room, and walk through these common issues that you may or may not have thought about to begin with. You can walk out of that room with consensus, with a shared vision, as well as a roadmap that’s customized for your success.

Listen to the podcast. Find it on iTunes/iPod. Read a full transcript or download a copy.

By: Dana Gardner, Principal Analyst, Interarbor Solutions Posted: 24th September 2010 Copyright Interarbor Solutions © 2010

Sonoa Systems, a provider of application programming interface (API) solutions, has changed its name this week to Apigee.

While Sonoa originally offered a free API tools and management platform, Apigee now offers three product lines for enterprises, developers, and API providers of all sizes. The company now serves more than 7,000 developers and some 140 enterprises with API management services. [Disclosure: Sonoa Systems is a past sponsor of BriefingsDirect podcasts.]

“By unifying the company under one brand and launching our premium line, we can better serve the full spectrum of companies and developers using APIs to power their apps, mobile and multichannel strategies and business partnerships,” said Chet Kapoor, CEO, Apigee.

The traffic has been brisk. Currently, 2,500 GB of data per month and 25k messages are processed per second on Apigee Tech, says the firm.

As I heard more about the role of APIs and how managing and defining that traffic and use patterns—both incoming and outgoing—I was reminded too of the Big Data analysis value so many companies are building out.

What if you were to be able to analyse real-time data with real-time API activities? This may not be for everyone, but many mobile, e-commerce and service providers—and a boat load of web-focused start-ups—could develop some super insights.

Joining the analysis from APIs, systems logs, and data could be a killer business intelligence benefit. It might also spur new revenue by selling that analysis if you happen to find yourself at the juncture of APIs and data and either business or consumer behavior. Viva la real time analytics at scale!

Among the new and rebranded Apigee products:

• Apigee Premium: Announced on Wednesday, Apigee Premium provides advanced features on top of the Apigee Free platform, including unlimited API traffic, advanced rate limiting and analytics, and developer key provisioning. Visit https://app.apigee.com/sign_up to sign up for the preview.
• Apigee Free: A free tools platform launched last year for developers and providers to learn, test, and debug APIs, get analytics on API performance and usage, and apply basic rate-limits to protect their services.
• Apigee Enterprise: An industrial-grade API platform for enterprises using APIs to fuel their mobile, multichannel, application and cloud strategies. Previously Sonoa Systems’ core product ServiceNet, Apigee Enterprise provides API visibility, control, management and security.

By: Dana Gardner, Principal Analyst, Interarbor Solutions Posted: 20th September 2010 Copyright Interarbor Solutions © 2010

Getting developers on board. That’s the challenge technologies from Linux to Android face every day. Genuitec has helped Eclipse overcome this challenge with Pulse. Indeed, more than one million developers around the world have now installed Pulse.

Pulse works to give software developers an efficient way to locate, install and manage their Eclipse-based tool suite, among other tools. The software essentially empowers developers to customize their installs while avoiding plug-in management issues—even when crossing operating systems. [Disclosure: Genuitec is a sponsor of BriefingsDirect podcasts.]

“When we envisioned Pulse in 2007, we knew the developer community badly needed an easy technology to help manage their Eclipse tools,” says Maher Masri, president and CEO of Genuitec, a founding and strategic member of the Eclipse Foundation. “Now with one million users, we can happily say Pulse is a great success story.”

The Pulse advantage
One of the advantages Pulse is pushing out to its one million developers is the ability to manage four years of Eclipse platform technologies from a single dashboard, including Eclipse 3.0, also known as Helios.

That’s no small feat, seeing how many enterprises standardize on older Eclipse versions, yet still demand an easy migration path to upgrade their projects, technical artifacts, and other mission-critical subsystems. Developers can even access Eclipse 3.7, also known as Indigo, as the milestones are rolled out in coming months.

This multi-year tool stack feature is part of the reason why Pulse has attracted so many Eclipse developers. Pulse is the only product on the market that supports this type of lifecycle-based stack management.

Getting to know Pulse
Pulse also provides a product family of offerings. There’s a Community Edition that’s free, a Managed Team Edition that aims at the needs of development teams, and a Private Label software delivery version designed for corporate use. Pulse Community Edition is free for individual developers, while Pulse Managed Team Edition is $60 annually. Pricing for Pulse Private Label, a software delivery and management platform, is based on individual requirements.

“Pulse, like many other powerful Eclipse-based technologies, continues to attract world-class developers to the Eclipse platform,” says Mike Milinkovich, executive director of the Eclipse Foundation. “As we continuously enhance our code base and march toward Eclipse 3.7 next summer, we’re pleased that Genuitec will continue to support developers using Eclipse with its Pulse management software.”

BriefingsDirect contributor Jennifer LeClaire provided editorial assistance and research on this post. She can be reached at http://www.linkedin.com/in/jleclaire and http://www.jenniferleclaire.com.

By: Dana Gardner, Principal Analyst, Interarbor Solutions Posted: 15th September 2010 Copyright Interarbor Solutions © 2010

• Technology, process and people must combine smoothly to achieve strategic virtualization benefits
• Converged Infrastructure Approach Paves Way for Improved Data Center Productivity
  
• HP's Bill Veghte on managing complexity amid converging IT 'inflection points'

By: Dana Gardner, Principal Analyst, Interarbor Solutions Posted: 15th September 2010 Copyright Interarbor Solutions © 2010

We've all heard about client virtualization or virtual desktop infrastructure (VDI) over the past few years, and there are some really great technologies for delivering a PC client experience as a service.

But today’s business and economic drivers need to go beyond just good technology. There also needs to be a clear rationale for change -- both business and economic. Second, there needs to be proven methods for properly moving to client virtualization at low risk and in ways that lead to both high productivity and lower total costs over time.

Cloud computing, mobile device proliferation, and highly efficient data centers are all aligning to make it clear that the deeper and flexible client platform support from back-end servers will become more the norm and less the exception over time.

Client devices and application types will also be dynamically shifting both in numbers and types, and crossing the chasm between the consumer and business spaces. The new requirements for business mobile use point to the need for planning and proper support of the infrastructures that can accommodate these edge, wireless clients.

To help guide business on client virtualization infrastructure requirements, learn more about client virtualization strategies and best practices that support multiple future client directions, and see why such virtualization makes sense economically, we went to Dan Nordhues, Marketing and Business Manager for Client Virtualization Solutions in HP's Industry Standard Servers Organization. The interview is conducted by BriefingsDirect's Dana Gardner, Principal Analyst at Interarbor Solutions.

Here are some excerpts:

Nordhues: In desktop virtualization, what really comes out to the user device is just pixel information. These protocols just give you the screen information, collect your user inputs from the keyboard and mouse, and take those back to the application or the desktop in the data center.

When you look at desktop virtualization, whether it’s a server-based computing environment, where you are delivering applications, or if you are delivering the whole desktop, as in VDI, to get started you really have to take a look at your whole environment -- and make sure that you're doing a proper analysis and are actually ready.

On the data center side, as we start talking about cloud, the solution is really progressing. HP is moving very strongly toward what we call converged infrastructure, which is wire it once and then have it provisioned and be ready to provide the services that you need. We're on a path where the hardware pieces are there to deliver on that.

But you have to look at the data center and its capacity to house the increased number of servers, storage, and networking that has to go there to support the user.

So now you get the storage folks in IT, the networking folks, and the server support folks all involved in the support of the desk-side environment. It definitely brings a new dynamic.

This is not a prescription for getting rid of those IT people. In fact, there is a lot of benefit to the businesses by moving those folks to do more innovation, and to free up cycles to do that, instead of spending all those cycles managing a desktop environment that may be fairly difficult to manage.

Where we're headed with this, even more broadly than VDI, is back to the converged infrastructure, where we talked about wire it once and have it be a solution. Say you're an office worker and you're just getting applications virtualized out to you. You're going to use Microsoft Office-type applications. You don’t need a whole desktop. Maybe you just need some applications streamed to you.

Maybe, you're more of a power user, and you need that whole desktop environment provided by VDI. We'll provide reference architectures with just wire it once type of infrastructure with storage. Depending on what type of user you are, it can deliver both the services and the experience without having to go back and re-provision or start over, which can take weeks and months, instead of minutes.

Also, really a hybrid solution could deliver in the future VDI plus server-based computing together and cover your whole gamut of users, from the very lowest task-oriented user, all the way up to the highest end power users that you have.

And, we're going to see services wrapped around all of this, just to make it that much simpler for the customers to take this, deploy it, and know that it’s going to be successful.

Why VDI now?

It’s a digital generation of millions of new folks entering the workforce, and they've grown up expecting to be mobile and increasingly global. So, we need to have computing environments that don’t have us having to report to a post number in an office building in order to get work done.

We have an increasingly global and mobile workforce out there. Roughly 60 percent of employees in organizations don’t work where their headquarters are for their company, and they work differently.

When you go mobile, you give up some things. However, the major selling point is that you can get access. You can check in on a running process, if you need to see how things are progressing. You can do some simple things like go in and monitor processes, call logs, or things like that. Having that access is increasingly important.

Delivering packaged services out to the end user is something that’s still being worked out by software providers, and you're going to see some more elements of that come out as we go through the next year.

And, of course, there's the impact of security, which is always the highest on customer lists. We have customers out there, large enterprise accounts, who are spending north of $100 million a year just to protect themselves from internal fraud.

With client virtualization, the security is built in. You have everything in the data center. You can’t have users on the user endpoint side, which may be a thin client access device, taking files away on USB keys or sticks.

It’s all something that can be protected by IT, and they can give access only to users as they see fit. In most cases, they want to strictly control that. Also, you don’t have users putting applications that you don't want ... on top of your IT infrastructure.

And there is really a catalyst coming as well in the Windows 7 availability and launch since late last year. Many organizations are looking at their transition plans there. It’s a natural time to look at a way to do the desktop differently than it has been done in the past.

Reference architectures support all clients

We've launched several reference architectures and we are going to continue to head down this path. A reference architecture is a prescribed solution for a given set of problems.

A lot of the deployment issue, and what makes this difficult, is that there are so many choices.

For example, in June, we just launched a reference architecture for VDI that uses some iSCSI SAN storage technology, and storage has traditionally been one of the cost factors in deploying client virtualization. It has been very costly to deploy Fibre Channel SAN, for example. So, moving to this iSCSI SAN technology is helping to reduce the cost and provide fantastic performance.

In this reference architecture, we've done the system integration for the customer. A lot of the deployment issue, and what makes this difficult, is that there are so many choices. You have to choose which server to use and from which vendor: HP, Dell, IBM, or Cisco? Which storage to choose: HP, EMC, or NetApp? Then, you have got the software piece of it. Which hypervisor to use: Microsoft, VMware, or Citrix? Once you chase all these down and do your testing and your proof of concept, it can take quite a substantial length of time.

We targeted the enterprise first. Some of our reference architectures that are out there today exist for 1,000-plus users in a VDI environment. If you go to some of the lower-end offerings we have, they are still in the 400-500 range.

We're looking at bringing that down even further with some new storage technologies, which will get us down to a couple of hundred users, the small and medium business (SMB) market, certainly the mid-market, and making it just very easy for those folks to deploy. They'll have it come completely packaged.

Today, we have reference architectures based on VDI or based on server-based computing and delivering just the applications. As I mentioned before, were looking at marrying those, so you truly have a wire-it-once infrastructure that can deliver whatever the needs are for your broad user community.

What HP has done with these reference architectures is say, "Look, Mr. Customer, we've done all this for you. Here is the server and storage and all the way out to the thin client solution. We've tested it. We've engineered it with our partners and with the software stack, and we can tell you that this VDI solution will support exactly this many knowledge workers or that many productivity users in your PC environment." So, you take that system integration task away from the customer, because HP has done it for them.

We have a number of customer references. I won’t call them out specifically, but we do have some of these posted out on HP.com/go/clientvirtualization, and we continue to post more of our customer case studies out there. They are across the whole desktop virtualization space. Some are on server-based computing or sharing applications, some are based on VDI environments, and we continue to add to those.

With any new computing technology, the underlying consideration is always cost or, in this case, a lot of customers look at it at a cost-per-seat perspective, and this is no different.

HP also has an ROI or TCO calculator that we put together specifically for this space. You show a customer a case study and they say, "Well, that doesn’t really match my pain points. That doesn’t really match my problem. We don’t have that IT issue," or "We don’t have that energy, power issue."

We created this calculator, so that customers can put in their own data. It’s a fairly robust tool, but we can put in information about what’s your desktop environment costing you today, what would it cost to put in a client virtualization environment, and what you can expect as far as your return on investment. So, it’s a compelling part of the discussion.

Obviously, with any new computing technology, the underlying consideration is always cost or, in this case, a lot of customers look at it at a cost-per-seat perspective, and this is no different, which is why we have provided the tool and the consulting around that.

On that same website that I mentioned, HP.com/go/clientvirtualization, we have our technical white papers that we've published, along with each of these reference architectures.

For example, if you pick the VDI reference architecture that will support 1,000-plus users in general, there is a 100-page white paper that talks about exactly how we tested it, how we engineered it, and how it scales with the VMware view or with Microsoft Hyper-V, plus Citrix XenDesktop.

• Thin Is In: The Enterprise Virtualization Inflection Point
• HP Data Protector, a Case Study on Scale and Completeness for Total Enterprise Data Backup and Recovery
  
• HP teams with Microsoft, VMware to expand appeal of desktop virtualization solutions

By: Dana Gardner, Principal Analyst, Interarbor Solutions Posted: 9th September 2010 Copyright Interarbor Solutions © 2010

Figuring that small- and medium-sized businesses (SMBs) want the best in IT advances too, HP on Wednesday unleashed a barrage of products and services that use integration, low-cost, and simplicity to bring cutting edge enterprise IT capabilities to the global mid-market.

The new products and services—ranging from the $329 HP ProLiant MicroServer to $424 minitower PCs to simplified virtualization, networking and storage bundles—come from multiple organizations across HP, but with a singular Goldilocks target of “Just Right IT” for SMBs. [Disclosure: HP is a sponsor of BriefingsDirect podcasts.]

The slew of value-oriented offerings is also designed to give HPs various global channel partners a new horse to ride into town on as the SMBs look beyond recession-reckoning for how to grow their operations while becoming more productive. The products and services are also available from HP directly.

HP is also putting financial muscle behind the channel partners and users by providing aggressive financing options leasing, life cycle asset management and upgrade services. HP Financial Services is the second-largest captive IT leasing company in the world, said HP. Leasing provides SMBs with flexibility (with no or low upfront payments) and a path to migrate to newer technology.

While the value and utilization benefits of virtualization have been quickly adopted by larger companies and IT departments, the use of hypervisors has been slower in SMBs. To help solve that, HP has developed more complete virtualization environments using Virtualization Smart Bundles with Microsoft Hyper-V Server 2008 R2. The bundles target storage, servers and networking virtualization technology uses.

The SMB-targeted worker productivity releases include:

• HP ProLiant MicroServer, an energy-efficient file server designed for businesses with up to 10 employees to centralize information and securely access files faster (at about half the size and 50 percent quieter than most entry-level servers).
• Web connectivity in the low-cost HP Officejet Pro 8500A e-All-in-One series and HP Officejet 7500A Wide Format e-All-in-One, which allow users to send print jobs from mobile devices as well as access content from the web without a PC.
• Slashed costs and energy use in the now-available HP 500B and 505B Series Business Desktop PCs, mini-towers installed with Windows 7 with Intel or AMD processors
• Simplified HP Insight with Microsoft System Center Essentials 2010 for monitoring and management of IT from a single console so midsize businesses can adopt or expand use of virtualized servers and storage.

The SMB-targeted storage management releases include:

• Storage advancements via the 10GbE iSCSI capabilities of the HP StorageWorks P2000 G3 Modular Smart Array (MSA), which speeds the server/storage connection bandwidth by 10 times.
• HP ITSM Assessment for Virtualized Environments Service for increased system availability and process improvements
• HP Data Protector Express 5.0 Software, designed for the general user for managing data backup and recovery on single servers as well as small networks in Windows, Linux and NetWare environments.
• Simplified shared storage with the HP P4000 Virtual SAN Appliance (VSA) so those using virtualized servers (deployed on Microsoft Hyper-V or VMware virtual machines) can move to shared storage without purchasing costly physical storage area network infrastructure.

The SMB-targeted networking and communications releases include:

• HP voice-over-IP and wireless offerings with the HP V-M200 802.11n Access Point Series, which connects up to 64 simultaneous mobile users to the network at wire-like speeds.
• HP VCX 9.5 IP Telephony system and 350x IP Phones (starting at $119), which enable the convergence of voice and data onto a single network infrastructure.

SMBs are where economists look for growth to emerge from recessions, and in developing countries. For years, though, large IT vendors have focused on the top ends of the IT market. It makes a lot of sense for HP to scale the technology and offerings down to the SMBs—which is a huge total market, poised for unprecedented growth in the world's most populous regions.

Fact is, too, that due to proliferating mobile devices and wireless networks, nearly all companies of any size need to deeply embrace technology and networking to remain competitive. Data explosion also makes it unavoidable to bring in managed storage and backup, not to mention the burgeoning requirements of security and managed access.

While many of us analysts harp on about the virtues and inevitability of cloud computing, for many small companies and in many regions, the promise of cloud cannot be considered until the basics of IT are modernized and managed.

Mobile devices alone can not take the place of a LAN and managed storage. In many ways, these new HP products and bundles—with their pricing and simplicity—can be seen as stepping stones for SMBs to soon be able to exploit the value and potential of cloud-based services, too.

And then we actually might see these SMBs leap-frog their larger corporate brethren, rather than be seen as a lagging market category, in regards to IT productivity and enablement. And wouldn't that be exciting?

By: Dana Gardner, Principal Analyst, Interarbor Solutions Posted: 3rd September 2010 Copyright Interarbor Solutions © 2010

The trap of unchecked virtualization complexity can have a stifling effect on the advantageous spread of virtualization in data centers.

Indeed, many enterprises may think they have already exhausted their virtualization paybacks, when in fact, they have only scratched the surface of the potential long-term benefits.

Automation, policy-driven processes and best practices are offering more opportunities for optimizing virtualization so that server, storage, and network virtualization can move from points of progress into more holistic levels of adoption.

The goals then are data center transformation, performance and workload agility, and cost and energy efficiency. Many data centers are leveraging automation and best practices to attain 70 percent and even 80 percent adoption rates.

By taking such a strategic outlook on virtualization, process automation sets up companies to better exploit cloud computing and IT transformation benefits at the pace of their choosing, not based on artificial limits imposed by dated or manual management practices.

To explore how automation can help achieve strategic levels of virtualization, BriefingsDirect brought together panelists Erik Frieberg, Vice President of Solutions Marketing at HP Software, and Erik Vogel, Practice Principal and America's Lead for Cloud Resources at HP. The discussion is moderated by Dana Gardner, Principal Analyst at Interarbor Solutions.

Here are some excerpts:

Vogel: Probably the biggest misconception that I see with clients is the assumption that they're fully virtualized, when they're probably only 30 or 40 percent virtualized. They've gone out and done the virtualization of IT, for example, and they haven't even started to look at Tier 1 applications.

The misconception is that we can't virtualize Tier 1 apps. In reality, we see clients doing it every day. The broadest misconception is what virtualization can do and how far it can get you. Thirty percent is the low-end threshold today. We're seeing clients who are 75–80 percent virtualized in Tier 1 applications.

Frieberg: The three misconceptions I see a lot are, one, automation and virtualization are just about reducing head count. The second is that automation doesn't have as much impact on compliance. The third is if automation is really at the element level, they just don't understand how they would do this for these Tier 1 workloads.

You're starting to see the movement beyond those initial goals of eliminating people to ensuring compliance. They're asking how do I establish and enforce compliance policies across my organization, and beyond that, really capturing or using best practices within the organization.

When you look at the adoption, you have to look at where people are going, as far as the individual elements, versus the ultimate goal of automating the provisioning and rolling out a complete business service or application.

When I talk to people about automation, they consistently talk about what I call "element automation." Provisioning a server, a database, or a network device is a good first step, and we see gaining market adoption of automating these physical things. What we're also seeing is the idea of moving beyond the individual element automation to full process automation.

As companies expand their use of automation to full services, they're able to reduce that time from months down to days or weeks. This is what some people are starting to call cloud provisioning or self-service business application provisioning. This is really the ultimate goal—provisioning these full applications and services versus what is often IT’s goal—automating the building blocks of a full business service.

This is where you're starting to see what some people call the "lights out" data center. It has the same amount or even less physical infrastructure using less power, but you see the absence of people. These large data centers just have very few people working in them, but at the same time, are delivering applications and services to people at a highly increased rate rather than as traditionally provided by IT.

Vogel: One of the challenges that our clients face is how to build the business case for moving from 30 percent to 60 or 70 percent virtualized. This is an ongoing debate within a number of clients today, because they look at that initial upfront cost and see that the investment is probably higher than what they were anticipating. I think in a lot of cases that is holding our clients back from really achieving these higher levels of virtualization.

In order to really make that jump, the business case has to be made beyond just reduction in headcount or less work effort. We see clients having to look at things like improving availability, being able to do migrations, streamlined backup capabilities, and improved fault-tolerance. When you start looking across the broader picture of the benefits, it becomes easier to make a business case to start moving to a higher percentage of virtualization.

One of the things we saw early on with virtualization is that just moving to a virtual environment does not necessarily reduce a lot of the maintenance and management that we have, because we haven’t really done anything to reduce the number of OS instances that have to be managed.

The benefits are relatively constrained, if we look at it from just a physical footprint reduction. In some cases, it might be significant if a client is running out of data-center space, power, or cooling capacity within the data center. Then, virtualization makes a lot of sense because of the reduction in asset footprint.

But, when we start looking at coupling virtualization with improved process and improved governance, thereby reducing the number of OS instances, application rationalization, and those kinds of broader process type issues, then we start to see the big benefits come into play.

Now, we're not talking just about reducing the asset footprint. We're also talking about reducing the number of OS instances. Hence, the management complexity of that environment will decrease. In reality, the big benefits are on the logical side and not so much on the physical side.

Frieberg: What we're seeing in companies is that they're realizing that their business applications and services are becoming too complex for humans to manage quickly and reliably.

The demands of provisioning, managing, and moving in this new agile development environment and this environment of hybrid IT, where you're consuming more business services, is really moving beyond what a lot of people can manage. The idea is that they are looking at automation to make their life easier, to operate IT in a compliant way, and also deliver on the overall business goals of a more agile IT.

Companies are almost going through three phases of maturity when they do this. The first aspect is that a lot of automation revolves around "run book automation" (RBA), which is this physical book that has all these scripts and processes that IT is supposed to look at.

But, what you find is that their processes are not very standardized. They might have five different ways of configuring your device, resetting the server, and checking why an application isn’t working.

So, as we look at maturity, you’ve got to standardize on a set of ways. You have to do things consistently. When you standardize methods, you then find out you're able to do the second level of maturity, which is consolidate.

Vogel: It becomes more than just talking about the hardware or the virtualization, but rather a broader question of how IT operates and procures services. We have to start changing the way we are thinking when we're going to stand up a number of virtual images.

When we start moving to a cloud environment, we talk about how we share a resource pool. Virtualization is obviously key and an underlying technology to enable that sharing of a virtual resource pool.

We're seeing the virtualization providers coming out with new versions of their software that enable very flexible cloud infrastructures.

This includes the ability to create hybrid cloud infrastructures, which are partially a private cloud that sits within your own site, and the ability to burst seamlessly to a public cloud as needed for excess capacity, as well as the ability to seamlessly transfer workloads in and out of a private cloud to a public cloud provider as needed.

We're seeing the shift from IT becoming more of a service broker, where services are sourced and not just provided internally, as was traditionally done. Now, they're sourced from a public cloud provider or a public-service provider, or provided internally on a private cloud or on a dedicated piece of hardware. IT now has more choices than ever in how they go about procuring that service.

But it becomes very important to start talking about how we govern that, how we control who has access, how we can provision, what gets provisioned and when. ... It's a much bigger problem and a more complicated problem as we start going to higher levels of virtualization and automation and create environments that start to look like a private cloud infrastructure.

I don’t think anybody will question that there are continued significant benefits, as we start looking at different cloud computing models. If we look at what public cloud providers today are charging for infrastructure, versus what it costs a client today to stand up an equivalent server in their environment, the economics are very, very compelling to move to a cloud-type of model.

Without the proper governance in place, we can actually see cost increase, but when we have the right governance and processes in place for this cloud environment, we've seen very compelling economics, and it's probably the most compelling change in IT from an economic perspective within the last 10 years.

Frieberg: If you want to automate and virtualize an entire service, you’ve got to get 12 people to get together to look at the standard way to roll out that environment, and how to do it in today’s governed, compliant infrastructure.

The coordination required, to use a term used earlier, isn’t just linear. It sometimes becomes exponential. So there are challenges, but the rewards are also exponential. This is why it takes weeks to put these into production. It isn’t the individual pieces. You're getting all these people working together and coordinated. This is extremely difficult and this is what companies find challenging.

The key goal here is that we work with clients who realize that you don’t want a two-year payback. You want to show payback in three or four months. Get that payback and then address the next challenge and the next challenge and the next challenge. It's not a big bang approach. It's this idea of continuous payback and improvement within your organization to move to the end goal of this private cloud or hybrid IT infrastructure.

Vogel: We've developed a capability matrix across six broad domains to look at how a client needs to start to operationalize virtualization as opposed to just virtualizing a physical server.

We definitely understand and recognize that it has to be part of the IT strategy. It is not just a tactical decision to move a server from physical machine to a virtual machine, but rather it becomes part of an IT organization’s DNA that everything is going to move to this new environment.

We're really going to start looking at everything as a service, as opposed to as a server, as a network component, as a storage device, how those things come together, and how we virtualize the service itself as opposed to all of those unique components.

It really becomes baked into an IT organization’s DNA, and we need to look very closely at their capability—how capable an organization is from a cultural standpoint, a governance standpoint, and a process standpoint to really operationalize that concept.

Listen to the podcast. Find it on iTunes/iPod. Read a full transcript or download a copy.