2.4 Back to the roots: source of all challenges

Although the symptoms described in the previous chapter are diverse, the root cause for most of them is the way companies handle their data.

The existence of many isolated databases ("data silos") impedes the free flow of information in a company. There is no way that a business ever becomes a successful digital enterprise if it holds on to its data silos. The main characteristics of such an enterprise IT landscape are tons of duplicated data pushed around over dozens of interfaces between systems, typically in long batch runs at night. The more silos a company has, the more difficult it becomes to maintain consistent information across all duplicates. In other words: when information is read from one of the data silos, you can never be sure that is it correct. Instead chances are high that it is outdated. There are estimations from independent analyst firms that up to 80% of data that is processed in an enterprise is duplicated. What a nightmare.

Companies have tried to solve the dilemma by bringing in dedicated applications which were in charge to keep all data synchronized. These are called “Enterprise Application Integration” (EAI) systems and a lot of money was spent on it with moderate success. Now companies urgently need more precise information on their business, their customers and partners based on exploding data volumes. The market calls it “Big data analytics” and what exactly is it: bringing in more expensive technology and with it even more complexity. What would we expect from such a solution if the old IT saying holds true: "garbage in, garbage out"?!

In order to get out of this deadly spiral of ever more technology and higher costs, companies need to address the root cause of the whole dilemma which is the existence of too many isolated data silos.

In other words: if companies change the way they handle their data many of the challenges described in the previous chapter will disappear!