'Attack of the Data Silos 2': Coming soon to an enterprise near you

On the heels of the consumerization of enterprise software and the growing ubiquity of easy-to-use analytics tools, silos appear to be coming back in all their former collaboration-stifling glory as individual teams and departments pick and choose different tools for different purposes and data sets without enterprise-level oversight.

In the short term, that can deliver exciting new capabilities for the users involved, who can finally get the quick business-intelligence insights they want without the involvement of data analysts or IT. Hand in hand with that new democratization of analytics power, however, can come duplication of data and efforts, lack of compliance, data-quality issues and, yes, those dreaded silos again, preventing data and insights from being shared with other parts of the organization.

"ERP suites were designed to be a replacement for separate departmental applications," said analyst Henry Morris, a senior vice president with IDC.

In such scenarios, integrated applications were meant to share the same database for both efficiency and access to a common, integrated set of data.

That, however, is not quite how it worked out.

"Several things thwarted this vision," Morris said. One is that some companies implemented multiple ERP suites; another is the rise of the cloud and software-as-a-service, making tools cheaper and easier to purchase without corporate approval. In both cases, that vision of a single, unified set of data and software fell by the wayside.

The silo problem is particularly common with desktop-based tools into which users upload whatever data they have access to, noted Forrester analyst Martha Bennett. "You can easily replicate the same issue you've already got with spreadsheets," she said. "Multiple versions of the truth, except it looks prettier."

But it can happen in big data projects as well, she said. "Even a successful project -- one that yields tangible results that are of value -- may ultimately be a failure if it ends up remaining a silo."

Either way, the potential consequences are many: time wasted arguing about whose results are right; misguided decisions; regulatory and compliance exposure; and higher cost. "Most importantly, the organization cannot become truly data-driven in its decision-making, which is likely sooner or later to lead to competitive disadvantage," Bennett said.

Allied Global, a provider of call-center services, is no stranger to data silos. With nine call centers in four countries employing about 1,800 people, the company recently replaced a proprietary enterprise software setup that consisted of multiple disjointed pieces spanning different areas of the company.

Getting the right information into the hands of the right people in a timely manner was an ongoing challenge, said David Rae, the company's president and CEO. "We definitely had silos," he said. "We were probably making more decisions based on intuition or gut feeling than solid facts."

An analytics group was responsible for generating standard reports, but analysts had to pull data from various sources and then cleanse and verify it. Users weren't always willing to wait. Instead, some would circumvent the analytics team and seek out the data themselves for manipulation in Excel.

Company data resided in multiple places, however, so it wasn't always clear how to get it. "People had to send an email or walk over and try to get the information from the person they believed had it," Rae explained.

Starting in 2013 Allied implemented finance and human-capital management systems from Workday, and Rae took pains to ensure silos wouldn't pop up again.

After training a group of 75 or so users in Workday's analytics and reporting capabilities -- as well as its unified data architecture -- Rae gave them free rein for a limited time to explore the new tools to their collective heart's content.

"I figured for the first three to four months it would be fine to let the horses run wild," he said.

After that time, though, Allied started to impose some structure, including processes ensuring consistent data validation and standardizing a set of popular reports.

"My absolute concern was that instead of just an analytics group and a bunch of keeners, we now had 75 people trained" in Workday's analytics, Rae said. "I didn't want somebody in our Phoenix office creating something relevant but not sharing it, and someone in Manila creating the same thing because they didn't know it existed."

A common set of tools can be important, but even more critical is a common set of data, IDC's Morris said.

"It's not the proliferation of separate BI tools that is the problem," he explained. "If you had an integrated, reconciled data set, you can certainly apply multiple BI tools to get at the data."

Such integration requires cooperation between IT and the lines of business, and that brings up what may well be the most important piece of the puzzle: governance.

By 2017, most business users will have access to self-service analytics tools, according to a recent Gartner report. Less than 10 percent of self-service BI initiatives, however, will be governed sufficiently.

"Departments have a ferocious appetite to get insights from their data, but they also need security and management's trust," said Nic Smith, SAP's senior director of marketing for analytics and the company's Lumira tool.

"It may not sound fashionable, but data governance is more important than ever," Forrester's Bennett said.

The business side of the company should own the data, she advised, while IT focuses on making that data available in the right format.

Time, however, is still of the essence.

"Getting the job on time often trumps getting the job done with the right tools using the right data but a week or month later," said Boris Evelson, a vice president at Forrester. "We strongly believe that single version of the truth is not absolute -- it's relative and contextual, and how much are you willing to pay for 100 percent accuracy"

Evelson recommends creating an enterprise data hub based on Hadoop or another similar, low-cost platform and then creating BI apps as "spokes" off of that hub.

Then, "pick your battles," he said. "Most of the enterprise data should be in the data hub," but "here you give up some of the controls: the data may not be very clean or integrated, but it's all in one place."

Whichever setup works best for now, it's a pretty safe bet this won't be the last time companies wrestle with the problem.

"The evolution of the business-intelligence market can be thought of as a pendulum," said Brad Peters, chairman and chief product officer at BI software provider Birst.

"We started at one extreme of the pendulum: an IT-owned, centrally governed model," he explained. "Then the pendulum swung to the other extreme in response to business-user demand. Today we're seeing the pendulum swing back towards the middle as IT and business leaders recognize the need to find a healthy balance."