Data analysis in business is not a new phenomenon. However, never before have we had so much data, from so many sources, available so quickly. At the same time, the technology to store, manipulate and make use of the data has never been more accessible. The explosion of data and the ability to use data to inform decision-making increasingly are referred to simply as “big data.” Big data has emerged both as an industry and as a practice within a growing number of companies that see the potential to derive new insights and ultimately gain competitive advantage from data. The rise of big data has received an enormous amount of press coverage, but we would like to draw attention to the organizational and talent investments required for big data to make a difference in the business.
Of course, realizing the potential of big data presents numerous technical challenges — from the computing and storage capabilities needed to keep up with the escalating growth to the platforms, tools and architecture required to make sense of the data. After decades of focusing on cost control, there is a growing buzz among IT teams, who view big data as a new and exciting opportunity to add value. Having a clear technology and data plan is critical, as is tackling potential privacy, legal or regulatory obstacles early.
However, realizing the potential of big data presents more than a technical challenge. As with other transformative technologies, companies need to consider whether they have the right executive talent to implement big data solutions and take full advantage of their potential. We at Spencer Stuart are frequently called upon to help companies evaluate and address new talent needs stemming from big data. We have completed more than 150 executive searches in the past few years, working with clients both to find specific leaders to drive big data initiatives and to create and scale big data technology and service providers. This includes engineering experts who have built their careers around data as well as statistics and analytics experts who bring in-depth knowledge of machine learning, data visualization, predictive analytics and other new approaches to distilling actionable insights from data. In addition, we have recruited hundreds of senior general managers and functional leaders who are embracing new methods of making business decisions — based on new forms of data analysis rather than instinct or past practice — and applying big data techniques to answer the business’ most important questions. Many of these leaders are going even further, looking for ways to transform the data exhaust from millions of mobile devices, decades of financial data or milliseconds of traffic data from around the globe into a central asset for their organizations.
Beyond the talent, the adoption of big data tools and technology has broad implications for the organization. Unlike other technology innovations, big data requires a deep integration between technologists and nontechnologists and dramatically different business processes to truly have an impact. One senior executive in the space recently asked us whether we had encountered an analogous trend with respect to talent and the organization. Two that come close are the introduction of the Internet to most businesses in the late 1990s — which arguably represented a more fundamental shift — and the broad enterprise adoption of mobile technologies. Neither fully captures the uniqueness of big data, but like big data, both were destructive of traditional business models and practices and have created billions of dollars in profit pools that previously did not exist.
Finally, for both, the demand for talent far exceeded the supply in the early years; there simply was no ready pool of executives and technology leaders with previous experience in the field. From the first webmasters to the titans of e-commerce and digital media, many of these executives and experts came from other fields with a relevant parallel. The same will be true of big data. Indeed, most of the executives today were not experts in big data 10 years ago; they were working in a variety of sectors, including relational databases, content management, enterprise resource planning (ERP) software, e-commerce, ad networks and many other highly quantitative and technology-driven businesses.
Through our work with clients and interviews with big data leaders, we have identified three fundamental requirements for realizing the potential of big data: organization-wide talent able to operate effectively in a data-driven world; a culture that promotes data-driven decision-making; and an organizational structure that promotes the shared use of data.
Build big data capabilities across your organization
If big data is to live up to its promise, companies will have to address talent needs at all levels of the organization, including: senior management and functional leaders who must evaluate and define the opportunities where data-driven insights could most impact the business; the data scientists able to extract meaningful insights from the data; the technologists required to build and incorporate new technologies; and an organization that holistically sees the potential of data not only to drive new insights about existing businesses, but as a source of new business models, products or services.
Where the data comes from: Putting the “Big” in Big Data
Start at the top
Companies face a variety of strategic, operational and cultural challenges to adopting big data approaches and tools. These can be overcome only if the board, CEO and senior executive team embrace the possibilities of big data. Senior leadership must direct big data resources to the most meaningful issues of the business and promote a culture that places quantitative analysis at the center of business decision-making. Further, they should consider the data of the organization a key asset to create new value.
Leaders must ensure that engineering teams can implement and exploit big-data-related technologies and that business heads can evaluate the opportunities to use data-driven insights to improve the business. Organizations also need data-savvy functional and business executives who are willing to integrate big data into business processes. In data-rich functions such as sales, customer service, operations and marketing, analytic abilities will be especially critical. “There’s a great opportunity out there to get ahead of your competition and we’re seeing our customers change market dynamics,” said MapR CEO John Schroeder, whose company helps organizations harness the power of big data analytics with Hadoop. “If you’re a laggard, you might not survive.”
Bring in the experts
A key constraint to extracting value from big data for many organizations is the lack of data science and engineering expertise. The complexity of data analytics requires individuals with years of academic or R&D experience. In addition, the most valuable data scientists also have in-depth domain knowledge, which can take years to acquire.
Organizations that view big data as a competitive advantage will need to hire data scientists and engineering leaders who can build and train big data teams. Unfortunately, there is a small pool of experienced talent. These executives can be principally found in a handful of high tech companies, IT services firms, startups and some progressive IT and analytics organizations in larger institutions. Not surprisingly, the competition to recruit these individuals has become increasingly intense, and the top talent wants to work on the most exciting challenges — companies with the most interesting data assets or the most innovative technology and tools.
We are seeing the coming of age of new roles such as “chief data officer” (CDO) and head of analytics reporting into the C-suite, as organizations strive to give more prominence and influence to these specialized technical leaders. Yahoo! named a chief data officer in 2004, one of the first companies to do so. Today the position is increasingly found at a broad range of companies, federal agencies, state governments and even cities, including Chicago, New York and Philadelphia. The CDO is generally responsible for defining the strategic priorities for the management and use of data throughout the enterprise, including new business and performance improvement opportunities.
Drive data literacy throughout the organization
In a recent survey by The Economist, 41 percent of respondents said they don’t have the right skills in their organization to process data more rapidly. Forward-thinking companies are putting plans in place to enhance the data literacy of employees at all levels of the organization. Employees across functions will be required to interpret and use big data-generated analytics; organizations, then, will have to improve their ability to hire, assess and develop individuals who are quantitative and who can integrate new processes for decision-making into their daily activities.
In addition, many businesses will need to develop and implement new tools that distill insights from the data into actionable information for employees. “An important opportunity for companies lies in dramatically enhancing their ability to extract the valuable predictive signals from the noise of big data and transform these signals into insights and prescriptive actions that can be ‘consumed’ by nontechnical employees at the front line,” said Opera Solutions CEO Arnab Gupta. “This requires a culture that embraces new processes and practices readily at all levels. The traditional notion of data management as capturing and storing data in a relatively static manner is giving way to the concept of managing an ongoing flow of data — not only the company’s own data but also data from external sources. This has extraordinary implications in terms of management technologies, management infrastructure, management perspective, the value of data and so on. Most enterprises today don’t have the capacity to deal with the flow of data and continually extract intelligence from that flow of data.”
Build a big data culture
For most companies, fully leveraging big data will require the adoption of new behaviors and approaches to solving business problems and, just as important, the organizational willingness to abandon traditional decision-making processes and tools in favor of predictive modeling techniques that provide actionable insights. This is incredibly powerful but if techniques are not well-understood, this could lead to badly inferred causal relationships and the cherry-picking of data and statistical relationships to drive a desired but not appropriate conclusion.
Over time, organizations that create a culture that values data-driven decision-making, analytical understanding and the spirit of continuous learning will excel in the use of big data.
Data-driven mindset
A large investment in data analytics and supporting technology is worthless unless the analysis is used during decision-making or to create new data-centric businesses. Increasingly, forward-looking senior leaders will insist that management decisions be based on hard analytic information and apply predictive analysis regularly to run the business. These leaders must be analytically savvy to understand the potential of big data and where to apply it. They must be curious and have the desire to invest in and experiment with big data techniques and tools. They must be willing to look at business problems in fresh ways — applying unconventional data sets and approaches to solve traditional problems.
The willingness to look for answers in unexpected places can lead to surprising insights. In one example, automatically generated data from commercial building elevators reflecting building activity has been used to predict whether tenants will renew their leases. “The best leading economic indicator of a building’s financial health is elevator traffic!” said Splunk Chairman and CEO Godfrey Sullivan. “Line executives are just now starting to be aware that new data sources are available to them and awakening to the possibilities. Big data is fundamentally changing people’s jobs. If executives are curious, they will plow into it. If they are not, they’ll be run over by the guy who did get curious.” Data generated from everything from mobile phones and social networks to fleet trucks and security cameras will continue to create new sources of value and insight.
As a corollary, decisions justified on intuition will be regarded as suspect. “Today, acceptance of data-driven decision-making varies widely by culture, and we do see people who rely on their subjective evaluation to know what is right and they can be skeptical of data,” observes 10gen CEO Max Schireson. “Over time, that mindset is going to become more and more of a hindrance to executive success.”
What you need to know: The questions about big data that boards and top leaders need to consider
A culture of experimentation
In a big data world, the decision-making processes that “worked in the past” must give way to a culture in which experimentation and continuous learning are embedded into the business processes. Advanced marketing organizations are at the forefront of this approach, undergoing a transformation driven by insights gained through experimentation. A traditional marketing campaign can spread over many months, with limited analysis of its efficacy. Today, social media and search campaigns can last mere days or even hours, with rapid adjustments and trials made based on immediately accessed data. In other areas, enormous stores of proprietary and publicly available customer data are powering recommendation engines, coupon-serving applications and mobile applications to drive sales. Despite these advances and their undeniable potential, many leaders admit their organizations may be collecting the wrong type of data with the wrong frequency to support real-time decision-making. The core framework for experimentation is also changing dramatically from simple user testing or pilots to massive multivariate testing across thousands if not millions of users. This experimentation is not only in the traditional arenas such as marketing or user experience, but also in the way data is amalgamated, analyzed and used.
Of course, data for the sake of data is not the answer either. Companies still need to have a clear objective in mind before starting a project: What questions are you trying to answer? What do you want to learn? Data isn’t a substitute for business leadership, but a helpful tool that can yield unexpected answers. Executives who are likely to be most successful in a big data environment will focus analytics on the business’ most critical challenges and opportunities, welcome the insight data provides, push to ensure there is appropriate rigor behind the analysis of this data, and impose disciplined processes to avoid misinterpreting or overreacting to initial observations. The environments that glean the most from big data will also look to create new opportunities and not just apply new tools to understand the current business with additional precision.
Address organizational barriers
One of the biggest impediments to using big data effectively for decision-making is the existence of organizational silos, particularly in larger organizations. Useful data is no longer found exclusively within a team or a function or even from a single source of “truth” such as financial planning and analysis, but across the organization, often in areas that are not traditional sources of data. Increasingly, internal data augmented with external sources will produce the insights that will drive new value and potential. Businesses are incorporating public sources of data and driving creative deals with other companies to obtain relevant data — for example, analyzing social media activity to flag possible fraud or analyzing the language in tweets to drive real estate development decisions.
Companies that are able to break down organizational barriers and data silos will be able to tap into rich stores of information that are valuable across functions and business units. The growing instrumentation of everything from jet engines to vending machines to home appliances is generating a treasure trove of information about how customers are using — or not using — products and services, and this information can provide insights for marketing, product management, customer service and beyond. For example, a cable set-top box providing a stream of data on customers’ use — what they search for, what they download — can be aggregated, separated from identifying information to protect privacy and analyzed not only to inform marketing decisions, but also to alert the company to service needs in a particular area.
Data silos can prevent this type of information sharing. Some silos may exist because of regulatory or privacy restrictions, while others may be a function of technical limitations or cultural barriers. While operationalizing data sharing presents significant technical challenges, cultural hurdles can be even more difficult to overcome, requiring employees at all levels to think and work in new ways. Overcoming these barriers requires both executive-level support and the adoption of standards and processes.
Tone from the top
Senior management and functional leaders play an important role in breaking down cultural barriers by creating a mandate for change. They can do this by embracing and evangelizing the potential of big data, building the right expertise and, importantly, acknowledging that these approaches are new and disruptive — instilling both the drive to change and the tolerance for the journey, which inevitably will have successes and failures. To ease the transition, some companies are using labs and skunkworks projects to experiment before trying to wholly transform the organization, or creating new functions trusted with aggregating data from throughout the business.
Organizational design
Drawing insight from big data requires collaboration across IT and engineering, marketing, finance, product and customer support — each of which may have distinct standards and processes for collecting, storing and using data. Just adding more dataliterate individuals across organizations is insufficient to address organizational flaws. Some companies have created governance councils to make decisions about who has access to data and how it’s used. Many are experimenting with how to structure the big data science function within the organization, considering whether data scientists should be embedded within functions and business units or operate as a centralized resource. Whatever the organizational structure, companies will need to establish processes that promote appropriate access to data, address the inherent privacy and security issues, and ensure that big data initiatives support the business’ strategic priorities.
The potential for big data to influence the development of digital products or services may prove to be one of the most telling examples of how organizational structure can help or hamper innovation. Traditional methods for gathering customer requirements can now be augmented with insights from mining social data and, increasingly, the product itself can be used to test large-scale reactions by observing actual use. By drawing on these different insights — which may be resident in different groups within the organization — product teams can make decisions about product direction from the usage patterns in tandem with classic feedback loops.
“A lot of these initiatives will crosscut functions and companies and will create new roles for people to drive analytics company-wide and to look at the whole stack — from technology, the data that’s produced and how that’s used in the business,” said Schireson.
Conclusion
Ensuring that big data creates value requires more than adopting new technology. It is also about having the right capabilities in the organization, having the right leadership and building a data-driven culture. The challenge for leaders is to build a big data culture and encourage employees at all levels of the organization to change their habits; for most organizations, this will be a long-term journey, requiring sophisticated change management skills.
To succeed, executives must develop and execute an aggressive plan that addresses the complicated organizational challenges to becoming a big data-enabled business. Having helped countless organizations navigate changing leadership needs, we understand that, in times of flux, organizations must be creative in finding skilled executives who meet the needs of the business. While difficult, we believe the most successful organizations will be the ones that tackle the big data problem holistically. In the words of a former technology executive from The Walt Disney Company, Nick Choat, “The real big data successes, I believe, will come from those executives who embrace data, not as a static asset such as a report or dashboard, but rather as an enabler of an exploratory journey into the information unknown.”