How to overcome 7 challenges to data science success

Data science offers great potential for any organization that wants to improve and/or automate decision-making. However, most data projects fail to produce much or any business value, leading to decisions based on simple heuristics, basic projections, or even intuition. Similarly, turning data science proof of concepts into enterprise-wide solutions requires a lot of effort.

But it does not have to be this way. Achieving success with data science starts by recognizing that the most critical elements go beyond data and technology. Rather, success depends on people, culture, and processes.

Bearing this in mind, here are seven challenges that can prevent successful data science endeavors and ways to overcome them.

1. Confusion about data science and how it can help.
Among the AI hype and jargon, it is easy to fall into the trap of thinking data science is the answer to everything. There are situations, however, where it is not the answer to solving a business problem. The reasons can include technical limitations, the impact on customer experience and ethical concerns. Therefore, to ensure expectations are reasonable and achievable, it is critical to educate management and team members about data science, how it works, and what it can or cannot do.

2. Lack of clarity on where, when and how to get started.
Demystifying data science provides necessary clarity to get started appropriately on the data journey. Not knowing where, when and how to begin can lead to ad hoc attempts to ‘do some data science.’ Such efforts often result in poorly designed and executed proof of concepts too far removed from key business challenges to provide value or visibility.

To be successful, data science teams must engage with management and the business team to understand and stay up to date on strategic and operational needs. Keeping an eye on the organization’s overall business needs will allow data scientists to scope out potential initiatives, prioritize them, and ensure time and focus are dedicated to activities that will move the needle. Data scientists also can develop forward-looking roadmaps that specify how initiatives will support an organization’s strategy in the long term.

For example, to improve data science capabilities of a client product, the team at RGA began reviewing the ‘as is’ state to outline the ‘to be’ state around a dozen use cases. The projects were prioritized based on impact and feasibility. Our team then identified people with the right experience and skill set, determined the IT investment, and anticipated necessary change management. Finally, we organized everything into a multiyear data science roadmap.

3. Inappropriate data team structure and management.
Despite the growing debate on how to structure and manage data teams, it seems that few organizations have created the right environment for their data scientists to succeed and develop. Often, data science teams sit in silos, such as research and development or center of excellence departments. Data science teams can be buried deep within the IT structures, far removed from business teams.

It is also common to find data science teams to be doctoral-level scientists without data and software engineering support managed by a lead data scientist who prefers to be an individual contributor, lacks people skills, or both. Instead, team leadership and structure should adopt a client-first mindset and operating model. At RGA, we have dedicated data science leaders who work closely with business teams and are accountable for delivering value.

4. Misalignment between data, business and technology teams.
The inappropriate structure and management of data science teams can often lead to communication problems that interfere with success. Data scientists rely on precise technical language and often do not speak the same language as business and technology teams. A lack of communication could lead to dysfunctional relationships that erode trust.

Therefore, data scientists should make an effort to speak everyday language with non-technical colleagues so they understand how data science works. Bridging the communication gap seems straightforward, but it is not always easy. Success also depends on data scientists continuing to educate business stakeholders and narrow knowledge gaps.

5. Resistance to change by management and end users.
Unfortunately, many data science initiatives focus on the science and technology first, fixating on model performance rather than being concerned with how it will be used in real life and by whom. Launching a data science solution, such as the next-best-action recommendation for advisors and expecting immediate adoption is a sure path to failure. Leaders should envision the bigger picture beyond the technical data science solution, anticipate issues and focus on embracing new ways of working. Ensuring proper adoption requires a user-centered design, strong executive sponsorship, and heavy doses of change management.

6. Unclear results and ROI from existing use cases.
There is nothing more demoralizing and detrimental to the success of a data science initiative than not knowing how well it performed. Therefore, it is critical to set clear objectives and measure value.

Although it is sometimes difficult to accurately estimate a data science project’s effect on the top or bottom line, unclear results and return on investment will make it difficult, if not impossible, to gain trust and funding to continue the data science journey. At RGA, no data science project moves forward without a business case demonstrating its potential value and ways to appropriately measure it.

7. Transforming use cases into solutions is complex.
Turning hand-cranked proof-of-concepts into enterprise-wide, always-on data science solutions is challenging for any company. Adopting agile development practices is necessary for effectively building data science solutions. Moreover, productionizing and rolling out these solutions not only requires the right software engineering support, but also proper business implementation and effective change management. None of this is possible, however, without human beings.

Conclusion
Although people see data science success through the lens of data and technology, it is the people, culture and processes that make the greatest difference between effective and ineffective data science endeavors.