Dec 2024 | Data and Innovation | Cloud, Data Science

As the data science talent gap grows, it’s important to have the right data and analytics platform to create job satisfaction and long-term retention.

An article by Paul Heaton

Head of Product and Proposition Marketing for Experian EMEA & APAC

Enabling data science talent with a unified platform

The financial services industry is increasingly reliant on data science talent to drive innovation, improve decision-making, and maintain a competitive edge. As the demand for data science talent grows, so does the challenge of retaining these highly skilled professionals.

According to Experian’s most recent survey of over 1,300 Financial Services and Telco decision-makers, 59% agree that access to specialist skilled labour will have a considerable impact on their business over the next one to two years. This highlights the critical role of skills retention for long-term growth.

To secure data science talent, financial institutions must enable them by providing an environment that empowers them to focus on their main strengths – removing obstacles that can hinder productivity and job satisfaction.

In this article, I’ll discuss how a centralised cloud-based analytics platform can lead to more engaged and motivated data scientists. And how this can also drive faster time to market for models and scores.

What are the key frustrations and challenges for data scientists?

Understanding the main pain points that affect data scientists is a useful starting point. These issues are some of the most serious concerns that impact the data science work environment. Without the right technology and infrastructure, their creativity and job satisfaction are reduced, which can lead to frustration, burnout and high turnover.

  • Data quality issues – poor data in, poor insight out. Missing, inconsistent or incorrect data can consume a considerable amount of time in cleaning and processing.
  • Poor access – delays and difficulty obtaining data due to siloed systems or restrictions.
  • Lack of integration – systems that are not integrated lead to fragmented insights
  • Inefficient tools – outdated tools that slow down productivity and lack of access to the latest best-in-class functionality
  • Lack of automation – frustration with spending time working on manual tasks (e.g., model deployment) that could be easily automated
  • Reliance on others – difficulty collaborating with other teams (e.g. IT) due to different priorities or workflows

How serious is the data science talent gap?

According to the World Economic Forum, the demand for AI/ML skills is estimated to grow by 40% over the next three years. Experian’s research shows that close to half (45%) of the data and analytics leaders we surveyed state that finding and retaining data science talent is a major pain point when it comes to developing and updating models. A similar proportion (48%) lack the in-house expertise to develop and manage the latest analytical models.

As businesses look for more agility – by optimising ModelOps and fast-tracking model production – having the skills to execute these projects has become a competitive differentiator. Organisations that struggle to retain the critical skills needed to deliver data-driven decisions are at a disadvantage compared to those that retain talent by empowering them with an end-to-end model platform.

Five ways to improve data science satisfaction and productivity

How can business leaders attract and retain data science talent? Although every organisation has a unique set of circumstances, the following five points can help bridge the skills gap and ensure future profitability.

1. Easy access to data

Advanced analytical models often incorporate datasets from a variety of alternative sources. The ability to share this data securely and at scale is a vital factor that impacts the performance and satisfaction of data science teams. Our research highlights that the biggest analytics challenge limiting the success of achieving wider business objectives is the inability to effectively connect different data assets.

Centralised cloud-based platforms streamline workflows by providing easy access to data and tools. Data sources are available via the platform and the integration and mapping of the data can be achieved with simple API connections. This reduces the time spent on data preparation and management, allowing data scientists to focus on high-value tasks such as analysis and model development. More than half (55%) of the data and analytics leaders we surveyed are prioritising the shift from siloed datasets into a single data lake/platform.

2. Data science accelerator programs

Progressive businesses are developing emerging data science talent in-house. This has multiple benefits, as it helps cover the talent gap while also developing young professionals with on-the-job skills.

In the short interview below, you can hear how Suveer Ramdhani, CEO of BluNova, has successfully enabled their data science team to innovate faster with a cloud-first approach. They have also tackled the challenge of securing data science talent head-on by integrating recent graduates into their established analytics and engineering teams.

3. Greater ownership

For data scientists, one of the most frustrating parts of the model deployment process is the handover to IT teams to recode the model ready for deployment into the decisioning software. With an effective ModelOps process, data scientists can easily register custom features and models and migrate to production instantly.

For example, Experian’s innovative containerisation engine makes it easy to instantly deploy models written in a range of coding languages to production environments, which eliminates elements such as recoding and speeds up testing. This empowers data scientists to own more of the process, reducing IT dependency.

4. Elastic data storage scalability

Cloud-based platforms offer scalability, enabling data scientists to handle large datasets and complex computations without worrying about infrastructure limitations. This flexibility allows them to experiment with new techniques and approaches, keeping their work interesting and engaging.

Nearly three-quarters (71%) of the technology leaders in our survey agree that the benefits of cloud, such as the ability to scale easily, along with cost savings and security, outweigh the risks. As businesses look to include new alternative datasets into their credit decisioning models, having the capacity to expand storage and compute power is essential.

5. Sandbox environments for experimentation

Data scientists thrive when they can unleash their creativity and experiment with innovative solutions without any risk to the integrity of their main data repository. To facilitate this, businesses need to provide sandbox environments that allow their data analysts to safely explore new techniques and software.

The number one priority for the data and analytics leaders in our survey (55%) was investment in cloud-based analytics tools – such as sandbox environments. The key from a platform perspective, is that the sandbox environment is seamlessly connected to other applications, which is often not the case, slowing down productivity.

Making the most of the Generative AI opportunity

Generative AI (GenAI) is changing how data scientists develop and monitor models. This technology represents an exciting opportunity to enhance productivity by automating manually intensive processes. However, as with any new technology, adoption and integration is challenging. More than half (54%) of the technology leaders in our survey state that they lack the technical talent and skills to take advantage of GenAI.

When we asked our respondents about the top GenAI use cases, nearly two-thirds (62%) said analysing alternative data sources to enrich predictive models. GenAI can significantly augment the tedious process of sifting through unstructured data to identify features used in model development.

Most business leaders we talk to are keen to enable their data teams with the latest GenAI capability, to augment performance. At Experian, we’ve already embedded GenAI into our platform to provide support to data scientists, from using natural language conversations to better understand the data, to coding and model development insights, and to simplify compliance effort with automation of documentation.

Conclusion: the key to retaining data science talent is to ‘enable’ and ‘empower’

Data scientists are generally creative problem solvers who thrive in situations that challenge their abilities, while also delivering tangible business value. To attract and retain these professionals, it’s essential to provide them with everything they need to be at the cutting edge of their field.

Cloud-based analytics infrastructure is the critical foundation that supports data science innovation and excellence. The versatility and security that cloud provides empowers analytics innovation, and ultimately more accurate data-driven decisions.

For more information about our latest research, conducted in partnership with Forrester Consulting, you can simply fill out the form below, and we will send you a complimentary copy of our latest report – Strategies for growth: Perspectives from the boardroom.