Managing data science initiatives as exploratory projects : A new approach to program management

Data is becoming increasingly ubiquitous in organisational life, with many business investments using data to inform the development of their business case, to deliver core elements of scope, and to track the realisation of benefits. Data science initiatives (DSIs) are defined as related projects or programs that involve the application of data science techniques and methods to address complex problems, generate insights or create new products or services. DSIs typically involve the collection, analysis and interpretation of large and complex datasets and require expertise in data science, statistics, machine learning and programming. While DSIs have emerged as a popular mechanism for extracting value from data, their track record has drawn substantial criticism from sponsors. The success rate of delivering DSIs is perceived to be low, with Gartner estimating that 85% of projects fail (Asay, 2017).The complexity of integrating various data sources, technologies and expertise; interdisciplinary nature requiring collaboration across multiple teams; strategic impact driving organisational transformation; long-term focus involving ongoing iterations and adaptations; and ongoing risk management of technical, regulatory and ethical considerations necessitated taking a program-based approach instead of project-based for their delivery. This highlights the need for effective program management frameworks to guide the successful delivery of DSIs and to realise the value they can potentially bring to an organisation.Program managers require a methodology and framework to guide them in the successful delivery of programs. A review of literature shows a lack of frameworks that program managers can adopt to manage DSIs and navigate the exploratory and innovative nature of such investments. The existing literature is inadequate in addressing the end-to-end program life cycle of DSIs, including the construction and funding of business cases, setting up appropriate governance, effectively delivering DSIs and then realising the value of DSIs through benefits tracking and investment decision-making. This highlights the need for research that addresses these gaps and provides program managers with the tools and guidance they need to effectively manage and deliver DSIs.This research posits that traditional practices used by program managers for conceptualising and managing ICT-enabled programs are not adequate for DSIs, which have unique characteristics. To build this argument, the research employed case studies of seven DSIs spanning six years at Transport for NSW (Transport), the statutory body responsible for transportation in the state of New South Wales in Australia. By analysing the seven case studies, the research aimed to understand the challenges and best practices in managing and delivering DSIs and to provide a framework for program managers to effectively navigate the unique characteristics of these initiatives. This research provides a valuable contribution to the field by addressing the gap in understanding how to effectively manage and deliver DSIs and by providing a practical framework for program managers to apply in their work.The research delved into the difference between exploration and exploitation and how managing DSIs as "exploratory projects” could improve the success rate of implementations. It offers an evidence-based delivery framework using agile methods, which allows for flexibility and adaptability in the face of uncertainty and changing requirements. To address the governance of DSIs, the research proposes a minimum viable governance framework using lean portfolio management and product governance, moving away from traditional portfolio, program and project governance methods which may not be as effective in managing DSIs