Next-Generation Project Management for AI: Taking PMP to the Next Level

Artificial intelligence and advanced data projects are in very high demand. There is a consistent and significant need for project managers as well as data scientists, machine learning engineers, and others who can successfully bring AI projects to completion. With over 70% of AI projects failing due for a wide range of reasons, organizations that are serious about putting AI into practice realize that implementing emerging technologies such as machine learning and artificial intelligence requires more than just great technology and great people; it requires IT project managers that understand how to best run and manage these projects for successful outcomes.

---

What makes AI project management challenging is that you can’t apply traditional application development or IT project management approaches to what are fundamentally data-driven projects.  Traditional application-centric project management approaches fall short by missing the core fundamental aspect of AI projects, which is that they are data-driven. The lifecycle of data, with all its incompleteness, inaccuracy, and redundancies requires taking an advanced approach that treats the needs for managing data separate from those of developing the systems that will leverage those advanced applications of data. 

Forward-thinking project managers are now applying advanced data-centric, agile project management methodologies that are more finely adapted to the needs of AI. Those individuals and organizations adopting such approaches are the ones seeing the most success with AI and advanced data applications.

Understanding the Seven Patterns of AI

To better understand AI project management skills, it’s important to first understand how AI is being applied. Most often, there is confusion about the nature of AI projects. When two different people are talking about AI they most likely aren’t talking about the same application. From chatbots to predictive maintenance, autonomous vehicles to  facial recognition, the use cases for AI are many.  While all those examples are considered AI, the nature of those projects, complexity of the application, and need for data and iteration are not the same. Therefore it’s important to focus in greater detail on the applications of AI to further get a sense of your project goals and what it is you’re trying to accomplish.

AI and data research firm Cognilytica devised the Seven Patterns of AI as a way to accelerate AI and ML projects, by giving project managers a tool to better understand the specific needs for each pattern to best manage the project. Rather than focusing on the abstract idea of AI, the seven patterns of AI focus on specific use cases for intelligent and cognitive applications of data. These seven patterns are: 

1. Autonomous Systems – Systems that are able to accomplish a task, achieve a goal, or interact with its surroundings with minimal to no human involvement. This is applied both to physical, hardware autonomous systems as well as software or virtual autonomous systems (software “bots”). The primary objective of the autonomous systems pattern is to minimize human labor.
2. Predictive Analytics & Decision Support – Using machine learning and other cognitive approaches to understand how learned patterns can help predict future outcomes or help humans make decisions about future outcomes using insight learned from behavior, interactions, and data. The objective of this pattern is helping humans make better decisions.
3. Conversational / Human Interaction – Machines interacting with humans through natural conversation and interaction including voice, text, images, and written forms. The objective is to facilitate communication interaction between machines and humans, as well as between humans and other humans. 
4. Pattern & Anomaly Detection – Using machine learning and other cognitive approaches to identify patterns in the data and learn higher order connections between information that can provide insight into whether a given piece of data fits an existing pattern or is an outlier and doesn’t fit. The primary objective of this pattern is to find which one of the things is like the other and which is not.
5. Recognition  – Using machine learning and other cognitive approaches to identify and determine objects or other desired things to be identified within some form of unstructured content. This content could be images, video, audio, text, or other primarily unstructured data that needs to have some aspect within it identified, recognized, segmented, or otherwise separated out into something that can be labeled and tagged. The primary objective of this pattern is to have machines identify and understand otherwise unstructured data. 
6. Goal-Driven Systems – Using machine learning and other cognitive approaches to find a solution through trial and error. The primary objective of this pattern is to find an optimal solution in use cases such as scenario simulation, game playing, resource optimization, iterative problem solving, bidding and real time auctions.
7. Hyperpersonalization  – Developing a unique profile of each individual, and having that profile learn and adapt over time for a wide variety of purposes, including displaying relevant content, recommending relevant products, providing personalized recommendations and guidance, personalized healthcare, finance, and other one-to-one insight, information, advice, and feedback. The primary objective of this pattern is using AI to treat each individual as an individual, not as a member of some grouping or bucketing into a broad category or classification. 

Each pattern in the seven patterns of AI represents projects that share similar objectives, technology basis, needs for data, and other aspects that once acknowledged will help to fill in the missing blanks as to how any project in that pattern should run. Identifying which pattern or patterns of AI your project is will help the project manager and project teams during all stages of AI development.

AI Projects are data projects

Successful AI project managers know that understanding how to manage AI projects requires understanding how to run data projects. Data is the heart of AI. Without data, machine learning systems would not be able to “learn”. While there is some application development involved in making AI projects work, this is often just a very small portion of the project, and not even the most important part of the project. More important aspects are data understanding, data preparation, model development, model evaluation, and aspects of iterating on data sets to find the optimal implementation.

Unfortunately, far too often, organizations are jumping into AI projects without first addressing an understanding of their data. And, some project managers are still running AI projects like they run application development projects. So, if you are supposed to run AI projects more like data projects, what are the steps involved to making this work?

An Iterative, Agile, Data-Centric AI Project Management Methodology

Experienced AI and advanced data project managers understand the power that following a proven methodology can provide for project success. Over two decades ago, a consortium of organizations developed the Cross-Industry Standard Process for Data Mining (CRISP-DM). However, no further development was done on that methodology. Since the release of that first approach to data-centric project management, others have developed aspects of agile methodology and AI-specific data science approaches. 

Combining CRISP-DM with Agile and adding AI-specific aspects through hundreds of real-world implementations comes a methodology optimized for the delivery of in-production, high value, successful AI projects. The Cognitive Project Management for AI Methodology (CPMAI) emerged as an approach to run AI and big data projects leveraging hard-learned best practices expertise learned from running real-world AI projects. 

CPMAI focuses on an AI project life cycle that involves an iterative approach to AI development and aims to provide a  step-by-step repeatable, documented, and agile methodology .It optimizes for AI and ML project success despite often difficult challenges in working with highly variable, large volumes of data. In the CPMAI methodology, the primary six phases organize in a logical flow the activities that aim to move each iteration of an AI project from the defined business needs to AI models that can meet those needs. An overview of the six CPMAI Phases and their objectives are as follows:

There are six primary CPMAI phases, all of which are iterative and data-centric:

CPMAI Phase I: Business Understanding – “Mapping the business problem to the AI solution.” 

The first step in any AI project is gathering an understanding of the business or organizational requirements. This includes not only functional requirements, but also requirements of continuously iterating data, performance requirements, and requirements for ethical and responsible AI. In this step, AI project teams focus on understanding the project objectives and requirements from a business perspective, then convert this knowledge into an AI and cognitive project problem definition and a preliminary plan is designed to achieve the objectives of applying the right pattern(s) of AI.

CPMAI Phase II: Data Understanding – “Getting a hold of the right data to address the problem.”

The second phase in an AI project following the CPMAI methodology is data understanding. The most important part here is understanding what data is required to address the business problem, whether or not that data is available, and what format it is in. Since data is the heart of AI, AI project teams need to make sure to have a firm understanding of their data before getting too far along in the project.

CPMAI Phase III: Data Preparation – “Getting the data ready for use in a data-centric AI Project.”

The third phase of CPMAI in an AI project is Data Preparation. Once you have figured out what data you have, now you need to make sure it’s usable for your project. Included in this step are data cleansing, data aggregation, data augmentation, data labeling, data normalization, data transformation and any other activities for data of structured, unstructured, and semi-structured nature. 

CPMAI Phase IV: Model Development – “Developing a model that addresses the business problem.”

The fourth phase of your AI project following the CPMAI methodology is creation and development of machine learning models. This includes model technique selection and application, model training, model hyperparameter setting and adjustment, model validation, ensemble model development and testing, algorithm selection, and model optimization. By the time we are ready to build your very first model you’ve already determined the business needs, the data requirements, and gotten the data in the right format and quality. Models that don’t adequately meet business needs given data capabilities will need to iterate, possibly back to prior CPMAI phases.

CPMAI Phase V: Model Evaluation – “Determining whether the AI solution meets the real-world and business needs.”

The fifth phase in the CPMAI methodology requires evaluating and testing the model, evaluating model performance measurement and improvement, and determining needs for ongoing model iteration. AI teams verify if the model meets requirements for accuracy, precision, and other metrics, evaluating concerns on overfit and underfit of models, evaluating the models against business Key Performance Indicators (KPIs) as well as determine means for model monitoring, iteration and versioning.

CPMAI Phase VI: Model Operationalization – “Putting the AI solution to use in the real-world, and iterating to continue its delivery of value:”

The final phase of an iteration of CPMAI is putting the developed model into operation. In this phase, AI project teams make sure to address model versioning and iteration, model deployment, model monitoring, model staging in development and production environments, and other aspects of getting the model in a position to provide value to meet the stated purpose.

Each of the above CPMAI phases are iterative with each other and allow for moving between phases during AI project lifecycle development depending on the need and challenges met in the real world. 

Enhancing Project Management Skills with CPMAI

In addition to providing methodology guidance, the Cognitive Project Management for AI (CPMAI) provides a workbook and templates that are useful for all AI and advanced analytics projects. Thousands of individuals have been CPMAI certified since 2017, and increasingly employers are adding CPMAI certification to their list of skills as they seek to fill out AI project management needs across their rapidly growing AI projects. CPMAI’s vendor-neutral, industry-agnostic approach lends itself well as a growing, robust, iterative step-by-step approach that is powering individuals and organizations on their path to AI success.