Artificial Intelligence

Competency Framework
A success pipeline from
college to university
and beyond
SEPTEMBER 2021
Table of Contents
Introduction 1
Message from the Team 1
Team Acknowledgments 2
Acknowledgments to Contributors 2
Acknowledgments 4
AI Competency Framework 5
Guiding Principles to Developing the Framework 5
Audience 6
Competency Domains at a Glance 7
How to Read the Competency Framework 7
Technical Domain 9
Business Domain 22
Human Domain 28
Using the Competency Framework 36
Narratives 36
Program Developer 36
Educator 38
Dawson College Use case 39
Implementing the AI Competency Framework for your Context 41
Conclusion 43
Glossary 44
References 50
Introduction
Message from the Team

This document is the principle outcome of an ambitious collaborative project between Concordia
University and Dawson College to co-construct an artificial intelligence (Al) competency framework
that supports the creation of success pipelines for learners from college to university to lifelong
learning. The aim is to provide a flexible tool for educators, program developers, recognition of acquired
competencies (RAC) coordinators and other stakeholders engaged with developing curriculum and
training programs that address ever-evolving Al talent needs. Growing evidence for an accelerating
rate of digital transformation further amplified by the worldwide coronavirus pandemic, motivated us
to look toward curriculum frameworks that are general in focus. For this reason we have clustered AI
competencies that may be used in varied higher education contexts along the pipeline for the pressing
needs of today while being adaptable and extensible for the future.

The AI competency framework is a result of extensive consultation with subject matter experts,
instructional designers, and program development professionals. It serves as a curriculum
development resource that can balance the technical, business, and human domains with ethics and
civics in educational programs and professional training. Since the competencies are modular, they can
be combined to fit specific objectives and program goals.

In the face of numerous challenges and in persevering through the pandemic conditions, our team
proudly celebrates the achievement of this work. It points to our shared responsibility in higher
education to accompany learners through the acquisition of skills and knowledge along paths toward
a lifetime of learning for the good of all. This also marks the beginning of a new phase of collaboration
with opportunities to partner, to innovate and to tackle education challenges in the age of AI. We hope
that this AI competency framework user guide will be a valuable addition to your AI education toolkit.

Sincerely,

Sherry Blok
Joel Trudeau
Robert Cassidy

AI Competency Framework 1
The Team
The team is composed of members of the Dawson College and Concordia University communities who
bring unique strengths and perspectives to the project.

Project Leads

Sherry Blok, Director of Programs, Concordia Continuing Education, Concordia University

Joel Trudeau, DawsonAI Artificial Intelligence Project Lead, Faculty, Dawson College

Robert Cassidy, Director, Centre for Teaching and Learning, Concordia University

Team Members

Alicia Apostolakos, Instructional Designer, Concordia Continuing Education, Concordia University

Ana Maria Isac, Instructional Designer, Concordia Continuing Education, Concordia University

Aniklet Zefi, Instructional Design Intern, Concordia Continuing Education, Concordia University

Laura Ragonese, Project Coordinator, Concordia Continuing Education, Concordia University

Myriam Dimanche, AI Project Coordinator, Dawson College

Salvatore Costanzo, Academic Coordinator, Concordia Continuing Education, Concordia University

Simona Ionescu, Business Intelligence and Information Analyst, Concordia Continuing Education,
Concordia University

Yingying Hu, Project Manager & Strategist, Concordia Continuing Education, Concordia University

Acknowledgement to Contributors
Advisory Committee

Benoit Desgroseillers, Finance Montréal

Brandiff Caron, Concordia University

Iulian Serban, Korbit Technologies

Michel Charest, Ville de Montréal

Mike Deutsch, Kids Code Jeunesse

Philip Mitsopoulos, Stradigi AI

Tamara Vandersluis, R2i.ca

AI Competency Framework 2
Competency Framework Developers

Abhishek Gupta, Founder and Principal Researcher, Montreal AI Ethics Institute, Machine Learning
Engineer and CSE Responsible AI Board Member, Microsoft

Adrian Gonzales, Concordia University

Jonathan Sumner, Dawson College

Michel Charest, Ville de Montréal

Tristan Glatard, Concordia University

Reviewers

Ary Bressane, Concordia University

Carol Hawthorne, Concordia University

Peter Croubalian, Plus5xp

Phil Mitsopoulos, Stradigi AI

Robert Stephens, Dawson College

Tanya Chickekian, Sherbrooke University

Winston Wang, Autodesk

Additional Contributors

Brandiff Caron, Concordia University

Christopher Cooke, Concordia Continuing Education, Concordia University

Fabien Cornu, Concordia University

Lindsay Vargas, Dawson College

AI Competency Framework 3
Acknowledgements

We express our gratitude to Pôle montréalais d’enseignement supérieur en intelligence artificielle (PIA)
for providing a grant to support the development of this artificial intelligence competency framework.
Our project embraces PIA’s mission to encourage higher education institutions within Montreal to
respond collectively in addressing training and development gaps that exist in the artificial
intelligence field.

The Concordia University and Dawson College project is one of 14 collaborative projects from partner
universities and colleges to develop training and competency framework projects.

A special word of thanks to Benoit Pagé for his support to our team during the project.

PIA was created to ensure that college and university

programs remain aligned with industry needs and to shed light
on the social and ethical considerations related to the rise of AI.
PIA supports post-secondary institutions to effectively create and
update academic programs in order to reflect current and future
AI needs.

We are also grateful for the support received from our institutions during the work on this project.

Concordia Continuing Education (CCE) offers distinctive

opportunities in the market to better answer the personal,
professional and organizational growth needs of our society.
One day at a time, we invite people from different backgrounds
and stages of life to take part in trend setting trainings that
generate concrete results.

Dawson College is the largest college in Quebec’s CÉGEP

network, home to 10,000 students in 50+ fields of study. We
provide a welcoming and stimulating environment in which to
learn and work, and where students are prepared to assume their
role as productive and responsible citizens of the world.

AI Competency Framework 4
AI Competency Framework
Guiding Principles to Developing the Framework

Community built: The competency framework was created in collaboration with experts
from the Montreal artificial intelligence (AI) and higher education ecosystems.
The framework was developed by industry professionals with field and academic
experience in AI guided by instructional designers.

Rigorous: The process for developing the competency framework involved several
iterations in order to ensure that each domain (technical, business, and human) presented
information that was clear and accurate.The first level of review was conducted by
instructional designers to ensure that the competencies were clear to a non-technical
audience and action-oriented. In the second level of review, each domain was sent out to
external reviewers (such as AI industry experts and educational experts) to ensure that the
content was accurate and well-defined within the Montreal context.

Human-Centered: Rather than focus on only the technical competencies required to work
in AI, this competency framework also includes the human competencies necessary to
excel in such a field.

Integrated: Ethical considerations and practices are critical to the field of AI.
Instead of treating ethics as a separate domain (similar to technical, business, and
human), this competency framework takes an integrated approach. Specifically, ethical
competencies are integrated into the technical, business and human domains in order to
demonstrate how ethical considerations are foundational to the AI field as a whole.

Accessible: Considering that the main audience for this framework do not typically have
technical knowledge of artificial intelligence, careful design decisions were made in order
to render the competency framework more accessible. These design decisions included
creating focus areas and themes to assist non-technical readers navigate the competency
framework, writing competencies that are action-oriented, and developing a glossary to
define technical terms.

AI Competency Framework 5
Audience
The intended audience for this document includes:

Primary

1. Educators

y Higher education faculty teaching in technical fields related to

artificial intelligence

y Higher education faculty looking at integrating AI competencies

in the curriculum

2. Program developers

y Curriculum developers

y Instructional designers

y Course developers

Secondary

1. Program administrators of artificial intelligence courses and programs

2. Student success centers looking at developing complimentary trainings for technical or

non- technical students

3. Training managers and human resource managers responsible for developing employees
working in artificial intelligence

4. Prior learning coordinators

y Recognition of acquired competencies (RAC) coordinators

y Validation of competencies (VoC) coordinators

AI Competency Framework 6
Competency Domains at a Glance

The goal of this AI competency framework is to outline the core competencies (knowledge, skills,
and abilities) required by AI practitioners in the technical, business, and human domains situated
in the Montreal AI context. With ethics being integral to the AI field and the work of AI practitioners,
ethical competencies have been integrated into the technical, business and human domains.

This competency framework is a tool for various educational roles including educators, program
developers, and prior learning coordinators involved in the program development process from
inception to implementation. This means that the competency framework is not meant to define
the competencies required for a specific role in AI (such as data scientist). Rather, it determines
the common set of competencies required by AI practitioners.

Technical Business Human

y Data y AI Initiative and Project Planning � Innovation

y Mathematics and Statistics y AI Initiative and Project Scaling � Teamwork

y Programming AI Technologies Professionalism

Theme

y �

y Machine Learning

y Deep Learning

y Infrastructure

y Libraries and Frameworks

How to Read the Competency Framework

In order to best serve our non-technical audience, each domain (technical, business, and human) of the
AI competency framework has been divided into six sections. The division allows for our non-technical
audience to familiarize themselves with the specific domain at a high-level first before exploring the
competencies at a more granular level.

AI Competency Framework 7
 Understanding Data 2
1 1.1 Employ different types of data and their representations 3

1.1.1 Contrast common data types 4

Data

1.1.1.1 Categorize data by type (such as categorical, ordinal, numerical) 5

1.1.1.2 Encode data by type (such as text, image, signals) 5

1.1.1.3 Verify that the categorization of the data respects those that are representative
of the communities (such as gender labels that don’t mischaracterize LGBTQ2S) 6

Each domain includes these six sections:

1 Focus Area: high-level terms used to describe the category of competencies presented.
For example, Data is a focus area for the technical domain

2 Focus Area Theme: each specific focus area is described using nouns referred to as themes
to further compartmentalize the competencies into their associated subject areas. For example,
Understanding Data is a theme used to describe the Data focus area.

3 Competency: main set of knowledge, skills, and abilities that are required of AI practitioners.
For example, Employ different types of data and their representations is a main competency
of the Understanding Data focus area theme.

4 Sub-Competency: supporting competencies required in order to fulfill the main competency.

For example, Contrast common data types is a supporting competency for the Employ different
types of data and their representations main competency.

5 Sub-Sub-Competency: most granular level of the framework describing specific actions

required to fulfill the main and supporting competencies. For example, Categorize data by type
is a specific action that supports the Contrast common data types sub-competency.

6 Ethical Competencies: ethics related competencies (at all levels) integrated into the
technical, business, and human domains. The AI competency framework represents ethics
competencies with this colour.

AI Competency Framework 8
AI Competency Framework
Technical Domain

AI Competency Framework Technical Domain 9

 Understanding Data
1.1 Employ different types of data and their representations
Data 1.1.1 Contrast common data types
1.1.1.1 Categorize data by type (such as categorical, ordinal, numerical)
1.1.1.2 Encode data by type (such as text, image, signals)
1.1.1.3 Verify that the categorization of the data respects those that are representative
of the communities (such as gender labels that do not mischaracterize LGBTQIA+)
1.1.2 Contrast common data structures
and Statistics
Mathematics

1.1.2.1 Organize data in a tabular structure

1.1.2.2 Manipulate data in a tabular structure
1.1.2.3 Organize data in a relational structure
1.1.2.4 Manipulate data in a relational structure
1.1.2.5 Identify the effects of the data manipulation from a representation perspective
(such as outlier trimming, removing data of people for whom all attributes are not captured)
1.1.3 Select appropriate data types and structures for a given context
Programming

1.1.3.1 Evaluate trade-offs for different data schemes (such as storage requirements, ease of query,
extensibility)
1.1.3.2 Tailor data scheme to project requirements and intended use case
1.1.3.3 Evaluate the environmental impact of different storage formats
(such as compressed formats, flat files)
1.2 Analyze typical uses of data in machine learning (ML) and AI
1.2.1 List common methods by which datasets are generated
1.2.1.1 Describe explicit data collection campaigns
Machine
Learning

1.2.1.2 Describe data aggregation campaigns

1.2.1.3 Evaluate the data to ensure that it meets ethical requirements
(with help from domain experts and social scientists)
1.2.2 Identify common applications of data
1.2.2.1 Interpret data as a tool for argumentation
1.2.2.2 Classify methods by which data are used to build models
1.2.2.3 List applications of ML and AI in different contexts
Learning

1.2.3 Analyze datasets for use in a ML or AI setting

Deep

1.2.3.1 Evaluate usefulness of a given dataset for the purposes of developing a specific ML or AI model
1.2.3.2 Curate an original dataset for the purposes of training and testing an ML or AI model
1.2.3.3 Include purpose limitation to meet ethical requirements (such as infeasibility of using a facial
recognition dataset collected in France for use in South Korea without significant updates)
1.2.3.4 Evaluate assumptions for second-hand and procured datasets that are present in its collection,
processing, and distribution
Infrastructure

1.2.3.4.5 Determine the data lineage of non-first party datasets

Legal and Ethical Implications
2.1 Assess the legal implications of data collection and use
2.1.1 Distinguish between common types of licenses attached to data and software
2.1.1.1 Identify the rights and responsibilities related to common open source licenses
2.1.1.2 Identify the rights and responsibilities related to working with closed licenses
and Frameworks

2.1.2 Apply relevant government regulations (such as PIPEDA)

Libraries

2.1.2.1 Identify regulations in the context of research projects involving data collection and machine
learning or artificial intelligence
2.1.2.2 Identify regulations in the context of commercial projects involving data collection and machine
learning or artificial intelligence

AI Competency Framework Technical Domain 10

 2.1.2.3 Evaluate that the data collection and storage respects relevant privacy regulations
and industry requirements
Data 2.1.3 Adhere to the rights of data producers and responsibilities of data consumers
2.1.3.1 Analyze the role and inherent value of data producers
2.1.3.2 Prioritize the rights of data producers
2.1.3.3 Prioritize the responsibilities of data consumers
2.1.3.4 Document the data producers’ and consumers’ responsibilities that will be adhered
to in an accessible document for internal and external stakeholders
2.2 Assess the ethical implications of data collection and use
and Statistics
Mathematics

2.2.1 Avoid unintended bias in data

2.2.1.1 Describe origins of bias
2.2.1.2 Identify negative effects of training bias in models
2.2.1.3 Apply best practices to minimize unintended training bias in models
2.2.2 Verify consent in data collection and use
2.2.2.1 Discern where explicit user consent is needed in the collection and use of data
Programming

2.2.2.2 Identify the role of research ethics boards

2.2.2.3 Apply best practices for ethical data collection and use
2.2.2.4 Identify internal stakeholders (such as legal) that can assist with determining legality
of use and sufficiency of data consent
Data Handling and Manipulation
3.1 Prepare data for use in an ML or AI project
3.1.1 Assess data quality
3.1.1.1 Compute quality metrics
Machine
Learning

3.1.1.2 Isolate problematic records

3.1.2 Improve data quality
3.1.2.1 Filter data
3.1.2.2 Impute missing data (as appropriate) based on type
3.1.2.3 Apply pre-processing techniques to image, audio, and video data
3.1.2.4 Verify that filtered data does not exclude marginalized communities (such as having missing
attributes because of lack of access to digital tools)
Learning

3.1.2.5 Verify that imputation of data does not assign non-representative values (such as gender labels
Deep

for LGTBQIA+ against how they wish to be represented)

3.1.3 Transform data
3.1.3.1 Coerce data types (such as dates)
3.1.3.2 Apply common transformations (such as groupby, reduce, map, pivot, merge, join)
3.1.3.3 Label data
Infrastructure

3.1.4 Address privacy concerns

3.1.4.1 Identify data, which alone or in aggregate, could potentially identify a person
3.1.4.2 Remove personally identifiable information (PII) unnecessary for the analysis
3.1.4.3 Tokenize remaining PII in a cryptographically secure fashion
3.1.4.4 Differentiate between linked and linkable data (PII alone does not encapsulate all personal data;
linked data = when data records are linked across different data sources; linkable data = when data
records have the potential to be linked across sources to give a richer profile)
and Frameworks

3.2 Manipulate data

Libraries

3.2.1 Isolate datasets from within a larger data structure (querying)

3.2.1.1 Filter data by direct selection (such as slicing using indices) or logical conditions
3.2.1.2 Compile data by higher-order operations

AI Competency Framework Technical Domain 11

 3.2.1.3 Use date/time variables
3.2.2 Calculate new properties
Data 3.2.2.1 Apply column-level operations
3.2.2.2 Apply row-level operations
3.3 Implement data security measures
3.3.1 Apply the principle of least privilege
3.3.1.1 Assign permissions to datasets
3.3.1.2 Enforce permissions to limit data access by user
and Statistics
Mathematics

3.3.1.3 Implement Role-Based Access Control (RBAC) in association with an Identity and Access
Management (IAM) of the organization to control data access
3.3.2 Use common encryption techniques
3.3.2.1 Store sensitive data at rest in encrypted form
3.3.2.2 Implement a key management strategy
3.3.2.3 Follow best practices as documented in NIST Cybersecurity Framework for up-to-date
recommendations
Programming

Data Representation
4.1 Analyze graphical visualizations of data
4.1.1 Interpret data presented graphically
4.1.1.1 Identify trends in scatter plots, histograms, bar and pie charts, and heatmaps
4.1.1.2 Interpret the significance of visual representation of data (such as scatter plots, histograms,
bar and pie charts, and heatmaps) within the context of a research question
4.1.2 Present data in an easily understandable form
4.1.2.1 Generate common visualisations (such as scatter plots, histograms, bar and pie charts,
Machine
Learning

and heatmaps)
4.1.2.2 Generate custom visualizations
4.2 Analyze tabular representations of data
4.2.1 Interpret data presented as a table
4.2.1.1 Explain common statistical measures (such as central tendency, spread, quartiles, extrema)
4.2.1.2 Interpret the significance of common statistical measures within the context of a research
question
4.2.2 Report statistical properties
Learning
Deep

4.2.2.1 Calculate measures of central tendency, spread, quartiles, and extrema

4.2.2.2 Calculate other properties of interest (such as counts, sums, frequencies, probabilities,
covariances)
4.2.2.3 Identify appropriateness of statistical measures to the domain and type of data to properly
handle underlying data distributions and scenarios like outliers
Examples of libraries and frameworks
Infrastructure

Python, NumPy, Pandas, Matplotlib

Probability and Statistics

1.1 Resolve problems using statistical methods and probability concepts
and Frameworks

1.1.1 Process data using descriptive statistics

Libraries

1.1.1.1 Represent frequency distributions in the form of tables and graphs

1.1.1.2 Calculate measures of central tendency, dispersion, and position
1.1.1.3 Interpret statistical distributions
1.1.2 Apply probability concepts in random situations

AI Competency Framework Technical Domain 12

 1.1.2.1 Calculate probability of random events
1.1.2.2 Interpret probability density functions
Data 1.1.2.3 Calculate joint probabilities
1.1.2.4 Calculate conditional probabilities (such as Bayes theorem)
1.1.3 Characterize a population
1.1.3.1 Apply central limit theorem
1.1.3.2 Calculate expectation, variance, and confidence intervals
1.1.3.3 Resolve problems using statistical inference
and Statistics
Mathematics

1.1.4 Quantify relationship between two variables

1.1.4.1 Calculate regression coefficients
1.1.4.2 Interpret significance of regression coefficients
1.1.4.3 Identify limitations and appropriateness for each of the statistical techniques to avoid
misrepresentations of the degree of insights that we can get from the data
Tensors
2.1 Analyze problems involving tensors
Programming

2.1.1 Determine result of common operations

2.1.1.1 Calculate relevant tensor properties (such as transpose, inverse, trace, rank, norms)
2.1.1.2 Calculate various tensor products (such as vector-vector, vector-matrix)
2.1.1.3 Calculate eigenvectors and eigenvalues
2.1.2 Use systems of linear equations
2.1.2.1 Identify linear dependency
2.1.2.2 Solve systems using relevant techniques (such as Gauss, Gauss-Jordan)
2.1.2.3 Explain the numerical conditioning of linear systems
Machine
Learning

Calculus
3.1 Analyze problems using differential calculus
3.1.1 Determine the derivative of a function
3.1.1.1 Calculate the derivatives of a function of a single variable
3.1.1.2 Calculate the partial and total derivatives of a multi-variate function at a point
3.1.2 Apply differential calculus to problems in AI
Learning

3.1.2.1 Use a Taylor series to locally approximate a function

Deep

3.1.2.2 Apply gradient-based optimization techniques to multivariate functions

3.2 Analyze problems using integral calculus
3.2.1 Determine the integral of a function
3.2.1.1 Calculate the indefinite integral of a function of a single variable
3.2.1.2 Calculate the indefinite integral of a multi-variate function
Infrastructure

3.2.2 Apply integral calculus to problems in AI

3.2.2.1 Calculate the definite integral of a multi-variate function over a region
3.2.2.2 Calculate convolutions
Numerical Methods
4.1. Interpolate and fit curves
4.1.1 Interpolate data points with polynomials (such as Lagrange, Newton, Neville)
and Frameworks

4.1.2 Interpolate data points with cubic splines

4.1.3 Fit data points with least-square polynomials (including line)
Libraries

4.2 Solve Ordinary Differential Equations (ODE) numerically

4.2.1 Identify the impact of truncation errors
4.2.2 Solve initial value problems with Euler and Runge-Kutta methods
4.2.3 Understand the determinants of numerical stability

AI Competency Framework Technical Domain 13

 4.2.4 Solve two-point boundary problems with finite differences
4.3 Solve optimization problems numerically
Data 4.3.1 Use Powell’s method to solve minimization problems
4.3.2 Use the downhill simplex method to solve minimization problems
Examples of libraries and frameworks
NumPy, SciPy, Matlab, GNU Octave, R
and Statistics
Mathematics

Core Language Skills

1.1 Write code using proper syntax and structure
1.1.1 Employ various data types and structures
1.1.1.1 Perform operations involving numeric types and strings
1.1.1.2 Perform operations with mixed types involving coersion
1.1.1.3 Manipulate array-like structures (such as instantiate, iterate, update, filter, map, reduce,
slice, copy)
Programming

1.1.1.4 Manipulate object-like structures (such as instantiate, iterate, update, filter, map,
reduce, copy)
1.1.1.5 Differentiate between mutable and immutable types
1.1.1.6 Design new object types using inheritance, mixins, and operator overloading
1.1.2 Work with files
1.1.2.1 Read files from local or remote sources
1.1.2.2 Write files to local storage
1.1.3 Apply control flow logic
Machine
Learning

1.1.3.1 Iterate over data structures

1.1.3.2 Use loops and conditional statements to control execution
1.1.4 Use functions and methods
1.1.4.1 Incorporate logic within functions
1.1.4.2 Contrast between pure and non-pure functions
1.1.4.3 Apply scope in the construction of closures
Learning

1.1.4.4 Compose operations through chaining

Deep

1.1.4.5 Write functions for asynchronous execution

1.1.5 Use common programming patterns
1.1.5.1 Use functional programming techniques to develop applications
1.1.5.2 Use object-oriented programming techniques to develop applications
1.2 Incorporate libraries
Infrastructure

1.2.1 Manage project dependencies

1.2.1.1 Install dependencies (such as by using pip, apt)
1.2.1.2 Document dependencies
1.2.1.3 Resolve dependency conflicts
1.2.1.4 Upgrade dependencies to address identified vulnerabilities or take advantage of improvements
1.2.1.5 Manage deprecations
and Frameworks

1.2.2 Work with third-party software and APIs

Libraries

1.2.2.1 Evaluate quality of opensource projects prior to adding as dependency

1.2.2.2 Interpret source documentation and source code
1.3 Improve code performance
1.3.1 Use benchmarks to improve performance

AI Competency Framework Technical Domain 14

 1.3.1.1 Determine appropriate benchmarks
1.3.1.2 Profile code execution (for memory use, database queries, run time, pdb)
Data 1.3.2 Remediate inefficient operations
1.3.2.1 Diagnose problematic routines
1.3.2.2 Consider alternative implementations
1.3.2.3 Implement code parallelization (where possible)
Software Development
2.1 Use version control
and Statistics
Mathematics

2.1.1 Employ the basic features of git

2.1.1.1 Work with forks and branches
2.1.1.2 Use commits to develop new code incrementally
2.1.1.3 Manage pull requests
2.1.2 Ensure code quality
2.1.2.1 Employ automated tools for linting, formatting, static analysis, style, and security checks
2.1.2.2 Conduct code reviews
Programming

2.1.3 Publish code to a repository

2.1.3.1 Organize releases using a standard nomenclature (such as semver)
2.1.3.2 Distribute over the web
2.1.4 Contribute to an opensource library
2.1.4.1 Work with ticketing systems
2.1.4.2 Conform to contribution requirements (such as documentation, coverage)
2.1.4.3 Use data lineage and version control systems for the AI lifecycle (such as Pachyderm, DVC,
Weights and Biases, MLFlow)
Machine
Learning

2.2 Apply software development process

2.2.1 Develop complex applications incrementally
2.2.1.1 Enumerate code requirements from design goals
2.2.1.2 Identify the minimum viable application
2.2.1.3 Develop with the smallest code footprint possible
2.2.2 Incorporate code requirements in unit tests
Learning

2.2.2.1 Organize tests into a logical structure that mirrors the codebase
Deep

2.2.2.2 Employ assertions to specify expected functionality

2.2.2.3 Break down complex tasks into smaller and more easily testable functions
2.2.2.4 Use test suite to add/refactor code while protecting against regressions
2.2.3 Implement automated testing through continuous integration
2.2.3.1 Configure a continuous integration pipeline within version control
Infrastructure

(such as CircleCI, Jenkins, TravisCI)

2.2.3.2 Automate testing and benchmarking
Databases
3.1 Build a relational database
3.1.1 Define new tables
3.1.2 Manipulate records
3.1.3 Execute complex queries
and Frameworks

3.2 Build a non-relational database

Libraries

3.2.1 Create records

3.2.2 Manipulate records
3.2.2 Execute complex queries

AI Competency Framework Technical Domain 15

 Examples of libraries and frameworks
Python, R, C/C++, Java, Julia
Data

Data Preprocessing
1.1 Prepare features for use in supervised or non-supervised learning tasks
1.1.1 Compute features for different types of data (such as categorical, numerical, time series)
1.1.1.1 Encode categorical data
and Statistics
Mathematics

1.1.1.2 Identify and correct errors in categorical data

1.1.1.3 Normalize/standardize features
1.1.1.4 Reduce dimensionality of high-dimension datasets
1.1.1.5 Compute features in time windows
1.1.1.6 Create a data dictionary (also called codebook) to document the various assumptions and inter-
pretations of features in the dataset
1.1.2 Evaluate features for use in ML models
Programming

1.1.2.1 Compute feature correlation matrices

1.1.2.2 Detect outliers from features
1.1.2.3 Measure feature importance from ML models
1.2 Establish data pipelines
1.2.1 Connect data sources to models
1.2.2 Use data structures native to machine learning libraries
1.2.3 Resample large datasets
Machine
Learning

Supervised Learning
2.1 Manage a supervised learning framework
2.1.1 Divide data into train, test, and validation sets
2.1.1.1 Apply k-fold validation
2.1.1.2 Apply leave-one-out validation
2.1.1.3 Apply validation in a multi-class context
2.1.2 Apply correct performance measures for regression, and binary and multi-class classifications
Learning

2.1.2.1 Identify correct measures

Deep

2.1.2.2 Evaluate model performance

2.1.3 Tune hyperparameters of classification and regression methods
2.1.3.1 Apply grid search
2.1.3.2 Apply optimization methods
2.1.4 Handle class imbalance
Infrastructure

2.1.4.1 Resample the training set to adjust class distributions

2.1.4.2 Simulate entries in the minority classes
2.1.4.3 Adjust class weights in classification methods
2.2 Apply supervised learning to specific tasks
2.2.1 Parametrize and apply classification methods
2.2.1.1 Contrast classification methods
and Frameworks

2.2.1.2 Select classification method for task

Libraries

2.2.1.3 Parameterize classification method

2.2.1.4 Apply ensemble methods
2.2.1.5 Apply semi-supervised learning methods
2.2.1.6 Use machine learning libraries for classification

AI Competency Framework Technical Domain 16

 2.2.1.7 Experiment with common classification tasks
2.2.1.8 Determine the inductive bias in the learning method selected to set
Data 2.2.2 Parametrize and apply regression methods
2.2.2.1 Contrast regression methods
2.2.2.2 Select regression method for task
2.2.2.3 Use machine learning libraries for regression
2.2.2.4 Experiment with common regression tasks
Unsupervised Learning
and Statistics
Mathematics

3.1 Manage an unsupervised learning framework

3.1.1 Apply correct performance measures
3.1.1.1 Identify correct measures
3.1.1.2 Evaluate model performance
3.2 Apply unsupervised learning to specific tasks
3.2.1 Parametrize and apply clustering methods
3.2.1.1 Contrast clustering methods
Programming

3.2.1.2 Select clustering method for task

3.2.1.3 Parameterize clustering method
3.2.1.4 Use machine learning libraries for clustering
3.2.1.5 Experiment with clustering datasets
Examples of libraries and frameworks
Scikit-learn, R
Machine
Learning

AI Ecosystems
1.1 Review technical perspectives on societal implications of AI
1.1.1 Review explainability techniques for AI models (such as SHAP, LIME)
1.1.2 Identify safety issues with AI models
1.1.3 Identify the technical determinants of racial, gender and social biases in AI models
1.1.4 Describe architectures for privacy-preserving AI deployments
Learning
Deep

1.1.4.1 Review homomorphic encryption

1.1.4.2 Review federated learning
1.1.4.3 Review differential privacy
1.1.5 Differentiate between narrow and global AI
1.2 Identify the relationship between AI and computational systems
Infrastructure

1.2.1 Distinguish between technological artifacts that use and do not use AI
1.2.2 Characterize the key technological ingredients for a successful AI project
1.2.3 Contrast AI and traditional IT approaches to various problems
1.3 Discuss the interaction between artificial and human intelligence
1.3.1 Analyze features that make an entity intelligent
1.3.2 Compare artificial and human intelligence in a range of problems
1.3.3 Position the need for human skills in an AI ecosystem
and Frameworks

1.3.4 Discuss the relation between AI models and biology

Libraries

1.4 Integrate AI systems in real-world contexts

1.4.1 Evaluate AI systems for specific applications
1.4.2 Assess the selected measures against heuristics or other established metrics in the domain
in consultation with external stakeholders and domain experts

AI Competency Framework Technical Domain 17

 1.4.3 Identify the role that a human can play (from a Human-in-the-loop (HITL) perspective)
and when to defer to human in the context of the AI system to make a decision
(say when the AI system is uncertain in its decision)
Data
1.4.4 Evaluate the relevance and representativeness of synthetic data to avoid issues of bias
(consult with domain experts)
Artificial Neural Networks
2.1 Use general multi-layer neural networks
2.1.1 Build multi-layer neural networks
2.1.1.1 Apply perceptrons
and Statistics
Mathematics

2.1.1.2 Build multi-layer neural networks

2.1.1.3 Select activation functions
2.1.1.4 Select loss functions
2.1.1.5 Understand optimizers
2.1.2 Apply multi-layer neural networks for supervised learning
2.1.2.1 Tune neural network hyperparameters
Programming

2.1.2.2 Configure dropout and regularization

2.1.2.3 Evaluate multi-layer perceptrons
2.1.2.4 Apply multi-layer perceptrons to regression problems
2.1.2.5 Apply multi-layer perceptrons to classification problems
2.2 Use specific deep learning models
2.2.1 Model data with Convolutional Neural Networks (CNNs)
2.2.1.1 Build Convolutional Neural Networks
2.2.1.2 Build graph convolutional networks
Machine
Learning

2.2.1.3 Train convolutional networks

2.2.1.4 Apply convolutional networks to image data
2.2.1.5 Apply convolutional networks to video data
2.2.2 Model data with Recurrent Neural Networks (RNNs)
2.2.2.1 Build Recurrent Units
2.2.2.2 Build long/short term memory units
2.2.2.3 Build Transformers
Learning
Deep

2.2.2.4 Train RNNs

2.2.2.5 Apply RNNs to textual data
2.2.2.6 Apply RNNs to time series data
2.2.2.7 Apply RNNs to sensors data
2.2.3 Generate data with deep learning models
Infrastructure

2.2.3.1 Build autoencoders

2.2.3.2 Build generative adversarial networks
2.2.3.3 Generate textual data
2.2.3.4 Generate signals (such as image, video, sound)
2.2.4 Learn behaviors with deep reinforcement learning (RL)
2.2.4.1 Apply value-based methods for deep RL
2.2.4.2 Apply policy gradient methods for deep RL
and Frameworks

2.2.4.3 Apply model-based methods for deep RL

Libraries

2.2.4.4 Benchmark deep RL

2.2.4.5 Apply deep RL to various domains (such as robotics, video games)
Examples of libraries and frameworks
Pytorch, Tensorflow, Keras

AI Competency Framework Technical Domain 18

 Data Storage
1.1 Manipulate data stored in files
Data 1.1.1 Manage file systems
1.1.1.1 Contrast between local, parallel and distributed file systems
1.1.1.2 Configure file systems
1.1.1.3 Optimize file systems
1.1.1.4 Work with files manually
1.1.1.5 Work with files programatically
and Statistics
Mathematics

1.1.2 Process data stored in various file systems

1.1.2.1 Aggregate data across different file systems
1.1.2.2 Manage large datasets that cannot fit in memory
2.1 Manipulate data stored in databases
2.1.1 Manage database systems
2.1.1.1 Contrast between relational dbs, key-value stores, document dbs, column dbs, graph dbs,
and RDF stores
Programming

2.1.1.2 Install various kinds of databases

2.1.1.3 Configure various kinds of databases
2.1.1.4 Optimize databases
2.1.1.5 Optimize data queries
2.1.2 Process data stored in various databases
2.1.2.1 Agglomerate data across different databases
2.1.2.2 Manage large datasets that cannot fit in memory
2.1.2.3 Build data models from data stored in databases
Machine
Learning

Parallel Computing
2.1 Use parallel processors to train data models
2.1.1 Contrast multi-threaded single-core CPU, multi-core CPUs, and Graphical Processing Units (GPUs)
2.1.1.1 Identify performance advantages of each
2.1.1.2 Select architecture for a given ML or AI application
2.1.2. Execute parallel programs
Learning

2.1.2.1 Handle logging and exceptions across processes

Deep

2.1.2.2 Manage file and database input/output (I/O) across processes

2.1.2.3 Handle code restarts
2.2 Use distributed clusters to train data models
2.2.1 Manage a cluster
2.2.1.1 Contrast High-Performance Computing and Big Data clusters
Infrastructure

2.2.1.2 Install a High-Performance Computing cluster (such as Slurm, Lustre)

2.2.1.3 Configure a High-Performance Computing cluster (such as Slurm, Lustre)
2.2.1.4 Install a Big Data cluster (such as Spark, Dask, Mesos, Hadoop)
2.2.1.5 Configure a Big Data cluster (such as Spark, Dask, Mesos, Hadoop)
2.2.2 Use a cluster to carry out ML or AI tasks
2.2.2.1 Execute applications on computing clusters
and Frameworks

2.2.2.2 Optimize programs for parallel distributed computing on clusters

Cloud Computing
Libraries

3.1 Use different types of cloud architectures

3.1.1. Contrast Infrastructure as a Service (IaaS) to Platform as a Service (PaaS) and Software
as a Service (SaaS)
3.1.1.1 Identify value proposition for each type of service

AI Competency Framework Technical Domain 19

 3.1.1.2 Select optimal architecture for a given research or commercial context
3.1.2 Build virtual environments for distribution to cloud
Data 3.1.2.1 Compare the different types of virtual environments
3.1.2.2 Build virtual environments for different languages (such as Python)
3.1.2.3 Build software containers (such as Docker, Singularity)
3.1.2.4 Build software containers (such as Docker, Singularity)
3.2 Deploy models at scale efficiently
3.2.1 Use services from the main cloud providers (such as AWS, Azure, Google)
and Statistics
Mathematics

3.2.1.1 Deploy virtual environments

3.2.1.2 Deploy ML data models
3.2.1.3 Operate ML data models
3.2.2 Scale services depending on workload
3.2.2.1 Monitor service demand
3.2.2.2 Adapt resource provisioning to service demand
3.2.2.3 Balance workload among resources
Programming

3.3 Establish data and infrastructure security

3.3.1 Ensure safety at rest
3.3.1.1 Implement secure authentication protocols
3.3.1.2 Define permission policies to control access levels to data and infrastructure
3.3.1.3 Implement permission policies to control access levels to data and infrastructure
3.3.1.4 Use Virtual Private Networks (VPN) to harden infrastructure security
3.3.1.5 Apply accounting practices to monitor system and data access and activity
Machine

3.3.1.6 Apply secure encryption techniques to stored data, at least that which contains PII
Learning

3.3.1.7 Use standards and best practices (such as ISO 27001, NIST CSF, ISO 27017)
3.3.2 Ensure safety in transit
3.3.2.1 Use highest SSL protocol supported (such as TLS 1.3) with up-to-date ciphers for web access
and enforce https
3.3.2.2 Use public key authentication for secure transmission (such as ssh/sftp)
3.3.2.3 Encrypt traffic within VPNs
Examples of libraries and frameworks
Learning
Deep

OpenStack, Docker, Singularity, VirtualBox, Hadoop, Spark, Mesos, Kubernetes, Dask, bash.

Tools
1. Select the appropriate tool
Infrastructure

2. Install the tool

3. Configure the tool
4. Find documentation for the tool
5. Run generic examples with the tool
6. Customize the tool for a specific purpose
7. Suggest improvements to the tool
and Frameworks

8. Modify the tool through open-source processes

Libraries

AI Competency Framework Technical Domain 20

AI Competency Framework
Business Domain

AI Competency Framework Business Domain 21

 Data and AI Project Scoping
1.1 Define potential AI use cases
1.1.1 Evaluate the organizational context
1.1.1.1. Identify departments, industries, and human structure within the organization
1.1.1.2. List roles and responsibilities for the involved stakeholders
AI Project Planning

1.1.1.3. List current and past data on AI projects within the organization
1.1.1.4. Identify executive project sponsors
1.1.1.5. Identify domain experts (preference for those with sociological training/experience
in the domain)
1.1.2 Map data journey
1.1.2.1. Find available data sources
1.1.2.2. Analyze data format, scope, and content
1.1.3 Enable decision makers to select a use case
1.1.3.1. Facilitate discussion with stakeholders to define the expected impact from use cases
1.1.3.2. Define use case prioritization grid based on feasibility and impact
1.1.3.3 Apply existing use case prioritization grid based on feasibility and impact
1.1.3.4. Draft a business case and related return on investment (ROI)
1.1.2.5. Identify externalities on customers and society
1.2. Evaluate data quality and availability
1.2.1. Establish a framework based on evaluation factors
AI Project Scaling

1.2.1.1. Define data quality evaluation factors per use case

1.2.1.2. Define data availability evaluation factors per use case
1.2.1.3. Define fairness metrics per use case (such as equalized odds and demographic parity)
1.2.1.4. Identify personally identifiable information (PII) per use case
1.2.2. Analyze the quality and availability of data sources inventory
1.2.2.1. Recommend high-level data improvement strategies
1.2.2.2. Define priorities based on available sources
1.2.2.3. Communicate current and future use case needs and actionables for each data source
(to improve 1 or more factors) to stakeholders
1.2.2.4. Document trade-offs for privacy considerations in data use per data source
1.3. Consolidate project requirements
1.3.1. Baseline existing project context (human, data, infrastructure, sponsor)
1.3.1.1. Estimate required resources for the specific project
1.3.1.2. Determine available and additional resources required for the project
1.3.1.3. Estimate budget for ethics considerations testing (such as privacy, bias, fairness,
accountability, transparency, and auditing considerations)
AI Technologies

1.3.2. Recommend training plan to bridge the skills gap for the project team and involved stakeholders
1.3.2.1. Define the required level of technical, business, hybrid, and ethics skills
1.3.2.2. Analyze the current level of technical, business, and hybrid skills
1.3.2.3. Analyze current level of ethics training for fairness, privacy, and other ethical concerns
1.4. Create a roadmap for agile AI projects
1.4.1. Estimate project activities and required effort
1.4.1.1. Divide project activities in different workstreams and tasks
1.4.1.2. Calculate required effort (such as number of people and number hours per week)
1.4.1.3. List major risks to the project
1.4.2. Negotiate Definition of Done (DoD) for deliverables with the project team
1.4.2.1. Confirm internal best practices to track project progress

AI Competency Framework Business Domain 22

 1.4.2.2. Define what is considered “done” and “not done” (depending on project)
1.4.3. Consolidate the project roadmap
1.4.3.1. Define project phases and key milestones
1.4.3.2. Negotiate frequency of executive/steering committee level follow-up meetings
and presentations
AI Project Planning

1.4.3.3. Present project planning to diverse audiences (such as business, technical, and executive)
1.4.3.4. Present the envisioned agile methodology (such as its advantages and limitations)
1.4.3.4. Incorporate insights from external stakeholders for ethics considerations in the project roadmap
Data and AI Project Execution
2.1. Manage internal technical and multidisciplinary teams
2.1.1. Lead AI implementations
2.1.1.1. Define roles and responsibilities
2.1.1.2. Facilite discussions on high-level working agreements
2.1.1.3. Monitor overall project progress
2.1.1.4. Monitor individual contributor workload
2.1.1.5. Establish working group on ethics considerations
2.1.2. Overcome existing and new roadblocks
2.1.2.1. Negotiate workarounds
2.1.2.2. Justify need for new resources
2.1.2.3. Establish channels between departments for ethics considerations
AI Project Scaling

2.1.3. Establish project management artifacts and tools

2.1.3.1. Facilitate agile/scrum ceremonies
2.1.3.2. Set up project management and productivity environment (such as by using Jira)
2.1.4. Coach individual contributors
2.1.4.1. Estimate required support based on situational leadership
2.1.4.2. Prepare action plan for customized support per member/role
2.1.4.3. Check in on contributors individually on a regular basis (such as every two weeks)
2.1.4.4. Ensure completion of ethics training prior to start of project work
2.2. Manage external and other business stakeholders
2.2.1. Establish realistic expectations amongst stakeholders
2.2.1.1. Communicate AI project outcomes using language that is appropriate to the target stakeholder
2.2.1.2. Define key AI project metrics and acceptable thresholds
2.2.1.3. Align expectations with the organizations’s strategic goals and business KPIs
2.2.1.4. Establish processes (such as surveys and focus groups) to solicit external stakeholder
feedback
2.2.1.5. Operationalize requirements using tools (such as Qualtrics)
AI Technologies

2.2.2. Implement a project status tracking system

2.2.2.1. Develop dashboards for KPI analysis
2.2.2.2. Maintain project and sprint backlog
2.2.3. Manage crisis, hype, and noise around the AI project
2.2.3.1. Prepare multi-stakeholder (proactive) project communication
2.2.3.2. Identify potential risks related to miscommunication
2.3. Deliver AI solutions
2.3.1. Organize project assets for delivery (such as code and documents)
2.3.1.1.Build code repositories (such as in Gitlab and GitHub) and related best practices
2.3.1.2. Prepare knowledge management platforms (such as Confluence)
2.3.2. Establish DevOps foundation for AI project lifecycle

AI Competency Framework Business Domain 23

 2.3.2.1. Define the continuous delivery (CD)/continuous integration (CI) process with the team
2.3.2.2. Implement code review, merge, and integration governance
2.3.2.3. Implement fairness and privacy checks aligned with solution update cycles
2.3.3. Enable knowledge transfer sessions
2.3.3.1. Organize material for internal (future) knowledge transfer
AI Project Planning

2.3.2.2. Organize material for knowledge transfer with external stakeholders/clients

2.3.3.3. Organize material for capabilities and limitations disclosures to stakeholders / clients

Business Development
1.1. Support technical presales activities
1.1.1. Evaluate organizational context
1.1.1.1. Analyze client activities, structure, and business
1.1.1.2. Select applicable AI examples from industry and/or previous projects
1.1.2. Facilitate client discussions and demonstrations
1.1.2.1. Create discussion material (such as presentation deck) to introduce context and notions
1.1.2.2. Prepare demonstrations based on existing platforms or code (if available)
1.1.3. Provide high-level guidance and recommendations for use case details
1.1.3.1. Select applicable AI examples and use cases
1.1.3.2. Calculate high-level time estimation for selected project(s)
AI Project Scaling

1.1.3.3. Define potential external funding sources (such as government via Scale AI and Mitacs)
1.1.3.4. Analyze limits of system capability and impacts on privacy and fairness
1.2. Lead documentation processes for AI adopter (client) procurement activities
1.2.1. Draft documentation for the request for proposal (RFP), request for quotation (RFQ), and request for
information (RFI) processes
1.2.1.1. Analyze client’s procurement document to extract AI needs
1.2.1.2. Estimate level of AI maturity using publicly available information
1.2.2. Coordinate internal stakeholders to collect available technical and business information (such as
project goals, technical environment, and special requirements)
1.2.2.1. Divide preparation into workstreams (such as techical information and governance)
1.2.2.2. Facilitate individual information sessions for stakeholders
1.2.2.3 Curate informative documentation for stakeholders
1.2.3. Deliver procurement-related information to AI adopters (clients)
1.2.3.1. Structure applicable offer of services
1.2.3.2. Collect related past examples to highlight project outcomes
1.2.3.3. Identify 3rd party data (such as buying prepared datasets to supplement local data)
AI Technologies

1.2.3.4 Identify tools support required (such as external software development kits (SDKs) that are
purchased as opposed to free and open source software (FOSS))
1.2.3.5. Evaluate compliance of 3rd party data and tools for ethics considerations
Partnership Development
2.1. Develop partnerships
2.1.1. Analyze the technological and business contexts of potential partners
2.1.1.1. Prioritize potential partners based on scope of activities
2.1.1.2. Analyze business nature, products, and organization of potential partners

AI Competency Framework Business Domain 24

 2.1.1.3. Analyze customer base of partner for ethics considerations (such as demographics)
2.1.1.4. Analyze legal ecosystem and regulatory market of partner
2.1.2. Draft potential synergies based on company activities and gaps
2.1.2.1. Analyze existing gap that could be closed with other partners
2.1.2.2. Select potential partner’s product or vertical of interest
AI Project Planning

2.1.2.3. Define business opportunity (such as new clients, products, savings)

2.2. Define opportunity for AI projects
2.2.1. Analyze technical gap between the company and the partner
2.2.1.1. Perform technical analysis of the partner’s solution
2.2.1.2. Define interfaces for integration and data exchange
2.2.1.3. Determine the provenance of data (1st and 2nd party) of the partner
2.2.1.4. Establish secure data handling mechanisms (such as following ISO 27001, NIST Cybersecurity
Framework) from partner
2.2.2. Define collaboration system for new AI projects
2.2.2.1. Draft governance structure (such as the roles and responsibilities)
2.2.2.2. Present the consolidated roadmap and communication plan
2.2.2.3. Establish liability framework for mishaps

AI fundamentals
AI Project Scaling

1.1. Apply technical concepts based on hybrid AI knowledge

1.1.1. Translate business-oriented use cases into AI and machine learning (ML) problems
1.1.1.1. Define high-level list of potential ML models
1.1.1.2. Select the shortlist of models and algorithms
1.1.1.3. Define model outcome benchmarking criteria
1.1.1.4. Define ethics benchmarks for privacy and fairness
1.1.1.5. Analyze capabilities and limitations of different algorithmic approaches with respect to ethics
considerations (such as privacy and fairness)
1.1.2. Link technical AI models and metrics to business goals
1.1.2.1. Present minimum acceptable results to all stakeholders
1.1.2.2. Quantify the number and duration of AI experimentation sprints with technical team members
(including data scientists, engineers, software developers)
1.1.2.3. Establish guardrails and thresholds for acceptable performance from an ethics perspective
(such as outright ban on pornographic content and hate speech using known slurs)
1.2. Select appropriate on-premises or cloud technologies based on context
1.2.1. Define the gap between current infrastructure and available options
AI Technologies

1.2.1.1. Analyze current infrastructure against computing requirements

1.2.1.2. Explain available options and advantages of cloud or hybrid implementations
1.2.1.3. Explain limitations of ethics analyses of 3rd party implementations (such as lack of access to
underlying training data, lack of control on the updation cycle of the 3rd party software)
1.2.2. Analyze expected impact of cloud for data and AI activities
1.2.2.1. Define infrastructure hard requirements based on AI and ML models
1.2.2.2. Select data storage options based on legal and regulatory (such as data privacy) requirements

AI Competency Framework Business Domain 25

 AI Projects and Technologies Budget
2.1. Consolidate cost structure based on technical requirements
2.1.1. Estimate human requirements per technical track (such as science, development)
2.1.1.1. Define required roles based on types of activity
2.1.1.2. Quantify level of effort per role during each stage of the project
AI Project Planning

2.1.2. Select best infrastructure options based on context

2.1.2.1. Calculate capital expediture (CAPEX) for new investments (such as software licenses)
2.1.2.2. Calculate operating expenditure (OPEX) for recurrent usage (such as cloud services)
2.1.2.3. Calculate external stakeholder engagement costs for ethics consultations
2.1.3. Choose potential/required tools for data and AI
2.1.3.1. Identify areas for data management (such as integration, prediction)
2.1.3.2. Compare features against cost for different vendors
2.1.3.3. Analyze external vendor technical documentation to understand privacy and
fairness impacts
2.2. Calculate total cost based on delta
2.2.1. Draft a multi-category budget for data and AI projects
2.2.1.1. Define framework for budget calculation (such as categories, allocated buckets)
2.2.1.2. Estimate realistic expenses based on project scope and budget framework
2.2.2. Obtain executive support for envisioned expenses
2.2.2.1. Categorize costs per involved requirement (such as IT, finance)
AI Project Scaling

2.2.2.2. Define timing, type of expense, and involved verticals per project stage
2.2.2.3. Periodically re-evaluate expenses in order to minimize market risk
AI Technologies

AI Competency Framework Business Domain 26

AI Competency Framework
Human Domain

AI Competency Framework Human Domain 27

 Empathetic Approach
1.1 Investigate an existing challenge that can benefit from AI techniques
1.1.1 Produce a list of existing challenges
1.1.1.1 Identify selection criteria for choosing the challenge (such as customer value, business value,
environmental benefit)
1.1.1.2 Define the categorical scale for scoring each selection criterion for a potential challenge (such
as 1=poor, 2=mediocre, 3=fair, 4=acceptable, 5=excellent)
Innovation

1.1.1.3 Write a list of potential challenges using the Business Model Canvas template (such as key
partners, activities, resources, value propositions)
1.1.1.4 Identify the goal for resolving the challenge (such as increasing accessibility in remote areas)
1.1.1.5 Identify the social dimensions (such as cultural, linguistic, ability) that the challenge might be
related to
1.1.2 Select the challenge that will be the focus of the innovation process
1.1.2.1 Score each potential challenge using an established categorical scale (such as 1=poor,
2=mediocre, 3=fair, 4=acceptable, 5=excellent)
1.1.2.2 Select the challenge (area of concern) with the highest score on pre-determined criteria
1.1.2.3 Evaluate each candidate challenge with respect to ethical considerations criteria (such as
privacy, fairness, bias, transparency)
1.2 Identify the users’ needs, wants, and objectives
1.2.1 Interview users to elicit needs, wants, and objectives
1.2.1.1 Conduct interviews to determine key user objectives, activities, difficulties, experiences, and
motivations
1.2.1.2 Observe users (such as mainstream users and extreme users) carrying out activities in their
Teamwork

physical environment
1.2.1.3. Conduct interviews with domain and local experts to determine cultural and contextual factors
(such as historical tensions, demographic specificities)
1.2.2 Develop an empathic understanding of user needs on an emotional and psychological level
1.2.2.1 Apply a flexible mindset by setting aside one’s own assumptions (such as by asking what, how,
and why)
1.2.2.2 Produce an empathy map composed of four labelled quadrants (says, thinks, does, and feels)
1.2.2.3 Apply the “5 Why’s” methodology to arrive at root cause of challenge
Problem Definition
2.1. Analyze selected challenge and user needs
2.1.1. Classify information acquired during the empathy phase
2.1.1.1 Present each observation visually as a collage of artifacts on a whiteboard
2.1.1.2. Identify commonly recurring patterns by grouping similar findings into clusters
2.1.2 Analyze observations in order to elicit a rough problem context
2.1.2.1 Evaluate each group observation using the “4Ws” (who, what, where, and why)
2.1.2.2 Identify the most prevalent issue or problem based on “4Ws” results
Professionalism

2.1.2.3 Apply an empathetic mindset to prevalent issue and problem identification

2.2 Develop a problem statement to fully understand the goal of the innovation process
2.2.1 Frame the final problem statement applying a user-centered approach
2.2.1.1 Define the problem statement using the “4Ws” (who, what, where, and why)
2.2.1.2 Refine the problem statement using the point of view (POV) format
2.2.1.3 Refine the problem statement further after consultation with domain experts and those
with lived experiences
2.2.2 Validate the final problem statement

AI Competency Framework Human Domain 28

 2.2.2.1 Evaluate final problem statement from a user perspective (such as by avoiding product focus
references, focusing on user needs instead, all the while ensuring the problem statement is sufficiently
broad to avoid excessive bias towards a specific user’s needs)
2.2.2.2 Evaluate final problem statement for manageability (such as sufficiently broad but focused
enough to be able to achieve a goal in a timely manner (feasibilitiy), avoids references to specific
technologies or solutions, actionable by a design team)
2.2.2.3. Evaluate the final problem statement for alignment with community (of target users) values
Innovation

(cultural and contextual)

Ideation Process
3.1 Produce potential AI Solutions to address the user problem
3.1.1 Develop a robust ideation process to maximize chances of success
3.1.1.1 Determine the number of participants for conducting the ideation process
(such as 8 participants)
3.1.1.2 Select an ideation session time frame (such as 3 sessions of 30 mins)
3.1.1.3 Determine ideation techniques to use based on participant experience levels with each
technique (such as mindmapping, brainstorming, storyboarding, reverse thinking, personas,
challenging assumptions, worst possible idea)
3.1.1.4. Evaluate the ideation technique to ensure inclusiveness (such as disability,
linguistic considerations)
3.1.2 Apply ideation techniques to explore and elicit potential solutions
3.1.2.1 Derive how might we (HMW) questions from empathy map and problem statement to initiate
idea elicitation
3.1.2.2 Apply several ideation methodologies to further elicit candidate solutions to the problem
3.1.2.3 Perform record keeping to ensure all ideas captured using a collaborative tool (such as Miro,
Teamwork

SessionLab, Ideaflip)
3.1.2.4. Create mechanism to make these records FAIR (findable, accessible, interoperable, reusable)
for stakeholders for future uses
3.2 Decide on a final solution to build upon
3.2.1 Evaluate potential solutions and ideas
3.2.1.1 Apply a voting scheme by participants to rank each candidate’s ideas or solution
3.2.1.2 Produce a short list of candidate ideas or solutions to carry forward to prototyping
(such as the top 5 ideas)
3.2.1.3 Apply ethical considerations (such as privacy, fairness, transparency) checklists to filter
shortlist candidate solutions
Prototyping and Testing
4.1 Create a prototype that integrates AI components
4.1.1 Design a low fidelity prototype of the product or service
4.1.1.1 Sketch principle canvas or user interfaces (such as input screens) using paper and pencil
4.1.1.2 Sketch principle components (such as buttons, result screens, web services) using
paper and pencil
Professionalism

4.1.2 Produce a medium fidelity (scaled-down version) prototype of the product using a wireframing tool
4.1.2.1 Implement static components (such as screens, buttons, text areas) using a wireframing tool
( such as Figma, Wireframe.cc)
4.1.2.2 Implement key dynamic behaviors (such as user flows, interactions) using a wireframing tool
4.2 Test the prototype
4.2.1 Perform concept testing on the prototype to gather feedback
4.2.1.1 Implement a test plan (containining fields such as personas, user flows, test cases)
4.2.1.2 Execute the test plan by users and score interaction results
4.2.1.3 Interview users about their feelings regarding the prototype

AI Competency Framework Human Domain 29

 4.2.1.4 Score how each user felt about the prototype
4.2.1.5 Implement mechanisms to report feedback anonymously to offer testers protections in
sensitive use-case violations
4.2.2 Assess the proposed prototype
4.2.2.1 Identify constraints and flaws based on test results (such as what needs to be reworked
or refined)
4.2.2.2 Identify novel and convenient features (such as what works well)
Innovation

4.2.2.3 Update the prototypes based on feedback (such as by using iterative processes, stopping when
sufficient refinement is achieved)
4.2.2.4. Group feedback into sociological (such as culture, history, age, gender, race) and technical
considerations (such as strength of privacy (through the value of epsilon in differential privacy)
appopriateness of the fairness metrics (equalized odds, reject option classification))
4.2.2.5. Prioritize sociological and technical considerations to address according to project budget
4.2.2.6. Identify red-lines (no-go zones which are determined to be non-negotiable) in feedback which
are show stoppers (such as the use of facial recognition in public spaces to flag criminal activity)

Communication
1.1 Develop a communication strategy for the AI project team
1.1.1 Investigate the AI team project context
1.1.1.1 Produce list of all stakeholders that will collaborate on the AI project (such as business analyst,
developer, data engineer, data scientist, scrum master, project manager, solution architect)
1.1.1.2 Define list of expected deliverables for each stakeholder (such as requirements specification,
Teamwork

architecture description, activities roadmap, code, documentation)

1.1.1.3 Define list of decisions that need to be communicated (such as sprint plans, target solution
architecture, performance metrics)
1.1.1.4 Define list of potential issues that will need to be communicated during the course of the project
(such as anticipated delays, technical difficulties)
1.1.1.5. Evaluate the list of stakeholders to include a diverse group of internal and external participants
that matches the target audience of the project
1.1.1.6. Evaluate baseline knowledge of selected stakeholders for base issues in ethics considerations
(such as privacy, fairness, transparency)
1.1.1.7. Define list of resources (such as videos, papers, handbooks) that will be provided to
stakeholders to fill knowledge gaps on ethics considerations
1.1.2 Design the communication strategy
1.1.2.1 Produce a Responsible, Accountable, Supporting, Consulted and Informed (RASCI)
Responsibility Matrix using list of defined responsibilities (such as decisions, deliverables, potential
issues) and stakeholders
1.1.2.2 Identify groups involving stakeholders with similar responsibilities
1.1.2.3 Define a set of team interactions (such as meetings, discussions, decision points) based on
Professionalism

identified groupings in the RASCI responsibility matrix

1.1.2.4 Define frequency of each team interaction
1.1.2.5 List available communication methods (such as phone, e-mail, videoconferencing,
in-person meetings)
1.1.2.6 Define communication methods for each team interaction (such as ad-hoc communications via
chat or e-mail, weekly team meetings using videoconferencing, 1:1 weekly meetings in person)
1.1.2.7. Identify limitations of technical and cost barriers for having inclusive participation from all
stakeholders
1.1.3 Implement the communication strategy

AI Competency Framework Human Domain 30

 1.1.3.1 Select technologies to use for each communication method (such as Gmail, Zoom, Slack,
JIRA, Confluence)

1.1.3.2 Communicate the strategy design and implementation details to all stakeholders before the
start of a project (such as the responsibilities, meeting frequencies, technologies to be used)
1.1.3.3. Gather feedback from stakeholders to identify any concerns on communication styles and
frequency (given considerations such their personal circumstances, special needs)
1.2 Apply active listening when engaging users and stakeholders
Innovation

1.2.1 Engage in the stakeholder communication

1.2.1.1 Use non verbal cues (such as occasional nodding, eye contact, leaning slightly forward)
1.2.1.2 Ask specific questions to ensure a common understanding among users and stakeholders
1.2.2 Identify key concepts from the stakeholder message
1.2.2.1 Restate the key ideas of the conversation to validate understanding of what is conveyed
1.2.2.2 Summarize key concepts of stakeholders message using sketching techniques (such as
free-hand diagrams, conceptual models using boxes and arrows)
1.2.3 Interpret the stakeholder’s message
1.2.3.1 Empathize with the speaker’s message
1.2.3.2 Identify the existing cognitive biases and team communication practices for stakeholders
(such as priming bias, affinity bias, confirmation bias)
1.2.4 React to the stakeholder message
1.2.4.1 Share similar experiences to show understanding of speaker message
1.2.4.2 Ask open-ended questions to stimulate further conversation about the topic of discussion
1.3 Present data using storytelling techniques
Teamwork

1.3.1 Prepare the narrative

1.3.1.1 Document the intial business problem and background (such as data sources involved,
anticipated business value derived from insights obtained from data analysis, constraints)
1.3.1.2 Determine your audience type (such as technical, business, legal, and executive role within
the organisation)
1.3.1.3 Predict possible questions your audience may have (such as questions around valuable
insights found in the data set, challenges, confidence level in accuracy of results)
1.3.1.4 Anticipate answers to possible questions by considering your audience type
(such as by tailoring your narrative to your audiences’ business acumen and technical level)
1.3.1.5 Identify key points you wish to make with data (such as comparing data sets, showing trends
in a data set, anamolies in data sets)
1.3.1.6 Document key points to reveal insights in a gradual manner (such as by beginning with an
introduction, then moving into the incident/problem, climax/main insight, resolution, and finally ending
with a conclusion)
1.3.1.7 Document best practices (such as avoiding misleading through the use of statistics
and visualizations) in using data to communicate with stakeholders
Professionalism

1.3.2 Incorporate visual elements

1.3.2.1 Verify the accuracy of the data in order to maintain credibility (such as duplicates, undetermined
values, unexplainable outlier values)
1.3.2.2 Use appropriate visualisations (such as charts, maps, infographics) to convey key points
of the narrative (such as by comparing, contrasting, demonstrating outliers and trends)
1.3.2.3 Manage cognitive load (such as by using only one visual element per slide, avoiding presenting
too much data at once)
1.3.2.4. Utilize techniques (such as confidence intervals) to convey the level of certainty in the results
that are being presented
1.3.3 Present the data story through narratives and visuals

AI Competency Framework Human Domain 31

 1.3.3.1 Share findings that reveal progressively deeper insights into the problem
1.3.3.2 Present major findings (“aha moments”) to audience
1.3.3.3 Propose recommendations and next steps to further motivate audience (such as empowering
audience to act or make decisions using the new findings and recommendations)
1.3.3.4. Present key ethical considerations from a risk perspective
Collaboration
Innovation

2.1 Apply emotional intelligence within the AI team and with stakeholders
2.1.1 Practice self-awareness of emotions during interactions with team members and stakeholders
2.1.1.1 Reflect on your reactions with people (such as by asking are you rushing to judgement
or stereotyping?)
2.1.1.2 Complete a self-evaluation (such as by asking what are your weaknesses, are you willing to work
to improve upon them?)
2.1.1.3 Assess your reactions to stressful situations (such as by asking do you become upset, do
you blame others, could you behave more calmly and assume a proactive-problem solving persective
instead)
2.1.1.4 Anticipate positive responses to stressful situations (such as by being more empathetic towards
fellow team members, suggesting methods for efficiently managing stressful situations)
2.1.1.5 Reflect on how your actions affect others (such as your behaviour during meetings)
2.1.2 Exercise self-regulation of emotions during interactions with team members and stakeholders
2.1.2.1 Analyze context (such as roles, expectations, objectives) before responding
2.1.2.2 Demonstrate a positive attitude during problem solving sessions (such as by sharing
insipirational moments where difficult problems were resolved with creativity and perseverance)
2.1.2.3 Adapt your responses to changing circumstances (such as after you encounter a problem,
Teamwork

proposing a brainstorm session to address immediate concerns)

2.1.2.4 Perceive feedback from peers in a constructive manner (such as by looking at feedback
as an opportunity for self-improvement, opportunity to learn from more experienced peers)
2.2 Foster social intelligence within the AI team and with stakeholders
2.2.1 Practice social awareness during interactions with team members and stakeholders
2.2.1.1 Practice empathy during individual and team interactions (such as considering the feelings
the other individual is exhibiting like anger or timidity)
2.2.1.2 Consider the needs and concerns of team members and stakeholders
2.2.1.3 Develop appropriate social behaviours depending on the team context
2.2.1.4 Identify the various roles and responsibilities within the team and organization (such as
managing requirements analysis, managing testing, determining project priorities, resolving escalations)
2.2.2 Engage in appropriate and positive social interactions with team members and stakeholders
depending on the context
2.2.2.1 Communicate in a clear and goal-oriented manner (such as by using words your team will
understand and quickly getting to the point)
2.2.2.2. Demonstrate accountability for your actions (such as by being on time for meetings, proactively
Professionalism

resolving issues that arise from your actions)

2.2.2.3 Address difficult situations for the well-being of the entire team (such as by offering to settle the
disagreement offline outside the meeting, proposing a common ground so both parties benefit)
2.2.2.4. Prepare a glossary of domain-specific terms for stakeholders to ensure shared understanding
Lifelong Learning
1.1. Develop a lifelong learning plan
1.1.1 Produce a list of subject areas to support a primary goal for lifelong learning
1.1.1.1. Determine the primary goal for pursuing lifelong learning activities (such as improving your
current job skills, obtaining a future position as a data scientist, starting your own company)

AI Competency Framework Human Domain 32

 1.1.1.2 Challenge the basis for choosing the primary goal for pursuing lifelong learning (such as by
asking is it rooted in a passion or genuine interest to learn a new skills, or potentially, is it only
a superficial curiousity because it is popular)
1.1.1.3 List subjects areas and skills that you are particularly passionate about or interested
in (such as computer vision, autonomous vehicles, robotics, privacy, fairness, transparency,
accountability, governance)
1.1.1.4 List required subject areas and skills to learn based on chosen goal after consultation
Innovation

(such as consulting a domain expert, career advisor, job descriptions)

1.1.1.5 Produce a final list of subject areas and skills to learn that take into consideration both interests
and goal requirements
1.1.1.6 Review the compiled list with social sciences and technical experts to ensure adequate coverage
1.1.2 Select required learning activites to pursue for lifelong learning
1.1.2.1 List learning resources that support education for the compiled list of subject areas and skills
(such as online learning websites, conferences, professional meetup sites, project collaborations,
professional certifications)
1.1.2.2. Identify specific learning activities by consulting list of compiled learning resources (such as
registering to a series of webinars or specific courses on an online learning site)
1.1.2.3 Classify the identified learning activities as either scheduled (such as scheduled webinars,
classes, conferences) or self-paced (anywhere, anytime such as reading articles)
1.1.2.4. Produce a calendar of learning activities (lifelong learning plan) for both scheduled and
self-paced activities to pursue in a specific time frame (such as scheduling times on a calendar
to dedicate to learning over the next year)
1.1.2.5 Determine your level of time commitment to learning
1.1.2.6. Determine case studies for ethical considerations to gain hands-on experience with applying
Teamwork

concepts (such as privacy, bias, transparency, accountability)

1.2 Implement the lifelong learning plan
1.2.1 Perform self-paced learning activities
1.2.1.1. Follow the latest developments by reading web articles (such as medium.com, weights and
biases, papers with code)
1.2.1.2 Complete self-paced online learning activities as per established calendar
(such as on DataCamp, PluralSight, KataCoda, Coursera, edx.org)
1.2.1.3 Watch video reviews from experts in the field of AI (such as 2 Minute Papers (Youtube), etc.)
1.2.1.4 Collaborate on open projects (such as in machine learning and AI) to learn as a team
such as by using open source projects on GitHub)
1.2.2 Perform scheduled learning activities
1.2.2.1 Participate at scheduled group meetups (such as in AI communities of practice, AI meetup.com)
1.2.2.2.Participate in webinars and webcasts (such as KdNuggets, Mila, IVADO)
1.2.2.3 Attend conferences (such as Open Data Science Conference (ODSC), Strata,
World Summit AI, NeurIPS)
Professionalism

1.2.2.4 Enrol in a formal education program in AI

1.2.2.5 Present at conferences or to communities of practice
1.2.2.6. Participate in online communities that are centered on ethical considerations of AI
such as MD4SG, Montreal AI Ethics Institute)

Goal Setting
2.1 Define the business objective for the AI project
2.1.1 Formulate the business objective

AI Competency Framework Human Domain 33

 2.1.1.1 Formulate a business value proposition that the AI project can deliver (such as reducing the
number of diseased trees that have to be cut down on an annual basis)
2.1.1.2 Reframe the value proposition as a business objective that can be realised by an AI project
(such as deploying a predictive model that detects diseased trees using video cameras)
2.1.1.3 Refine the statement to convey an inspirational tone that encourages the team to work towards it
2.1.1.4 Express the business objective statement qualitatively (such as by using descriptive language
and not precise numbers or quantities)
Innovation

2.1.1.5 Set a specific time frame for achieving the business objective (such as successfully deploying a
predictive model that detects diseased trees using video cameras mounted onto a fleet of dump trucks
that cover the entire city on a daily basis by end of the third quarter in 2021)
2.1.1.6. Define the societal externalities for ethical considerations (such as privacy, bias, transparency)
that may arise from trying to achieve the business objective
2.1.2 Validate the business objective
2.1.2.1 Verify that the objective provides business value for stakeholders (such as by automatically
detecting diseased trees the burden on inspectors is lightened by not have to manually survey the field
daily)
2.1.2.2. Verify that the objective is feasible, realistic, and attainable (such as by ensuring the objective
can be reached within a specific time frame)
2.1.2.3. Verify that the objective is controllable by the team (such as by ensuring the project team can
execute the key activities required to achieve the objective)
2.1.2.4 Verify that the statement is expressed qualitatively
2.1.2.5. Verify a contigency plan in case the system needs to be taken offline and the impact of
such a takedown on critical services for vulnerable people
2.2 Define the expected key results for achieving the business objective
Teamwork

2.2.1 Formulate the expected key results that will ascertain the business objective has been met
2.2.1.1 Define a list of potential key result statements using performance criteria by which the objective
can be evaluated (such as an inference latency of less than two seconds, predictive model memory
footprint fits on a mobile phone)
2.2.1.2 Define a list of potential key result statements using quality criteria by which the objective
can be evaluated (such as by determining accuracy using mean Average Precision (mAP), F1-score)
2.2.1.3 Define a list of potential key result statements using revenue or savings criteria (if applicable)
that can be achieved from deploying the implemented AI project (such as reducing annual inspection
costs by $100K per year)
2.2.1.4 Select key result statements from previously defined lists of potential statements that best
support the achievement of the business objective (such as striving to strike a balance between
performance, quality, and revenue perspectives based on specific customer needs and expectations)
2.2.1.5 Establish scoring criteria for evaluating selected key results (such as 1.0 = extremely ambitous
outcome that may be nearly impossible; 0.7 = progress that is difficult, but ultimately attainable;
0.5 = almost what we hope to achieve, but not quite ; 0.3 = what can be achieved with minimal effort;
0 = no progress, unacceptable result)
Professionalism

2.2.1.6 Define the rationale for computing final score of business objective achievement (such as by
asking should only performance and quality key results be mandatory for computing the final scoring
or should all key result statements should be used?)
2.2.1.7. Define a list of quantitative ethics key results statements (such as epsilon value for differential
privacy, equalized odds for fairness)
2.2.1.8. Define a list of qualitative ethics key results statements (such as respecting indigenous
perspectives on holding data on sovereign land)
2.2.2 Validate the key results statements
2.2.2.1 Verify that the key results are measurable milestones (such as obtaining a 95% accuracy rate
on a predictive model, revenue amount in dollars, percentage improvement in quality)

AI Competency Framework Human Domain 34

 2.2.2.2 Verify that key results describe outcomes and not activities (such as by not expressing as a list
of tasks but conveying a concrete, substantial, and measurable result)
2.2.2.3 Verify that key result statements are demonstratable in terms of progress (such as the
possibility to quantify progress with an accuracy rate, measuring a numerical or categorical metric)
2.2.2.4 Assign an owner for the key result statements
2.3 Monitor the progress of the business objective using key results
2.3.1 Assess progress during check-ins
Innovation

2.3.1.1 Assess the confidence level for completing each key result statement at expected delivery date
(such as 0-20% = unlikely; 21-50% = high risk of failure; 51-75% = difficult areas must be addressed
quickly; 80+% = on track)
2.3.1.2 Define a list of priority activities to focus on for subsequent tasks (such as obtaining more
annotated images, upgrading server hardware)
2.3.1.3 Communicate prioritized activites for subsequent tasks to the team and stakeholders
2.3.1.4. Define a method of incorporating feedback from stakeholders to adjust for ethical
considerations (such as the emergence of new categories of hate speech that need to be caught
by the content moderation system)
2.3.2 Perform grading of key results to stay on track
2.3.2.1 Assess the progress of the business objective against each key result statement using
previously agreed upon scoring criteria (such as 1.0 = extremely ambitous; 0.7 = ultimately attainable;
0.5 = mediocre; 0.3 = falling short ; 0 = no progress)
2.3.2.2 Assess which components, deliverables, or risks must be addressed
2.3.2.3 Define a list of priority activities to be performed for subsequent tasks
2.3.2.4 Communicate short-term priorities (to address shortcomings) to team and stakeholders
Teamwork

2.3.1 Perform final grading of key results

2.3.2.1 Assess the progress of business objective against each key result (KR) statement using
previously agreed upon scoring criteria (i.e. 1.0 = overachieved the KR; 0.7 = KR achived satisfactorily;
0.5 = mediocre result, not acceptable; 0.3 = little progress, not acceptable; 0 = complete fail)
2.3.2.2 Establish final score of business objective using previously agreed upon scoring rationale (such
as it must only contain two key results of 0.7 or greater or all key results greater than 0.7 for
establishing final business objective achievement)
2.3.2.3 Obtain a consensus from team for all key result scores (such as a majority vote)
2.3.2.4 Obtain a consensus from the external stakeholders for the ethical considerations related
key words
2.4.2 Conduct a retrospective (post-mortem) meeting with team and stakeholders
2.4.2.1 Discuss actions that supported achieving the objective
2.4.2.2 Discuss actions that were not as effective in supporting the achievement of the objective
2.4.2.3 List improvements that can be leveraged for the next AI business objective (such as adding
or omitting activities, using supplemental tools)
2.4.2.4 Define a prioritized plan with a timeline to address the improvements that need to be made
Professionalism

AI Competency Framework Human Domain 35

Using the AI Competency
Framework
Narratives
The goal for this section is to provide users of the AI competency framework with different contexts
in which the identified knowledge, skills, and abilities can be applied. To accomplish this, fictional
characters based on interviews have been developed for program developers and educators.
In the same way that students are diverse, each with their own lived experience and knowledge,
the context is critical for these narratives.

The narratives describe the role of a program developer and an educator in a CÉGEP, Dawson College,
and a continuing education, Concordia Continuing Education (CCE) context. Based on this description,
relevant applications of the competency framework are identified for performing specific program
developer and educator responsibilities. This is useful to direct users of this document working in
similar contexts to different applications of the framework for their own roles.

Program Developer
Zoe leads program development as an instructional designer and as a subject matter expert in AI.
They develop and update programs in AI at the continuing education and CÉGEP levels.
Zoe understands that developing and updating programs is dependent on the context and the goals
of the respective institutions where they work. Zoe is a generalist and understands both the granular,
such as content structure at the course level, and the broad, such as main competencies and learning
outcomes at the program level.

Zoe leads a program development team and serves as the bridge between developers and the
approvers making final decisions. They are a strong connector and communicator and leverage
the skills of several people on their team including instructors with disciplinary expertise, professionals
such as instructional designers or curriculum developers, and project managers to develop
and update programs.

AI Competency Framework 36
Ultimately, Zoe is responsible for leading the creation of a roadmap for the journey students take from
the day they enter an AI program to what students are able to do at the end of the program by meeting
the exit profile and program learning outcomes.

As a program developer, Zoe is also responsible for:

y Constructing the scope of the program including how much it will cover

y Designing the learning pathways for students

y Creating a curriculum map

y Developing program evaluation tools

y Establishing an iterative feedback and testing loop for the program development process

y Drafting the program level learning outcomes and core competencies

y Developing student entrance and exit profiles

y Assessing relevance of program for industry needs

Job Titles

Zoe leads program development as an instructional designer in continuing education, and as a subject
matter expert and faculty member in CÉGEP.

Competency Framework

The AI competency framework is a common set of core competencies required by all AI practitioners.
As a program developer for AI, Zoe envisions using the AI competency framework to fill in gaps where
the current program and courses do not reflect the core knowledge, skills, and abilities industry seeks
and to ensure that the program is relevant.

Here are two areas in Zoe’s role as a program developer where the AI competency framework could
meaningfully contribute:

1. Developing target competencies for exit profiles including program-level

learning outcomes

2. Developing resource documents for curriculum development

AI Competency Framework 37
Educator
Julien is an experienced educator who teaches and develops courses in AI at both the CÉGEP and
continuing education levels in Montreal. Julien interacts directly with their students and attempts to
translate complex concepts inherent in AI into easy-to-understand content for a diverse group.
Julien adheres to teaching specific course-level learning outcomes and course competencies and
uses these to create relevant learning activities. Additionally, Julien ensures students meet course
competencies and learning outcomes by creating a specific set of assessments. They often engage
in additional research on content and teaching strategies to curate content for students after
understanding the student needs. In continuing education, Julien integrates their relevant industry
experience to the classroom and even coaches students on approaching the job market and connecting
students to industry.

Although Julien adheres to a curriculum, they find creative ways to facilitate student journeys from
when students arrive to what they must be able to do when they leave the course. Ultimately, Julien
might begin with established course outlines, competencies, and learning outcomes but they are still
required to transform those into a learning experience for students.

Julien also takes on additional responsibility by developing courses in AI and emerging technologies.
When Julien develops courses, they are responsible for:

y Understanding the goal of the course meaning what we ultimately want to accomplish

y Developing course learning outcomes

y Integrating core program competencies and learning objectives into a specific course

y Planning the structure of the course

y Aligning course content with learning outcomes

y Validating iterations of the course with an instructional designer

y Formatting the course design in a way where other instructors can implement the course

Job Titles

Julien educates students in the classroom and supports course development as a faculty member in
CÉGEP and as an instructor in continuing education.

AI Competency Framework 38
Competency Framework

The AI competency framework is a common set of core competencies required by all AI practitioners.
As an AI educator, Julien ensures that their courses prepare students for a career in AI. Julien feels that
because the framework covers all of the core competencies an AI practitioner should possess, they can
select the competencies that are relevant to their context and that respect their constraints including
time, budget, and institutional processes to apply in the classroom.

Here are two areas in Julien’s role as an educator where the AI Competency Framework could
meaningfully contribute:

1. Creating learning activities for students

y In practical terms this involves combining several sub-competencies and sub-sub

competencies from the framework to help develop an activity

2. Developing assessments based on course learning outcomes and competencies

y In CÉGEP, this is more likely done at the course committee level

Dawson College Use Case

Given that program competencies are prescribed by the Ministry of Higher Education, a major challenge
at the college level is to find ways to scale AI offerings and formalize AI curriculum pathways. In light of
this, Dawson College is actively building capacity to support the integration of AI competencies
in curricula. The goal is to expose as many students as possible to AI concepts through both general
education and in courses across the sciences, the social sciences, and the arts.

In the specific case of the CÉGEP-level Science Program, some AI competencies align with those
already present in the curriculum, notably the majority of those in the Mathematics & Statistics focus
area. Moreover, other competencies are easily integrated as learning activities and do not require
formal program revision to implement. For general education courses, AI provides a fertile context for
exploration in the humanities and is an appealing subject for complementary courses, which allow
for a flexible, multidisciplinary approach to AI themes.

Below is a sketch of how technical competencies from the framework could be integrated into
the current Pure & Applied Science profile at Dawson College with many courses already covering
Mathematics & Statistics competencies, and a project-based programming course diving deeper into
the technical details needed for working with data and building models.

AI Competency Framework 39
 Term I Term II ​ Term III​ Term IV​
Mechanics ​ Waves & Modern Physics​ Electricity & Magnetism​ Option​
General Chemistry​ Chemistry of Solutions​ General Biology Probability & Statistics​
Math & Stats: 1.1

Calculus I​ Calculus II​ Linear Algebra​ Programming​

Math & Stats: 3.1​ Math & Stats: 3.2 Math & Stats: 2.1 Math & Stats: 4.1, 4.2, 4.3 ​

Programming: 1.1​

Data: 3.1, 3.2, 4.1, 4.2

Machine Learning: 1.1, 2.2, 3.2 ​

English​ English​ English​ English​

PhysEd​ PhysEd​ PhysEd​ French​
Humanities​ Humanities​ Complementary​ Humanities​
Deep Learning: 1.1, 1.3
Data: 2.2

Complementary​ French​ ​ ​

AI Technical Focus Areas: Data Math & Stats Programming Machine Learning Deep Learning Infrastructure

Furthermore, all students must successfully complete an independent study capstone project
as a requirement for graduation. The goal is for students to demonstrate synthesis of program
competencies by applying their learning to a contextually rich problem. As such, these projects provide
a wealth of opportunity for embedding more advanced topics within a concrete task and are ideal for
interdisciplinary fields like AI.

At the time of writing, the CÉGEP Science and Social Science programs are undergoing revision across
Québec. This is an exceptional opportunity and Dawson College is exploring the creation of new profiles
within these programs that could organize a range of AI-related curricula already developed by faculty
through its AI learning community of teaching and research fellows. Additionally, a new AI-themed
certificate to be offered in 2022-23, similar to a university specialization, will allow any student to
flavour their program experience with AI topics and projects. The AI competency framework developed
through this project is a valuable tool for the development of such initiatives.

AI Competency Framework 40
Implementing the AI
Competency Framework
for your Context
While the guide primarily focuses on the profiles of educators and program developers, this section
addresses considerations for other users of the competency framework to take into account when
implementing your use case.

Program administrators of AI courses and programs

y Identify growth opportunities such as new programs and courses

y Evaluate the relevance of current programs and offerings

Student success centers looking at developing complimentary trainings for technical or non-technical students

y Provide students with career counselling:

à Support students in selecting the appropriate degree that meets their goals

à Assist students in developing an education plan

à Support students in exploring their strengths, interests and passion

y Provide students with career advising:

à Assist with resume writing

à Help students prepare for interviews

y Develop resources for students to explore career pathways

y Develop career development workshops to introduce AI pathways

Training managers and human resource managers

y Identify knowledge and skills gaps in current organizations to develop training programs
for internal employees or create recruitment strategies for internal and external hiring

y Identify goals and formal and informal experiences for career development planning

AI Competency Framework 41
 y Baseline main and supporting objectives for performance planning

y Identify the skills, knowledge, experience, and attitudes required in a job description

Prior learning coordinators

y Use the framework as a baseline to validate and certify competencies

y Identify the competencies required for a specific field and compare it to candidate’s
previous experience and knowledge

y Use it as a reference document for competencies that a candidate should master

AI Competency Framework 42
Conclusion
Concordia University and Dawson College leveraged the opportunity to collaborate on this project as
a way to continue our work in aligning AI related teaching and learning in the classroom to the skills,
knowledge, and abilities students are expected to possess as AI practitioners in the workplace.
This AI competency framework also supports a success pipeline for learners from college to university
to life-long learning who are all at different stages in their learning journeys. Finally, it attempts to
address the evolving AI talent needs and serves as a base for curriculum development to balance
technical, business, human, and ethical competencies in AI training programs and recognition
of prior learning.

We need to continue to break down the silos in the way we conceptualize learning in AI because it is not
just about developing technical competencies but requires AI practitioners to possess business and
human skills along with ethical skills.

The challenge ahead is to move beyond simply creating this competency framework that identifies
the core competencies AI practitioners should possess in a Montreal context to understanding and
using these competencies to help develop or update programs and courses for students to acquire
the knowledge, skills, and abilities needed to become AI practitioners. Institutions must help students
develop these competencies to prepare them to work as AI practitioners, but for many it would require
substantial adaptations to the curriculum because most curriculums were not designed to develop
these skills. To do this, higher education institutions can make use of their adaptive advantages to
build out existing curriculum and investigate the creation of new curriculum in different models and
modalities to implement the identified competencies and evaluation tools.

AI Competency Framework 43
Glossary
1st party (primary) data “Information that you collect yourself, rather than get from somewhere else”
(Cambridge Dictionary, n.d.-f)

2nd party (secondary) data “Data that is publicly available, rather than data that an organization collects for
itself” (Cambridge Dictionary, n.d.-g)

agile “An iterative approach to project management and software development that
uses small, but consumable, increments to deliver work” (Atlassian, n.d.) resulting
in an ability to respond to change quickly

algorithm “A set of mathematical instructions or rules that, especially if given to a computer,

will help to calculate an answer to a problem” (Cambridge Dictionary, n.d.-a)

algorithmic bias “A phenomenon that occurs when an algorithm produces results that are
systemically prejudiced due to erroneous assumptions in the machine learning
process.” (Pratt, 2020)

application programming “A tool used to share content and data between software applications” (MIT
interface (API) Libraries, n.d.)

architecture description “A formal explanation of an information system organized to support reasoning

behind the structural properties of the system.” (The Open Group, n.d.)

artificial general intelligence “Artificial general intelligence (AGI) as opposed to narrow intelligence, also
(AGI) known as complete, strong, super intelligence, Human Level Machine Intelligence,
indicates the ability of a machine that can successfully perform any tasks in an
intellectual way as the human being.” (Ranschaert et al., 2019, p. 350)

big data “Big data is high-volume, high-velocity and/or high-variety information assets that
demand cost-effective, innovative forms of information processing that enable
enhanced insight, decision making, and process automation.” (Gartner, n.d.-a)

business model canvas A strategic management template which breaks down key elements of a current
business, or future intended venture, and how it will generate revenue. (University
of Toledo, n.d.)

cloud computing “Cloud computing is a style of computing in which scalable and elastic IT-enabled
capabilities are delivered as a service using internet technologies. Examples of
services include Infrastructure as a Service (IaaS), Platform as a Service (PaaS)
and Software as a Service (SaaS).” (Gartner, n.d.-b)

clustering “Clustering is a task to organize data into groups based on certain properties.
Clustering analysis is widely used in data mining for pattern recognition, image
analysis, and computer graphics, among others.” (Ranschaert et al., 2019, p. 352)

AI Competency Framework 44
code review “A code review is a process where someone other than the author(s) of a piece of
code examines that code.” (Google, n.d.)

concept testing “The process of asking a group of people for their opinion about a new product or
advertisement” (Cambridge Dictionary, n.d.-b)

confidence intervals “The purpose of taking a random sample from a lot or population and computing
a statistic, such as the mean from the data, is to approximate the mean of the
population. How well sample statistics estimate the underlying population value
is always an issue. A confidence interval addresses this issue because it provides
a range of values which is likely to contain the population parameter of interest.”
(NIST/SEMATECH, n.d.)

confidence level (as per “A qualitative indicator of whether the person leading the OKR believes it is still
Objectives and Key Results (OKR) achievable at a prescribed period in time.” (Perdoo, 2018)
framework)

continuous delivery (CD) “Continuous Delivery is the ability to get changes of all types—including new
features, configuration changes, bug fixes and experiments—into production, or
into the hands of users, safely and quickly in a sustainable way.” (Humble, n.d.)

continuous integration “The practice of automating the integration of code changes from multiple
contributors into a single software project. It’s a primary DevOps best practice,
allowing developers to frequently merge code changes into a central repository
where builds and tests then run. Automated tools are used to assert the new
code’s correctness before integration.” (Rehkopf, n.d.)

data integrity “Data integrity is the overall completeness, accuracy and consistency of data.
This can be indicated by the absence of alteration between two instances or
between two updates of a data record, meaning data is intact and unchanged.”
(Techopedia, 2020b)

data quality “Data quality is an intricate way of measuring data properties from different
perspectives. It is a comprehensive examination of the application efficiency,
reliability and fitness of data, especially data residing in a data warehouse.”
(Techopedia, 2017)

data storytelling Conveying facts and statistics collected together for analysis using a narrative
people can understand, not just numbers and charts. (Stackpole, 2020)

deep learning “A subset of machine learning where systems ‘learn’ to detect features that are not
explicitly labelled in the data.” (Information Commissioner’s Office, 2020)

definition of done (DoD) “The team agrees on, and displays prominently somewhere in the team room, a
list of criteria which must be met before a product increment “often a user story”
is considered “done”. Failure to meet these criteria at the end of a sprint normally
implies that the work should not be counted toward that sprint’s velocity.” (Agile
Alliance, n.d.)

descriptive statistics “The term “descriptive statistics” refers to the analysis, summary, and presentation
of findings related to a data set derived from a sample or entire population.
Descriptive statistics comprises three main categories – Frequency Distribution,
Measures of Central Tendency, and Measures of Variability.” (Corporate Finance
Institute, n.d.)

AI Competency Framework 45
design thinking “An innovative problem-solving process grounded in a set of skills frequently
applied to the development of new products and services.” (Linke, 2017)

DevOps “DevOps is the combination of cultural philosophies, practices, and tools that
increases an organization’s ability to deliver applications and services at high
velocity: evolving and improving products at a faster pace than organizations
using traditional software development and infrastructure management
processes. This speed enables organizations to better serve their customers and
compete more effectively in the market.” (Amazon Web Services, n.d.)

emotional intelligence “The ability to understand the way people feel and react and to use this skill to
make good judgments and to avoid or solve problems” (Cambridge Dictionary,
n.d.-c)

empathy map “An empathy map is a collaborative visualization used to articulate what we know
about a particular type of user. It externalizes knowledge about users in order to
1) create a shared understanding of user needs, and 2) aid in decision making”
(Gibbons, 2018)

fairness metrics “Fairness metrics are often used to verify that machine learning models do not
produce unfair outcomes across racial/ethnic groups, gender categories, or other
protected classes.” (Carey, 2020)

high-performance computing “High Performance Computing most generally refers to the practice of aggregating
computing power in a way that delivers much higher performance than one
could get out of a typical desktop computer or workstation in order to solve large
problems in science, engineering, or business.” (USGS, n.d.)

human-in-the-loop (HITL) “The integration of a human workforce in the AI pipeline in order to train and
perspective validate models in a continuous way.” (Humans in the Loop, n.d.)

identity and access management “Identity and access management (IAM) is the discipline that enables the right
(IAM) individuals to access the right resources at the right times for the right reasons.”
(Gartner, n.d.-c)

key result (as per OKR framework) A measurable outcome, usually in the form of a quantitative statement, that
measures progress towards a given objective. (Perdoo, n.d.)

key results scoring criteria A standardized scale of values used to grade key result statements and if an
objective was fully achieved. (re:Work, n.d.)

knowledge management “The way in which knowledge is organized and used within a company, or the
study of how to effectively organize and use it” (Cambridge Dictionary, n.d.-d)

knowledge transfer “The process of communicating knowledge that has been developed in one part
of an organization to other parts of the organization or to customers.” (Macmillan
Dictionary, n.d.)

machine learning (ML) “The process of computers changing the way they carry out tasks by learning from
new data, without a human being needing to give instructions in the form of a
program” (Cambridge Dictionary, n.d.-e)

AI Competency Framework 46
narrow artificial intelligence “A specific type of artificial intelligence in which a technology outperforms humans
in some very narrowly defined task. Unlike general artificial intelligence, narrow
artificial intelligence focuses on a single subset of cognitive abilities and advances
in that spectrum” (Ranschaert et al., 2019, p. 358)

neural network “Also known as artificial neural network, neural net, deep neural net; a computer
system inspired by living brains.” (Ranschaert et al., 2019, p. 358)

NIST cybersecurity framework “This voluntary framework consists of standards, guidelines and best practices to
manage cybersecurity risk.” (NIST, 2013)

objectives and key results (OKR A goal-setting framework for defining and tracking qualitative goals and their
framework) outcomes. (White, 2018)

outlier trimming “The process of removing or excluding extreme values, or outliers, from a data set.
Data trimming is used for a number of reasons and can be accomplished using
various approaches.” (SAGE Research Methods, 2017)

personally identifiable “Any representation of information that permits the identity of an individual to
information (PII) whom the information applies to be reasonably inferred by either direct or indirect
means.” (U.S. Department of Labor, n.d.)

persona “A realistic and reliable representation of your target audience using fictional
characters based on qualitative and quantitative research” (Usability.gov, n.d.)

pre-processing “The process of transforming data prior to using it for training a statistical model.”
(Information Commissioner’s Office, 2020)

prototype fidelity “Refers to how closely an early test product or solution matches the final product
or solution.” (Jain, n.d.)

RACI “RACI is a powerful tool that clarifies individual or group roles for each task in a
project or business process, and it creates a simple language to discuss roles and
responsibilities within an organization.” (RACI Solutions, n.d.)

RASCI Responsibility Matrix Abbreviation for Responsible, Accountable, Support, Consulted, Informed
Responsibility Matrix. “It is a matrix used to identify and define roles and
responsibilities for each stakeholder working on a specific project.” (Reeves, 2019)

reinforcement learning “A type of dynamic programming that trains algorithms using a system of reward
and punishment. The algorithm is exposed to a total random and new dataset
and it automatically finds patterns and relationships inside of that dataset”
(Ranschaert et al., 2019, p. 360)

relational databases “A relational database organizes data into tables which can be linked—or related—
based on data common to each.” (IBM Cloud Education, 2019)

request for information (RfI) “Request for information (RfI) is made when you’re looking for information or
you’re not sure what solution might solve your problem.” (Hulsen, 2019)

request for proposal (RfP) “Request for proposal (RfP) is made when you’re ready to shop around and
evaluate many factors before making a choice.” (Hulsen, 2019)

request for quotation (RfQ) “Request for quotation (RfQ) is made when you know exactly what you want and
why but need to explore all of the financial details.” (Hulsen, 2019)

AI Competency Framework 47
role based access control (RBAC) “Access control based on user roles (i.e., a collection of access authorizations
a user receives based on an explicit or implicit assumption of a given role). Role
permissions may be inherited through a role hierarchy and typically reflect the
permissions needed to perform defined functions within an organization. A given
role may apply to a single individual or to several individuals.” (CSRC, n.d.)

scrum “Scrum is a lightweight framework that helps people, teams and organizations
generate value through adaptive solutions for complex problems. Scrum describes
a set of meetings, tools, and roles that work in concert to help teams structure and
manage their work.” (Schwaber & Sutherland, 2020)

situational leadership model “The Situational Leadership® Model is a timeless, repeatable framework for
leaders to match their behaviors with the performance needs of the individual or
group that they are attempting to influence.”
(The Center for Leadership Studies, n.d.)

social intelligence The capacity to know yourself and others while communicating and forming
relationships with empathy and assertiveness. (Garcia-Bullé, 2019)

solution architecture “provides the ground for software development projects by tailoring IT solutions to
specific business needs and defining their functional requirements and stages of
implementation.” (LeanIX, n.d.)

sprint “Fixed length events of one month or less to create consistency and enable
predictability by ensuring inspection and adaptation of progress toward a team or
product goal.” (Scrum.org, n.d.)

supervised learning “A machine learning task of learning a function that maps an input to an output
based on examples of correctly labelled input-output pairs.” (Information
Commissioner’s Office, 2020)

tensor “A generalization of vectors and matrices and is easily understood as a

multidimensional array. Many of the operations that can be performed with
scalars, vectors, and matrices can be reformulated to be performed with tensors.
As a tool, tensors and tensor algebra is widely used in the fields of physics and
engineering. A set of techniques known in machine learning for the training
and operation of deep learning models can be described in terms of tensors.”
(Brownlee, 2018)

tokenization “The process of turning a meaningful piece of data, such as an account number,
into a random string of characters called a token that has no meaningful value if
breached. Tokens serve as reference to the original data but cannot be used to
guess those values.” (McAfee, n.d.)

training, validation, and test sets Training dataset: “The sample of data used to fit the model.” (Shah, 2017)

Validation dataset: “The sample of data used to provide an unbiased evaluation

of a model fit on the training dataset while tuning model hyperparameters. The
evaluation becomes more biased as skill on the validation dataset is incorporated
into the model configuration.” (Shah, 2017)

Test dataset: “The sample of data used to provide an unbiased evaluation of a final
model fit on the training dataset.”
Shah, 2017)

AI Competency Framework 48
unsupervised learning “Unsupervised learning, also known as unsupervised machine learning, uses
machine learning algorithms to analyze and cluster unlabeled datasets. These
algorithms discover hidden patterns or data groupings without the need for human
intervention.” (IBM Cloud Education, 2020)

use case “a description of how a person who actually uses that process or system will
accomplish a goal. It’s typically associated with software systems but can be used
in reference to any process.” (Study.com, n.d.)

waterfall “The waterfall model is a sequential software development process model that
follows defined phases. It enforces moving to the next phase only after completion
of the previous phase.” (Techopedia, 2020a)

AI Competency Framework 49
References
The following sources were consulted to develop the competency framework:

Technical Domain
Calzada Prado, J., & Marzal Garcia-Quismondo. (2013). Incorporating data literacy into information literacy programs: Core
competencies and contents. LIBRI, 63(2), 134-123. Francois-Lavet, V., Henderson, P., Islam, R., Bellemare, M.G.,
& Pineau, J. (2018). An introduction to deep reinforcement learning. Foundations and Trends in Machine Learning,
11(3), 219-354. http://dx.doi.org/10.1561/2200000071

GitHub. (2021). Interactive deep learning book with multi-framework code, math, and discussions. Retrieved May 12, 2021,
from GitHub - d2l-ai/d2l-en: Interactive deep learning book with multi-framework code, math, and discussions.
Adopted at 175 universities. https://github.com/d2l-ai/d2l-en

Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.

Long, D., & Magerko, B. (2020). Proceedings of the 2020 CHI Conference on Human Factors in Computing
Systems. ACM. https://doi.org/10.1145/3313831.3376727

Ministère de l’Éducation et de l’Enseignement supérieur. (2017). Programme d’études préuniversitaire Sciences de la

nature. Gouvernement du Quebec.

Percival, H. (2014). Test-Driven Development with Python. O’Reilly Media.

Ramalho, L. (2015). Fluent Python. O’Reilly Media.

Wing, J. M. (2019). The Data Life Cycle. Harvard Data Science Review, 1(1). https://doi.org/10.1162/99608f92.e26845b4

Business Domain
Jfgagne. (2020). Global AI talent report 2020. Jfgagne. https://jfgagne.ai/global-ai-talent-report-2020/

Human Domain
Advani, V. (2019). The roles of design thinking in artificial intelligence applications. Retrieved May 13, 2021, from https://www.
mygreatlearning.com/blog/the-role-of-design-thinking-in-artificial-intelligence-applications/

Briggs, S. (2014). 25 practices that foster lifelong learning. InformED. Retrieved May 13, 2021, https://www.opencolleges.edu.
au/informed/features/lifelong-learning/

Cox, J.D (Ed). (2011). Proceedings of ASBBS annual conference: Las Vegas. ASSBS. http://asbbs.org/files/2011/
ASBBS2011v1/PDF/C/CoxJ.pdf

Dam, R.F., & Siang, T.Y. (2021). What is ideation – and how to prepare for ideation sessions. Retrieved May 13, 2021,
from https://www.interaction-design.org/literature/article/what-is-ideation-and-how-to-prepare-for-ideation-sessions

Doyle, A. (2020). Important active listening skills and techniques. Retrieve May 13, 2021, from https://www.thebalancecareers.
com/active-listening-skills-with-examples-2059684

AI Competency Framework 50
Dykes, B. (2016). Data storytelling: The essential data science skill everyone needs. Forbes. Retrieved May 13, 2021,
from https://www.forbes.com/sites/brentdykes/2016/03/31/data-storytelling-the-essential-data-science-skill-
everyone-needs/?sh=594aa0e752ad

Fanelli, M. (n.d.). The skills needed for data storytelling. Eyeota. Retrieved May 13, 2021, from https://www.eyeota.com/blog/
data-storytelling-skills

Fereira, E. (2016). The effectiveness of communication in software development project management. https://www.
semanticscholar.org/paper/The-effectiveness-of-communication-in-software-Ferreira/5813ab905f91254e766d172b5
6bbfc027aba4daf

Forbes Insights Team. (2019). How to build a great AI team. Forbes. Retrieved May 13, 2021, from https://www.forbes.com/
sites/insights-intelai/2019/05/22/how-to-build-a-great-ai-team/?sh=31574b5e426a

FS. (n.d.). Active listening: The master key to effective communication. FS. Garcia-Bulle, S. (2019). What is social intelligence
and why it should be taught in schools. Observatory of Educational Innovation. Retrieved May 13, 2021, from https://
observatory.tec.mx/edu-news/social-intelligence

Goldstein, M. (2013). Mindful listening. [Conference Paper Presentation]. PMI Global Congress 2013. New Orleans, LA. Newton
Square, PA. https://www.pmi.org/learning/library/important-project-manager-mindful-listening-5901

Golhar, A. (2018). 10 ways to increase your emotional intelligence. Inc. Hernandez, V. (2017). 30 of the latest and greatest
data storytelling tools. Medium. Retrieved May 13, 2021, from https://medium.com/@ToTheVictor/nicar30tools-
da83db0f3dc6

Kihlstrom, J.F., & Cantor, N. (n.d.). Social intelligence. Berkeley. Retrieved May 13, 2021, from https://www.ocf.berkeley.
edu/~jfkihlstrom/social_intelligence.htm

Lamorte, B. (2016). A brief history of scoring objectives and key results (OKRs). LinkedIn. Retrieved May 13, 2021, https://www.
linkedin.com/pulse/brief-history-scoring-objectives-key-results-okrs-ben-lamorte/

Leyden, A. (n.d.). 10 ways to engage in lifelong learning. Teachthought. Retrieved May 13, 2021, from https://www.
teachthought.com/learning/10-simple-ways-to-engage-in-lifelong-learning/

Martina, A. (2018). Active listening: What is it and how to improve it. Medium. Retrieve May 13, 2021 from https://medium.
com/@antoniomartina/active-listening-what-is-it-and-how-to-improve-it-ba532cad781b

Mind Tools Content Team. (n.d.). Emotional intelligence: Developing strong people skills. Mind Tools. Retrieved May 13, 2021,
from https://www.mindtools.com/pages/article/newCDV_59.htm

ProSchool. (n.d.). Data storytelling: A critical skill for data scientists. ProSchool. Retrieved May 13, 2021, from https://www.
proschoolonline.com/blog/data-storytelling-critical-skill-data-scientist

Smith, S.J. (2019). How to use OKRs for your AI team. Eckerson Group. Retrieved May 13, 2021, https://www.eckerson.com/
articles/objectives-key-result-google

Stackpole, B. (2020). The next chapter in analytics: Data storytelling. MIT Management. Retrieved May 13, 2021, from https://
mitsloan.mit.edu/ideas-made-to-matter/next-chapter-analytics-data-storytelling#:~:text=%2C%20and%20
Visuals.%E2%80%9D-,The%20skill%20of%20data%20storytelling%20is%20removing%20the%20noise%20
and,attention%20on%20the%20key%20insights.&text=%E2%80%9CPart%20of%20the%20skill%20is,visualization%20
piece%2C%E2%80%9D%20Dykes%20said.

Stevens, E. (2021). What is design thinking? A comprehensive beginner’s guide. Retrieved May 13, 2021, from The Coaching
Room. (2018). Developing your emotional and social intelligence. The Coaching Room. Retrieved May 13, 2021,
from https://www.thecoachingroom.com.au/blog/developing-your-emotional-and-social-intelligence#:~:text=In%20
his%20interview%20on%20leadership,is%20more%20about%20self%2Dmastery.&text=A%20leader%20needs%20
to%20be,what%20social%20intelligence%20is%20about.

Van Edwards, V. (2020). Social intelligence. Science of People. Retrieved May 13, 2021, from https://www.scienceofpeople.
com/social-intelligence/

Zawadski, J. (2018). The power of goal-setting in data science. Towards data science. Retrieved May 13, 2021,
https://towardsdatascience.com/the-power-of-goal-setting-for-your-data-science-project-9338bf475abd

AI Competency Framework 51
The following sources were consulted to develop the glossary:

Agile Alliance. (n.d.). Definition of done. Agile Alliance. Retrieved September 28, 2021, from https://www.agilealliance.org/
glossary/definition-of-done/

Amazon Web Services (AWS). (n.d.). What is DevOps? Amazon Web Services, Inc. Retrieved September 28, 2021, from https://
aws.amazon.com/devops/what-is-devops/

Atlassian. (n.d.). What is agile? Atlassian. Retrieved September 27, 2021, from https://www.atlassian.com/agile

Brownlee, J. (2018, February 13). A gentle introduction to tensors for machine learning with NumPy. Machine Learning Mastery.
https://machinelearningmastery.com/introduction-to-tensors-for-machine-learning/

Cambridge Dictionary. (n.d.-a). Algorithm. In Cambridge Dictionary. Cambridge University Press. Retrieved September 27, 2021,
from https://dictionary.cambridge.org/dictionary/english/algorithm

Cambridge Dictionary. (n.d.-b). Concept testing. In Cambridge Dictionary. Cambridge University Press. Retrieved September 28,
2021, from https://dictionary.cambridge.org/dictionary/english/concept-testing

Cambridge Dictionary. (n.d.-c). Emotional intelligence. In Cambridge Dictionary. Cambridge University Press. Retrieved
September 28, 2021, from https://dictionary.cambridge.org/dictionary/english/emotional-intelligence

Cambridge Dictionary. (n.d.-d). Knowledge management. In Cambridge Dictionary. Cambridge University Press. Retrieved
September 28, 2021, from https://dictionary.cambridge.org/dictionary/english/knowledge-management

Cambridge Dictionary. (n.d.-e). Machine learning. In Cambridge Dictionary. Cambridge University Press. Retrieved September
28, 2021, from https://dictionary.cambridge.org/dictionary/english/machine-learning

Cambridge Dictionary. (n.d.-f). Primary data. In Cambridge Dictionary. Cambridge University Press. Retrieved September 27,
2021, from https://dictionary.cambridge.org/dictionary/english/primary-data

Cambridge Dictionary. (n.d.-g). Secondary data. In Cambridge Dictionary. Cambridge University Press. Retrieved September 27,
2021, from https://dictionary.cambridge.org/dictionary/english/secondary-data

Carey, V. (2020, December 15). Fairness metrics won’t save you from stereotyping. Medium. https://towardsdatascience.com/
fairness-metrics-wont-save-you-from-stereotyping-27127e220cac

Corporate Finance Institute. (n.d.). Descriptive statistics. Corporate Finance Institute. Retrieved September 28, 2021, from
https://corporatefinanceinstitute.com/resources/knowledge/other/descriptive-statistics/

CSRC. (n.d.). Glossary. NIST. Retrieved September 28, 2021, from https://csrc.nist.gov/glossary/term/RBAC

Garcia-Bullé, S. (2019, September 23). What is social intelligence and why it should be taught at schools. Observatory: Institute
for the Future of Education. https://observatory.tec.mx/edu-news/social-intelligence

Gartner. (n.d.-a). Gartner glossary: Big data. Gartner. Retrieved September 27, 2021, from https://www.gartner.com/en/
information-technology/glossary/big-data

Gartner. (n.d.-b). Gartner glossary: Cloud computing. Gartner. Retrieved September 27, 2021, from https://www.gartner.com/en/
information-technology/glossary/cloud-computing

Gartner. (n.d.-c). Gartner glossary: Identity and access management (IAM). Gartner. Retrieved September 28, 2021, from https://
www.gartner.com/en/information-technology/glossary/identity-and-access-management-iam

Gibbons, S. (2018, January 14). Empathy mapping: The first step in design thinking. Nielsen Norman Group. https://www.
nngroup.com/articles/empathy-mapping/

Google. (n.d.). Code review developer guide. GitHub. Retrieved September 28, 2021, from https://google.github.io/eng-
practices/review/

Hulsen, D. (2019, October 29). RFI vs RFQ vs RFP: Which should it be? RFP360. https://rfp360.com/rfi-rfp-rfq/

Humans in the Loop. (n.d.). What is a human in the loop? Humans in the Loop. Retrieved September 28, 2021, from https://
humansintheloop.org/what-is-a-human-in-the-loop/

AI Competency Framework 52
Humble, J. (n.d.). What is continuous delivery? Continuous Delivery. Retrieved September 28, 2021, from https://
continuousdelivery.com/

IBM Cloud Education. (2019, August 6). Relational databases. IBM. https://www.ibm.com/cloud/learn/relational-databases

IBM Cloud Education. (2020, September 21). What is unsupervised learning? IBM. https://www.ibm.com/cloud/learn/
unsupervised-learning

Information Commissioner’s Office. (2020, July 30). Glossary. Information Commissioner’s Office; ICO. https://ico.org.uk/for-
organisations/guide-to-data-protection/key-dp-themes/guidance-on-ai-and-data-protection/glossary/

Jain, A. (n.d.). What is fidelity of a prototype? CommonLounge. Retrieved September 28, 2021, from https://www.
commonlounge.com/discussion/9de500b38d264803aba07026f61a65ad

LeanIX. (n.d.). The definitive guide to solution architecture. LeanIX. Retrieved September 28, 2021, from https://www.leanix.net/
en/wiki/ea/solution-architecture

Macmillan Dictionary. (n.d.). Knowledge transfer. In Macmillan Dictionary. Retrieved September 28, 2021, from https://www.
macmillandictionary.com/dictionary/british/knowledge-transfer

McAfee. (n.d.). Tokenization vs. Encryption. McAfee. Retrieved September 28, 2021, from https://www.mcafee.com/enterprise/
en-ca/security-awareness/cloud/tokenization-vs-encryption.html

MIT Libraries. (n.d.). APIs for scholarly resources. MIT Libraries. Retrieved September 27, 2021, from https://libraries.mit.edu/
scholarly/publishing/apis-for-scholarly-resources/

NIST. (2013, November 12). Cybersecurity framework [Text]. NIST. https://www.nist.gov/cyberframework

NIST/SEMATECH. (n.d.). What are confidence intervals? Engineering Statistics Handbook. Retrieved September 28, 2021, from
https://www.itl.nist.gov/div898/handbook/prc/section1/prc14.htm

Perdoo. (n.d.). The ultimate OKR guide. Perdoo. Retrieved September 28, 2021, from https://www.perdoo.com/okr-guide/

Perdoo. (2018, August 21). OKR confidence levels. Perdoo. https://www.perdoo.com/resources/okr-confidence-levels/

RACI Solutions. (n.d.). What is RACI? An introduction. RACI Solutions Blog. Retrieved September 28, 2021, from https://www.
racisolutions.com/blog/what-is-raci-an-introduction

Ranschaert, E. R., Morozov, S., & Algra, P. R. (2019). AI: A glossary of terms. In Artificial Intelligence in Medical Imaging:
Opportunities, Applications and Risks. Springer International Publishing. https://doi.org/10.1007/978-3-319-94878-2

Reeves, S. (2019, September 11). A comprehensive guide to the RASCI/RACI model. GoodCore Blog. https://www.goodcore.
co.uk/blog/a-guide-to-the-raci-rasci-model/

Rehkopf, M. (n.d.). What is continuous integration? Atlassian. Retrieved September 28, 2021, from https://www.atlassian.com/
continuous-delivery/continuous-integration

re:Work. (n.d.). Tool: Grade OKRs. Re:Work. Retrieved September 28, 2021, from https://rework.withgoogle.com/guides/set-
goals-with-okrs/steps/grade-OKRs/

SAGE Research Methods. (2017). Data trimming. In M. Allen (Ed.), The SAGE Encyclopedia of Communication Research
Methods. SAGE Publications, Inc. https://doi.org/10.4135/9781483381411.n130

Schwaber, K., & Sutherland, J. (2020). The scrum guide. ScrumGuides.org. https://scrumguides.org/docs/scrumguide/
v2020/2020-Scrum-Guide-US.pdf

Scrum.org. (n.d.). What is a sprint in scrum? Scrum.Org. Retrieved September 28, 2021, from https://www.scrum.org/
resources/what-is-a-sprint-in-scrum

Shah, T. (2017, December 6). About train, validation and test sets in machine learning. Towards Data Science. https://
towardsdatascience.com/train-validation-and-test-sets-72cb40cba9e7

Stackpole, B. (2020, May 20). The next chapter in analytics: Data storytelling. MIT Sloan. https://mitsloan.mit.edu/ideas-made-
to-matter/next-chapter-analytics-data-storytelling

Study.com. (n.d.). What is a use case? Study.Com. Retrieved September 28, 2021, from https://study.com/academy/lesson/
what-is-a-use-case-definition-examples.html

AI Competency Framework 53
Techopedia. (2017, January 24). Data quality. Techopedia. http://www.techopedia.com/definition/14653/data-quality

Techopedia. (2020a, March 31). Waterfall model. Techopedia.Com. http://www.techopedia.com/definition/14025/waterfall-

model

Techopedia. (2020b, August 24). Data integrity. Techopedia. http://www.techopedia.com/definition/811/data-integrity-

databases

The Center for Leadership Studies. (n.d.). Situational leadership. The Center for Leadership Studies. Retrieved September 28,
2021, from https://situational.com/situational-leadership/

The Open Group. (n.d.). TOGAF: Frequently asked questions. Retrieved September 27, 2021, from http://www.opengroup.org/
public/arch/p1/togaf_faq.htm

University of Toledo. (n.d.). Business model canvas. University of Toledo. Retrieved September 27, 2021, from https://www.
utoledo.edu/rocketinnovations/entrepreneur-starter-kit/business-model-canvas.html

U.S. Department of Labor. (n.d.). Guidance on the protection of personal identifiable information. Retrieved September 28, 2021,
from https://www.dol.gov/general/ppii

Usability.gov. (n.d.). Personas. Usability.Gov; Department of Health and Human Services. Retrieved September 28, 2021, from
https://www.usability.gov/how-to-and-tools/methods/personas.html

USGS. (n.d.). What is high performance computing? USGS: Science for a Changing World. Retrieved September 28, 2021, from
https://www.usgs.gov/core-science-systems/sas/arc/about/what-high-performance-computing

White, S. K. (2018, September 4). What is OKR? A goal-setting framework for thinking big. CIO. https://www.cio.com/
article/3302036/okr-objectives-and-key-results-defined.html

AI Competency Framework 54
Design by Raha Jahromi-Zadeh