In collaboration with

the Markkula Center for Applied Ethics

at Santa Clara University, USA

Responsible Use
of Technology:
The Microsoft Case Study
WHITE PAPER
FEBRUARY 2021
 Cover: Getty Images/Nikolay Pandev

Inside: Unsplash/Florian Krumm; Getty Images/Piranka

Contents
3 Foreword

4 1 Introduction and challenge

5 2 The Microsoft experience

6 3 How to change a culture

6 3.1 Governance

7 4 Microsoft AI principles

8 5 Responsible AI Standard and processes

9 6 Tools for responsible innovation

9 6.1 Judgment call

10 6.2 Envision AI workshop

10 6.3 Impact assessment

11 6.4 Community Jury

11 6.5 Machine learning tools with ethical impact

15 7 Measuring culture change

16 8 Analysis of best practices

17 9 What remains to be done

18 Conclusion

18 Box 1 Project Tokyo: Applying responsible innovation practices

20 Contributors

21 Endnotes

© 2021 World Economic Forum. All rights

reserved. No part of this publication may
be reproduced or transmitted in any form
or by any means, including photocopying
and recording, or by any information
storage and retrieval system.

Responsible Use of Technology: The Microsoft Case Study 2

February 2021 Responsible Use of Technology:
The Microsoft Case Study

Foreword

Brian Green, Don Heider, Kay Firth-Butterfield, Daniel Lim,

Director, Technology Executive Director, Markkula Head, Artificial Intelligence Project Fellow, Artificial
Ethics, Markkula Center for Center for Applied Ethics, and Machine Learning; Intelligence and Machine
Applied Ethics, Santa Clara Santa Clara University, USA Member of the Executive Learning, World Economic
University, USA Committee, World Economic Forum LLC; Seconded
Forum LLC from Salesforce

The World Economic Forum Centre for the Fourth intention–action gap, we aim to provide leaders
Industrial Revolution was launched in 2017 with with practical tools for how they might: 1) educate
the mandate to co-create policy and governance and train their employees to think more about
frameworks through a multistakeholder approach to responsible technology; 2) design their organization
accelerate the adoption of emerging technologies. to promote more ethical behaviour and outcomes;
The Centre’s platforms include areas such and 3) design and develop more responsible
as artificial intelligence and machine learning, technology products.
blockchain, data policy and internet of things. At
the heart of this work is the drive towards action, It is with this last goal in mind that the World
transparency, ethics and global public good. Economic Forum and the Markkula Center for
Applied Ethics at Santa Clara University publish
Today, the global Coronavirus pandemic continues this White Paper, the first in a series of case
to cast a dark shadow over all facets of society. studies highlighting tools and processes that
As social distancing measures became a necessity facilitate responsible technology product design
to preserve public health, digital transformation and development. This initial document on the
became a requirement for most businesses to “Responsible Use of Technology: The Microsoft
simply survive. The urgency to maximize the Case Study” will be followed by other companies’
benefits of technology, while mitigating the examples of ethical practices and tools in future
risks and harms, has never been greater. It is papers. We thank Microsoft for having shared
incumbent on all organizations that design, develop, their responsible innovation tools, practices and
procure, deploy and use technology to do so in a expertise for this effort. It is our hope that this
responsible manner. document will inspire others to contribute to the
Forum’s Responsible Use of Technology project by
In our numerous conversations with leaders sharing tools and methods that businesses have
across the various sectors, we’ve learned that a created for the same purpose.
gap exists between organizations’ desire to act
ethically and their understanding of how to follow To achieve these ambitious goals requires the
through on their good intentions. We refer to this collaboration of all global stakeholders. The World
as an intention–action gap. To this end, the Centre Economic Forum, committed to improving the state
is focused on providing practical resources for of the world, is the International Organization for
organizations to operationalize ethics in technology. Public-Private Cooperation. The Markkula Center
This initiative, which began in 2019, with active for Applied Ethics at Santa Clara University in
participation from civil society, governments and California, a key partner in this project, has over
companies, made the case for both human- 30 years’ history and experience in promoting
rights-based and ethics-based approaches to the ethical deliberations. Together, we are pleased to
responsible use of technology. To help bridge this collaborate towards this ambitious vision.

Responsible Use of Technology: The Microsoft Case Study 3

1 Introduction and
challenge
Our society is undergoing a Fourth Industrial The effort to operationalize ethics in technology for
Revolution,1 in which powerful technologies such any organization should be a continuous journey
as artificial intelligence (AI), the internet of things with companies always looking for ways to improve.
(IoT) and augmented reality have the potential to Microsoft’s experience is no exception. The purpose
create lasting societal benefits. However, without of this paper and this series, in the context of the
the right guard rails, these technologies can also World Economic Forum’s Responsible Use of
cause immense harm. Organizations have a Technology project, is to surface lessons that can
responsibility to design, develop, procure, deploy help organizations advance their own responsible
and use technologies in a responsible manner. As innovation practices. The project aims to identify
an initial step towards more responsible innovation, areas for improvement in tools and processes
dozens of groups from industry, civil society and that can help drive more ethical considerations in
government have published ethical technology technology product development. It may even inspire
principles, particularly on AI.2 These values and others who have created new methods for this cause
frameworks offer a foundation for what responsible to share their work, either in this series or elsewhere.
technology innovation outcomes might look like. Yet
the need to define how technology products can be
ethically made remains.3

This paper is the first in a series of case studies

that investigate how companies have begun to
incorporate ethical thinking into the development of
technology. It focuses on Microsoft Corporation and
will be followed by papers describing efforts in other
companies. Through a series of interviews with
Microsoft executives and employees, combined
with secondary research, this paper presents an
outside perspective of Microsoft’s current and
evolving efforts to build on the ethical values and
culture of the company with tools and practices in
its product engineering organization.

Responsible Use of Technology: The Microsoft Case Study 4

2 The Microsoft
experience
When Satya Nadella became Chief Executive understanding. The hope was that the more Tay
Officer of Microsoft in 2014, he brought with him interacted with humans in conversations on Twitter,
the concept of the “growth mindset”. This concept the “smarter” the AI-powered bot would get.
originated from the work of Stanford University However, soon after its debut, a group of users
professor Carol Dweck, whose research led her to maliciously targeted Tay, causing Tay to respond
discover the powerful idea of mindsets. Dweck’s with inappropriate and denigrating responses.
work demonstrates that success in human Microsoft quickly withdrew Tay from the public
endeavours can be highly influenced by how one and issued an apology.5 This episode, as well as
thinks. Someone who has a fixed mindset is less others, was pivotal in motivating the organization to
likely to succeed than someone with a growth incorporate ethical considerations into its product
mindset.4 From the beginning of his tenure as innovation process.
CEO, Nadella instilled the growth mindset into
Microsoft’s culture. This mindset encourages Technologies are powerful, and that power can
curiosity, experimentation, hard work and learning, be used for good or for ill. The designers of new
and repeating the process. This evolution in technologies must therefore think carefully about
Microsoft’s culture created a corporate environment how technologies are built and help to increase
that promoted innovation and introspection on the the likelihood that they will be used for the benefit
impact of technology on society. of society. Microsoft is one company that is taking
steps to develop quality products that empower
At every level, people at Microsoft already encounter customers to achieve more in a way that also allows
and respond to ethical questions in their work. One them to be used responsibly.
notable example is the release of a chatbot named
Tay in 2016, an experiment on conversational

Responsible Use of Technology: The Microsoft Case Study 5

3 How to change a culture
Driven by a commitment to strengthen trust with Tense” describes it well: this decision to make socially
customers and society more broadly, the leadership aware and ethical technology is not merely about
and employees of Microsoft have taken a values- profit but about the world created for humans to live
based approach to building and deploying technology in. Technology developers are not only technologists.
responsibly. While this approach is certainly more They are human beings who must live in the world
demanding in terms of time, money and effort, the that technology creates. Keeping this fact in the
company believes it has the best return on investment foreground ensures that the bigger issues of the social
over the long term. and environmental impact of technology remain close
at hand, and directs technologists towards developing
What motivated Microsoft to take this approach more humane technologies that can truly benefit the
to solving this challenge? Satya Nadella’s 2016 common good of all humankind and the planet.
“Partnership of the Future”6 article in Slate’s “Future

Building trust in technology is crucial … it starts with us taking

accountability … for the algorithms we create, the experiences we
create, and ensuring there is more trust in technology with each day.
Satya Nadella, Chief Executive Officer, Microsoft7

Importantly, Microsoft’s choice to emphasize and mitigated, while maximizing benefits to

ethics was not only a top-down decision but one society. The building blocks for this responsible
that emerged within the company at many levels. AI programme include a governance structure to
The company’s approach, which is based on its ensure accountability and enable progress; a set
AI principles (see the next section), focuses on of rules defining its Responsible AI Standard; and
proactively establishing guard rails for AI systems training, tools and practices that allow employees to
that will ensure that potential risks are anticipated operationalize the principles and rules.

3.1 Governance

Microsoft’s AI governance approach follows a to promote awareness and education on a given

“hub-and-spoke” model that helps the company subject. The Responsible AI Champs programme
integrate privacy, security and accessibility into its was implemented in 2020 and includes experts in the
products and services. Three teams play critical areas of security and open source, among others.
roles in this governance approach. First, the Champs bring attention to the available responsible
Aether Committee comprises working groups of AI tools and processes. They also help teams identify
scientific and engineering experts to advise on and consider ethical and societal issues in their work.
responsible AI issues and the enactment of the They have been crucial for leading a culture shift
company’s AI principles. Second, the Office of towards thinking more about ethics.8
Responsible AI manages the policy, governance,
enablement and sensitive uses functions. Third, the Another important spoke in the governance approach
Responsible AI Strategy in Engineering (RAISE) team is Ethics & Society, a team within RAISE tasked
enables Microsoft’s engineer groups to implement with taking a design thinking approach to ethics. It
responsible AI processes through the adoption of has played a key role in the creation of some of the
systems and tools. responsible innovation tools described in this paper.9

As for the “spokes” in this governance approach, Shifting the culture of a corporation is a monumental
Microsoft has found consistent success in deploying a feat, but it is certainly not impossible, and the
“Champs” model, in which respected domain experts resources in industry, civil society and academia for
across teams and regions are appointed by leadership thinking about these efforts are excellent.10

The most critical next step in our pursuit of AI is to agree on an

ethical and empathic framework for its design.
Satya Nadella, Chief Executive Officer, Microsoft11

Responsible Use of Technology: The Microsoft Case Study 6

4 Microsoft AI principles
To promote a culture shift and infuse its AI work with ethical awareness, Microsoft developed six
overarching ethical principles for AI: fairness, reliability and safety, privacy and security, inclusiveness,
transparency and accountability.12

Fairness Reliability and safety Privacy and security

AI systems should treat AI systems should perform AI systems should be secure

people fairly reliably and safely and respect privacy

Inclusiveness Transparency Accountability

AI systems should empower AI systems should be People should be

and engage people understandable accountable for AI systems

These six principles act as a mental tool or framework in which to organize thinking about ethics at Microsoft.
While specifically phrased as responsible AI principles, they have relevance for much of the work in the
technology industry. A brief explanation of each principle follows:

Fairness Transparency
Focuses on developing systems that treat Seeks to create technology that is intelligible and
everyone in a fair and balanced way. This principle explainable, not only to those who are developing
acknowledges that defining and mitigating fairness the technology but also to those who will be
issues for a system depend on understanding the using it, or will be affected by it. Stakeholders
system’s purpose and context of use, and that a should be able to interpret and understand what
system’s fairness reflects decision-making during a technology is doing and why it is acting that
both development and deployment. way. This allows product teams to contextualize
and improve results.
Reliability and safety
Means developing systems that are robust and Accountability
capable of maintaining safe operations even in Means that people take responsibility for the way
worst-case scenarios. This principle encompasses technology operates and for the impact of that
consideration of the harms that might come from operation on society. This includes considering
a technology, and ways employees can strive to the structures that can be implemented to ensure
minimize those risks, so technologies can give the accountability at multiple levels, including design,
greatest benefits to their users. development, sales, marketing and use, as well
as advocacy for the regulation of technologies
Privacy and security when warranted.
Seeks to protect data and use data in a way that
is secure for all stakeholders. Privacy is a basic Simply having principles does not change a culture
right and protecting it is crucial for ensuring that unless those principles are made concrete through
stakeholders can trust companies with their data. tools and practices that help employees work
Data must be secure at all stages and, to further through how to think ethically. To advance these
this end, actions must be taken to institutionalize principles and make sure they are implemented
privacy and security for the data companies are into the company’s workflows, Microsoft
responsible for. developed several tools for incorporating applied
ethics in technology. All of these tools serve an
Inclusiveness ethical end; some are more procedural, while
Makes sure that no one is left out of the design, others are more technical in nature.
development, deployment and use of technology.
Communities across the full spectrum of humanity
should be meaningfully engaged and empowered
by technology, and technology should not be
limited to only a few privileged communities. This
inclusion should not only involve building for, but
building with, the diverse stakeholders.

Responsible Use of Technology: The Microsoft Case Study 7

5 Responsible AI
Standard and processes
With the goal of turning principles into practice, will reinforce a human-centred approach, as well
Microsoft created the Responsible AI Standard in as strong engineering and research foundations.
2019 as part of a process of learning how best to For each requirement in the 2.0 Standard,
enact Microsoft’s AI principles across the company. Microsoft is building implementation methods
The standard outlines a set of steps that teams that teams can follow to support the creation of
must follow to support the design and development responsible AI systems.
of responsible AI systems. A key part of the
standard is a set of responsible AI considerations One of the Responsible AI Standard’s requirements
with examples that guide teams through the AI is for sensitive use cases that meet predefined
system development life cycle. The standard has review criteria to be reported and escalated to the
been piloted with 10 internal engineering groups Office of Responsible AI. If there is no previous
and two customer-facing groups. Although teams precedent to draw upon, members of that office
found the responsible AI considerations and participate in a process of deliberation.
examples helpful, they requested more specificity
on the requirements and criteria they could apply to In 2019, Microsoft also created the Introduction
their situations. Based on this feedback, version 2.0 to Responsible AI training course for staff, which
of the standard was developed and is now being covers the sensitive use process, the Responsible AI
previewed with employees ahead of its roll-out Standard and the foundations of its AI principles. The
across the company more broadly. This version training is now mandatory for all employees.

Responsible Use of Technology: The Microsoft Case Study 8

 6 Tools for responsible
innovation
Technology ethics is fundamentally an applied, at Microsoft’s Responsible AI Resource Center.
practical discipline. It is not about theory, although Together, institutionalized into a structure and applied
theory informs practice. Technology ethics, as a systematically, they can help to improve products
practical pursuit, benefits from having very specific and prevent the potentially damaging effects of
and applicable tools to aid thinking, analysis, technologies. Applying these tools, regularly and
stakeholder engagement and decision-making. The routinely, can help to shift a culture, especially
tools presented here – Judgment Call, Envision AI in conjunction with techniques such as securing
workshops, impact assessments, Community Jury, leadership buy-in, establishing prosocial norms and
Fairlearn, InterpretML and the error terrain analysis implementing ethical reminders.13 These responsible
– each address a specific aspect of the ethics of innovation tools have been applied to improve
technology development. Most of these tools, Microsoft technologies, such as spatial analysis,
guidelines and resources are available to the public speech consent and Custom Neural Voice.14

6.1 Judgment call

Source: Microsoft

To help cultivate empathy during its product The game has a number of benefits:
creation process, Microsoft’s Ethics & Society
team created the Judgment Call game.15 The – Engineers, product managers, designers
game is an interactive team-based activity that and technology executives consider the
puts Microsoft’s AI principles of fairness, privacy perspectives of the impacted stakeholders and
and security, reliability and safety, transparency, imagine the potential outcomes of their product
inclusiveness and accountability into action. on these stakeholders.
During the game, each participant is given a
card that assigns them a role as an impacted – Although the game does not replace the
stakeholder of a digital product (e.g. product valuable benefits of interacting directly with
manager, engineer, consumer). Each is also stakeholders, it builds empathy, especially early
given a card that represents one of Microsoft’s in the product design process.
AI principles, and a card with a number from 1
to 5, representing the stars in a ratings review. – Roles are arbitrarily assigned to participants
Participants are asked to write a review of the due to the random distribution of the cards. The
digital product from the perspective of their game’s dynamics create a safe environment for
assigned role, principle and rating number. Each product team members to discuss potentially
player is asked to share and discuss their review. sensitive ethical topics.

Responsible Use of Technology: The Microsoft Case Study 9

 6.2 Envision AI workshop

Source: Microsoft

Developed by the Project Tokyo team (see the Tokyo. They learn the human-centric design
Project Tokyo text box below) from Microsoft approach to AI and the resources available to them
Research, Engineering Learning & Insights and to identify the potential effects of the technology on
the Office of Responsible AI, the Envision AI the stakeholders. Participants apply these lessons
workshop is an exercise that educates Microsoft to completing an impact assessment. The Envision
teams on how to conduct an impact assessment, AI workshop helps Microsoft empower teams to
a process required in the Responsible AI Standard. conduct ethical deliberations on their own and take
In an interactive and engaging setting, Envision AI responsibility for the implications of the products
participants examine real scenarios that occurred they create.
while developing the assistive AI system in Project

6.3 Impact assessment

Conducting an impact assessment is a required assessments are reviewed by peers and executives
step in the product development process of all AI at the company. This process is facilitated and
projects at Microsoft. Teams complete an extensive required by the company’s Office of Responsible
questionnaire, which takes into account the AI. It serves as an important tool to help ensure
intended use cases of a product and its potential the responsible development and deployment of AI
impacts on stakeholders, and a self-assessment across Microsoft.
of the potential risks. The completed impact

Responsible Use of Technology: The Microsoft Case Study 10

 6.4 Community Jury

Source: Center for New

Democratic Processes, USA

Community Jury is a technique that allows moderator is important to allow every voice to be
project teams to directly interact with impacted heard. Moderators need to provide ample time for
stakeholders.16 A group of representative knowledge sharing, deliberation and co-creation.
stakeholders from diverse backgrounds are Finally, it is important to disseminate a report on the
recruited and selected to be jury members. During Community Jury outcome that summarizes the key
a Community Jury session, project teams provide insights for transparency.
the jury members with an overview of the product’s
purpose and its potential use cases, benefits and The benefits of the Community Jury technique from
harms. Participants share information and discuss an ethical perspective are multifaceted. Product
their perspectives of the product’s impacts with the teams and impacted stakeholders are brought
facilitation of a neutral moderator. As key themes together to learn from each other’s perspectives.
emerge from the discussion, the participants jointly The proximity and connection helps build
define the opportunities and challenges presented community and empathy. Especially for product
by the technology. This process can also lead to teams, stakeholder engagements raise their
co-created solutions. awareness of issues otherwise not readily apparent
when the new technologies were conceived.
The planning process starts by aligning goals and This process can build consensus among teams
outcomes with the project teams. It is important and their communities on the challenges and
that product teams allocate time in the product opportunities that technological innovations may
development process to conduct Community Jury pose. It also presents a vital opportunity for teams
sessions. Based on the project objectives, jury to improve their products and solutions to benefit a
recruitment and selection should be diverse and larger group of stakeholders.
inclusive. Strong session facilitation by a neutral

6.5 Machine learning tools with ethical impact

Some of Microsoft’s tools for considering ethics are and researchers to help them assess and improve
technical devices to understand, assess and mitigate fairness in machine learning.17 Fairlearn has two
the ethical risks of machine learning models. They main components: 1) a set of fairness assessment
serve multiple ethical AI principles – namely, that metrics and an interactive data visualization
AI must be fair, reliable, inclusive, transparent and dashboard, which provide an understanding of
accountable. These software tools are constantly how particular groups may be adversely affected
being developed, refined and changed. by models (this allows a comparison of fairness
and performance metrics between models); and
Fairlearn 2) unfairness mitigation algorithms for a variety of
Fairlearn is an open-source toolkit designed for AI tasks, as well as definitions of fairness to allow
data scientists, developers, business stakeholders deeper thought in this context.18

Responsible Use of Technology: The Microsoft Case Study 11

 FIGURE 1 Fairlearn tool dashboard

Source: Microsoft

Fairlearn’s creators note that the reasons models InterpretML

can behave unfairly are many, including social, Microsoft developed the InterpretML open-source
technical and combined sources of unfairness.19 toolkit to make machine learning models more
Fairlearn focuses on the negative impacts of models transparent, intelligible and interpretable.22 It can
on groups of people, such as those defined in help to provide both “global” explanations about
terms of race, gender, age or disability status. It overall model behaviour and “local” explanations for
contains understanding and mitigation tools to individual model predictions. InterpretML contains
assess quality of service, such as, for example, “glass box” (or inherently explainable) models,
whether a voice recognition system works as well including explainable boosting machines and
for one group as for another. The tools can also decision trees, as well as a number of tools that
assess allocation, for instance whether a system help to explain “black box” models. The toolkit also
that identifies job applicants unfairly recommends supports “what-if” explanations for model outputs;
applicants of one ethnicity over another. most recently, it also added “diverse counterfactual
explanations”, which compute the most similar data
Since fairness is a deeply sociotechnical concept, instances that have received different prediction
using Fairlearn does not guarantee that a model outcomes. A counterfactual analysis generates
can be made perfectly fair. It only promises to help explanations for individual outputs or predictions by
identify and mitigate fairness issues involving harms identifying the smallest change to the input features
of allocation and harms in quality of service. The that would cause the model or system to produce a
authors of the tool rightly note that perfect fairness desired output or prediction.
is not possible, not only because of the nature of
data from the real world but also because of the Transparent, intelligible and interpretable models
theoretical incompatibilities of some definitions are desirable because they make models much
of fairness. Importantly, Fairlearn is specifically easier to understand and debug. They also
aimed at mitigating fairness-related harms towards provide advantages that include making the model
protected groups (e.g. different ethnicities).20 In one easier to explain to others, such as end users,
case, Fairlearn significantly improved fairness for helping to discover sources of fairness issues and
loan decisions.21 clarifying options for informing unfairness mitigation
techniques, and making models clear for compliance
with relevant regulatory obligations.

Responsible Use of Technology: The Microsoft Case Study 12

 These technical improvements should Overall, making machine learning more transparent
be recognized for what they are: ethical helps to make it more trustworthy, and builds trust
improvements. For example, buggy software and reputability.
can be potentially dangerous and harmful in
numerous ways. These harms are ethically Error terrain analysis for machine learning
significant, and being able to remove them The error terrain analysis for machine learning tool
ultimately contributes to a more ethical product. (often called simply the Error Analysis tool) is similar
The opportunity to discover and mitigate the to InterpretML in that it works to debug machine
fairness issues of a model is also a significant learning models. However, its specific role is to
ethical benefit, both for recognizing where past help debug exactly those classifications within a
wrongs may have been systemic (and, for these, model that might be subject to errors.23 The tool
a historical wrongs recompense may be in order) enables data scientists to identify cohorts of data
and for mitigating these wrongs. And prioritizing with high error rates versus the benchmark rate
the clear and timely fulfilment of regulatory and to visualize how the error rate distributes. Via
obligations can greatly simplify processes in the integration with interpretability techniques and
long term, saving labour and money that could be visualizations, users can further diagnose the root
better directed elsewhere. causes of the errors by gaining deep insights into
the data or model.

FIGURE 2 Error Analysis tool data displays

Source: Tool images provided by Microsoft

Responsible Use of Technology: The Microsoft Case Study 13

The Error Analysis tool solves a core technical As with InterpretML, the Error Analysis tool could
challenge in machine learning, which is to be construed as simply a technical fix in order to
determine exactly where a model misclassification gain accuracy in machine learning development.
occurs. For example, in a dataset of faces for a But in certain cases, such as facial recognition,
facial recognition model, the tool might discover failures in classification are ethically significant.
that skin tone correlates with misclassification of For example, inaccuracies with respect to race
sex, or that beards or hair colour are a cause of and sex can both reflect deep fairness issues in
various other misclassifications. By pinpointing society and perpetuate injustice at a systemic
exactly where these misclassifications occur, level, not to mention their being offensive. And
developers can determine how to fix the model to other sorts of inaccuracies in processing visual
improve its accuracy. data – for example with autonomous vehicles –
could cause safety concerns.
The Error Analysis tool enables machine learning
practitioners to efficiently analyse and debug Once again, technical sophistication and ethics
machine learning models’ errors to accelerate go hand in hand. Ethical mistakes caused by
improvements in model accuracy and reliability. technical errors can be prevented by eliminating
They can discover new insights about relationships these errors from products. As one Microsoft
between data features and model performance, interviewee stated, “We don’t use our customers
dive deeper beyond overall performance metrics as testers!” Instead, the company strives to test
by performing a disaggregated evaluation, and and adjust a product thoroughly before it ever
discover how errors are distributed in various gets to the customer (though, of course, it does
cohorts. They can further leverage this knowledge receive customer feedback on improvements that
to deploy mitigation techniques and improve the remain to be made, in the form of comments,
model or the data. press, error reports, etc.).

Responsible Use of Technology: The Microsoft Case Study 14

7 Measuring culture
change
Periodic goal setting and performance evaluations to write coherent, individualized core priority
are essential exercises to build alignment and statements and implement the company’s
measure progress at any organization. Microsoft has responsible AI work successfully.
extended these organizational techniques to include
considerations of ethics and responsible behaviours. Microsoft’s Ethics & Society team learned some
important lessons from this effort. First, for over
In 2018, a product team under the leadership three months, every person on the team was
of Alex Kipman (inventor of Kinect and the tapped to participate in the delivery of weekly
HoloLens) was working on reducing error rates workshops and informational sessions. Their
in facial recognition technology at Microsoft. purpose was to help each individual employee
Inspired by this work, Kipman’s colleagues in the understand why the new responsible AI
Ethics & Society team suggested to management commitment existed and how it was related to
that the facial recognition project team add a their day-to-day role in the company. The team
“core priority” related to reducing bias to each discovered that some employees whose work
team member’s annual performance goal. They focused on the “bottom of the tech stack” (e.g.
also suggested that the team commit to external on hardware or drivers) struggled to understand
audits. This biannual goal setting and individual how “ethics” and responsible AI were relevant
performance review exercise at Microsoft is to their work. The Ethics & Society team helped
called a “Connect”. Kipman championed this these employees use the new core priority in their
idea, expanding the practice beyond his facial performance review as a mechanism to prioritize
recognition team to his entire organization, which quality and take the time to ensure those further
included all the AI cognitive services and mixed up the stack can use the tech responsibly. It was
reality teams at Microsoft. He supported an much easier to set core priorities and success
18-month culture change process to incorporate indicators with ethical considerations for people
this new responsible AI and ethics core priority who worked on AI that has a more directly
into each employee’s performance review. perceivable social impact.
Every employee was evaluated twice a year,
in part on their responsible AI commitments The Ethics & Society team also learned that
and how they implemented them. Because this managers need to have an evaluation framework
was a significant new individual requirement so they can measure an employee’s performance
and organizational commitment, a virtual team for promotions and bonuses. Finally, the team
was formed with representation from across discovered that when upper management turned
Kipman’s organization (e.g. Human Resources, their own responsible AI core priorities from their own
Engineering, Communications) to work on a individual performance reviews into organization-wide
culture change plan and roll-out strategy. The Objectives and Key Results (OKRs) on responsible
Ethics & Society team, which initiated this AI, organizational change accelerated.
programme, launched an educational initiative
consisting of workshops, informational sessions, This initial pilot to implement a common
presentations and pamphlets with the goal of responsible AI core priority across Alex Kipman’s
educating Kipman’s 1,400-person team on how organization was seen as a success within Microsoft.

Responsible Use of Technology: The Microsoft Case Study 15

 8 Analysis of best practices
The Markkula Center for Applied Ethics has been at the forefront of applied ethics for over 30 years. Among its
contributions to technology ethics are its Ethics in Technology Practice materials, which include an analysis of
Best Ethical Practices in Technology.24 In practice, Microsoft has implemented or begun to implement most of
them. For example, Microsoft’s Responsible AI Principles directly connect to six of the best practices.

FIGURE 3 Markkula Center for Applied Ethics best practices in technology applied by Microsoft

Microsoft Markkula Center for Applied Ethics

Responsible AI Principles Best practices

Fairness aligns with 12. Consider disparate interests, resources
and impacts
Reliability and safety connects to 10. Practice disaster planning and crisis
response
Privacy and security matches with 13. Design for privacy and security
Inclusiveness connects to 14. Invite diverse stakeholder input
Transparency aligns with 11. Promote the values of autonomy,
Source: World Economic
transparency and trustworthiness
Forum and Markkula Center
for Applied Ethics at Santa
Accountability directly relates to 08. Establish chains of ethical responsibility
Clara University and accountability

By turning best practices into principles, Microsoft Envision the technical ecosystem: The name
assures that they are highly visible and likely to “Ethics & Society” hints at Microsoft’s recognition
contribute to and influence ethical conversations. that it is part of a sociotechnical ecosystem. The
company’s willingness to share its responsible product
Other Markkula Center best practices that Microsoft innovation tools publicly is also a strong indication of
has incorporated include: its desire to contribute to the common good.

Keep ethics in the spotlight: The formation of the Treat technology as a conditional good: By
Ethics & Society team in 2017 and the expansion of choosing to develop and release some technologies
its activities (e.g. the goal to create Responsible AI but not others,25 Microsoft shows that it believes
core priorities for every Cloud + AI team member) technology is not an unconditional good. Not all
illustrate that Microsoft has moved beyond technologies ought to exist. Rather, the technologies
compliance towards inspiring culture change. that should exist are those that help people and
have positive social impact, while others should be
Highlight the human lives and interests behind selected against, regulated or perhaps even banned.
the technology: Microsoft’s user-first approach
to product development demonstrates a focus on Make ethical reflection and practice standard,
people rather than on technology. Ethical tools, pervasive, iterative and rewarding: As Microsoft
such as impact assessments, Judgment Call and rolls out ethical practices across its organization,
Community Jury institutionalize this focus as well. it is institutionalizing its ethical tools and scaling
these resources to increasingly large groups. Ethical
Consider downstream (and upstream and lateral) reflection is becoming more common through the
risks for technologies: All of Microsoft’s ethical Responsible AI Champs programme, company-
tools are centred on considering and mitigating wide education training and RAISE activation
ethical risks. Even the more technical tools, Fairlearn, and scaling, and more rewarding through the
InterpretML and the error terrain analysis, serve this implementation of responsible AI OKRs.
function by keeping an eye on the risks of bias and
other problems in machine learning. Model and advocate for ethical tech practice:
By being an industry leader in integrating ethical
Do not discount non-technical actors, interests thinking into its product life cycle and through
and expectations: Microsoft’s Community Jury leadership in supporting organizations, Microsoft has
exercise is specially designed to bring diverse acted to model and advocate for ethical practices in
community voices into the product development effort. technology. Its willingness to share some of the tools
and practices is also indicative of this best practice.

Responsible Use of Technology: The Microsoft Case Study 16

9 What remains to be done
Microsoft has made a commitment to responsible Microsoft is a partner in the World Economic
innovation, ethics and trustworthiness. And yet, Forum Responsible Use of Technology project. Its
while the scope of this commitment could be commitment is also reflected in its participation and
limited solely to internal improvements, Microsoft leadership in the Partnership on AI to Benefit People
has chosen to do more. Employees of the and Society and the Vatican’s Rome Call for AI Ethics,
company’s various ethics functions consistently among other initiatives. Technology does not exist in
state their hope that the entire technology a vacuum or apart from society. Technology always
industry will advance its ethical commitment. In affects real people and human institutions. As human
this way, the company acknowledges that ethics technological power grows, the impacts of these
is not a zero-sum game. Ethics seek to make powerful technologies should be carefully considered
the world a better place for everyone, and the and controlled before the effects occur. These
more people and organizations strive to become technological impacts, as well as the preparations,
and act ethically, the better it is for everyone. controls and considerations designed to govern
Microsoft’s commitment sets an example for the them, are inherently ethically charged. Therefore,
technology industry as a whole – that technology technology companies must consider ethics as a part
should be designed, developed, deployed of their business. And because all companies are now
and used ethically. The tools it has developed technology companies, all companies should think
and shared publicly are only the beginning for more closely about how technology ethics is involved
Microsoft and, it hopes, the world. in their work.

As AI continues to rapidly evolve, it’s critical that we think

carefully about the complex task of building and using it
responsibly. This is a never-ending journey. As we develop new
technology, we must preserve timeless values.
Brad Smith, President, Microsoft26

Even with the steps that Microsoft has taken investigating expanding ethical tools and processes
to operationalize ethics in recent years, it is to the entire organization. As with any major project,
not immune to regulatory and public scrutiny, the company will proceed in phases, the specifics
especially as legal frameworks continue to evolve of which are being developed.
to satisfy public sentiments. For example, the
concerns raised by European regulators about the Many other companies in the technology industry
privacy policy and practices in Office 365 are well are also pursuing efforts to institutionalize ethical
documented.27 In response, Microsoft updated thinking in the product development process.
its Online Services Terms for commercial cloud Some of these companies will be part of the World
customers.28 But these issues are not unique Economic Forum’s series of case studies on the
to Microsoft alone. The company has plans for Responsible Use of Technology.
future developments in the area of ethics and is

Responsible Use of Technology: The Microsoft Case Study 17

 Conclusion
The rapid adoption of Fourth Industrial Revolution technologies is permeating every aspect of society.
Simultaneously, the world is navigating through a global pandemic that is causing stress while creating
opportunities for change. As global stakeholders cooperate to manage the direct consequences of the crisis,
designing, developing, distributing, deploying and using technology responsibly are of paramount importance.

Making technology ethical will require efforts not only from technology companies but from many types of
organizations worldwide. The World Economic Forum and the Markkula Center for Applied Ethics at Santa
Clara University share Microsoft’s journey in this endeavour in an effort to inspire and enable organizations
with similar intentions to benefit from its experience. This case study aims to promote discussion, critiques,
as well as efforts to build upon Microsoft’s work. The World Economic Forum and its partners in this project
hope more organizations not only operationalize ethics in their use of technology but also share their own
experience with the global community.

BOX 1 Project Tokyo: Applying responsible innovation practices

In Cambridge, United Kingdom, a 12-year-old boy what is possible, the team learned that people who
named Theo is sitting in a friend’s kitchen wearing are blind or have low vision are extremely skilled at
a modified Microsoft HoloLens headset. Theo is using their senses to determine what is happening
blind. When he turns his head to face a person in around them. This is called sense-making. Yet,
the room, the name of the person is played in his with less redundant information, there is a great
headset along with a bump sound. This is a Microsoft deal of uncertainty about whether their judgements
research prototype that uses artificial intelligence and are accurate. That is where technology can come
augmented reality to assist people who are blind or in. By providing additional information, it can help
have low vision, code named Project Tokyo.29 blind people and those with low vision feel more
confident in their nuanced skills about making
In 2016 a team from Microsoft Research, led by
sense of the world in their own way.
Cecily Morrison, set out to explore explore how AI
that uses computer vision could extend people’s
The research team discovered that social
capabilities rather than replace them. As early
information was the hardest for their participants to
adopters of AI technologies, the team worked
gather and the most important to them. A research
alongside people who are blind or have low vision to
project led the team to focus on enabling the social
innovate. According to Morrison, “our team wanted
to imagine a future with AI that enabled people to agency of blind children in schools, helping them
extend their own capabilities, using a human-centric understand users in their immediate vicinity.
approach to developing algorithms and interactive
AI experiences. A human-centric approach must The AI system Microsoft Research developed uses
also be a process of responsible innovation, a key a modified HoloLens worn on the user’s head to
pillar of our thinking as we developed the research.” scan a 160-degree field of view. A phone or cloud
server processes images from the sensors to detect
Morrison and her team of researchers began people’s position, gaze, pose and identity. It then
Project Tokyo by observing athletes and spectators communicates this information in spatial audio to
on their trip to the Paralympic games in Brazil. the user, with sound coming from the direction of
From observing those who push the boundaries of the person being identified.30

On the left: Image of the adapted HoloLens device; On the right: Schematic description of the core
functionality of the AI system

Source: Interactions,
“Interpretability as a Dynamic
of Human-AI Interaction”31

Responsible Use of Technology: The Microsoft Case Study 18

 Microsoft researchers made several ethical had enabled people to spontaneously socially
decisions when developing the HoloLens system. negotiate the use of a technology in the moment,
Understanding the tension between inclusiveness avoiding the need for regulation in this context.
and privacy, Morrison and her team decided
to discard image, timestamp and location data Working through the responsible innovation
to avoid misuse by a malicious actor. But that process, the research team changed their mindset
alone was not enough to make social interactions from thinking about the user to thinking about
with a camera comfortable. To ensure a positive interactions between users – a fundamental shift
Sources: Interview with experience for bystanders identified by the system, in perspective that can enable the innovation of
Cecily Morrison, Senior the research team added a LED light strip to products that are thoughtful to the wider social
Researcher, Microsoft the top of the HoloLens. A white light tracks the context in which they will exist and influence.
Research Lab, Cambridge, movement of the nearest bystander. Once the
United Kingdom; Roach, system identifies that person to the user, the LED The Office of Responsible AI at Microsoft is now
John, “Using AI, people who flashes green. When testing the device with Theo using Project Tokyo as a case study to challenge
are blind are able to find and his schoolmates, the children understood this its product teams to think about how engineering
familiar faces in a room”, interaction immediately without explanation, and decisions can impact people and society in ways
Microsoft, Innovation Stories, began to play hide-and-seek. The research team they may not have previously considered.
28 September 2020.

Responsible Use of Technology: The Microsoft Case Study 19

Contributors
Lead contributors

Brian Green
Director, Technology Ethics, Markkula Center for Applied Ethics, Santa Clara University, USA

Daniel Lim
Project Fellow, Artificial Intelligence and Machine Learning, World Economic Forum LLC; Seconded from
Salesforce

Emily Ratté
Project Coordinator, Artificial Intelligence and Machine Learning, World Economic Forum LLC

Acknowledgements

Kathy Baxter
Principal Architect, Ethical AI Practice, Salesforce, USA

Kay Firth-Butterfield
Head, Artificial Intelligence and Machine Learning; Member of the Executive Committee, World Economic
Forum LLC

Steven Mills
Partner and Chief AI Ethics Officer, Boston Consulting Group, USA

Ben Olsen
Lead, Responsible Innovation, Education and Activation, Facebook, USA

Kay Pang
Senior Director and Associate General Counsel, Global Markets Compliance Officer, VMware, USA

Ann Skeet
Senior Director, Leadership Ethics, Markkula Center for Applied Ethics, Santa Clara University, USA

Leila Toplic
Head, Emerging Technologies Initiative, NetHope, USA

Thor Wasbotten
Managing Director, Markkula Center for Applied Ethics, Santa Clara University, USA

Responsible Use of Technology: The Microsoft Case Study 20

 Endnotes
1. Schwab, Klaus, “The Fourth Industrial Revolution: What it means and how to respond”, World Economic Forum, Agenda,
14 January 2016, https://www.weforum.org/agenda/2016/01/the-fourth-industrial-revolution-what-it-means-and-how-to-
respond (accessed10 February 2021).
2. Jobin, Anna, Marcello Ienca and Effy Vayena, “The global landscape of AI ethics guidelines”, Nature Machine Intelligence,
vol. 1, no. 9, 2 September 2019, pp. 389-399.
3. Skeet, Ann, and Jim Guszcza, “How businesses can create an ethical culture in the age of tech”, World Economic Forum,
Agenda, 7 January 2020, https://www.weforum.org/agenda/2020/01/how-businesses-can-create-an-ethical-culture-in-
the-age-of-tech (accessed 10 February 2021).
4. Dweck, Carol, Mindset: The New Psychology of Success, Random House, 2006.
5. Lee, Peter, “Learning from Tay’s introduction”, Official Microsoft Blog, 25 March 2016, https://blogs.microsoft.com/
blog/2016/03/25/learning-tays-introduction (accessed 11 February 2021).
6. Nadella, Satya, “The Partnership of the Future: Microsoft’s CEO explores how humans and A.I. can work together to
solve society’s greatest challenges”, Slate, 28 June 2016, https://slate.com/technology/2016/06/microsoft-ceo-satya-
nadella-humans-and-a-i-can-work-together-to-solve-societys-challenges.html (accessed 11 February 2021).
7. Nadella, Satya, “Build 2017” keynote address, Microsoft Build Conference 2017, Seattle, Washington, 10 May 2017.
8. O’Brien, Tim, Steve Sweetman, Natasha Crampton and Venky Veeraraghavan, “How global tech companies can
champion ethical AI”, World Economic Forum, Agenda, 14 January 2020, https://www.weforum.org/agenda/2020/01/
tech-companies-ethics-responsible-ai-microsoft (accessed 11 February 2021).
9. Lane, Mira, “Responsible Innovation: The Next Wave of Design Thinking”, Medium, Microsoft Design, 19 May 2020,
https://medium.com/microsoft-design/responsible-innovation-the-next-wave-of-design-thinking-86bc9e9a8ae8
(accessed 11 February 2021).
10. World Economic Forum, in collaboration with Deloitte and the Markkula Center for Applied Ethics at Santa Clara
University, “Ethics by Design: An organizational approach to responsible use of technology”, White Paper, December
2020, http://www3.weforum.org/docs/WEF_Ethics_by_Design_2020.pdf (accessed 11 February 2021).
11. Nadella, “The Partnership of the Future”, op. cit.
12. Microsoft, “Responsible AI”, including videos, 2020, https://www.microsoft.com/en-us/ai/responsible-
ai?activetab=pivot1:primaryr6 (accessed 11 February 2021).
13. World Economic Forum, Deloitte, Markkula Center for Applied Ethics, “Ethics by Design”, op. cit.
14. Boyd, Eric, “Build powerful and responsible AI solutions with Azure”, Microsoft, Azure Blog, 22 September 2020, https://
azure.microsoft.com/en-us/blog/build-powerful-and-responsible-ai-solutions-with-azure; Roach, John, “Microsoft gives
users control over their voice clips”, Microsoft, AI Blog, 15 January 2021, https://blogs.microsoft.com/ai/microsoft-gives-
users-control-over-their-voice-clips; Culler, Leah, “Are you talking to me? Azure AI brings iconic characters to life with
Custom Neural Voice”, Microsoft, AI for Business, 3 February 2021, https://blogs.microsoft.com/ai-for-business/custom-
neural-voice-ga (all accessed 11 February 2021).
15. Ballard, Stephanie, Karen Chappell, Mira Lane and Oscar Murillo, “Judgment Call”, Microsoft, 12 May 2020, https://docs.
microsoft.com/en-us/azure/architecture/guide/responsible-innovation/judgmentcall (accessed 11 February 2021).
16. Noah, Ben, Lori Li and Arathi Sethumadhavan, “Community Jury”, Microsoft, 18 May 2020, https://docs.microsoft.com/
en-us/azure/architecture/guide/responsible-innovation/community-jury (accessed 12 February 2021).
17. Bird, Sarah, et al., “Fairlearn: A toolkit for assessing and improving fairness in AI”, Microsoft, May 2020, https://www.
microsoft.com/en-us/research/uploads/prod/2020/05/Fairlearn_WhitePaper-2020-09-22.pdf (accessed 12 February 2021).
18. See Fairlearn [website], “Quickstart”, https://fairlearn.github.io/v0.5.0/quickstart.html (accessed 12 February 2021).
19. Bird, et al., “Fairlearn”, op. cit. pp. 1-2.
20. Ibid., p. 6.
21. Microsoft, “EY Platform uses Fairlearn to help customers gain trust in AI”, 15 May 2020, https://customers.microsoft.com/
en-us/story/809460-ey-partner-professional-services-azure-machine-learning-fairlearn (accessed 12 February 2021).
22. Sameki, Mehrnoosh, et al., “InterpretML: A toolkit for understanding machine learning models”, Microsoft, 18 May 2020,
https://www.microsoft.com/en-us/research/uploads/prod/2020/05/InterpretML-Whitepaper.pdf; for additional resources,
see https://interpret.ml and “interpretml/interpret” on GitHub, https://github.com/interpretml/interpret (all accessed
12 February 2021).
23. Barraza, Rick, et al., “Error terrain analysis for machine learning: Tools and visualizations”, International Conference
on Learning Representations (ICLR), 6 May 2019, https://slideslive.com/38915701/error-terrain-analysis-for-machine-
learning-tool-and-visualizations?ref=speaker-16330-latest; Nushi, Besmira, Ece Kamar and Eric Horvitz, “Towards
Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure”, Association for the Advancement
of Artificial Intelligence, 2018, https://besmiranushi.com/docs/accountable_AI_hcomp_2018.pdf (both accessed
12 February 2021).

Responsible Use of Technology: The Microsoft Case Study 21

24. Vallor, Shannon, and Brian Green, “Best Ethical Practices in Technology”, Markkula Center for Applied Ethics at Santa
Clara University, 2018, https://www.scu.edu/ethics-in-technology-practice/best-ethical-practices-in-technology
(accessed 12 February 2021).
25. Greene, Jay, “Microsoft won’t sell police its facial-recognition technology, following similar moves by Amazon and IBM”,
The Washington Post, 11 June 2020, www.washingtonpost.com/technology/2020/06/11/microsoft-facial-recognition
(accessed 12 February 2021).
26. Quote provided to the World Economic Forum for this paper.
27. Tung, Liam, “Microsoft warned: Your cloud contracts are ‘serious concern’ says EU privacy watchdog”, ZDNet,
22 October 2019, https://www.zdnet.com/article/microsoft-warned-your-cloud-contracts-are-serious-concern-says-eu-
privacy-watchdog (accessed 12 February 2021).
28. Brill, Julie, “Introducing more privacy transparency for our commercial cloud customers”, Microsoft, EU Policy Blog,
18 November 2019, https://blogs.microsoft.com/eupolicy/2019/11/18/introducing-privacy-transparency-commercial-
cloud-customers (accessed 12 February 2021).
29. Roach, John, “Using AI, people who are blind are able to find familiar faces in a room”, Microsoft, Innovation Stories,
28 September 2020, https://news.microsoft.com/innovation-stories/project-tokyo (accessed 12 February 2021).
30. Thieme, Anja, et al. “Interpretability as a Dynamic of Human-AI Interaction.” Interactions, vol. 27, no. 5, October 2020,
pp. 40-45.
31. Ibid., Figure 1, 2020, https://interactions.acm.org/archive/view/september-october-2020/interpretability-as-a-dynamic-of-
human-ai-interaction (accessed 15 February 2021).

Responsible Use of Technology: The Microsoft Case Study 22

The World Economic Forum,
committed to improving
the state of the world, is the
International Organization for
Public-Private Cooperation.

The Forum engages the

foremost political, business
and other leaders of society
to shape global, regional
and industry agendas.

World Economic Forum

91–93 route de la Capite
CH-1223 Cologny/Geneva
Switzerland
Tel.: +41 (0) 22 869 1212
Fax: +41 (0) 22 786 2744
[email protected]
www.weforum.org