Proceedings of the 55th Hawaii International Conference on System Sciences | 2022

Fluid Socio-Technical (Trans)formation of an AI system

Joona Ruissalo Esko Penttinen Aleksandre Asatiani

Aalto University School of Business Aalto University School of Business University of Gothenburg
[email protected] [email protected] [email protected]

Abstract development, we ask: “Why do AI system development

The paper applies a flow-oriented perspective to and deployment projects experience delays in
examine how temporal conditioning of the flows of knowledge work organizations?”
people and digital technologies dynamically shape Instead of looking at the AI system design and
socio-technical formation and the transformation development as originating from actors participating in
process of an AI (artificial intelligence) system. it and exercised in interactions among them, we consider
Drawing on an in-depth case study of a financial how temporal conditioning of the flows of people and
accounting services company that was developing and digital technologies fluidly shapes the formation and
deploying an AI system in its work process enabled transformation of an AI system [4]. We take a flow-
forming a flow-based genealogical account of the fluid oriented approach to explore how the dynamics of
process of socio-technical (trans)formation. This allows creation, sensing, and undergoing shape the formation
to explore how delays to AI system deployment can arise of the idea, design, and development of an AI system.
from impediments to the dynamics of creation, sensing, This enables gaining a sense of the fluid socio-technical
and undergoing of possibilities for action wherein the (trans)formation of such a system, which allows us to
flows of practices and actions involved cannot reach explain how and along which flowing lines of action the
favorable conditions to form correspondence along the system is created and continues (trans)forming.
(trans)formative system-development path. To see how the temporal conditioning unfolds in AI
systems’ development and deployment, we sampled an
1. Introduction organization that successfully implemented such a
system in its work process. Drawing on the flow-
oriented approach made us able to identify how
Organizations are keen to deploy models based on
generative and degenerative dynamics in the process of
machine learning (ML) in their processes, since these
AI system development among involved practices and
models yield higher degrees of effectiveness and
technologies can cultivate or stifle conditions for
efficiency than human-based operations do [1]. The
meaningful action along the (trans)formative path.
shift in which organizations are implementing ML
models has recently been fueled and enabled by the
proliferation of Big Data. The principle behind utilizing 2. Literature review
ML models in knowledge work is usually
straightforward: identify a potential use case, design and Amid the resurgence of AI research [3], precise
develop a specialized AI system for it, and deploy it for definition of AI remains difficult to pin down. For our
use after testing. Knowledge work organizations purpose, we use a definition offered by Kaplan and
predominantly consist of professionals such as data Haenlein [5], in which AI is a system that is able “to
scientists or accountants who create, utilize, and interpret external data correctly, to learn from such data,
disseminate knowledge and information. However, and to use those learnings to achieve specific goals” (p.
there exist managerial difficulties [2, 3] associated with 17). While this is not the most thorough definition, it
ML models which often result in delays in development. describes the type of system addressed here perfectly.
These difficulties are related to requirements connected Scholars’ growing attention to AI in organizational
with the quantity and quality of training data required, context has led to a gradual shift from a predominantly
difficulties in foreseeing how the model’s accuracy may technical perspective to an increasingly socio-technical
change during development, problems in scaling ML one [6, 7]. The traditional AI paradigm suggests that
models, and the fragile nature of ML models. powerful machines could process data and produce new
Motivated by the proliferation of ML utilized in AI knowledge without domain experts’ involvement [8];
systems and the persistent delays observed in their however, more recent arguments contend that

URI: https://hdl.handle.net/10125/80181
978-0-9981331-5-7 Page 6964
(CC BY-NC-ND 4.0)
organizational AI cannot be considered a “plug-and- and continuously woven into new paths [13].
play” technology [3] and that it requires close Accordingly, entities are seen as always in the making
collaboration on the part of AI developers and domain as they are swept forward and animated along the flows
experts throughout the development process [7, 9]. of action. The flow-oriented approach marks an
Notwithstanding contemporary AI systems’ important shift in the notion of time, from a
remarkable capabilities, certain limitations render a chronological order (chronos) to kairological time
vision of self-sufficient machine experts non-viable. (kairos) extending beyond a single timeline. Attention
Moreover, these limitations cannot be overcome by moves to kairotic timing and the varying temporal
means of technology alone. Most of the problems lie qualities of flows of action. Such temporal qualities as
instead in epistemological misconceptions of AI’s rhythms, tempos, intensities, timeliness, and
capabilities and in the restricted nature of the social directionalities have their conditioning aspects and
context in which the AI is developed. Systems based on thereby entail specific (trans)formative dynamics.
AI notoriously suffer from the frame problem [10]: AI Figure 1 illustrates different temporal qualities of flows,
agents cannot competently perform an action in for example, Action A and B sharing same tempo, and
environments for which they lack predetermined rules. Action C illustrates how iterative loops cause shifts in
Therefore, seemingly all-powerful algorithms may be rhythm and directionality of action. Instead of time
unable to deal with dynamic events that would be trivial elapsing in a uniform chronological past-present-future
for humans to handle. Domain experts can significantly order, kairotic timing brings to the fore timely moments
contribute to the development process by helping adjust and timing of those moments as confluences of various
the frame to fit the task. Organizational AI developed in flowing lines of action, each with particular temporal
isolation from the social context in which the domain qualities, arise along a shared path [4] – such as that of
experts operate may lead to systems failing to account an AI system’s development process. These flowing
for broader implications of their use [9, 11]. That could lines of action and their confluences that shape the
produce especially troubling consequences given the patterns of lines in a continuous open-ended process can
lack of diversity in AI research and industry [2]. For be depicted as a kairotic meshwork [14] as exemplified
example, the impact of implementing AI in an in Figure 1.
organization varies between stakeholders; therefore,
excluding some stakeholders from the development
process could bring about perceived breach of social
contracts and, in the end, failure of the project [12].
Hence, these processes become more complex than
those in traditional IT development projects [6, 7, 9].
Such complexities may lead to unexpected hurdles in
the development and deployment of an AI system,
ultimately delaying the project.
To understand the process of developing the ML
Figure 1. Time as a kairotic meshwork
models and AI systems that set in motion changes and
travails in organizations, use a theory of socio-technical Correspondence is a notion developed by Ingold
transformation proposed by Mousavi et al. [4]. It offers [14, 15] for what occurs when multiple flowing lines of
an alternative way to view how flows of people and action are being woven together such that specific
digital technologies temporally condition the ways in (trans)formative dynamics of creation, sensing, and
which the flowing lines of action are continuously actualization may emerge. In Figure 1, Action A and B
becoming. From such a flow-oriented, rather than are in loose correspondence and the lines of action fall
actor-centric, perspective, the originators of formation out of it but remain closely converged and synchronized.
and transformation are not the self-contained entities Furthermore, there is initially close correspondence
involved in the ongoing process; instead, the among Action C and D as they are tightly coiled around
socio-technical (trans)formation is seen as stemming one another, but as the lines of action fall out of
from the becoming and conditioning of historical and correspondence, the coiling unravels. To give rise to the
new lines of action that create new possibilities for (trans)formative dynamics in socio-technical flows,
action along the flows as they contingently enter however, the flowing lines of action must gain
confluence, converge, and become corresponding or fall correspondence, through three modalities: timing,
out of correspondence with one another. attentionality, and undergoing. These modalities entail a
Two concepts central to the flow-based approach moment of kairotic timing, an attentional orientation,
are correspondence and flow. Flow refers to a quality and an experience of undergoing, per Mousavi Baygi et
that comes about as previous lines of action are absorbed al. [4]. Timing affords creating conditions for new

Page 6965
possibilities for action along flows, attentionality for 3.1. Data collection
sensing possibilities for action, and undergoing for
actualizing them along a (trans)formative path. As We conducted 53 semi-structured interviews
correspondence gives rise to the dynamics of creation, between January 2020 and June 2021. Of these
sensing, and actualization, the flowing lines of action interviews, 33 were with accountants who were using
can become co-responsively interwoven, in their the AI system; 17 were of senior accountants impacted
conceptualization. The process (trans)forms the by the system; and one interview each with the director
temporal qualities and trajectories of socio-technical of the SSC, the team leader for accounts payable and
flowing lines of action that comprise and shape the path receivable, and a data scientist who had a central role in
of an AI system project. designing, developing, and deploying the AI system.
Overall, the flow-oriented approach allows one to These interviews lasted 40-105 minutes. In parallel with
account for the ongoing nature of AI system the interviews, we attended meetings with the team at
(trans)formation and its vibrancy as several flowing the AI innovation unit. While these were not recorded,
lines with their temporal qualities enter mutual they kept us informed and yielded insight into the
confluence. Therefore, an AI system’s design, overall process of the AI system’s design and
development, and deployment entails corresponding development. Topics discussed related to what the team
dynamics along various co-responsive flows of action. is currently aiming to achieve with the ML models and
From this perspective, we can explain how the lines of AI system and how they are progressing with the design
action (trans)form an AI system project and the entities and development process. Also, we received
involved in it. In turn, directionalities of action shift organization-internal documents that, for example,
along a socio-technical (trans)formative path and can outline the unit’s mission and introduce the team.
lead to delays in the AI system project if correspondence
cannot be gained among pertinent lines of action. 3.2. Data analysis

3. Method With the data analysis, we set out to identify central

events, various practices forming part of the AI system’s
We took an exploratory approach to our (trans)formation process, and experiences of
preliminary analysis, applying the novel IS theory of individuals, and form a flow-oriented genealogical
socio-technical transformation. We used purposeful storyline for the emergence and unfolding of the design
sampling [16] to identify an appropriate empirical and development of that system and its algorithmic
setting for this study. We set criteria to select a suitable action. Genealogy is a specific type of historical account
case organization: 1) the organization had to be planning [17] and a mode of inquiry that spotlights contingent
to deploy ML model(s) for use in its knowledge-work correspondences that turn out to be fundamental to the
processes; 2) the employees using the AI system had to process of ongoing (trans)formation [4]. A flow-
be knowledge workers, and; 3) we had to be able to oriented genealogy thus aids in teasing apart the
collect data over the course of the AI system project to dynamics of the conditions that give rise to possibilities
analyze how the process unfolds at the various levels of for action along the flows [4].
the organization. We selected a large Finnish company, The same researcher conducted all the interviews,
referred to as CloudAccounts (pseudonym), which met taking notes, and thus became immersed in the data and
these criteria. The company specializes in delivery of deeply familiar with the material’s nuances. A flow-
financial accounting and payroll-administration services oriented perspective made it possible to carry out
to other firms. CloudAccounts sought to utilize ML preliminary analysis and produce a detailed illustrative
models in its processes for this work. The company was narrative of the case company’s socio-technical
structured with several local offices, in Finland, that transformation. We began by preparing a storyline of the
offer financial services and had also established a shared overall process based on our interview with a data
service center (SSC) to concentrate specific lines of scientist who had been involved in the AI design and
work in one location, where dedicated teams could development from the beginning. Interview notes
handle financial accounting processes for accounts informed us of how the accountants’ experiences and
payable, accounts receivable, and general-ledger perceptions of the AI system evolved over time and
accounting. Moreover, the firm set up an AI innovation what the interviewees saw as important moments or
unit to develop and deploy ML models to automate and events over the course of this process thus far. This
augment financial accounting work tasks by means of material allowed us to take their views into account and
the historical data accumulated over the years. incorporate them into the storyline [18]. We then
considered the general storyline in light of the flow-
oriented approach, by considering the three modalities

Page 6966
of correspondence (timing, attentionality, and 4.1.1. Formation of an AI system. In November 2019–
undergoing) in parallel with extracts from interviews of February 2020, the CloudAccounts innovation unit,
key informants (p. 62) that pertained to piloting and consisting of two data scientists, two service designers,
testing of the AI system. This helped reveal several and the head of the unit, begins planning the pilot phase
distinct views of the design and development process. for the new AI system, which is still in the early stages
Finally, all the authors examined the findings from this of the design and development process. In February, to
preliminary analysis, to catch possible inconsistencies. get the piloting going, a small group of accountants who
handle the accounts payable process and its related task
4. Findings of processing purchase invoices are chosen from the
CloudAccounts financial services SSC to begin testing
By means of a genealogical story and the the fledgling AI system and give feedback. Because the
sensitizing lens of the three modalities of center’s accounts payable teams are responsible for the
correspondence, we illustrate how the conditions for the handling of hundreds of client companies’ purchase-
AI system were formed, how it was brought into being invoice processes, their tasks include making correct
and continues (trans)forming, and why it ended up account postings based on the purchase invoices’
taking nearly two years to shape the AI system to information and checking (and validating the
correspond more closely with the requirements and correctness of) each posting and the VAT values in
temporal qualities of the accounting practices. those invoices. For these work tasks, the accountants
have been, for the most part, working through large
4.1. The making of conditions for an AI system volumes of purchase invoices by making account
postings manually in the AIS. Since they carry out these
tasks daily, the accounting specialists and accountants
The move toward creating and actualizing the
have become attuned to the activities of this accounting
envisioned AI system was made possible by an earlier
practice, thus possessing sufficient experience and
decision by CloudAccounts to put more emphasis on the
expertise to point out how the AI system should be
development of its cloud-based accounting information
adjusted to accommodate the needs of said practice.
system (AIS). Maintaining that focus for more than five
Proceeding from the piloting feedback, obtained in
years, in total, was crucial to transforming the financial
February to August 2020, the data scientists make
accounting data flow largely from a paper-based
gradual adjustments to the ML models. However, to
endeavor into an electronic, digital one over that time.
understand how the accountants undertake the daily
This gradual shift in data flow created favorable
processing of purchase invoices, the innovation unit
conditions for collecting digital structured data created
personnel involved in the design and development
by CloudAccounts clients’ business transactions, data
process deem it necessary to become more acquainted
that could be utilized in training the AI system’s models.
with the flows of action involved in financial
We begin our story in June 2019, when a new
accounting, especially purchase-invoice processing.
innovation unit was formed with the purpose of
Data scientist Veronica later recalls, “Through that
actualizing the potential of AI in accounting practice.
cooperation [with accountants], we found out so many
An introduction posted on company’s intranet reveals
different insights we had, like a lot of unnecessary and
the ambitious mission of the unit: “Our goal is not only
even [harmful] constraints on the data we were getting.
to rethink the current processes, but also to simplify
And yeah, it helped us a lot, a lot a lot – so many
various decision-making problems. Using historical
different bugs. Help of pilots and accountants was, I
data CloudAccounts collected over the years, we can
don't know, golden.”
teach machines to think like people in some specific
situations. Imagine how different and easy the
4.1.2. Transformation of an AI system. Having
workflows might become if we are able to use
improved the ML models used in the AI system in
automatized knowledge and help of previous
accordance with the feedback and grown acquainted
generations!” The innovation unit’s more immediate
with intricacies of financial accounting, the unit takes
goal is to kick off the design and development of a
another step forward with the development process: in
specialist AI system that can handle tasks constituent to
September 2020, it moves from piloting to testing the
purchase-invoice processing. Automating this
system in production use for around 100 client
processing is a prominent business case for use of ML
companies. It takes only a few weeks to notice,
models, in that it includes repetitive work. The company
however, that the AI system is producing inconsistent
saw potential for an ML model to be trained on large
and inaccurate outputs. This alarms the accountants
volumes of data to deal with relatively straightforward
since the incorrect outputs have knocked the work out
tasks such as predicting a correct account posting and,
of rhythm and increased the workload in cases in which
on this basis, trigger new algorithmic action.

Page 6967
the AI system is in use. The AI developers receive outputs, and the AI system is ready for full production
negative feedback because the system necessitates much deployment in April 2021.
more checking and validation of account postings and
VAT values than previously needed. 4.2. Timing and seizing timely opportunities
Working on the issues related to the ML models’
accuracy, the innovation unit makes a significant Proceeding from the above storyline, we can see
breakthrough in November 2020. Thus far, the data how different historical lines of action and their timely
scientists had worked with a limited test-environment kairotic correspondences put the idea of an AI system’s
database. Gaining access to the production database in design and development in motion. The storyline also
October 2020 adds the full set of historical data from outlines how the innovation unit’s activities of design
each client company to the ML models’ training. Thanks and development, in combination with the feedback
to accuracy improvements, some of the accountants received from accountants, conditioned the
begin again seeing the AI system as a potential tool to (trans)formation of these flows of action and, moreover,
improve their work: as December 2020–February 2021 led to deploying an AI system for purchase invoice
unfolds, it is now corresponding better with their processing that created a new flow of algorithmic action.
practices. Now, an accountant can activate the AI Several historical lines of action started to come
system used in the AIS client-specifically based on the into confluence and form correspondence in the timely
system’s estimates of what its accuracy would be for the creation of conditions favoring AI system development
client in question. The purpose of the AI system is to initiative to gain prominence. In addition to increasing
supply predicted values for account postings and VAT use of AIS gradually at CloudAccounts, new
amounts for purchase invoices to increase the level of knowledge-work automation technologies such as
automation in the invoice handling process. Those ML- robotic process automation and AI systems, had been
based predictions that do not exceed a set confidence- gaining prominence in financial accounting.
level threshold are highlighted, and the system lists them Furthermore, the company was facing competitive
in a new widget in the AIS for the accountant to check. pressure to enhance its operations, and the top
Here, the user either corrects or approves the proposed management had identified a strategic need to step up
postings and VAT values. its digital transformation via new automation
This breakthrough notwithstanding, a considerable technologies. As these historical lines of action began to
ongoing issue with one ML model is leading to converge and correspond, the conditions were created
erroneous VAT-deduction postings that create problems for establishing an AI innovation unit.
for both the accounts-payable process and the general- As the new flowing line of AI system design and
ledger accounting process, which uses the purchase development practice and the flow of accounting
invoice data to generate monthly reports. This issue has practice began converging and briefly entered
persisted for several months, and when, in December correspondence for the first time during the piloting,
2020, the AI system is brought into use for a large new possibilities for action started to form. The
percentage of the CloudAccounts’ client companies, following quote illustrates an opportune correspondence
accountants grow increasingly frustrated with the among situational lines of action that cultivated
situation since they have no easy way to check and favorable conditions during the AI system development:
validate the VAT deductions. To ameliorate the Veronica: Before COVID-19, when we were at the
situation, the AI systems’ developers add a new column office – and we have flexi-logic [hot desking] and
to the value-validation reports, specifically for VAT usually next to our team there were sitting random
deductions. This gives the accountants an overview people but there was one lady who had been doing
whereby they can compare the values with historical accounting for, like, decades […] – we asked the lady,
postings to check for deviations. Though this addition to if she is interested to help us, because we really need it.
the report renders the various activities more aligned […] She was really curious about the way we work and
again, the underlying problem with the VAT-deduction the […] algorithms we are trying to design, and her
ML model remains. input was really valuable. I remember that, out of that,
In March 2021, Veronica, who had been I figured out that kind of invoice-type model. It was
investigating the VAT-deduction problem for months, through cooperation with her I came up with it
suddenly is able to spot a couple of bugs that are eventually. Sometimes it's enough just to see what's the
producing a combination of problems in the ML model. first line and then just to make [a] conclusion [on]
With the bugs fixed, it all begins to work as intended. whether to go “Next” [purchase invoice] or not.
Nearly two years after establishment of the innovation Because before joining [CloudAccounts], I've never had
unit, the ML models are finally producing accurate anything to do with accounting, so I had to kind of
[chuckles] learn these sorts of things from scratch.

Page 6968
 As this extract attests, the emergence of the ML move from contingent confluences to the beginnings of
models used in the AI system was not solely an outcome convergence and greater resonance with one another,
of deliberate design and development practice; the creating fertile ground for correspondence.
models were shaped and formed on the foundation of Even though the technical development of the ML-
the correspondence among the flowing lines of practice based algorithm at CloudAccounts and the ways of
that then carried on transforming. Although the AI applying the ML and AI platforms’ features can be
innovation unit worked intensively to find solutions, attributed to the design and development team’s data
without the timely feedback of accountants and the scientists, success required the AI innovation unit’s
opportune moment along the flowing lines of action, the personnel to become exposed – and, thereby, attuned –
trajectory of the design and development practice would to accounting practice. Likewise, the accountants who
not have become more aligned with accounting practice. participated in piloting, testing, and giving feedback had
Thus, as timing was off among the flowing lines of to become exposed and attuned to the AI system’s
action, this in part led to delays in the AI system project algorithmic action and, to some extent, to AI system
as the lines of action could not gain correspondence. development practice. In this process, the accountants
Another important turning point in the trajectory of had the important role of teaching the data scientists and
designing and developing the AI system was when data service designers basics of accounting, so that they
scientists gained access to the AIS’s production could become attuned with the intricacies and rhythms
database. Because the data scientists initially had to of the relevant accounting practice.
work with test-environment data only, the ML models’ As all the new flowing lines of design and
training was insufficient, and the resulting low accuracy. development practice for the AI-based system, and the
This led to AI system producing erroneous outputs. algorithmic action, met with the historical flow of
While this disturbed the accounts-payable accountants’ accounting practice in a contingent confluence for the
day-to-day work rhythm on account of the additional first time in piloting, the initial impetus was created for
checking and validation activities, enhancing the ML the design and development practice’s greater alignment
models with timely data flow did improve the relevant with accounting practice. As this quote from an
model’s accuracy: though algorithmic action still accountant David illustrates, the innovation-unit staff’s
yielded faulty outputs at this point, there were far fewer efforts to understand the purchase-invoice processing
errors than before. This accentuates the point that the more deeply became frustrating on occasion: Because I
data scientists’ delayed access to production data and have been part of piloting and testing from the
the timing being off created difficulties and problems beginning, the data scientists lean on me quite often and
that rippled all the way through the accounting work as ask me – and I have to look into – things I don’t know
the conditions were not met for algorithmic action to about: why it’s been done this or that way. […] At the
gain fuller correspondence with accounting practice. beginning, I was immensely interested in being part of
In sum, this outline highlights how possibilities for this kind of project, and when I was responsible for
action either emerged or were suppressed via the testing that was interesting, but now [chuckles] it is a
conditions created by the timing of various individual bit frustrating every now and then since I haven’t done
flows of action. Also, it reveals the centrality of timing that client’s postings and I would need to know why it
to rendering contingent lines of actions and their has been done in that way because “you did it like this
temporal qualities more aligned with one another. While before, so why is it now done like that?”
kairotic correspondences create new action possibilities, This example showcases the data scientists’ work
mistiming can impede the formation of convergence and to attune themselves to the accounting practices first, so
correspondence among the flows of action leading to that they would be able to tune the ML models such that
delays in the AI system project. the algorithmic action would converge more fully with
accounting practice. The tuning proved helpful later too,
4.3. Attentionality and sensing possibilities for when they moved on to testing the AI system with a
action larger number of clients and then implementing it for
those clients that met the prerequisites for activating the
Our approach enables demonstrating also how AI system. Attuning thus aids in sensing possibilities for
exposure and attunement to the various flowing lines of action and thereby permits the design and development
action and their temporal qualities gives rise to process to progress so that the converging lines of action
conditions for sensing possibilities for action. Thus, we end up corresponding with one another at some point.
unveiled how being attentive to the temporal qualities However, attuning itself can be a timely process and if
(e.g., rhythm and the intensity of the flows of action) can it is not accounted for in the AI system project schedule,
shape formation of correspondence. The resulting it can lead to unexpected delays. Nevertheless, as the
perspective affords explaining how the flows of action innovation unit became more aware of the requirements

Page 6969
of accounting practice, this assisted its staff in sensing time to sense possibilities for action that emerge along
new possibilities for action. For example, when the data the flows.
scientists set the confidence level to 75% ensure that the
system catches errors in AI-generated postings, the 4.4. Undergoing and actualizing
resulting excessive highlighting of purchase invoices as (trans)formative opportunities
uncertain not only increased the accountants’ workload
significantly but also threw off their attunement to the Finally, undergoing a (trans)formative process
flow of algorithmic action. Instituting data-driven opens new paths of becoming whereby one may
confidence thresholds later, instead of a static threshold rediscover and reinvent oneself. The innovation-unit
led to the AI system highlighting uncertain purchase- staff and accountants alike had to become exposed and
invoice posting suggestions more accurately. attuned to the flows conditioning the (trans)formation
Veronica: We thought that “OK, let it be so that if connected with the AI system before they could sense
[the confidence for a posting prediction] is more than timely possibilities for action.
75%, we do not highlight [the prediction as uncertain Both the accountants and the staff of the innovation
and if it’s] less we do.” Then we started having many unit involved in the system’s design and development
different unhappy accountants [chuckles] […]. So I process found themselves swept away and animated by
thought that “well, having this number 75% just out of the flows of action from time to time as they were
a hat, it's not a good idea.” So I had a subproject to undergoing the process of actualizing the new
make this threshold to be data-driven on a daily basis possibilities for action that had emerged. As they
[…]. According to our assessments, accountants underwent this (trans)formative process, they were
became much happier with that when we removed the rediscovering and reinventing their paths of becoming.
75% and started being more data-driven. As the following two quotes articulate, the AI system’s
This points to another facet of tuning: as the data design and development was not merely a process of just
scientists were growing attuned to the intricacies and getting ML-based algorithms to work; it also featured
requirements of accounting practice, the ML models much more work, beyond data-science activities, than
reciprocally were becoming more tuned to the originally anticipated. Veronica: I've been developing
accounting practice’s flow – for example, through new, some models in a lab. […] I had some image that was
improved features and via training from all the far from reality. But eventually what I learned is that
production data. With implementation of the AI “actual” data science as […] we think [of it] usually
system’s improved ML models and more sophisticated while we play with models and algorithms and try to
features such as data-driven thresholds, the algorithmic tune this or that, well, it's like [that] less than 10% of
action became better aligned with the temporal rhythms the time. Majority of time will be spent engineering,
and tempos of the accounts payable process. This, in communicating, or getting to know this or that,
turn, led to accountants becoming more attentive to the arranging different activities. My perception has been
flow of algorithmic action, since it was now part and changing with each and every step.
parcel of the accounts-payable process. The improved Veronica: Imagine: we had at some point around
process required them not just to validate the correctness hundreds of companies that were piloting the feature
of purchase-invoice postings but also deal with the […]. I had to kinda go inside to retrain everything, to
uncertain posting predictions that were listed by the new use backtesting approaches to check [thing] out, and it's
widget in the cloud-based AIS’s user interface. very time-consuming. It's very, very time-consuming,
Overall, there must be changes in the conditioning and it's not like “check this and that” but “then check
attentional flows before the flows of action can through the whole history what was there before,” like
converge and start resonating with one another such that resimulate the situation [with a particular invoice] […].
correspondence can take shape. These changes can I learned that this is a part of a process. It's not just me
come to pass through greater attunement to the practices and the data and tuning, playing with it. It's just...
involved and their related temporal qualities. This fact handling very many different requests from many
points to the centrality of attentionality for sensing the different teams from managerial to pilots, from
possibilities for action. Therefore, the picture is far engineers, architects to database people, and so on.
wider than one of deliberate intention or planned goals These musings highlight how individuals and the
shaping technology use which can be difficult to take innovation unit needed to rediscover and reinvent their
into consideration in the schedule of the AI project. paths of becoming during the process of undergoing.
Therefore, unexpected delays can occur as attuning to This took time as the possibilities for action were
sense possibilities for action can be a lengthy process. It created during the process which can be seen as delays
is crucial to also understand that AI systems’ design and from the viewpoint of the AI project, but for the
development is a fluid, continuous process and it takes individuals it was a necessary learning process to

Page 6970
become attuned with the other lines of action. scientists became animated by the flowing line of
Furthermore, because there were plenty of other accounting practice, the flow of algorithmic action itself
activities for the four innovation-unit personnel to attend was transformed. With that action beginning to find a
to, the design and development split into several streams shared rhythm and start forming closer contingent
of action. Veronica: Eventually it was not just correspondence with accounting practice, it began to
innovation unit plus developers; it was innovation-unit condition and animate accounting practice in a new way.
team, developers, architects, engineers, marketing
people, and so on. We were coordinating all our 5. Discussion and conclusions
activities on a weekly basis, which was great, and we
had a really great project manager who was pushing Drawing on the notions outlined in the presentation
everything to be done as it should be. of findings, we can see that as new lines of action are set
While it was the innovation unit that set the AI in motion, they exert both generative and degenerative
design and development practice in motion and had sole effects on the work in an organization. Thus, the new
charge of it at first, that practice has continued with possibilities of action can be actualized in the
various streams of AI system design and development undergoing of the continuous process. Next, we discuss
action, and the innovation unit’s role is still being why delays occurred in the AI system project and reflect
rediscovered accordingly. A comment from Veronica on how we applied flow-oriented genealogy.
encapsulates the process of fluidly etching these furrows
through the landscape: “From one team-based project it 5.1. Delays in socio-technical (trans)formation
really transformed to something really well organized,
of an AI system
and that's why we managed to finally go live in spring
[April 2021].”
Though each flowing line of action comprises three
Moreover, the flow of the AI system’s algorithmic
distinct modalities of correspondence, the figure below
action now corresponds more closely with accounting
depicts only confluences and kairotic timing among the
practice at CloudAccounts, and the trajectory of
main lines of action being shaped in the (trans)formation
accounts-payable activities has been altered
between the innovation unit’s establishment to the AI
simultaneously. As one of the company’s senior
system’s deployment in production use. In Figure 2, line
accountants noted, the AI system has come to
A refers to accounting practice, line B to the flow of
correspond more closely with the temporal qualities of
financial transactions’ data to the AIS, line C to ML
the accountants’ work than it did when first tested in
models’ algorithmic action, and line D to the AI
production use for a limited set of client companies, and
system’s design and development practice. The
she also pointed out how she saw the further
moments shown are when piloting began (1), when
transformation of the role of accountant. Kate: Lately,
testing and limited use of the AI system in a production
[the AI system] has been improving tremendously, and
environment started (2), the design and development
[it] will likely improve much more. I just remember back
team getting access to all historical data (3), and full
when I started [accounting] studies [and] the lecturer
production deployment of the system (4).
was saying that AI is coming and replacing accountants,
and I can say that will never happen, but it is interesting
to see what will happen. But, of course, my work has
changed such that you aren’t necessarily a person
processing accounting information anymore but the
validator and expert.
This crystallizes the ways in which accountants at
CloudAccounts were rediscovering and reinventing
their paths of becoming as they were both swept away
and animated by the flows of action and becoming to
embody (trans)formative possibilities along the Figure 2. Kairotic meshwork of the AI system
corresponding flow of algorithmic action. project
All in all, the findings attest that, amid undergoing
the design and development of the AI system, both the The figure expresses the idea that accounting
innovation unit’s and the accountants’ action practice (line A) and the transaction-data flow (line B)
possibilities were continuously (trans)forming. While grew duly synchronized once the accountants had a firm
the AI system’s algorithmic action was disruptive to grasp of the work tasks that need to be done in a certain
accounting practice at first, it also created possibilities order for meeting the deadlines of monthly financial and
for action that were actualized in the process. As the data regulatory reporting. Therefore, there was close

Page 6971
correspondence among the two historical lines of action. action, hence leading to further delays in the AI system
To put in motion the new AI system project, the project. The disruption to accounts-payable tasks’
innovation unit brought into being the design and rhythm and intensity was accompanied by changes to
development practice (line D), and that line of action the directionality and trajectory of accountants’ practice
created the ML models’ algorithmic action (line C) a in that they were now supposed to use the AI system in
while later. The AI system project began with the unit’s their work for suitable client companies, thus stepping
data scientists having control of ML models’ (line C) onto the transformative path toward gradual rediscovery
outputs and understanding what they wanted to achieve and reinvention of accounting practice.
with the system. Close correspondence formed among Another important moment in the process came
the lines accordingly. when the innovation unit gained access to the full
However, once these two lines of action met the production database. With the historical data, they could
other two for the first time as piloting of the AI system make a new major iteration in the AI system’s
began (1), lines C and D fell out of correspondence, development, depicted by the looping of line D. Also,
since the ML models were not producing the intended the innovation unit was becoming one of the streams in
outputs. To get design and development practice more design and development practice at point 3 in the figure.
aligned with accounting practice, the innovation unit’s This created grounds for correspondence of lines C and
experts began making sense of the intricacies and D while also adjusting the trajectory of these flowing
requirements of financial accounting tasks related to lines of action to be more aligned with A and B. Finally,
purchase-invoice processing [1]. Hence, lines D and A at the moment of full deployment (4), lines A and B
were in momentary correspondence when piloting was were becoming loosely interwoven with lines C and D,
in progress, forming suitable conditions for making a condition likely to create new, generative possibilities
design and development practice converge with for action. Through this transformation, all the lines of
accounting practice and creating new possibilities for action began either converging or corresponding more
action. For the data scientists to condition accounting closely – albeit not in a fully synchronized manner,
practice through the alteration of financial transaction since the AI system never completely matched the
data flow with new algorithmic action, they had to requirements and temporal qualities of accounting
become conditioned by the very same lines of action. practice. Initially, line A was the dominant conditioning
This highlights why delays occur in an AI system flow as it required D to move, into greater convergence
project: Both the AI developers and accountants had to with it. Convergence grew through timely actions in
“grow into knowledge” [14] through cultivation of attuning to accounting practice, which allowed
kairotic correspondences which were not fully taken possibilities for action to emerge along the flows and led
into account in a chronological timeline that was to moments of kairotic correspondence. While
planned for the AI design and development project. simplified, this flow-oriented genealogical account [4,
Therefore, issues in the three modalities of 17] illustrates how one can trace the historical lines of
correspondence in a flowing line of action hinder the action and their confluences that create the conditions
possibilities of gaining correspondence among the leading to fluid socio-technical (trans)formation of an
different lines of action that in turn are experienced as AI system and its algorithmic action such that they
delays. Nonetheless, when line D is more attuned with become corresponding and co-responsive with other
line A, it can sense the action possibilities along the pertinent flowing lines of action along the same
flows as it undergoes the process. This allows transformative path.
actualizing the action possibilities. By point 2 in the Therefore, as this study underlines, an AI system
figure, line C (algorithmic action) has begun to converge design and development project is not just a technical
and form correspondence with line D. endeavor which is why the initially planned schedule for
Nevertheless, since this moment coincided with the the project gave in several times: To achieve the set
system’s testing in production use and its lack of goals of the AI project, both the AI developers and
correspondence with accounting practice, accountants accountants had to become attuned to the different lines
experienced disruptions in their work and the workload of action by undergoing the (trans)formative process.
rose because the ML models had overly strict This allowed them to actualize the timely action
confidence thresholds and yielded erroneous posting possibilities as they were conditioning and being
predictions. With the small innovation unit needing to conditioned by the pertinent flowing lines of action. The
manage various other activities, outside technical flow-oriented genealogical research allows to consider
development, its attention was split across several issues the multiple lines of action that condition the AI project
during the process, which reduced the possibility of and the becoming of the involved entities as well as the
timely action. Ultimately, this caused issues in creating, three modalities of correspondence that allow to better
sensing, and actualizing generative possibilities for

Page 6972
grasp why AI projects are complex and lengthy 6. References
processes.
Finally we would like to reflect on the use of the [1] Asatiani, A., P. Malo, P. Rådberg Nagbøl, E. Penttinen,
socio-technical transformation theory [4]. We began by T. Rinta-Kahila, and A. Salovaara, “Sociotechnical
creating a historical storyline that allowed us to point out Envelopment of Artificial Intelligence: An Approach to
matters related to each three modalities of Organizational Deployment of Inscrutable Artificial
correspondence. Once the historical storyline was Intelligence Systems.”, Journal of the Association for
created, we found out that the easiest part was in Information Systems 22(2), 2021, pp. 325–352.
[2] Whittaker, M., K. Crawford, R. Dobbe, et al., AI now
establishing the timing and mistiming of flowing lines
report 2018, AI Now Institute at New York University New
of action as respondents stated clearly if they were York, 2018.
pleased or dissatisfied about how the AI system [3] Keding, C., Understanding the interplay of artificial
development process had unfolded. This allowed to intelligence and strategic management: four decades of
point out when the lines were gaining correspondence research in review, Springer International Publishing, 2020.
or falling out of it. However, it was challenging to tease [4] Mousavi Baygi, R., L.D. Introna, and L. Hultin,
out the modality of attentionality as it was a much more “Everything Flows: Studying Continuous Socio-
difficult endeavor to trace the different lines of action Technological Transformation in a Fluid and Dynamic
with their specific temporal qualities as they became to Digital World”, MIS Quarterly 45(1), 2021, pp. 423–452.
[5] Kaplan, A., and M. Haenlein, “Siri, Siri, in my hand:
condition one another in the process. Thus, getting to
Who’s the fairest in the land? On the interpretations,
grips with how attunement to the lines of action illustrations, and implications of artificial intelligence”,
unfolded and how that allowed to sense possibilities for Business Horizons 62(1), 2019, pp. 15–25.
action required from us attentiveness towards the [6] Pachidi, S., H. Berends, S. Faraj, and M. Huysman,
constant change in conditioning attentional flows. Once “Make way for the algorithms: Symbolic actions and change
modalities of timing and especially attentionality were in a regime of knowing”, Organization Science 32(1), 2021,
established, it was easy to expound from the interview pp. 18–41.
data how the lines of action and involved entities were [7] Asatiani, A., P. Malo, P.R. Nagbøl, E. Penttinen, T.
undergoing the process and became (trans)formed along Rinta-Kahila, and A. Salovaara, “Challenges of Explaining
the Behavior of Black-Box AI Systems”, MIS Quarterly
the (trans)formative path.
Executive 19(4), 2020, pp. 259–278.
[8] Agrawal, A., J. Gans, and A. Goldfarb, Prediction
5.2. Future research and managerial machines: the simple economics of artificial intelligence,
implications Harvard Business Press, 2018.
[9] van den Broek, E., A. V Sergeeva, and M. Huysman,
As future research, comparing flow-oriented “When the Machine Meets the Expert: An Ethnography of
genealogical research approach to other socio-technical Developing AI for Hiring”, MIS Quarterly, .
[10] Salovaara, A., K. Lyytinen, and E. Penttinen, “High
approaches and discussing their advantages and
Reliability in Digital Organizing: Mindlessness, the Frame
disadvantages provides a potential avenue for future Problem, and Digital Operations”, MIS Quarterly 43(2),
inquiry. This would allow to further weave the new 2019, pp. 555–578.
approach of socio-technical (trans)formation and the [11] Hagendorff, T., and K. Wezel, “15 challenges for AI: or
becoming ontology into the fabric of information what AI (currently) can’t do”, AI & SOCIETY 35, 2020, pp.
systems research. 355–365.
Unlike traditional IT projects, new AI system [12] Wright, S.A., and A.E. Schultz, “The rising tide of
projects bring about potentially transformative shifts to artificial intelligence and business automation: Developing
work practices that take time for the organization to an ethical framework”, Business Horizons 61(6), 2018, pp.
823–832.
absorb. Furthermore, the AI system development
[13] Chia, R., “Essai: Time, Duration and Simultaneity:
projects are characterized by constant testing of tuned Rethinking Process and Change in Organizational Analysis”,
ML algorithms based on ML model performance and Organization Studies 23(6), 2002, pp. 863–868.
domain expert feedback which is why the progress of AI [14] Ingold, T., The Life of Lines, Routledge, New York,
system projects might not be a straightforward process 2015.
due to temporal dynamics. Facilitating active [15] Ingold, T., Being Alive: Essays on Movement,
communication and collaboration in the socio-technical Knowledge and Description, Routledge, London, 2011.
transformation process is a key to cultivate and foster [16] Patton, M.Q., Qualitative Evaluation and Research
favorable conditions, since problems in understanding Methods, SAGE Publications, London, 2001.
[17] Foucault, M., “Nietzsche, Genealogy, History”, In P.
how to steer the system’s and models’ development into
Rabinow, ed., The Foucault Reader. Penguin,
fuller alignment with work-practice requirements could Hammondsworth, 1984, 87–90.
well lead to consistent delays and postponing system [18] Silverman, D., Interpreting qualitative data: A guide to
deployment. the principles of qualitative research, SAGE, London, 2011.

Page 6973