Design (Science)Research

1
Design Science Research basics
 Process model

 Artifact types:
 result of the research work

 Artifact structure
 content of the research approach

 Evaluation:
 evaluation criteria
 evaluation approach
2
Process model
 a problem-solving paradigm:
 seeks to create innovations that define the ideas, practices,
technical capabilities, and products through which the analysis,
design, implementation, and use of information systems can be
effectively and efficiently accomplished [Tsichritzis 1997;
Denning 1997]

3
4
5
 Design Science research process

knowledge
flows + operation and goal knowledge

circumscription
process
steps
Awareness of
Suggestion Development Evaluation Conclusion
problem

logical
formalism
abduction deduction

6 [Takeda,1990]
In this model all design begins with Awareness of a
problem.

Design research is sometimes called “Improvement

Research” and this designation emphasizes the problem-
solving/performance-improving nature of the activity.

Suggestions for a problem solution are abductively

drawn from the existing knowledge/theory base for the
problem area (Pierce, 1931).

An attempt at implementing an artifact according to the

suggested solution is performed next. This stage is
shown as Development in the diagram.
7
 Partially or fully successful implementations are then
Evaluated (according to the functional specification
implicit or explicit in the suggestion).

Development, Evaluation and further Suggestion are

frequently iteratively performed in the course of the
research (design) effort.

The basis of the iteration, the flow from partial

completion of the cycle back to Awareness of the Problem,
is indicated by the Circumscription arrow.

Conclusion indicates termination of a specific design

project.
8
New knowledge production is indicated in the Figure by
the arrows labeled Circumscription and Operation and
Goal Knowledge.

The Circumscription process is especially important to

understanding design research because it generates
understanding that could only be gained from the specific
act of construction.

Circumscription is a formal logical method (McCarthy,

1980) that assumes that every fragment of knowledge is
valid only in certain situations.
9
Further, the applicability of knowledge can only be
determined through the detection and analysis of
contradictions – in common language, the design
researcher learns or discovers when things don’t work
“according to theory."

This happens many times not due to a

misunderstanding of the theory, but due to the
necessarily incomplete nature of ANY knowledge
base.

The design process, when interrupted and forced

back to Awareness of Problem in this way,
contributes valuable constraint knowledge to the
understanding of the always-incomplete-theories that
abductively motivated the original design.
10
Artifacts
 are not exempt from natural laws or behavioral theories

 artifacts rely on existing "kernel theories" that are applied,

tested, modified, and extended through the experience,
creativity, intuition, and problem solving capabilities of the
researcher [Walls et al. 1992; Markus et al. 2002]

11
Design Science Research outputs
[March & Smith, 1995]
 Constructs
 conceptual vocabulary of a problem/solution domain
 Methods
 algorithms and practices to perform a specific task
 Models
 a set of propositions or statements expressing relationships
among constructs
 abstractions and representations
 Instantiations
 constitute the realization of constructs, models and methods in a
working system
 implemented and prototype systems
 Better theories
 artifact construction

12
 Design research outputs

constructs
better theories
emergent theory about models
embedded phenomena
abstraction models
abstraction methods
knowledge as constructs
operational principles
better theories

abstraction

artifact as situated implementation instatiations

methods
constructs
[Purao , 2002]

13
 Gregor and Hevner (2013)
14
Specific Examples
 Provide a unifying framework
 Resolve a long-standing question
 Thoroughly explore an area
 Contradict existing knowledge
 Experimentally validate a theory
 Produce an ambitious system
 Provide empirical data
 Derive superior algorithms ( in IR, Machine learning..
 Develop new methodology (for software develop..)
 Develop a new tool (DM/BI, KBS, Stemmer…

15
Design science research perspectives
 The structure

 Evaluation criteria

 Evaluation approach

16
Artifact structure
 Structure of the artifact
 the information space the artifact spans

 basis for deducing all required information about the artifact

 determines the configurational characteristics necessary to

enable the evaluation of the artifact

17
Evaluation criteria
 Evaluation criteria
 the dimensions of the information space which are relevant for
determining the utility of the artifact

 can differ on the purpose of the evaluation

18
Evaluation approach
 Evaluation approach
 the procedure how to practically test an artifact

 defines all roles concerned with the assessment and the way of
handling the evaluation

 result is a decision whether or not the artifact meets the

evaluation criteria based on the available information.

19
Cont...
 Quantative evaluation:
 originally developed in the natural sciences to study natural
phenomena
 approaches:
 survey methods
 laboratory experiments
 formal methods (e.g. econometrics)
 numerical methods (e.g. mathematical modeling)
 Quantiatative data Sources
 Questionair
 Database and servers

20
Cont...
 Qualitative evaluation:
 developed in the social sciences to enable researchers to
study social and cultural phenomena

 approaches:
 action research
 case study research
 ethnography
 grounded theory

 qualitative data sources:

 observation and participant observation (fieldwork)
 interviews
 documents and texts
 the researcher’s impressions and reactions

21
Constructs

Structure Evaluation criteria Evaluation approach

meta-model of construct deficit ontological analysis

the vocabulary construct overload

construct redundancy

construct excess

22
 Methods

Structure Evaluation criteria Evaluation approach

process-based meta appropriateness laboratory research
model completeness field inquiries
intended applications consistency surveys
conditions of case studies
applicability action research
products and results of
practice descriptions
the method application
interpretative research
reference to constructs

23
Models

Structure Evaluation criteria Evaluation approach

 domain correcteness syntactical validation
 scope, purpose completeness integrity checking

 syntax and semantics clarity sampling using selective

 terminology flexibility matching of data to actual

 intended application simplicity
external phenomena or
trusted surrogate
applicability
integration tests
implementability
risk and cost analysis

user surveys

24
Instantiations
Structure Evaluation criteria Evaluation approach
executable implementation functionality code inspection
in a programming language
usability testing
reference to a design model
reliability code analysis
reference to a requirement
specification performance verification

reference to the supportability

documentation
reference to quality
management documents
reference to configuration
management documents
reference to project
management documents

25
Conclusion

Good research results require a careful

design of the research methodology and
considerable evaluation efforts

26