SCS703: Advanced Research

Methods in Computing
Doctor of Philosophy in Information Technology
and
Doctor of Philosophy in Computer Science

PREPARED BY:
Dr. Geoffrey Mariga
Murang’a University of Technology
[email protected]
Objectives of this Course …
• What is Research and a research problem?
• How to identify and define a useful research
problem?
• How to do a literature survey?
• How to select a research method for a given
research problem?
• How to carry out a research study?
• What are the data analysis techniques?
• How to conclude or draw conclusions?
• How to present your research work/study?
• How to write a formal research article?
…and others
Course Organization
• Lectures
• Discussion and review/feedback
sessions
• Presentations
• Exercise sessions
• Exams (Continuous assessment: 40%,
End of semester exam: 60%)
Introduction to Research Methods in Computing

Defining Research:
- Scientific investigation involving systematic search for
pertinent information on a specific topic for the purpose of
discovering new knowledge, interpreting existing
knowledge or revising erroneous or incomplete knowledge.
Research comprises:
- Defining and redefining problems,
- Formulating hypothesis or suggested solutions;
- Collecting, organising and evaluating data;
- Making deductions and reaching conclusions; and
- Carefully testing the conclusions to determine whether they fit the
formulated hypothesis.
Research is, thus, an original contribution to the existing stock
of knowledge, with the help of study, observation, comparison
and experiment, making for its advancement.
Research is divided into two general categories:
(1) Basic research is inquiry aimed at increasing scientific
knowledge, and
(2) Applied research is effort aimed at using basic research
for solving problems or developing new processes,
products, or techniques.
Motivation in Research
The possible motives for doing research may be either one or more of
the following:
1. Desire to get a research degree along with its consequential
benefits;
2. Desire to face the challenge in solving the unsolved problems, i.e.,
concern over practical problems initiates research;
3. Desire to get intellectual joy of doing some creative work;
4. Desire to be of service to society;
5. Desire to get respectability.
Other reasons:
- Government directive,
- Employment conditions,
- Curiosity about new things,
- Desire to understand causal relationships,
- Social thinking and awakening,
and the like may as well motivate (or at times compel) people to
perform research operations.
Researching Skills’
In this ‘Age of Information’ the following skills are
necessary when undertaking research:
• How to find the ‘right’ information: Searching Skills
• How to evaluate it: Reviewing Skills
• How to develop/improve it: Engineering/Scientific Skills
• How to report it clearly and accurately: Writing
Skills
• How to make money out of it: Business Skills
One Fundamental Aspect of Research is “Asking
Questions”
• As many as one can…
• Framing Questions
• Identifying
Sensible/Meaningful/Useful/Relevant/Important
Questions
• Investigate those questions
• Report those questions
Sensible Questions means asking about
…
• How to find the information
• How to evaluate it
• How to report it clearly and accurately
• How to improve it
• How to make money out of it
…. the list is not exhaustive

For example, the W5H2 Analysis

• (What) to find the information; • (Which) to evaluate it
• (Where) to report it clearly and accurately;
• (Why) How to improve it; • (When) How to make money
out of it; • (How)…; • (How Much)…
Need for Research
• You will do Research willingly or unwillingly…for sure…
• So why not to have an ‘edge’ over others by exploring it
• Improve your presentation and communication skills…
• Improve clarity and reduce confusion
• It’s more of a life Style and decision making
• More sensible/useful life… more satisfaction
• Better citizen of the world…probably more human

What is NOT Research…

1. Authority: Enforce obedience, orders
2. Tradition: Customs or beliefs from generation to generation
3. Common sense: Sound judgement
4. Media myths: Traditional story
5. Personal experience
Research Approaches
There are two basic approaches to research:
- Quantitative approach and
- Qualitative approach.

Quantitative Approach
Involves generation of data in quantitative form which can be
subjected to rigorous quantitative analysis in a formal and
rigid fashion.

Further classified into:

- inferential,
- experimental and
- simulation approaches to research.
Inferential Approach
- Involves inferring characteristics or relationships of a population from
sample.
- Usually means survey research where a sample of population is studied
(questioned or observed) to determine its characteristics, and it is then
inferred that the population has the same characteristics.

Experimental Approach
- Some variables are manipulated to observe their effect on other
variables.

Simulation Approach
- Involves the construction of an artificial environment within which
relevant information and data can be generated.
Qualitative Approach

Concerned with subjective assessment of attitudes, opinions

and behaviour.

Qualitative Approach to research is a function of researcher’s

insights and impressions.

Such an approach to research generates results either in:

- non-quantitative form or,
- A form which are not subjected to rigorous quantitative
analysis.

Generally, the techniques of focus are group interviews,

projective techniques and depth interviews are used.
 Research Process
 Realize there is a problem
 Look to see if anyone has solved it
 Develop a plan to solve it
 Solve it
 Evaluate your solution
 Disseminate your solution
 Scientific Methods in Computer Science
- What is specific for computing research is that its objects of
investigation are artifacts (computer-related phenomena
observed in a scientific investigation or experiment that is not
naturally present but occurs as a result of the preparative or
investigative procedure).
- These artifacts change concurrently with the development of
theories describing them and simultaneously with the
growing practical experience in their usage.
- Computer science can be divided into Theoretical,
Experimental and Simulation CS, which are three
methodologically distinct areas. One method is however
common for all three of them, and that is modeling.
 Scientific Methods in Computer Science
Modeling
- Modeling is a process that always occurs in science, in a sense
that the phenomenon of interest must be simplified, in order
to be studied.
- A model has to take into account the relevant features of a
phenomenon. It obviously means that we are supposed to
know which features are relevant. That is possible because
there is always some theoretical ground that we start from
when doing science.
- A simplified model of a phenomenon means that we have a
sort of description in some symbolic language, which enables
us to predict observable/measurable consequences of given
changes in a system. Theory, experiment and simulation are
all about (more or less detailed) models of phenomena.
 Scientific Methods in Computer Science
A lot of research in computing involves software
development (SE).
SE is a multi-disciplinary field
We need to investigate
 Which tools, techniques, and processes they use?,
 What social and cognitive processes surrounding
them?
 How individual software engineer develop software?
 How teams and organizations coordinate their
efforts?
 Computer Science
• Origins
– Mathematics,
– Engineering, and
– Commercial practice.
• Evolved into
– Theoretical,
– Experimental and
– Design (or user) orientated aspects.
• Balance and synthesize these aspects
• Mathematics (what is?), CS (how to?)
 Computing Directions
- Computer Science
• most basic field of computing, using algorithms to communicate
information.
• Practical realm of CS involves programming languages and computer
graphics, for instance, and the theoretical realm involves things like
complex computational theory and very large-scale integration.
- Information Systems
• About acquiring, formatting, holding, and retrieving information.
• Emphases is on both the computer systems that contain the information
and to the people who process the information into the system.
• It is more business and people focused than the other two on this list,
though all have business applications.
- Information Technology
• Involves the software and hardware aspects of our technological world.
• The subject is IT if the purpose is creating, exchanging, accessing, or
securing digital data.
 Experimental Computer Science (ECS)
• ECS is the creation of, or the experimentation with or
on, nontrivial hardware and software systems
– These systems, taken broadly, are called computational
artefacts.
• ECS process:
– Form a hypothesis
– Construct a model and make a prediction
– Design an experiment and collect data
– Analyze results
 Why should we experiment?
• Experiments cannot prove anything with
absolute certainty
• But they are good for:
– Reducing uncertainty about theories, models, and
tools
– Leading to new insights and whole new areas of
investigation
– Quickly eliminating fruitless approaches
 Experimental Hypothesis
• Any scientific research should state a hypothesis,
then
– Provide evidence for/against
– Conclude whether it is supported or refuted
• Should be:
– Precise, explicit statement (stated clearly and in detail, leaving no
room for confusion or doubt).
– Falsifiable: Able to be proved
• Run experiment to confirm or refute
• Ensure that the experiment really tests the
hypothesis
 Elaborating the Hypothesis
• Hypothesis might be that system/theory/technique/ parameter P is:
– Good for task X
– Better than rivals Q and R for task X
• According to:
– Behaviour - correctness or quality of solution
– Coverage - range of problems to which it applies
– Efficiency - resources consumed
• Evidence can be theoretical, experimental or both
– Theoretical evidence - theorem based
– Experimental evidence - testing on a range of examples
 Designing an Experiment
• Specification needs to be complete and explicit
• Make sure the experiment really tests the hypothesis
• Requirements:
– Controlled - other factors must be kept constant
– Quantitative - provide numbers
– Coverage - are tests representative of the full range of the
hypothesis
 Analyzing and Reporting Results
• Analyze the measured data:
– Does statistical evidence really support (or refute) the hypothesis?
– Make sure differences are not due to chance or natural variability
• Be Careful:
– Better to admit to flaws in your methodology
– Don’t generalize without adequate support
• Report everything:
– Procedures, results and conclusions
– So that others can replicate the experiment
– And build on your conclusions
 Message - Prove your Claims
• which means that you have to have claims to prove
• and evidence to back you up
• and evidence is almost always convincing numbers
• from well constructed, all influences considering, set
of experiments
• that are discussed
• and from which a series of conclusions are drawn
 Information Artefacts
• Synthetic phenomena created by people (not nature)
• Artefacts  an instance or implementation of one or more
computational phenomena
– Subject of study, and/or
– Apparatus with which to conduct the study
• Artefacts are Extraordinarily Complex
– Construction
– Dynamic behaviour
 Examples of Artefacts
• Hardware systems: computers, chips and circuit boards
• Software systems: compilers, editors, expert systems, computer-aided
design
• Graphic images and animations, robots, hard-to-construct data files
(execution traces, Utah Tea Pot)
• Programming languages, architectures, protocols, and methodologies
(object-orientation, spiral approach, ...)
 Tips
• Keep a note book!
• Keep even apparently insignificant aspects of
procedure constant between observations
– Time a program, and then repeat the timing while moving
the computer's mouse
• Data collection and presentation
– Repeat observations several times
– Raw data must be included in lab reports
• Data analysis requires (some/lots) of statistics