A Short Guide To Research Methods For M.Sc. and Ph.D. Degrees
A Short Guide To Research Methods For M.Sc. and Ph.D. Degrees
A Short Guide To Research Methods For M.Sc. and Ph.D. Degrees
Research Methods
for
M.Sc. and Ph.D. Degrees
David Vernon
Department of Computer Engineering
Revision 1.2
February 2007
Table of Contents
1. Getting Started 1
2. Moving Along 4
3. Reading 7
4. Writing 9
4.1 Good Writing Discipline 9
4.2 Good Writing Style 9
Research is Difficult Doing an M.Sc. or Ph.D. is hard work and getting started is
perhaps the most difficult part. A research degree focuses on you
becoming an expert in a particular topic and adding to the body of
scientific and engineering knowledge on that topic. However,
research is also concerned with learning, and in particular with
learning to work independently and being able to develop your own
understanding of any given topic. Starting with very little, where do
you begin? The following are some pointers on how to get started.
Understand the Problem Your supervisor will have provided you with an outline description
of your research topic. To begin with, and to test your
understanding of this outline, you should try to expand on the
problem statement. In your own words, try describing informally
exactly what the final system should do, what data it will take as
input, what data it will produce as output, and how these input and
output processes are accomplished. Next, try to say exactly how
the input data is transformed in order to produce the required
output. Describe why this problem is relevant. Say why it is
important to solve it. What are the consequences of finding a
solution? What does solving this problem mean?
Start Reading Your supervisor will suggest some initial reading: journal papers,
conference papers, book chapters. Read them all.
Start Writing Arm yourself with a pen and write a short synopsis of every paper
and book chapter you read. Write down the key message (one or
two sentences) and the main contribution (one or two paragraphs).
It is also worth writing down one or two quotes from the paper if
they provide some important insight into the topic. This is hard
work. Don’t underestimate it but do be aware of how important it is
to do it. The very act of writing helps crystallize ideas. Remember:
summarize every paper you read. Don’t be tempted to copy the
paper abstract: the point of the exercise is for you to express your
understanding of the paper in your own words. This is a necessary
part of the learning experience. You won’t learn if you don’t write it
down in your own words.
Start Talking Talk to your supervisor: ask questions, try to explain your ideas, tell
him what you are finding hard (and easy). Talk to other students:
explain to them what the goal of your work is, what the central
problem is, how you are going to try to solve it, how other people
have tried to solve it, what difficulties they encountered, why their
approach isn’t as good as the one you are trying to use. This is
hard, but it gets easier with practice. Now, try to write down these
ideas. Use a pen and paper first. Type the notes later. Give a
short presentation to your group.
Start Simple Carve up the problem. Is there a simpler version you can tackle? If
so, do that first, and then move on to the more complicated version
later. In other words, try to structure your research goals so that
there is (a) an essential but fairly easily achieved goal, (b) a
desirable but somewhat harder or risky goal, and (c) the ideal
goal, something that would represent a real breakthrough. This
gives you both a way of structuring your work and a safety net if
everything doesn’t go as expected (and it won’t).
Formalize As early as possible, you need to try to formalize the problem you
are working on. You won’t be able to do this straight away
because you will need to get a grounding in the background theory
first. But you should at least make an attempt so that, after having
1
www.simonsingh.net/Andrew_Wiles.html
2
read around the topic, the next time you try you will see how much
progress you have made.
Learn About Tools As you study for your degree and pursue your research, you will
become and expert in two different domains: the problem domain
and the solution domain. The problem domain refers to the
theoretical issues by which we can model and solve the problem.
This is the primary focus of research. At the same time, you also
need to know how to handle the tools that will enable you to
implement the solution. These might include configuration tools,
compilers, libraries, programming languages, operating systems,
application programming interface (API) definitions, class libraries,
toolkits, networking infrastructures and protocols, hardware
platforms, input devices (data acquisition), output devices (control
systems). These tools belong to the solution domain and they
give you an opportunity to learn other things in parallel to your
formal research in the problem domain. Typically, they are more
straightforward to deal with so will give you a chance to feel you
are making some progress early on.
3
2. Moving Along
Make Notes The best way to organize your research is to keep a log book of all
work in progress. Get a notebook and write everything you do on
the project into this log book every day. Every thought and
observation you have on your project should go into this book,
along with notes of meetings with your supervisor, results,
theoretical developments, calculations, references, anything that is
relevant to your work.
Your Own Ability you have to have a lot of self-belief to do it. At the outset, there is
a lot you don’t know. You don’t understand the problem, you don’t
know how other people have approached it, you don’t know how
successful they were, you don’t know if you will be able to solve it
yourself, and you don’t know how long it will take. All that is certain
is that you have to spend many hours every day, often with no
reward, chipping away at the problem, hoping for progress. To get
through this, you need to believe in your own ability. You need to
constantly remind yourself that you can do it. It will take hard work,
determination, and sometimes a little help from others, but you can
do it.
Believe In You must also believe that the topic you are working on is
Your Problem worthwhile. If you are not convinced that solving this problem
matters, don’t even think about working on it. Find another topic.
You should also enjoy working on it: it should represent a
challenge that motivates you to get out of bed each morning (well,
maybe not every morning, but most mornings!) There is no
substitute for enthusiasm in a research degree, and you will need
lots of it to get you through. So, if you find yourself working on
something that simply does not fire your imagination, consider
switching topic. Talk to your supervisor and do it as soon as
possible. Don’t delay.
Practice, Practice, Practice Research is a skill and can be learned. However, just like every
other skill, you have to practice it to get better. You can’t improve
just by thinking about it. Research is not a spectator event: it is a
contact sport.
4
Ph.D. Topics Evolve Don’t be afraid to re-scope your research as you learn more about
it. Your supervisor will help guide you in this. If it turns out that the
problem is much more difficult that it seemed at first, it needs to be
redefined. A Ph.D. thesis typically will ask and attempt to answer a
hard question. Make sure it answers convincingly. If it doesn’t,
either you need to generate a better answer or ask a slightly
different question.
Jump Start Your Day Getting started first thing in the morning is often difficult so, when
you stop the night before, set yourself a task for the morning that is
easy and even enjoyable. For example, you might update your
bibliography, comment some code, or draw a diagram. That way,
you will reduce the temptation to avoid starting work early by doing
other ‘important’ tasks such as checking email or browsing the
web. By the way, browsing the web is not research, not unless you
are searching for some specific paper and in that case you should
be using Google Scholar.
Dealing with Criticism You will inevitably have to take a lot of criticism during you time as
a research student. Being wrong and being ignorant2 goes with
the territory. Remember too that you can’t learn without mistakes:
learning means improving, and improvement implies that you start
out being less capable. Criticism is just a way of pointing this out
so don’t take offence. Criticism is a positive act, not a negative
one. Criticism allows you to learn and to improve and that’s what
you are here for. Most criticism will be valid but some may not be.
Learn to trust you own judgment. If you think the criticism is invalid,
say so, but be prepared to defend your position cogently, rationally,
and quietly. Most invalid criticism is caused by a
misunderstanding. This means you may not have explained your
ideas clearly in the first place. Try to explain them better and then
see if the same criticism is forthcoming.
Don’t Give Up Even when everything looks black and your problem looks
impossible, keep on going. However, give yourself lots of mini-
holidays: take ten-minute breaks away from the daily grind (that’s
ten minutes, not ten days!). Don’t keep thinking about a problem all
the time.
2
Ignorant does not mean stupid. It means lacking knowledge.
5
Be Prepared Frequently, the solution you are looking for will come to you when
For Inspiration you least expect it and often when you are not thinking about the
problem. So always keep pen and paper handy, just in case!
Get Yourself a Theory! The great power of science and engineering is that it allows us to
predict how systems will behave. To be able to predict something,
though, you must have a model: an abstract formulation. This is
how science and engineering works. Mathematics is the bed-rock
of engineering so if at all possible try to describe your problem in
mathematical terms. If this isn’t possible, at least try to formalize
the problem and use rigorous arguments: identify your axioms and
state your hypothesis clearly and in a manner that can be refuted
(falsified is the more correct term). You can never prove a
hypothesis (only a theorem) but any worthwhile hypothesis will
have a clearly stated test that will show it is incorrect.
6
3. Reading
Recursive Reading Your supervisor will have given you an initial list of papers to read.
Get copies and read them. They will refer to other papers. Get the
main papers cited and read these. This is a recursive procedure.
Eventually, you will begin to see closure and you won’t be
encountering many new papers. At that point, you have covered
the topic in depth.
Different Levels of Not all the articles you collect will be equally relevant or important.
Reading Consequently, it’s not efficient to give each of them the same
attention. But it is not easy to know how relevant it is until you read
it. So how do you proceed? To solve this dilemma, you should
know that there are three levels of reading: shallow reading,
focused reading, and deep reading.
Shallow Reading
Some papers will be mainly background material providing the
context to the research. The topics are relevant to your work but
not directly related. Often it is sufficient to read just the abstract,
introduction, and conclusion, and then write a one or two line
summary of the main issue addressed.
Focussed Reading
Other papers are directly relevant and provide, for example,
alternative ideas on the topic of your research. These papers
should be read thoroughly, perhaps twice. You should aim to write
a one-paragraph summary after you have read it. These papers
will typically go in your literature survey.
Deep Reading
Finally, some papers, typically 10 to 15 for an M.Sc. and 15-30 for
a Ph.D., will be absolutely central to your work and you will need to
read them several times to really understand them. You may need
to work through some examples and you will probably have to refer
to other papers or texts to understand some of the concepts
described. You should write a 1-2 page summary of the content of
these papers. If they contain mathematical results, you should
include these too, explaining each term and the importance of the
results. After many careful readings, you should know as much
7
about the topic as the author. A good test of your understanding of
a paper is to see if you can give a short presentation on it and
explain it to other people in your group.
Build a Bibliography For everything you read, insert the full citation of the paper or book
chapter in your bibliography so that you can refer to it in your
subsequent writing.
Make sure you keep a full citation index, i.e., you must record
exactly where every article you copy comes from. Typically, you
need to record the title of the article, the authors, the name of the
magazine/journal/book, the volume and number of the journal or
magazine, and the page numbers. If it’s a chapter in a book and
the author of the chapter is different from the editor of the book,
you need to record both sets of names.
Variety Note that the ‘reading in’ phase of the project can last quite a long
time (there’s a lot of reading and writing to be done) and it can help
to overlap with some of the other early tasks, such as learning
about the solution domain.
Reading Means Writing Finally, it is very important that you realize that, in order to fully
understand anything that you read, you must write it up in your own
words. If you can’t express or speak about a given idea, then you
haven’t truly understood it in any useful way. This is so important
that it’s worth saying a third time:
8
4. Writing
Writing is not easy. There are two aspects to this: the discipline of
writing and the style of writing. Discipline refers to the physical act
of writing: the process of assembling ideas and getting them down
on paper. Style refers to elegance and simplicity: the power of your
writing to communicate an idea.
Use pen and paper. Write things down long-hand. Later on,
write these notes up more neatly and in a more organized
fashion. Once you get good at this, you can go straight from
long-hand notes to typed document, but it’s very helpful at the
beginning to first create an intermediate long-hand version.
†
Several of these guidelines are adapted from Strunk and White
1979.
9
reading are sequential processes. Therefore, you have to
construct the meaning of your message, piece by piece, in a linear
time-line. However, the meaning you intend to convey may
emerge from many sources, not all related in a nice orderly
fashion. This creates a problem for the writer: how to order the
messages contained in each sentence effectively. The job of a
writer is to make the sequence of pieces as mutually-relevant as
possible so that the story or message builds naturally, each piece
adding to the previous one. When the pieces are presented out of
order (e.g. parenthetical expressions in the wrong place, or two
related sentences split by a third) the impact on the reader is
lessened. And remember the golden rules of good writing: keep to
the point and keep it simple.
10
to follow the story line. Strike a balance but favour brevity over
complexity.
If you use pictures and diagrams (and you should), make sure
each one has a self-contained explanatory caption. Never
refer to a picture or diagram in the main text without saying
what it is. For example, never say “Figure 2.3 shows the
results of the noise test” and then carry on to another topic.
Help the reader. Summarize the content of the figure in a short
sentence: “Figure 2.3 shows the results of the noise test.
These results demonstrate the robustness of the system to
Gaussian noise with a standard deviation of 2.3 or less.” If you
have copied the figure from a book or article you must cite the
source.
11
Don’t take short-cuts. Explain what you mean. Don’t leave the
reader to struggle trying to figure out the real meaning of your
carefully constructed but complicated sentence. He may
conclude there is none. Explain all acronyms the first time you
use them.
3
Blaise Pascal, "Lettres provinciales", letter 16, 1657.
12
5 Writing Scientific Papers
The Importance of The ultimate test of M.Sc. or Ph.D. research is whether or not it is
Writing Papers worthy of publication in a journal or in the proceedings of a
conference. Consequently, research papers are the primary output
of a research degree, not software. Software may be needed to
validate the ideas in the paper and software may be the whole
point of developing the idea, but the contribution to knowledge is
the idea itself, encapsulated in a paper. Indeed, it should be
possible to re-write or re-generate the software based on the
information contained in the paper.
Know Your Reader Assume the person reading your paper is intelligent but ignorant
(i.e. they don’t know everything about the topic so part of the role
of the paper is to bring them up to speed quickly). Assume the
person reading your paper misunderstands thing easily so make
sure the argument is really clear. Make it easy for them to say:
‘nice idea, good model, great validation; yes, the community would
like to know about this’. If someone doesn’t understand your paper,
assume it is your fault, not theirs. Find out where they got lost and
improve it.
Structure The structure of your paper (or thesis) is critical to the impact you
make in your writing. Remember that you are trying to convey a
message to the reader and, since this message is likely to be quite
complicated, you must assist the reader by making your points clearly
and in a logical order. Think of it as telling an exciting story: you
want to tell enough early on to attract the reader’s interest but not too
much otherwise he will become confused. You want to build up to a
climax, incrementally revealing more and more of your message, but
doing so in a way which builds on what you have already said. This
is what we mean by a logical structure: breaking up the ‘story’ into a
linear sequence of messages which follow naturally one from the
other and which lead to an interesting conclusion (i.e. a conclusion
you couldn’t have guessed from reading page 1).
13
state its relevance or importance, to offer a model of the subject
you are investigating, and to provide either some theoretical or
empirical evidence that the model is valid. You should also provide
an assessment of how well it works.
Don’t Start at the Although papers are intended to be read from beginning to end,
Beginning and the argument should flow linearly from beginning to end, it isn’t
always best to write the paper in that order. Often, it is a good idea
to start by describing the technique and write the introduction later,
once you’ve established the core message, and possibly after you
have drawn your conclusions. The abstract should always be
written last, once everything is clear and in place.
Do Your Best and then Be prepared to write, and re-write, many times. It can take up to
Improve It ten attempts just to get a good first draft of a paper. Once you do
have a good draft, ask other people to read it. Do not ask people
to read early drafts: it’s impolite and says to them “I think my time
is more valuable than yours so I’m not going to bother giving you
my best attempt”. Give them your best and thank them for their
time. Under no circumstances use your supervisor as a proof-
reader to correct mistakes or get hints on how to improve structure
before you are certain that the paper can’t be improved! You will
probably be wrong, but that should be your goal.
Reviews Journal papers, and most conference papers, are subject to peer
review. This means that two to four referees read the paper and
provide a critical assessment of its contents. These reviewers are
normally experts in the field. You need to convince them that there
is merit in your work. This is hard: they are very busy people and
don’t have the patience and motivation to read poorly-written text,
no matter how important the issue. Make it easy for them.
14
model? Did you provide enough evidence of the validity of the
model, e.g., by providing quantitative tests? How did you establish
the robustness, generality, or limitations of the technique?
Choose Your Forum Quite often, journals and conferences have different standards,
Carefully with journals requiring a more substantial contribution to
knowledge. This is not always the case: it can be just as difficult to
publish in some top-flight conferences as it is to publish in a
journal. In any case, choose your conference and journal carefully.
Make sure there is a good match between the subject matter of a
journal or conference and the topic of your paper.
Try, Try, Try Again If your paper is rejected, don’t give up. Take on board the
reviewers’ comments and improve the paper. Then submit it
somewhere else.
15
6. Ph.D. and M.Sc. Dissertations
You should design your own thesis to the level of detail given in the
proposed structure, adapting it to your own particular needs. Note
that you should do this before you start writing anything. It’s just
like designing a piece of software: the sooner you start coding, the
longer it will take (and the more likely it is to be wrong). Try to
achieve modularity and independence amongst your chapters and
sections. At the same time, remember you are trying to convey a
convincing message to the reader. Again, it’s like telling a good
story: you have to keep the reader interested and he has to be able
to follow the story-line (which means there has to be one: make
sure there is). Keep the thread of the story going continuously,
from section to section, and from chapter to chapter by providing
link sentences or paragraphs. At the end of a chapter, for
example, remind the reader of the important messages, tell him
why they are important, and then say what you need to look at
next, and why, in order to continue with the story. That’s your cue
for the next chapter.
16
accepted for publication in a journal. This should be your target.
The research described in an M.Sc. thesis should have a good
chance of being accepted for a relevant conference.
17
the system and more to the overall objectives, methodologies, and
findings of the dissertation.
7. Looking Forward
Congratulations on having completed the guide. At this point, you
might be wondering if all this research is worth it. Why bother?
Why do all this work? Here’s why. Along the path of a research
degree, you grow. You become able to do things – hard things –
that you could only dream of doing before: developing a new model
or algorithm of your own, learning how to master a new technique,
seeing simplicity in a complex equation, having and being able to
convince others of your own view on an issue. But these are the
little rewards that accompany the process. The big reward comes
after the degree is complete and after the papers have been
published. This is when you realize that you have changed and
that you can now tackle more or less any problem, with complete
confidence. The unknown becomes a challenge and the reward is
success. This success stays with you for the rest of your
professional life.
Acknowledgements
Much of the advice on research methodology is based on [Bundy
et al. 2004] while the pointers on good writing have their roots in
[Strunck and White 1979].
18
Further Reading
19
Appendix I - Typical Structure of a Thesis
Title Page
Abstract
What is the subject matter of the thesis: what did you do?
Motivation: why is it important?
Significance: what contribution does the thesis make?
The abstract should be approximately 200 words long. It
normally takes at least ten revisions to achieve a good abstract.
The abstract should be written after the thesis has been
completed.
Table of Contents
Chapters
Sections
Acknowledgements
Chapter 7. Evaluation
References
Appendices
20