JISC DEVELOPMENT PROGRAMMES

Project Document Cover Sheet

CORE User Requirement Study
Project
Project Acronym CORE Project ID
Project Title Collaborative Orthopaedic Research Environment
Start Date 01 November 2004 End Date 31 October 2006
Lead Institution University of Southampton
Project Manager & Dr Gary Wills
contact details Intelligence, Agents, Multimedia Group
School of Electronics and Computer Science
University of Southampton
Southampton, SO17 1BJ
Project Web URL www.core.ecs.soton.ac.uk
Programme Name Virtual Research Environments (05/04) Strand III
(and number)
Programme Manager Dr Maia Dimitrova

Document
Document Title CORE User Requirement Study
Reporting Period
Author(s) & Project Role Matthew Stenning (Co-Investigator), Simon Grange (Co-
Investigator), Yee Wai Sim (Co-Investigator), Chu Wang (Co-
Investigator), Lester Gilberts (Technical Manager), Gary
Wills (Project Manager)
Date 25/08/2005 Filename ecstr_iam05_007.doc
ISBN 0854328440
Technical Report Number ECSTR-IAM05-007
Access † Project and JISC internal „ General dissemination

Document History
Version Date Comments
0a 18 August 2005 Initial draft of report
1a 25 August 2005 Reviewed by Gary Wills and Simon Grange
 CORE – CORE User Requirement Study – Version 1a – 25 August 2005

CORE User Requirement Study

M. Stenning, S. Grange, Y. W. Sim, C. Wang, L. Gilbert, G. B. Wills
School of Electronics and Computer Science
University of Southampton
E-mail: [email protected], [email protected],
{gbw, lg3, yws01, cw2}@ecs.soton.ac.uk
Technical report Number: ECSTR-IAM05-007
ISBN: 0854328440

Abstract
This report describes a study undertaken as part of the Collaborative Orthopaedics
Research Environment (CORE) project to determine the user requirements for a
Virtual Research Environment (VRE). The study was conducted through a
consultation process that involved semi-structured interviews and online survey.
Requirements of users and their recommendations are presented according to the
study’s objectives. The result of this study will be used to inform the development of a
VRE demonstrator in the CORE project.

1 Introduction
The dynamic and technology driven research community, shares its information
through approaches initiated by Gutenberg’s printing press and recognizable
conceptually to scientists since the 18th century [3]. Until recently, it has been the
case that scientific findings are captured, summarized, and shared through
manuscripts. Nevertheless, the Web technologies have dramatically affected the
practices of science already been by the Internet. For example, for biologists, the
genome sequence which is gigabytes in magnitude is now available online [2]. Data is
something that they find on the Web, not only in the lab. The rapidly expanding
computing and storage capabilities of the federated Grid, and advances in optical
networks are accelerating the trends of disseminating and sharing scientific findings
over the Web.

Collaborative Orthopaedics Research Environment (CORE) [5] is a JISC funded

project, which aims to provide a loosely coupled infrastructure that combines clinical
practice, education and research into one working environment. The CORE builds on
the work carried out under the Virtual Orthopaedic European University (VOEU)
project [15].

The architecture of VOEU is integrated and tightly coupled, making it difficult to

expand as user requirements change. In view of this, the CORE takes the foundation
of Virtual Research Environment (VRE) from VOEU and re-builds it into a Grid/Web
services based VRE demonstrator using a Service-Oriented Architecture (SOA) [9].
According to JISC, a VRE can be considered as “a set of applications, services and
resources integrated by a standards-based, service-oriented framework which will be
populated by the research and IT communities working in partnership” [13]. Readers
can refer to [4] and [13] for additional background material and rationale for creating
a VRE.

Page 1 of 12
 CORE – CORE User Requirement Study – Version 1a – 25 August 2005

The objective of this study is to ascertain the issues and requirements for provision of
Grid/Web services that relate to the storage, access, use and reuse, of research data in
repositories, and information from digital libraries and its dissemination, under a VRE.
The extent to which specific users can achieve specified goals with effectiveness,
efficiency and satisfaction using a VRE is largely driven by the requirements of the
users. Hence the study of user requirements plays an important role when developing
a VRE. The end users in this study are researchers and clinicians working in an
orthopaedic (bone) laboratory, who typify a group of e-scientists working on
collaborated projects and a group of e-learners studying in joint-partnership
institutions.

The VRE in the CORE project will incorporate the basic science disciplines of
molecular and tissue biology, engineering and computing with allied medical
disciplines of pharmacology, prosthetics, trials management and the clinical
disciplines concentrating upon musculoskeletal applications in rheumatology and
orthopaedics. This report describes the user requirements for such a VRE in
accordance with the objectives outlined below. Recommendations for each objective
are also included in this report to guide further development activities in the CORE
project.

2 Objectives of the Study

The objectives of this user requirement study are as follows:
1. To identify the user requirements for a VRE.
2. To elucidate enhancements for the planned functions in the CORE.
3. To assess the current practice of discovering, locating and using research
findings (i.e. publications and experimental data) to focus the enhancement of
such processes through the VRE.

3 Methodology
3.1 Semi-structured interviews
The resources available for this study lend itself to a semi-structured interviewing
technique that is an adaptation of methods used by Wood [16] and Spradley [14]. The
important principles underlying such an interviewing technique, as identified by
Wood, are concerned mainly with the nature of the expert’s knowledge. End users of
a VRE are usually experts in their research domain, which the VRE is intended to
support. There is a body of literature in cognitive psychology [12] on the nature of
expertise that has implications for how one should work with experts to gain an
understanding of the way they accomplish their work within a specialised domain.

The aspect of expertise, which the authors find relevant to this study, is the potential
for experts to exercise translation competence. Interviewers are often novices in the
investigated domain. In ethnographic research settings, translation competence [14]
may occur when cultural experts translate their view of their cultural to an outsider.
The more an expert translates for the convenience of an interviewer, the more the
expert’s view becomes oversimplified and distorted. In an effort to avoid the errors
that might result from translation competence, Spradley advocates an approach to
questioning in which the interviewer makes minimal assumptions about experts’

Page 2 of 12
 CORE – CORE User Requirement Study – Version 1a – 25 August 2005

knowledge, and which uses information that they provide as the basis for further
questioning [14].

In view of the translation competence issues, the authors conducted the interviews by
first using very general probing techniques to persuade experts to talk freely about
their domain in a global sense. Expert’s language is recorded and then examined for
category labels and other domain-specific linguistic cues. The authors probe experts
for additional, related information by then using domain-specific terms. For this
reason, the interviewing technique described in this report is considered semi-
structured rather than structured [6]. Instead of designing a priori set of questions to
be asked in a specified order, the authors have various types of questions at their
disposal to be used in opportunistic ways, depending on the demands of the situation.

As stated earlier in this report, end users in the CORE project typify e-Scientists and
e-learners in their research environment; hence, five professionals who work as
researchers in the computer science field and medical field were invited to take part in
this study. The participants were chosen based on their expert knowledge in the
research areas of digital library, information and learning technologies, and the
biomedical domain. Their expertise is appropriate in finding out the requirements
involved with providing Grid/Web services that facilitate the sharing of resources (for
the purpose of research and education) in data repositories and digital libraries.

Starting from the foundation of the VRE in VOEU, a user case scenario (see Figure 1)
was worked out to explore possible functions and application settings. The scenario
described several concrete activities of a fictitious user such as formalising a trial
protocol, selecting a dataset, analysing a dataset and discussing results.

Figure 1: Workflow in the VRE framework

The authors approached each interview by first giving a brief overview of the CORE
project and the planned basic functions such as formalising the trial protocol, storing
and analysing the data, submitting and reviewing articles, and then discussing
experiment findings in a forum. This was then followed by a presentation of the user

Page 3 of 12
 CORE – CORE User Requirement Study – Version 1a – 25 August 2005

case scenario described above. Each interview was generally an hour in duration, and
permission was asked to record the interviews for later analysis and interpretation.
The purpose of the interviews was to stimulate individual reflection upon experience
and act as a starting point to engage participants in identifying their requirements for a
VRE. Some key common questions were included in order to identify strategies for
discovering, locating and using/reusing of research findings (see Appendix 1).

3.2 Online survey

The aim of the online survey was to identify end user requirements in the
collaborative research environment. A survey questionnaire containing nine questions
(see Appendix) was designed to be completed by participants via email. A total
number of 17 orthopaedic surgeons (including 14 Higher Surgical Trainee (HST), one
consultant and two surgeons who did not disclose their grade) took part in this survey.
HSTs are qualified surgeons training to be consultants. Their study is work based, and
they are rarely co-located with others. During the six years of training, they usually
move post twelve times, and they have to keep a logbook (e-Portfolio) as required by
the learning contracts with their professional Colleges.

Participants who took part in the online survey responded to questions in a way that
indicate that they were ‘tech-savvy’, i.e. they use computers at home and work, and
they discover and locate research materials using the Web. HSTs have to conduct
research and publish papers as part of their training. Hence, all of the participants
(except one) are involved in research projects, where 13 of them conduct individual
audits/projects for publication, and 3 others undertake part-time postgraduate degree
in conjunction with their training.

4 Major Findings
In general, the participants were very positive about the VRE and made many
suggestions as to how to enhance the functions of the planned VRE. This section
maps out the major findings derived from the semi-structured interviews and online
survey with the objectives listed in Section 2. Recommendations for each objective
will also be included in this section. Some of the recommendations overlap the
objectives, reinforcing the outcomes of the study.

4.1 Objective 1: To identify the user requirements for a VRE

4.1.1 Requirements
ƒ All of the interviewed participants agreed that any scientific effort will have
diachronic features, i.e. collaborative research activities may extend through time,
enabling the influence of these activities to carry on beyond their timescale,
disseminating research findings beyond the original boundaries of the
collaboration.
ƒ One of the interviewed participants also identified that there is a loose coupling
that exists between researchers as educators and their students, notability in the
context of higher and further education. The findings of research activities can be
used to educate the next generation of researchers.
ƒ In view of the research activities, the idea of centralising and sharing relevant
resources under a VRE is well accepted by all participants. This is because the

Page 4 of 12
 CORE – CORE User Requirement Study – Version 1a – 25 August 2005

sharing of resources can keep others in the research community up-to-date with
the current development.
ƒ An interviewed participant foresees digital library as an important medium for
focusing various channels into a single portal. She also recognises the need for
researchers to undertake experiments, deposit scientific data, and produce pre-
prints using a VRE.
ƒ There were mixed feelings about sharing resources among the participants. This
was highlighted by the tension between the concept of sharing and sense of
ownership of the resources. Some participants welcomed the idea of sharing as it
would make their life easier; while others see that it is pointless to share resources
unless some form of rewards were involved. One of the interviewees implied that
resources per se do not make scientific sense; it is what the researchers do with
these resources and how they are being used that adds scientific value to content
materials.
ƒ The VRE should be made easy to use for users who may not be computer literate,
as pointed out by all interviewed participants. For example, if users are asked to
complete multiple fields (metadata) when depositing articles, they may become
frustrated and give up on using the VRE.
ƒ The comments made by vast majority of the interviewed participants and survey
respondents indicated the requirement to contextualise resources within a
research framework that is relevant to the subject discipline. This emphasises the
importance of providing resources that are of easy access to the users, delivered
to them regardless of their computer platforms. Some degree of personalisation or
user profiling in order to narrow the users’ inquiry on particular subjects seemed
to be envisaged by all the participants.
ƒ A participant from bone laboratory identifies the need to utilise Grid services in
running simulations and analysing large-scale experimental data.
ƒ Majority of the survey respondents undertake research as part of the requirements
of surgeons’ Continuing Professional Development. However, a large number of
them have no experience in collaborative research. The majority of those who
conduct research collaboratively commented that a lack of space for sharing
resources poses difficulties for them when coordinating research activities and
pooling results from trials. Three of them found that collaborative research yields
good results but time consuming compared to an individual project.
ƒ When asked what are the essential facilities and functions that survey respondents
required in a VRE, the majority of them answered that access to shared scientific
data and analysis tools is important in the VRE. Furthermore, three of the
respondents emphasised that the accessibility to the shared resources should be
made easy and simple.

4.1.2 Recommendation
ƒ Sharing of research resources, i.e. scientific data and publications, appears to be
an attractive idea. This is supported by the comments given by survey
respondents that express how time consuming it is to conduct collaborative
research without a space to share scientific data and results. However, issues of
both personal and institutional intellectual capital must be addressed. For example,
initiatives and support to enable resource sharing should be targeted at the
institutional level.
ƒ Portals build on the same technology used for Web sites, but enhance the
functionality and flexibility to cater for the demands of specific classes of user [1].

Page 5 of 12
 CORE – CORE User Requirement Study – Version 1a – 25 August 2005

A useful definition from Andy Powell of UKLON is: “A portal is an online

facility that provides a personalised, single point of access to resources that
supports the end user in one or more tasks (i.e. discovery, learning and
research)” [1]. Hence, a portal is an appropriate approach to construct a VRE. It
can provide personalisation to users and allow them to access the VRE using Web
browsers, which is platform independent.
ƒ A clear requirement priority is the design of a user-oriented portal, aimed at the
non-computer specialist, with accessible resources that are easy to browse, upload
and download, as suggested by the survey respondents.
ƒ Grid services make sense when running large-scale analysis and simulation since
they provide secure and managed access to distributed computational power.
Hence Grid services should be included in the VRE as part of its functions to
support research activities.

4.2 Objective 2: To elucidate enhancements for the planned

functions in the CORE
4.2.1 Requirements
ƒ During the interviews, participants were asked to identify what would enhance
the use and access to a VRE. The major concern of the participants was how to
integrate a VRE into their working environment and working practice. For
instance, problems could arise if users were asked to submit scientific data in
XML format, which might be unfamiliar to them. Most of the participants are
reluctant to change their work output formats in their research projects.
ƒ Three of the interviewed participants in this study also express the need for
interaction between users and VRE administrators. This means resources should
be made dynamic and responsive, where users can add/remove links to resources
in the VRE.
ƒ Supporting storage and retrieval of raw scientific data was a welcome thought to
all participants. The interviewed participants agreed that only a small percentage
of the data generated by many scientific experiments appears in, or is referenced
to the published literature. Hence, providing links between raw scientific data and
articles is a desirable feature that could enhance the functionality of VRE. One of
the interviewed participants added that not only data, which show positive
outcomes, should be included in the VRE but those with negative results should
also be made accessible.
ƒ An interviewed participant felt that the process of formalising trial protocols
should be made flexible. For instance, users should be allowed to choose trial
protocols from a list of available options and only include those that are relevant
to them in the trial procedures. Such an approach will not only give users the
flexibility in conducting their trials, but simultaneously offer them a chance to use
protocols that might be unknown but yet prove useful to them.
ƒ The ability to view scientific data in various ways, i.e. numerically or graphically,
was another feature that majority of the interviewed participants felt was useful.
For example, a biologist might want to visualise scientific data graphically while
a mathematician would prefer to see the same set of data numerically.
ƒ Populating the VRE with vast amount of resources is an essential requirement to
attract users and promote its usage. An interviewed participant suggested that
prototype systems should be built to collect and audit the data generated by

Page 6 of 12
 CORE – CORE User Requirement Study – Version 1a – 25 August 2005

scientific activities. These data can in turn be used to populate the VRE once it is
constructed.
ƒ In the online survey conducted, the majority of respondents suggested that the
inclusion of access to a shared database, statistical analysis tools and authoring
tools (i.e. word processors) are essential in the VRE. All respondents, who
answered the question that explores how they integrate research resources into
their work, claimed that they used Microsoft Office (mostly Microsoft Excel) and
SPSS software.

4.2.2 Recommendation
ƒ Users have preconceived expectations of VRE, not only as a source of research
resources, but also as a focus to share good practice. Providing examples of
excellence with tools and templates can do this.
ƒ It is important to ensure that the portal involve potential users through the process
of content development. A participatory-based approach (resources developed,
evaluated and tested by the community) should be considered.
ƒ A loosely coupled architecture, i.e. one that based on SOA, should be used to
build the VRE, as this allows the flexibility of adding, removing or modifying
extra services when user requirements change.
ƒ It appears that constructing a prototype system to collect and audit scientific data
is beneficial. Hence such system should be taken into consideration when
constructing the VRE.
ƒ The online survey shows that the most frequently used software in respondents’
research activities is Microsoft Excel. Hence, the VRE should provide support for
storing and retrieving scientific data in Excel format (files with xls extension).
ƒ All interviewed participants and survey respondents agreed that the planned
functions in the VRE, as discussed in section 3.1, are essential.

4.3 Objective 3: To assess the current practice of discovering,

locating and using research findings (i.e. publications
and experimental data) to focus the enhancement of such
processes through the VRE
4.3.1 Requirements
ƒ Most participants and survey respondents claimed that Google search engine is
the first place that they will be start looking for research materials. However, the
interviewed participants felt uncomfortable about using some of the resources
found this way due to unclear copyright and quality of these materials, or the
links to the resources might be remove without notice. Google Scholar [7] is
another new search engine that attempts to address some of these issues. It
enables users to find articles from a wide variety of academic publishers,
professional societies, preprint repositories and universities, as well as scholarly
articles available across the Web.
ƒ On a few occasions, the interviewed participants reported to visit particular Web
sites to locate resources. It appears that resources found in this way are somewhat
random, which means that finding resources is not an easy task and very few
knew about existing collections and supports services. This highlights the need to
centralise resources, for example, via a portal.

Page 7 of 12
 CORE – CORE User Requirement Study – Version 1a – 25 August 2005

ƒ All of the survey respondents (except one) are using PubMed [11], which is a
search engine for the National Library of Medicine that includes over millions of
citations from MEDLINE and other life science journals for biomedical articles.
Ovid [10] is another resource on the Web that a few survey respondents visit to
discover and locate research materials.

4.3.2 Recommendation
ƒ A strategy to manage the quality of resources should be devised to guarantee the
flow of new materials. This could involve expert input and feedback on content
renewal and, as suggested by participants in this study, peer review. On top of
that, it is also important to derive a mechanism to maintain and check the
resource links’ integrity.
ƒ It will be useful to be able to access the VRE (portal) from popular search engines
such as Google. Metadata in the VRE should be used effectively so that research
resources in the portal can be retrieved via search engines.
ƒ Providing a cross searching facility between Google and PubMed could be
another desirable feature as most of the participants in this study employ these
search engines to find research resources available on the Web.

5 Conclusions
The current development of Web technologies increases the effectiveness of
collaboration between scientists. Digital libraries lie in the heart of these technologies,
acting as an information grid that consists of a collection of resources for learning and
teaching, data repositories for research purposes, or as archives of diverse cultural
heritage materials [8]. The VRE has been implemented as a Web-based environment
for supporting a critical subset of the e-science cycle: the collation and analysis of
experimental results, the organisation of internal project discussions, and the
production of appropriate outline documents depending on the requirements of
conferences and journals selected for dissemination [15].

It is essential to understand (through further research and evaluation) the users’

experience and requirements for a VRE in order to promote its usage. This user
requirement study has involved a total of five professionals contributing to the semi-
structured interviews and 17 orthopaedic surgeons responded to the online survey.
Reactions toward the proposed VRE in the CORE project were on the whole positive
and users generally agreed that there is need to centralise and shared research
resources. In addition, the results from this study have reinforced the necessity of
adopting portal technologies in developing the VRE demonstrator.

The valuable suggestions of all participants and outcomes of their interviews will be
used to gain a thorough understanding of user requirements, which should be taken
into consideration when designing the Grid/Web services based demonstrator in the
CORE project.

6 Acknowledgements
The authors would like to thank all the interviewed participants and online survey
respondents who have contributed time and insights to the interviews.

Page 8 of 12
 CORE – CORE User Requirement Study – Version 1a – 25 August 2005

Reference
1. Allan, R., Awre, C., Baker, M. and Fish, A. (2004) Portals and Portlets 2003.
National e-Science Centre, article available at:
http://www.nesc.ac.uk/technical_papers/UKeS-2004-06.pdf
2. Bairoch, A. and Apweiler, R. (2000) The SWISS-PORT Protein Sequence
Database and its Supplement TrEMBL in 2000. Nucleic Acids Research, 28(1),
pp. 45-48.
3. Castells, M. (2001) The Internet Galaxy: Reflections on the Internet, Business,
and Society, Oxford University Press, Inc., New York, USA.
4. Cox, A. (2004) Building Collaborative eResearch Environments. JISC, article
available at: http://www.jisc.ac.uk/index.cfm?name=event_report_eresearch
5. CORE Project Proposal. University of Southampton, article available at:
http://www.core.ecs.soton.ac.uk/overview/
6. Gordon, S. E. and Gill, R. T. (1992) Knowledge acquisition with question probes
and conceptual graph structures. In Lauer, T. W., Peacock, E. and Graesser, A. C.
(Eds.) Questions and information systems, Lawrence Erlbaum Associates,
Hillsdale, New Jersey, pp. 29-46.
7. Google Scholar, available at http://scholar.google.com/
8. Lyon, L. (2002) Emerging information architectures for distributed digital
libraries. In Proceedings of the International Conference of Digital Library: IT
Opportunities and Challenges in the New Millennium, Beijing, China.
9. Newcomer, E. and Lomow, G. (2005) Service-Oriented Architecture. Addison
Wesley.
10. Ovid, available at http://www.ovid.com/site/index.jsp
11. PubMed, available at http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?DB=pubmed
12. Reimann, P. and Chi, M. T. H. (1989) Human Expertise. In Gilhooly, K. J. (Ed.)
Human and machine problem solving, Plenum Press, New York, pp. 161-191.
13. Roadmap for a UK Virtual Research Environment: Report of the JCSR VRE
Working Group (2004). JISC, article available at:
http://www.jisc.ac.uk/uploaded_documents/VRE roadmap v4.pdf
14. Spradley, J. P. (1979) The ethnographic interview. Holt, Rinehart and Winston,
New York.
15. Wills, G., Miles-Board, T., Bailey, C., Carr, L., Gee, Q., Hall, W. and Grange, S.
(2005) The Dynamic Review Journal: a Scholarly Archive. New Review of
Hypermedia and Multimedia, 11(1), pp. 69-89.
16. Wood, L. E. (1997) Semi-Structured Interviewing for User-Centered Design.
Interactions, 4(2), pp. 48-61.

Page 9 of 12
 CORE – CORE User Requirement Study – Version 1a – 25 August 2005

Appendix 1
The following are key common questions asked during the interviews:

ƒ What are the requirements that you will envisage in a VRE?

ƒ In your opinion, what developments would enhance the planned functions of the
planned VRE in CORE?

ƒ What strategies do you use to discover and locate research materials?

ƒ What collections, portals, services and networks are you aware of in the area of
your research?

ƒ How do you integrate discovered research resources, i.e. raw scientific data, into
your research practice?

ƒ What are the lessons learnt from your experience in projects that involve research
resources sharing and discovery?

Page 10 of 12
 CORE – CORE User Requirement Study – Version 1a – 25 August 2005

Appendix 2
VRE User Requirements Questionnaire
Please delete question answers as necessary

1) Are you currently involved in a research project?

a) Taken a year out from clinical training for dedicated research
b) Part time post graduate degree in conjunction with clinical training
c) Audit / project for publication
d) No involvement in a project at present

2) What form of IT do you use when undertaking research?

a) PC at home
b) PC at work
c) Special equipment e.g. dedicated facilities in a university
d) None

3) What are the experiences and lessons learnt from your experience in projects
that involve collaborative research?

4) In your opinion, what developments would enhance the proposed virtual

research environment?
a) Statistical tools
b) On – line study environment (e.g. Orthoteers)
c) A trials manager
d) Other (please specify)

5) What resources do you use to discover and locate research materials?

a) Google
b) Scholarly Google
c) Pub Med
d) Other: - (Please specify)

6) What IT supports and resources are you aware of in the area of your
research?
a) Hospital
b) University
c) Other (please specify)

Page 11 of 12
 CORE – CORE User Requirement Study – Version 1a – 25 August 2005

7) How do you integrate discovered research resources, i.e. raw data, into your
research practice?
a) Do you use PC’s or Macs yes no

b) What file formats do you use

8) What are the essential facilities and functions that you think are required in
the Virtual Research Environment?

9) Do you think the virtual research environment would be useful in your

research activities?

Thank you for your time

Page 12 of 12