Development and Evaluation of The Team Work Skill in University Contexts. Are Virtual Environments Effective?
Development and Evaluation of The Team Work Skill in University Contexts. Are Virtual Environments Effective?
Development and Evaluation of The Team Work Skill in University Contexts. Are Virtual Environments Effective?
* Correspondence:
[email protected] Abstract
2
Universidad Nacional de Educacin
a Distancia (UNED), Madrid, Espaa Soft skills refer to those personal competencies affecting the way we interact with
Full list of author information is people and they include aspects such as communication, listening, negotiating, team
available at the end of the article work, leadership, ethics, commitment, etc. (Computerworld 35:24, 2001; Dubrin,
Coaching and Mentoring Skills (NetEffect Series), 2004; J Soc Work Pract 14:149-158,
2000; InfoWorld 20:104, 1998; J Coll Teach Learn 2:1-6, 2005; Comput Can 24:21-22,
1998; Waller, Soft skills for lawyers, 2007). The role of soft skills in terms of personal
growth, employability, social development, social participation, change adaptation,
etc. has been widely acknowledged although there is no agreement regarding the
differential importance of some soft skills over others. However, the latter point is
relative given that it depends on the field of reference. Despite the fact that these
competencies are highly valued in the labor market, universities seem to overlook
this demand throughout students education process and the development and
assessment of these competencies have been given but scant attention. Further,
the increasing emphasis on the use of ICTs in university education and a gradual
trend towards blended-learning and virtual environments have prevented the
development of these competencies as they are thought to require face-to-face
interaction amongst classmates and between students and teachers. This study
presents the results of an innovative proposal that intends to break these barriers
by focusing on promoting the learning and assessment of team work skills in
virtual environments. These results prove the suitability of the program when it
comes to developing and evaluating these skills in a b-learning environment.
Keywords: Blended-learning, Soft skills, Team work, Problem-based learning
Introduction
Competency is a polysemous term which has been defined from different perspectives
(European Commission 2008; Zabala and Arnau 2007), but the most widely accepted
definition seems to be the concept of competency as a set of abilities, knowledge,
procedures, techniques and attitudes a person owns and which are necessary to a)
carry out professional tasks in a specific job in an efficient way; b) solve problems in
an autonomous, free and creative way and c) collaborate in work organization and in
the socio-labor environment. In the Final Report (stage I) of the Tuning project
(Gonzlez and Wagenaar 2003) competences [sic] and skills are understood as
2016 Garca et al. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International
License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium,
provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and
indicate if changes were made.
Garca et al. International Journal of Educational Technology in Higher Education (2016) 13:5 Page 2 of 11
therefore, the formal university syllabus should systematically include the possibility
of developing interpersonal and evaluation skills in a controlled and supervised
environment (Woeran and Kronsteiner 2007).
work teams is grouping. Therefore, teams are set up with 35 members in accordance
with a diagnosis of students learning styles (Vermunts Learning Styles Inventory)
(Vermunt 1992), the aim being setting up self-regulating teams. That is, groups are
intra-heterogeneous in terms of autonomy level but inter-homogeneous in relation to
all the groups in the classroom so as to ensure balance and effectiveness in all teams.
Students are assigned a functional role with specific tasks they will be responsible for
within their team.
The learning process starts with the presentation of a project or mission that students
have to solve by searching for information, analyzing and summarizing it, writing,
making decisions, planning, organizing work and cooperating in a virtual environment.
The virtual format allows for working remotely, coordinating time zones and reconcil-
ing the working pace of all the team members. As previously mentioned, the system
runs under Moodle modified with new plug-ins and with a private server in order to
facilitate the management and evaluation of team work, as well as the specific skills to
solve the learning problem proposed. Given that competency is evaluated by carrying
out tasks and is developed throughout the learning process, evaluation needs to be
anchored in this process in order to collect evidence favoring competency achievement
(Villardn 2006).
Therefore, the competencies that students have to apply in action are defined, and
the evaluation criteria and indicators that will allow assessment of the acquisition of
competencies through students answers to situations taken from a real context are
specified. To sum up, the Evalsoft system has been created to be developed cyclically in
five stages (Fig. 1). It is a continuing process, which means that once the mission is
completed, a second iteration with a new mission can be initiated.
Team building and role assignment (Stage 2) are carried out based on students inter-
personal skills scores and on the perception of satisfaction and compatibility amongst
team members. A quarterly subject can be developed through two or three iterations
depending on the extension and level of difficulty of the project or mission. Moodle
provides a series of collaborative instruments that are used in Stage 3 to facilitate work
development and help complete the mission or project: glossary and private group
chat through the whole course of the mission and a private group forum and private
group wiki that can be independent in each stage depending on the difficulty and
differentiation of the project tasks. Each post created by each group member can be
improved, extended and corrected by classmates and, thus, the mission is carried out
cooperatively.
Objectives
The objective is to improve university students team work skills (soft skills) in a
b-learning environment (Evalsoft system) designed for undergraduate courses. It is,
therefore, important to check the level of reliability that soft skills evaluation can
achieve in virtual contexts and, consequently, it is worth considering whether or
not team work skills can be reliably developed and evaluated in b-learning environ-
ments. Furthermore, it is also important to assess whether the Evalsoft system can
be an accessible, efficient and realistic tool allowing university teachers to incorpor-
ate the learning and evaluation of generic competencies students will require for
their future career advancement.
Variables
Students learning context either on-site or b-learning with Evalsoft software is the
independent variable for the analysis and study work, team work skills being the
dependent variable. The agreed definition of team work in this study is to become
integrated, collaborate and cooperate with others in an active and efficient way to
achieve common objectives (Ruiz Morales 2013).
Instrument
An instrument to evaluate team work skills was designed and assessed based on a
review of the specialized literature (Bloxham and Boyd 2007; Ibarra and Rodrguez
2007). The object of this instrument is for students to self-evaluate their performance
on a 110 scale in accordance with six categories related to this skill (supporting and
motivating group members, agreement on and acceptance of team rules, participative
attendance at group meetings, contribution of ideas and research material on the
subject, analysis and preparation, contribution to the group cooperative processes).
Two contrasting criteria were set for each of the categories (semantic differential scale)
in order to help students with their self-evaluation process.
Data analysis
Statistical analysis was carried out with the SPSS pack (version 19.0). The instruments
internal consistency was assessed with Cronbachs Alpha, and team work skill differ-
ences between on-site and b-learning contexts were studied through an analysis of
covariance.
Results
The instrument shows a very good internal consistency level (George and Mallery
1995), Cronbachs alpha reached a 0.837 value (over 0.8). We carried out a covariance
analysis and checked the starting position in order to analyze changes in the team work
skill across different learning contexts. The results obtained reveal the environments
capacity to affect skill level. Table 1 shows that the group variable is statistically
significant (p-value < 0,000), which means there are significant differences between
the intervention group and the non-intervention group. More specifically, the
estimated values for these differences is +3.207. That is, the results in the intervention
group are on average 3.207 points higher than those obtained in the non-intervention
group (Table 2).
Table 3 and Fig. 3 show that the the non-intervention group mean results are below
50 points, whereas the intervention group exceeds this threshold.
The results show positive empirical evidence for the b-learning context (intervention
group) in the development of team work skills. Therefore, it is safe to state that the
implementation of e-learning and evaluation tasks in a b-learning context has a direct,
positive and significant effect in the learning of team work skills.
Conclusions
Although previous studies have pointed out difficulties in the evaluation of team work,
commitment and leadership in b-learning environments (Herradn Diez et al. 2009),
this study reveals that team work soft skills reach higher achievement levels through
the Evalsoft system in a b-learning environment than in the traditional classroom
context using the virtual campus only as a document repository, these results being in
line with the work by Guitert et al. (2007).
In this regard, PBL articulated with e-tasks, simulated context, collaboration and com-
munication tools (forum and wiki) seem to facilitate the learning and development of this
skill. More specifically, e-tasks favor collaborative e-learning; that is, tasks related to
learning by doing (planning, building, designing, proposing) in turn facilitate learning
by discovery (searching, questioning, exploring) and enable e-evaluation as a form of
reflection and improvement (self-evaluation, peer-evaluation) so that students experi-
ence skill learning based on their involvement and co-responsibility in team work.
Adaptation to students needs and interests is one of the keys to the success of this
work system, characterized by an integrated use of new technologies and by the close
link to students future career course. A detailed analysis of the constitutive elements of
this system shows that the wiki and the e-forum are the key tools provided by Evalsoft
in terms of the development of team work skills observed in the intervention group.
Specifically, the wiki can be key to students team work as it enables and facilitates the
52
51
50
49
48
47
46
Intervention Non-Intervention
Fig. 3 Differences between Intervention and Non-intervention groups. Team work skill
PhD in Education at the Universidad Complutense de Madrid (2013) and Computer Science Teacher. He has taken part
in six funded research projects and published several articles and conference papers. In 2005 he was appointed to a
full time position at the UNET, where he teaches undergraduate and graduate courses. He specializes in research
methodology and e-evaluation of generic competences at the university level.
Universidad Nacional Experimental del Tchira
C/3 N 2 02
San Cristbal-5015
Estado Tchira
Venezuela
Competing interests
The authors declare that they have no competing interests.
Acknowledgement
This study is part of the Evalsoft project: Evaluation of soft skills at the university level. Comparative study of virtual and
attendance-based environments. Ref: EA2009-0020. Programa de Estudios y Anlisis para la mejora de la calidad de la
enseanza superior y de la actividad del profesorado universitario. Direccin General de Universidades de la Secretara
de Estado de Universidades (Ministerio de Educacin).
Author details
1
Universidad Complutense de Madrid (UCM), Madrid, Espaa. 2Universidad Nacional de Educacin a Distancia (UNED),
Madrid, Espaa. 3Universidad Nacional Experimental del Tchira (UNET), Caracas, Venezuela.
References
lvarez ML, Fidalgo R, Arias-Gundn O et al (2009) La eficacia de las metodologas activas en el rendimiento del
alumnado de magisterio. In: Actas do X Congresso Internacional Galego-Portugus de Psicopedagogia.
Universidade do Minho, Braga
Aquilino AJ, de Andrs J, Nieto C y otros (2006). Definicin de competencias especficas y genricas del ingeniero en
informtica [artculo en lnea]. Aula Abierta. [Fecha de consulta: 2 de junio de 2014]. http://di002.edv.uniovi.es/~
albizu/articulos/ice06b.pdf
Benito M, Romo JM, Portillo J (2008) Estudio de la asignatura Web 2.0: Redes sociales impartidas a las universidades
del G9. In: Rodrguez Gmez G (ed) Evaluacin de competencias con herramientas de interaccin dialgica
asncronas. Servicio de Publicaciones de la Universidad de Cdiz, Cdiz, pp 276297
Bloxham S, Boyd P (2007) Developing Effective Assessment in Higher Education. Open University Press, Maidenhead
Cameron K, Whetten D (2002) Developing management skills. Pearson Prentice Hall, Upper Saddle River
Comisin Europea (2014). Policies and Activities. Agenda for New Skills and Jobs [artculo en lnea]. [Fecha de
consulta: 1 de junio de 2014] http://ec.europa.eu/social/main.jsp?langId=en&catId=958.
Comisin Europea (2008) El Marco Europeo de Cualificaciones para el aprendizaje permanente (EQF-MEC). Oficina de
Publicaciones Oficiales de las Comunidades Europeas, Luxemburgo
Corominas E, Tesouro M, Capell D et al (2006) Percepciones del profesorado ante la incorporacin de las
competencias genricas en la formacin universitaria. Revista de Educacin 341:301336
Dash J (2001) Schools push soft skills for info security majors. Computerworld 35:624
DuBrin, A J (2004). Coaching and Mentoring Skills (NetEffect Series). Rochester Institute of Technology, Prentice Hall,
Instock, New Jersey
Echazarreta C, Prados F, Poch J y otros (2009). La competencia El trabajo colaborativo: una oportunidad para
incorporar las TIC en la didctica universitaria. Descripcin de la experiencia con la plataforma ACME (UdG)
[artculo en lnea]. UOC Papers. [Fecha de consulta: 22 de junio de 2014]. http://www.uoc.edu/uocpapers/8/dt/esp/
echazarreta_prados_poch_soler.pdf.
Ezeiza A, Palacios S (2009) Evaluacin de la competencia comunicativa y social en foros virtuales. RELIEVE 15(2):115
George D, Mallery P (1995) SPSS/PC+ step by step: A simple guide and reference. Wadsworth Publishing Company,
Belmont
Gonzlez Maura V, Gonzlez Tirados R (2008) Competencias genricas y formacin profesional: un anlisis desde la
docencia universitaria. Revista Iberoamericana de Educacin 47:185209
Gonzlez Pareja A, Caldern Montero S, Galache Laza T. y otros (2006). Uso de wikis para la realizacin de trabajos
colaborativos en el aula [artculo en lnea]. En: XIV Jornadas de ASEPUMA y II Encuentro Internacional. [Fecha de
consulta: 22 de junio de 2014]. http://www.uv.es/asepuma/XIV/comunica/118.pdf.
Gonzlez J, Wagenaar R (eds) (2003) Tuning Educational Structures in Europe. Informe Final. Fase Uno. Universidad de
Deusto, Bilbao
Gorman H (2000) Winning hearts and minds?Emotional labour and learning for care management. J Soc Work Pract
14(2):149158
Guitert M, Romeu T, Prez-Mateo M. (2007). Competencias TIC y trabajo en equipo en entornos virtuales [artculo en
linea]. RUSC. Vol. 4, nm. 1. [Fecha de consulta: 1 de junio de 2014]. http://www.uoc.edu/rusc/4/1/dt/esp/guitert_
romeu_perez-mateo.pdf.
Hernndez Trasobares A, Lacuesta Gilaberte R. (2007). Aplicacin del aprendizaje basado en problemas (PBL) bajo un
enfoque multidisciplinar: una experiencia prctica. En: J. C. Ayala Calvo (ed.). Conocimiento, innovacin y
emprendedores: camino al futuro, pgs. 3043, Universidad de la Rioja, La Rioja
Herradn Diez R, Blanco Cotano J, Prez Yuste A et al (2009) Experiencias y metodologas en asignaturas b-learning
para la formacin y evaluacin en competencias genricas en Ingeniera. La cuestin universitaria 5:3345
Garca et al. International Journal of Educational Technology in Higher Education (2016) 13:5 Page 11 of 11
Hoffman J (2003) Student-Created Graphic Organizers Bring Complex Material to Life. CollegeTeaching 51:105
Ibarra MS, Rodrguez G (2007) El trabajo colaborativo en las aulas universitarias: reflexiones desde la autoevaluacin.
Revista de Educacin 344:355375
Isaacs N (1998) Use job interviews to evaluate soft skills. InfoWorld 20(14):104
Llorente M C (2008). Blended-learning para el aprendizaje en nuevas tecnologas aplicadas a la educacin: un estudio de
caso. Tesis doctoral indita Universidad de Sevilla, Sevilla.
Maes J, Weldy T, Icenogel M (1997) A Managerial Perspective: Oral Communication is Most Important for Business
Students in the Workplace. J Bus Commun 34(1):6780
Marcelo Garca C, Perera Rodrguez VH (2007) Comunicacin y aprendizaje electrnico: la interaccin didctica en los
nuevos espacios virtuales de aprendizaje. Revista de Educacin 343:381429
Medina A (2001) Los mtodos en la enseanza universitaria. In: Garca-Valcrcel A (ed) Didctica universitaria. La
Muralla, Madrid
Murnane R (1996) Teaching the new basic skills: Principles for educating children to thrive in a changing economy.
Free Press, Nueva York
Nealy C (2005) Integrating soft skills through active learning in the management classroom. J Coll Teach Learn 2(4):16
Palmer Pol A, Montao Moreno J, Palou Oliver M (2009) Las competencias genricas en la educacin superior.
Estudios comparativos entre la opinin de los empleadores y acadmicos. Psicothema 21(3):433438
Prez Pueyo A, Clemente JA, Lpez Pastor VM (2009) Evaluacin formativa y compartida en el espacio europeo de
educacin euperior (EEES). In: Lpez Pastor VM (ed) Evaluacin formativa y compartida en el espacio europeo de
educacin superior. Propuestas, tcnicas, instrumentos y experiencias. Narcea, Espaa
Redondo J M, Ortn F, Zapico D y otros (2009). Elaboracin de repositorios de contenidos a travs de wikis como
tcnica de evaluacin. En: XV JENUI. Barcelona, 810 de julio de 2009. http://upcommons.upc.edu/bitstream/
handle/2099/7886/p107.pdf
Riesco Gonzlez M (2008) El enfoque por competencias en el EEES y sus implicaciones en la enseanza y el
aprendizaje. Tendencias Pedaggicas 13:79104
Rodrguez G, Ibarra MS (2010) Caracterizacin de la e-evaluacin orientada al e-aprendizaje [documento no publicado].
Programa de Formacin y Asesoramiento, Madrid
Ruiz Morales Y (2013). Evaluacin de las competencias genricas en la universidad: estudio comparativo en entorno b-
learning y presencial. Tesis doctoral
Salmern H, Rodrguez S, Gutirrez C (2010) Metodologas que optimizan la comunicacin en entornos de aprendizaje
virtual. Comunicar XVII(34):163171
Sancho Thomas P, Fuentes Fernndez R, Fernndez-Manjn B (2009). NUCLEO an adaptive role game based learning
scenario. Learning Technology IEEE Computer Society Technical Committee on Learning Technology, pgs. 9:(3)811.
Sancho Thomas P, Fuentes Fernndez R, Fernndez-Manjn B (2009) Learning teamwork skills in university
programming courses. Comput Educ 53:517531
Sancho Thomas P, Torrente J, Marchiori E J y otros (2011). Enhancing Moodle to Support Problem Based Learning.
En: IEEE Global Engineering Education Conference (EDUCON2011), pgs. 11771182. Amn (Jordania), 46 de abril de
2011. doi:10.1109/educon.2011.5773296.
Santos R (2009). La Wiki-Webquest: Una actividad colaborativa en la asignatura de Nuevas tecnologas aplicadas a la
educacin [artculo en lnea]. Revista Universitaria. Vol. 7, nm. 5. [Fecha de consulta: 14 de abril de 2011] http://
redu.net/redu/index.php/REDU/article/view/171/pdf.
Schulz Y (1998) New world puts emphasis on softer skills. Comput Can 24(45):2122
Tobn S (2006) Competencias en la educacin superior. Polticas de calidad. ECOE, Bogot
Vaquerizo B, Renedo E, Valero M (2009). Aprendizaje colaborativo en grupo: Herramientas Web 2.0. En: XV Jornadas de
Enseanza Universitaria de la Informtica. Facultat d'informtica de Barcelona , Barcelona
Vermunt JD (1992) Learning styles and directed learning processes in higher education: towards a process-oriented
instruction independent thinking. Swets and Zeitlinger, Amsterdam / Lisse
Villanueva A (2009). Uso de wikis en Ingeniera Informtica [artculo en lnea]. Revista de Docencia Universitaria. Vol. 7,
nm. 5. [Fecha de consulta: 14 de abril de 2011] http://redu.net/redu/index.php/REDU/article/view/168/pdf.
Villardn GL (2006) Evaluacin del aprendizaje para promover el desarrollo de competencias. Educatio Siglo XXI 24:5776
Waller J (2007) Soft skills for lawyers. Chelsea Publishing Limited, Londres, Recurso Electrnico
Woeran B, Kronsteiner H (2007). EQ Interpersonal Skills work based learning Key soft skills acquired online during a
practical placement. DANUBE European Training & Technology. PLATO Conference DANUBE Experiences
Zabala A, Arnau L (2007) Cmo aprender y ensear competencias. Gar, Barcelona