A topic map based learning management system to facilitate meaningful grammar learning: the case of Japanese grammar learning

Abstract

This study investigates the effect of studying with topic maps provided by a self-developed language learning support system on (a) learning perception, (b) learning achievement and (c) variation in learning attitude and motivation, from the perspective of prior learning attitude, motivation and learning style. An analysis was conducted on the learning data of 90 participants for examining pre-test and post-test scores, learning perception, learning style indicators, attitude and motivation rankings before and after the study of target Japanese grammar content with the support of the system. The findings suggest that learners with high attitude/motivation perceived greater development of the habit of “learning from the comparison of related knowledge” and felt more satisfied with the learning situation in the system environment. Furthermore, the learners' attitude towards Japanese grammar learning and their motivation toward Japanese language learning were significantly stronger after the learning activity. It is also observed that (a) learner attitude towards Japanese grammar after studying with the system only had a positive correlation to prior attitudes; and (b) learner motivation towards Japanese language after studying with the system correlated highly with prior motivation and perception of development of the habit of “learning from the comparison of related knowledge.” Finally, from a learning style perspective, the analysis result suggests that “Global learners” expressed a stronger feeling about the usefulness of the comparison function of the system and their increment in motivation after using the system was higher than for “Sequential learners.” Moreover, between learning style indicators in the Sequential/Global dimension and level of learner expertise in the reported mental load scale, a significant interaction effect was found.

Introduction

Grammar teaching uses techniques to highlight specific grammatical forms, aiding learners in understanding, processing, and internalizing them (Ellis, 2006). Some studies claimed that grammar teaching has minimal effect on language acquisition, provided that learners are sufficiently motivated and have access to appropriate data (Corder, 1967; Krashen, 1981). However, a number of empirical studies (Pica, 1983; Long, 1983; White et al., 1991; Norris & Ortega, 2000) demonstrate that although grammar instruction was not a guarantee that learners would acquire what they had been taught, the acquisitional processes of both instructed and naturalistic learning were the same, and instructed learners generally achieved higher levels of grammatical competence than naturalistic learners.

Whether the appropriate stage of grammar teaching should be during the early language learning stage or during the stage that learners have begun to form interlanguages is a topic of debate. Despite this, most researchers agree that grammatical knowledge can provide the basis for input processing required in the competence development of a target language (Ellis, 2005; Lightbown, 1991) and there is ample evidence to demonstrate that grammar learning is essential for promoting target language learning (Ellis, 2006).

To support grammar learning in the last decade, numerous studies have proposed various intelligent tutor systems. Primarily, most of them have been implemented to support grammar learning in the English language. For example, a web-based English grammar learning support system was developed by Goto et al (2008) which can recommend exercises to learners based on their understanding of knowledge as indicated by their answers. Fill-in-the-blank exercises are arranged into knowledge networks in the system, where two exercises are connected by an inclusive relation if one of them includes more new grammatical information than the other. To promote the learning motivation of English language learners, Sunar et al (2013) developed a system which automatically generates multiple-choice cloze exercises. Part-of-Speech (POS) tags and a few specific words are used to formulate English grammar rules, and the system defines the semantic relations among units of English textbooks with four types of relations (is-a relation, part of relation, group-by relation, and example-of relation). Su (2017) integrated a spell checker and a grammatical parser technique to build a computerized English grammar instructional dialogue environment which can provide learners with supportive feedback after they respond to grammar questions.

However, few intelligent tutor systems have been developed to support Japanese grammar learning and none of the previous studies have conducted experiments to investigate the learning style, learning attitude and motivation together with the learning achievement and learning perceptions while using Japanese grammar learning support systems. Therefore, in this study, a topic map based learning management system, CLLSS (a customizable language learning support system) has been developed, aiming to facilitate meaningful grammar learning through visualisations. Moreover, a case study was conducted on 90 undergraduate students majoring in Japanese to explore the effectiveness of CLLSS.

In a previous study (Wang et al., 2014), it was found that students who studied with CLLSS achieved significantly better learning achievements than those who just did self-study with textbooks after studying the same target content. In this paper, we will further examine learning achievement, learning perception and the variation of learning attitude and motivation in light of four learning conditions: prior learning attitude, prior learning motivation, learning style, and level of learner expertise. The following research question will be discussed:

• 1. What is the correlation between prior learning attitude/motivation and learning perception/achievement after studying with CLLSS?

• 2. Does the attitude and motivation of the students change after studying with CLLSS?

• 3. Does the learning performance (including learning perception scales, changes in attitude and motivation, and learning achievement) depend on learning styles and level of learner expertise?

The main contribution of this work is a) to present a learning analytics approach to conduct the investigation of learning styles, the variation of learning attitude and motivation, together with learning achievement and learning perception, and b) to provide insights based on the learning analytics results toward better designs for language learning support systems in response to individual profiles, mainly referring to attitude, motivation and learning styles. By considering the initial learning styles, attitudes, and motivations of the learners before engaging in a learning activity supported by CLLSS, analyses of learner data were conducted to assess changes in learning perception, achievement, attitude, and motivation after the activity.

Related work

Learning attitude, learning motivation and learning styles

To study the effectiveness of a learning support system, in addition to comparing learning performance with and without the system support, it is essential to analyse and discuss learning performance with consideration of individual differences in prior knowledge, learning attitude, learning motivation, and learning style. This kind of analysis and discussion will afford a better understanding of learner performance changes from various perspectives.

Learning attitude reflects the way a learner approaches the learning target and may be positive or negative. It has components of learner curiosity and willingness to explore. Learning motivation refers to the internal and external impetus that prompts the learner to study and acquire new skills. The improvement of learning attitude or motivation were often used to prove the effectiveness of e-learning systems (Huang & Chen, 2013; Hwang & Chang, 2011), and these factors have been specifically identified as contributors to the acquisition of second language proficiency (Gardner, 2000; Gardner & MacIntyre, 1993).

If a learner can sustain their motivation, they may expend more effort on the learning activity and may be more likely to optimize their performance. On the other hand, learning attitude influences a learner’s choice of action and responses to the task. Therefore, a learner who maintains a positive attitude towards learning is more likely to have strong motivation to study. However, attitude is not the only variable that affects learning motivation. Extrinsic motivation, for example, a strong desire to get rewards or recognition from others, can also evoke a strong effort to learn.

Learning style is defined as "the ways in which an individual characteristically acquires, retains, and retrieves information" by Felder and Henriques (1995). There are over 70 learning style models in the literature (Coffield et al., 2004). Different models are used by various studies to classify learners into supposedly distinct groups or to assign learners graded scores on single or multiple dimensions (Pashler et al., 2008). For collecting learning style distribution data in the preparatory phase of the experiment, this study adopted a questionnaire written in Chinese, translated from the Index of Learning Styles (ILS) questionnaire of 44 questions (Soloman & Felder, 1999). The ILS questionnaire was designed based on the Felder-Silverman learning style model (Felder & Silverman, 1988) and its Chinese version was suggested to be reliable, valid and suitable for capturing learners’ behavioural tendencies (Wang and Mendori, 2015).

FSLSM (Felder–Silverman Learning Style Model) identifies four dimensions of learning style: Active/Reflective, Sensing/Intuiting, Visual/Verbal, and Sequential/Global. It should be noted that these learner characteristics are dimensions rather than categories. The Active/Reflective dimension is concerned with information processing: Active learners tend to gain understanding through active trial or discussion or by explaining the material to others, whereas Reflective learners tend to observe and reflect. The Sensing/Intuiting dimension is concerned with information perception: Sensing learners prefer to perceive data via the senses, whereas Intuiting learners prefer to do so by accessing memories or insights. The Visual/Verbal dimension is concerned with information reception: Visual learners prefer that information be presented in the form of diagrams, flow charts, pictures or films rather than written words, a form preferred by Verbal learners. The Sequential/Global dimension is concerned with information understanding: Sequential learners gain understanding in logical, linear steps, whereas Global learners need the overall picture of a subject before mastering details.

Map based learning support systems

In the domain of educational technology, maps such as concept maps and knowledge maps are often used as learning materials. Concept maps organize concepts hierarchically, derived from a central question, to facilitate comprehension by arranging more general concepts at the top of the map and more specific concepts below (Novak & Canas, 2008), meanwhile a knowledge map utilizes a common set of labelled links to connect ideas intentionally (O’Donnell et al., 2002). On the other hand, topic maps describe knowledge structures and link them to corresponding resources, serving as a crucial technology for enabling knowledge management (Kuang & Luo, 2010). In other words, topic maps primarily function as metadata for learning materials or objects, which aim to link the represented knowledge structures with corresponding resources (Anonymised for review).

Using a concept map model, Utami and Yuliyanto (2020) observed significant differences in students' learning outcomes and motivation levels before and after being taught the atomic structure material in chemistry. In language learning and reading comprehension, concept maps and knowledge maps are often used in article or paragraph summarisation. Hwang et al (2019) utilised a concept mapping-based summarisation strategy in a flipped learning classroom to improve the reading comprehension of fifth graders from Mandarin classes at an elementary school in Taiwan. The experiment result suggests that employing concept mapping directly improved the students’ summarisation ability and indirectly improved their reading comprehension skills.

Topic map based systems, such as a blogging system built by Huang and Chen (2013) which includes knowledge navigation by using “topic maps” to structure the knowledge representation of a “Citizenship and Society” course and to link the corresponding blog entries as supplementary learning resources, were also found to improve learning performance and learning attitudes. For more general education, Dicheva and Dichev (2006) propose the “topic maps editor and viewer” (TM4L) system which adopts a bottom-up structure where learning materials are classified based on knowledge concepts that are clustered in multiple contexts/themes to provide multiple viewpoints. However, whilst TM4L facilitates learners in comprehending relationships between resources, it lacks organisation of learning materials that specifically address the relations between concepts.

CLLSS: a topic map based language learning system to facilitate meaningful learning

Ausubel's learning psychology theories (Ausubel, 1963; Ausubel et al., 1978) state that meaningful learning is achieved when new knowledge integrates with existing frameworks of the learner. Strategies organising information hierarchically in the brain can significantly enhance learning outcomes (Bransford, 2000; Tsien, 2007; Wang et al., 2020), but individual differences exist in learning styles, habits, attitudes, and motivation towards different target learning content.

To facilitate meaningful learning of language patterns, this research proposed a systematic description of information about language concepts (including the relationships between concepts) in a map structure, using ontology techniques (Wang et al., 2013; 2014). Accordingly, the first topic map based language learning support system CLLSS was developed to update teaching steps via ontology modifications, automatically manipulate the ontology of any specific language, and visualise the relations between knowledge concepts and their corresponding learning materials, offering a more personalised learning experience compared to traditional learning management systems. This system aims to encourage the learner to build up their conceptual framework based on their prior knowledge and gradually cultivate the habit of “learning from the comparison of related knowledge”.

To evaluate the effectiveness of CLLSS, this research conducted a case study utilising a self-developed Course-centered Ontology for Japanese Grammar (Wang et al., 2014) which organises grammar points (GPs) in a hierarchical network structure. The system automates the extraction of relationships between GPs and visualises them in a relation map. As shown in Fig. 1, the CLLSS 2.0 interface features a left panel which includes a catalogue search and keyword search, and a right panel which provides a visual representation of each selected GP and all its related GPs linked by relations in a topic map. This allows instructors to flexibly arrange learning materials on the right relation panel, while learners can easily navigate related grammar concepts and the corresponding learning materials.

Fig. 1

The main interface of CLLSS 2.0 where nodes 1–4 represent grammar points

For example, the topic map displayed in Fig. 1 enables the learner to compare GP “ ~ tekudasaimasenka” with its 4 related GPs through relations “hasNecessaryPrior (to Node 1), “isHonorificOf” (to Node 2), “isMoreRespectifulThan” (to Node 3) and “isSimilarWith” (to Node 4). When hovering the mouse over each representation of a GP or relation, the learners can check properties of that GP (for example, the properties “normalForm”, “pattern” and “example” of “ ~ tekudasaimasenka” in Fig. 1) or name and direction of that relation (for example, “isSimilarWith” relation from “ ~ tekudasaimasenka” to Node 4 in Fig. 1). Moreover, from each representation of a GP or relation, the learners can also access further related learning materials including explanations and practices. In other words, the topic map in the right relations panel functions as the metadata of the learning materials.

Experimental procedures and measure techniques

Participants and experimental procedure

The participants of this study consist of 38 male and 52 female first year undergraduate Japanese language majors from 3 different classes at a Chinese university. Before the experiment, all the students had already studied Japanese for 8 months. Figure 2 shows the experiment procedure for the study in this paper. The measurement techniques in this experiment included learning achievement tests and questionnaires to measure students’ learning attitude and motivation, learning perception, habits, preferences, and so on.

Fig. 2

The experimental procedure

In the preparatory phase, all participants took the ILS questionnaire, a pre-test and Questionnaire 1 which involved learning attitude and motivation, and the habit of “learning from the comparison of related GPs”. Because the participants from 3 different classes were taught by three different instructors, students from each class were assigned to be the experimental group and the control group according to their learning style in the Sequential/Global dimension and their learning habits of “learning from the comparison of related GPs”, so as to minimize the group composition differences. The experimental group consisted of 34 learners who didn’t have the habit (“N-learners”) and 26 learners who had the habit (“H-learners”), while the control group contained 16 “N-learners” and 14 “H-learners.”

In this experiment, five GPs “ ~ hoshigaru,” “ ~ tagaru,” “ ~ tekuru,” “Causative Sentence,” and “ ~ hazuda,” which have one, two, three, five and seven related GPs respectively, were chosen as the target learning content. After 25 min of training, the experimental group was guided in the use of the comparison function provided in the CLLSS 2.0 relations panel (as displayed in Fig. 1) to study the target contents in a computer-assisted language learning lab. Participants were required to compare the target content with their prior GPs, displayed in the relations panel, and were encouraged by an expert teacher to compare in particular “ ~ hoshigaru” with “ ~ hoshii”, “ ~ tagaru” with “ ~ tai”, “ ~ tekuru” with “ ~ teiku”, “ ~ hazuda” with “ ~ darou”, “Causative Sentence” with “ ~ temorau” by accessing related explanations and practice. Meanwhile, the control group in another classroom studied with a textbook. After 1 hour of study, both groups took a post-test designed for the assessment of the student's knowledge of the target content.

The test sheets were created by two experienced teachers. The pre-test aimed to evaluate the student's prior knowledge of Japanese. It contained ten fill-in-blank items, twenty-five single-choice items and ten translation items with a perfect score of 100. The post-test contained fifteen fill-in-blank items with a perfect score of 90. Those fifteen items were designed to assess the student's knowledge of target contents after the learning activity.

Finally, both groups completed Questionnaire 2 to assess their cognitive load (Sweller et al., 1998). Unlike the control group, the experimental group was required to answer some additional questions on Questionnaire 2 involving other learning perceptions, and learning attitude and motivation after the learning activity.

Measure techniques

Attitude and motivation questionnaire

In this study, all 60 students in the experimental group were required to complete a 14-item attitude and motivation questionnaire before and after the learning activity. The design of the 14 items, written in Chinese, was based on items from instruments used in other studies (Hwang & Chang, 2011; Pintrich & De Greet, 1990), with some modifications. Participants were instructed to respond to each item on a 6-point scale (1–3: strongly to slightly disagree, 4–6: slightly to strongly agree).

Factor analysis was performed on all 120 responses to the 14 items. On the basis of the results, two scales were constructed: attitude towards Japanese grammar learning (AJG, 7 items, α = 0.891) and motivation towards Japanese language learning (MJL, 7 items, α = 0.892), accounting for 63.1% of total variance. In all calculations reported here, the mean of the scores of the items in a given scale represents the ranking of that scale. The attitude towards Japanese grammar learning scale was constructed from the responses to the 7 items in Table 1, and the scale for motivation towards the Japanese language course consisted of the 7 items in Table 2 which include the expectation of the Japanese course (M1, M3, M5 and M6) and the perceived intrinsic value of the course (M2, M4 and M7).

Table 1 Learning attitude questionnaire towards Japanese grammar

Table 2 Learning motivation towards Japanese language course

Learning style questionnaire

The learning style profiles indicated by the results of the ILS questionnaire are presented in (Table 4, Wang et al. (2014)). The experimental group consisted of 25 Sequential learners and 35 Global learners while the control groups consisted of 10 Sequential learners and 20 Global learners.

Learning perception questions

In addition to the attitude/motivation questionnaire, after the post-test all 60 participants in the experimental group responded to another questionnaire consisting of 15 items related to learning perception. Factor analysis of these 15 items revealed 5 distinct scales: (a) the effect on developing the habit of “learning from the comparison of related knowledge” (2 items, α = 0.787); (b) mental effort (2 items, α = 0.692); (c) mental load (2 items, α = 0.764); (d) technology acceptance (2 items, α = 0.711); and (e) learning mode satisfaction (7 items, α = 0.822).

The first scale (6-point, 1 = not at all to 6 = very much) consists of the following two items:

• H1: Now that you have used the system, do you think learning with the support of the system can affect your development of the habit of “learning from the comparison of related knowledge”?

• H2: Do you think the system provides an environment supportive of “learning from the comparison of related knowledge”?

The details of the next four scales are as described in (Wang et al., 2014). Question content related to mental effort and mental load was modified based on an adaptation of the items of the questionnaire on cognitive load (Sweller et al., 1998). Responses to those four items are on seven-point Likert scales (1 = not at all to 7 = very much). Responses to items referring to technology acceptance and learning mode satisfaction are both on six-point Likert scales (1–3: strongly to slightly disagree, 4–6: slightly to strongly agree). The two items related to technology acceptance were modified based on questionnaires used in Chu et al. (2010b) and Davis (1989) while the seven items related to learning mode satisfaction were modified based on those used in the questionnaire in Chu et al. (2010a).

Analysis of learner data

The three major research questions explored in this study are discussed below.

The first research question

The first research question concerns the correlation between prior learning attitude/ motivation and learning perception/achievement. Here we consider Pre-AJG and Pre-MJL as individual difference variables before the learning activity.

Analyses were conducted to determine the correlation between the rankings of attitude and motivation before the experiment, the ranking of the other five perception variables (considering the mean rankings for items in each perception scale as one dependent variable), and post-test achievement. The highest correlation (r = 0.466) was found between the post-test score and Pre-AJG; on the other hand, the correlation (r = 0.294) between Post-test score and Pre-MJL was also significant. Furthermore the two learning perception scales, “effect on developing the habit” and “learning mode satisfaction” were found significantly correlated with Pre-AJG (r = 0.338 and r = 0.370) and Pre-MJL (r = 0.433 and r = 0.454).

ANOVA tests (considering the mean rankings for items in each perception scale as one dependent variable and post-test score as a dependent variable) were further performed to dichotomize Pre-AJG and Pre-MJL by mean value to form two low/high categorical variables. The results are displayed in Table 3. ANOVA results were significant for the “effect on the habit” and “learning mode satisfaction” scales. This suggests that after studying with CLLSS, learners with high level of AJG before studying were more likely to perceive an effect on development of the habit of “learning from the comparison of related knowledge” (F(1,58) = 5.398, p < 0.05) and reported greater satisfaction with the learning mode (F(1,58) = 6.805, p < 0.05) than learners with low level of AJG; moreover, compares to learners with low level of MJL before the learning activity, learners with high level of MJL reporter greater effect on developing their habit (F(1,58) = 8.230, p < 0.01) and greater satisfaction with the learning mode (F(1,58) = 11.098, p < 0.01) in CLLSS.

Table 3 ANOVA results for learners with Low/High attitude and motivation

Furthermore, significant differences were detected in the “Mental effort” scale (F(1,58) = 5.264, p < 0.05) and the post-test achievement (F(1,58) = 4.267, p < 0.05) from ANOVA tests for learners with different level of Pre-AJG; this suggests that compared to learner with low level of attitude towards Japanese grammar learning before the learning activity, learners with high level of attitude reported significantly lower required mental effort and achieved better on the grammar test after studying with CLLSS.

The second research question

The second question addressed in this paper is whether or not the participants’ attitude and motivation changed after studying with CLLSS. Learning attitude and motivation can be considered not only as conditions for learning but also as the results of a learning experience (Winne & Perry, 2000). Therefore, here we consider Post-AJG and Post-MJL as a result of studying with CLLSS.

Table 4 shows the t-test results of participant responses to the attitude/motivation questionnaire before and after the learning activity. It can be seen that after the learning activity, the participants had significantly higher scores for both attitude towards Japanese grammar learning (t = − 7.02, p < 0.001) and motivations towards Japanese language learning (t = − 5.62, p < 0.001). This suggests that CLLSS improved not only learner attitude towards Japanese grammar learning, but also learner motivation toward Japanese language learning.

Table 4 T-test result of Pre- and Post- attitude/motivation ranking

We went on to investigate whether the variation in learning attitude and motivation was related to the prior rankings or learning perception and achievement. Table 5 shows the computed Pearson's correlation coefficients for rankings of attitude and motivation scales before and after the learning activity. As predicted, the learner's AJGs were associated with MJLs (Pre: r = 0.698; Post: r = 0.550). Furthermore, the D-AJG (the deviation of AJG) and D-MJL (the deviation of MJL) were negatively correlated with Pre-AJG (r = − 0.628) and Pre-MJL (r = − 0.677), respectively; this suggests that learners with lower level of AJG and MJL before the learning activity were more likely to get a greater increase in their AJG and MJL after the learning activity.

Table 5 Corrections among AJG and MJL

It also can be seen from Table 5 that Post-AJG and Post-MJL were positively correlated with Pre-AJG (r = 0.795) and Pre-MJL (r = 0.711), respectively. In terms of learning perception, Post-AJG and Post-MJL were found to be strongly related to the “effect on developing the habit” scale (r = 0.402 and r = 0.553) and the “satisfaction for learning mode” scale (r = 0.352 and r = 0.519). Therefore, to examine the independent relations, two separate regression analyses were run: one for Pre-AJG, the two scales of learning perception and the post-test achievement on Post-AJG; and one for Pre-MJL, the two scales of learning perception and the post-test achievement on Post-MJL. Regression analysis of Post-AJG (r = 0.795) revealed that except for Pre-AJG (partial r = 0.795), none of the variables was significantly correlated to Post-AJG. Post-MJL (r = 0.762) was positively correlated to both Pre-MJL (partial r = 0.630) and the scale of “effect on developing the habit” (partial r = 0.386). The other variables were not significantly correlated to Post-MJL when included in the regression analysis with Pre-MJL and the scale of “effect on developing the habit.” These results suggest that learning attitude towards Japanese grammar after the learning activity can be predicted from attitude before the learning activity, and that motivation towards Japanese Language after studying with CLLSS can be predicted from prior motivation and perceived effect on the development of the habit of “learning from the comparison of related knowledge.”

It is also noteworthy that eight participants' post-rankings on questions A5, A6 or M5 were slightly lower than their pre-rankings and that all their pre-rankings on those three questions were greater than 4. In a subsequent interview, all eight participants reported that since the system already provided all the related knowledge, they felt no need to actively search for more by themselves. This indicates that the CLLSS environment should be modified to encourage learner engagement in the construction of relation maps. Version 2.0 of CLLSS, used in this study, simply displayed information about the concepts related to the target concept and the relations between them. The participants made comparisons between concepts in a passive receptive manner. It should be beneficial to encourage the learner to actively recall their prior knowledge and design their own relation map and then to compare it with that displayed by the system. This active process is expected to improve learners’ construction of their own knowledge frameworks and also to prevent a decrease in willingness to explore related knowledge. A visualization environment which can support the learners’ active construction of their own knowledge frameworks will be developed in future work.

The third research question

The third research question concerns the relationship between learning style and level of learner expertise and learning performance (including 5 learning perception scales, changes in attitude and motivation, and learning achievement). Here, learning style and level of learner expertise before the experiment (pre-test) are considered as learning conditions.

In the previous work (Wang et al., 2014), among all factors of learning perception and learning achievement, significant difference between “Sequential learners” (N = 25) and “Global learners” (N = 35) was only found for “technology acceptance” (MANOVA result: p < 0.05). Individual univariate analysis indicated that that significant difference was caused by the difference in reported “Perceived usefulness” (Mean of Sequential = 4.56, Mean of Global = 5.02, p < 0.025); this suggests that most “Global learners” had a stronger sense than “Sequential learners” that the comparison function provided by CLLSS 2.0 supported the improvement of their learning performance.

In this study, we begin with an investigation of the relationship between learning perception/achievement and the other three learning style indicator dimensions (Active/Reflective, Sensing/Intuiting, and Visual/Verbal). Four MANOVA analyses (considering the two items of the first 4 learning perception scales as two dependence variables) and two ANOVA analyses (considering the mean rankings for the 7 items in the last perception scale as a dependent variable and post-test score as a dependent variable) were conducted for detailed examination of each dimension. Significant difference was only found between Active and Reflective learners in their reported Mental Load (MANOVA result: F (2, 57) = 3.189, p < 0.05). Individual univariate analysis indicated a high probability that this significant difference was caused by the difference in reported “Perceived pressure” (Mean of Active = 2.50, Mean of Reflective = 1.94, F(1,58) = 3.688, p < 0.05), although for both types of learners the average rating was much less than 4 (neutral); this suggests that during study with the comparison function provided in CLLSS 2.0, most “Active learners” felt more pressure than “Reflective learners” despite the fact that both learner groups reported low pressure. This result is expected since during the learning activity in this experiment, most of the time the participants studied alone with CLLSS. Unlike reflective learners, who prefer a more individual working mode, active learners prefer to come to an understanding of target knowledge through group work, so it is to be expected that reflective learners would experience less pressure than active learners. This is consistent with the interview results in which 5 participants, identified as stable active style learners, reported that they preferred discussion with others while using CLLSS.

Our second area of investigation is the relationship between learning style and changes in attitude and motivation. As can be seen in Table 6, significant differences were only found between “Sequential learners” and “Global learners” in motivation change (F(1,57) = 5.513, p < 0.05). This suggests that after using CLLSS, the average motivation of “Global learners” increased more than that of “Sequential learners.”

Table 6 The experimental group’s attitude/motivation changes from a learning style perspective

A third analysis was conducted to determine whether there was an interaction effect between the learning style indicator in the Sequential/Global dimension and the level of learner expertise in learning perception/achievement. Due to the small and unbalanced number of participants in the other 3 learning style dimensions, the investigation of the interaction effect was not appropriate.

Figure 3 shows the distribution and the quartile of the pre-test scores of the experimental group students. We considered the score on the pre-test as representative of the extent of the student’s fundamental knowledge of the target content. To categorize students in terms of extensive knowledge or limited knowledge, the pre-test scores of experimental group participants were divided into three levels (high ≥ 85, 74 < middle < 85, low ≤ 74) according to the quartile of the scores on the pre-test. In this paper, we only investigated the cases of learners whose pre-test scores were identified as high or low level.

Fig. 3

Distribution and quartiles of pre-test scores

Six 2*2 ANOVA analyses were conducted on the five perception scales (considering the mean rankings for items in each perception scale as one dependent variable) and on the post-test score, respectively; significant interaction effect (F(1,28) = 5.487, p < 0.05) was found only between learning style indicator in Sequential/Global dimension and pre-test score levels for reported mental load. Further 2*2 MANOVA analyses considering the two mental load scale items (“Perceived distraction” and “Perceived pressure”) as two dependent variables were conducted for detailed examination. MANOVA result indicates a high probability (F(2,27) = 2.805, p < 0.1) that mental load is dependent on learning style indicator in Sequential/Global dimension and pre-test score levels; further ANOVA results indicate significant interaction effect (as shown in Fig. 4, F(1,28) = 5.782, p < 0.025) between learning style indicator in the Sequential/Global dimension and pre-test score levels in the rating of “Perceived distraction,” but no significant interaction effect (F(1,28) = 3.967, p > 0.05) in the rating of “Perceived pressure.”

Fig. 4

Interaction effect between Sequential/Global group and level of pre-test in perceived distraction (*p < 0.05)

Despite the small number of participants, the results of individual univariate analyses indicate a significant difference in the rating of " Perceived distraction" (F(1,28) = 5.119, p < 0.05) between Sequential and Global learners with high prior knowledge; while using CLLSS, the experienced learners who were identified as Global learners (Mean = 2.50, Mean < 4, i.e. 4 is neutral point, S.D = 1.871) reported higher distraction than did those who were identified as Sequential learners (Mean = 1.22, Mean < 4, S.D = 0.441) despite the fact that both types reported low distraction. This may be explained by the observation that when both Global and Sequential learners have an understanding of most of the information in a relation map, Sequential learners, who are accustomed to learning bit by bit until they get the whole body of information, can easily decide their learning steps. However, it is difficult for Global learners to decide the order in which to study the whole picture of the information, since there are multiple possibilities; for them this leads to greater distraction. This deduction will be confirmed by means of log data or eye-tracking data in our future work.   

Conclusion and discussion

This paper analyses learner data for learning perception and achievement, attitude and motivation changes after a CLLSS supported learning activity, from the perspective of learning style, and attitude and motivation before the activity. The following points suggested by the results are worthy of consideration.

• (1) Considering learning attitude and motivation before the learning activity as individual difference variables, it appears that learners with high attitude and motivation perceived a greater effect on the development of the habit of “learning from the comparison of related knowledge” and felt more satisfied with the learning mode in the CLLSS environment. Moreover, compared to learners with a low level of attitude towards Japanese grammar before the learning activity, learners with a high level of attitude reported significantly less mental effort during learning with CLLSS and achieved better on the post-test.

• (2) In consideration of learning attitude and motivation after the learning activity as learning outcomes, it was observed that both learner attitude towards Japanese grammar learning, and learner motivation toward Japanese language learning were significantly higher after studying the target Japanese grammar with CLLSS. This can be considered a benefit of using CLLSS. Furthermore, learners with lower level of attitude towards Japanese grammar learning before the learning activity are more likely to get a greater improvement in attitude after the learning activity. Motivation toward Japanese language learning displayed a similar pattern. These findings suggest that learners with higher attitude and motivation tend to have relatively stable learning status, while learners with lower attitude and motivation had more scope for improvement in motivation.

  
  

  Moreover, learner motivation toward Japanese language learning was positively associated with attitude towards Japanese grammar learning. This finding is in accordance with the socio-educational model of second language acquisition (Gardner, 2000; Gardner & MacIntyre, 1993), which is used to examine the relationship between student motivation, attitude and second language achievement.

  
  

  It is also found here that (a) learner attitude towards Japanese grammar after studying target grammar content with CLLSS had a significantly higher correlation to prior attitude than to learning achievement and other learning perceptions; and (b) learner motivation towards Japanese Language after studying with CLLSS was strongly correlated to both prior motivation and perceived effect on developing the habit of “learning from the comparison of related knowledge.” This suggests that learners with high attitude and motivation before the learning activity also had a higher attitude and motivation after studying with CLLSS; moreover, for learners with the same level of motivation towards Japanese language learning before the learning activity, the more effect on the development of the habit they perceived, the higher their motivation. Therefore, to enhance learning motivation, future work on topic map system design should focus on means of increasing the perceived effect of habit development. However, other factors could influence variations in learning attitude and motivation and this calls for further investigation.

• (3) Regarding the relationship between learning performance and learning style, it is observed that (a) most “Global learners” had a stronger feeling than “Sequential learners” that the comparison function provided by CLLSS 2.0 supported improvement in their learning performance; and (b) most “Active learners” felt more pressure than “Reflective learners” while using the comparison function provided by CLLSS 2.0. In terms of the relationship between learning attitude/motivation and learning style, “Global learners” showed a greater increase in motivation after using CLLSS 2.0 than “Sequential learners”. Furthermore, the significant interaction effect of learning style indicator in Sequential/Global dimension and level of learner expertise on learning performance was only found for the reported mental load scale. Despite the fact that both groups reported low pressure, experienced learners identified as “Global learners” reported greater distraction than those identified as “Sequential learners”. In the future, support for learners with different learning styles needs to be deliberately designed to address the above issues.

A limitation of this study is that the participants involved in the experiment only interacted with the topic map-based system for one hour to learn five target grammar concepts. Therefore, only the short-term effectiveness of the system was explored and discussed in this paper. Future experiments should be designed to encourage participants to interact with the system for several weeks to examine its long-term effectiveness.