打开APP
userphoto
未登录

开通VIP,畅享免费电子书等14项超值服

开通VIP
AJET 19(1) Treleaven (2003) - evaluating a communicative model for web mediated collaborative learning
Australian Journal of Educational Technology
2003, 19(1), 100-117.
AJET 19

Evaluating a communicative model for web mediated collaborative learning and design

Lesley Treleaven
University of Western Sydney
Collaborative learning and its associated processes of social interaction have been given increasing emphasis in innovative education as pedagogical research has established the importance of active student learning. Furthermore, the possibilities for conceptualising, designing, facilitating, monitoring and assessing active student learning have been transformed by the technological innovations and proliferation of web mediated learning. The effectiveness of these new learning environments in generating and sustaining successful collaborative learning now requires systematic evaluation from within these social interaction processes themselves.

This study responds to the call for theory driven empirical research into online learning. The paper investigates the usefulness of a Communicative Model of Collaborative Learning (CMCL) in analysing the linguistic interactions between students in a web mediated environment, in order to evaluate their online collaborative learning. A symptomatic set of student postings, on an electronic bulletin board in an undergraduate management subject at a ‘new‘ Australian university, demonstrates how students co-created knowledge about managing information overload. The analysis shows that not only is the CMCL a useful pedagogical tool for evaluating collaborative learning in student postings, the flow between their contributions, and knowledge co-creation, it can also be applied to further improve and test the design of discussion for web mediated forums.


Introduction

The nature and extent of evaluation in the literature of web mediated learning has been reviewed by Mason (1992), Alavi (1994), Gunawardena, Carabajal and Lowe (2001) and Treleaven (forthcoming). Three broad categories of evaluation studies can be identified (Treleaven, forthcoming): phenomenographic studies focusing on students‘ experiences, instructional method studies assessing the effectiveness of different e-learning tools and techniques, and sociocultural studies evaluating the teaching and learning processes employed in this social context.

In much of this literature, there is an implicit focus on evaluation. Thus the orientation of phenomenographic studies, as Gosling (2000) indicates, is to study students‘ experiences and perceptions of their online learning (Sanders and Morrison-Shetlar, 2001; Sullivan, 2001; Gallini and Barron, 2002; Askov and Simpson, 2001; Weiner, 2002). Few, however, are focused on the value of their collaborative learning.

Typically, instructional method studies are more targeted investigations assessing the effectiveness of different online learning tools and techniques for collaborative learning, and thereby student outcomes, in a wide range of settings (for example, Collings and Pearce 2002; Chen, Liu, Ou and Lin, 2001; Chang, 2001; Nakhleh, Donovan and Parril, 2000; Chalk, 2000). However, such research tends not to focus on whether it is collaborative online learning, in particular, rather than some other features of the course design or implementation, that is contributing to student outcomes.

Sociocultural studies cluster broadly speaking around three different research interests: pedagogical studies (Stacey and Rice, 2002; Ronteltap and Eurelings, 2002; Tu and Corry, 2001; Kumpulainen, Salovaara and Mutanen, 2001), linguistic studies (Weasenforth, Biesenbach-Lucas and Meloni 2002; Collot and Belmore, 1996; and Yates, 1996), and cross-cultural studies (Bates, 2001; Gunawardena, Nolla, Wilson, Lopez-Islas, Ramirez-Angel and Megchun-Alpizar, 2001; Lauzon, 2000; Warschauer; 1998; Herring, 1996). Sociocultural studies share an understanding that learning occurs in a social context and as such, evaluation of learning needs to take into account the nature of the learning environment and social interaction processes, as well as the tools and techniques employed. What distinguishes the sociocultural studies from studies that give more emphasis to ‘instruction‘ via tools and techniques to ‘deliver‘, is a recognition that meaning, and therefore learning also, is socially-constructed. Thus the social context, especially, but not only its cultural and cross-cultural demographics, within which meaning is produced, becomes an integral part of the research.

The study in this paper is broadly sociological, investigating the pedagogical implications for web mediated learning and teaching through a linguistic approach. Collaborative learning strategies require more social interaction and engagement between learners than do traditional methods (Rust & Gibbs, 1997; Topping & Ehly, 1998). As such, collaborative learning strategies, especially where constructively aligned with assessment (Biggs, 1996; Boud, 1999) produce deeper learning of concepts and theories, and the co-creation of knowledge. However, the process of restructuring learning from teacher centred to student centred is critical (Spiller, 1998). Not only do content, process, and assessment require redesign but also students and teachers are repositioned. Web technologies and appropriate strategies offer new possibilities in such reorientation to active learning (Bonk & Dennen, 1999; Freeman, 1997).

Though pedagogical studies have compared the effects on discussion and student performance of computer supported learning versus traditional, face to face learning environments (Alavi, 1994; Arbach, 2000a), there has been less emphasis on understanding the collaborative learning processes themselves from within these environments. Instructional method studies from teachers‘ perspectives investigate how student learning processes and learning outcomes can be monitored and assessed using student web portfolios and software support (Chen, Liu, Ou & Lin, 2001). Other recent publications, such as Salmon (2000), contribute to practitioners‘ development of e-moderating these new environments created and shaped by both staff and students. The Australian CUTSD project (Phillips, 2002) has focused on developing learner centred evaluation of computer facilitated learning in higher education. Yet employing a theoretical framework that informs a sociological evaluation of collaborative learning processes from the perspective of student learners actively engaged within web based discussion forums, such as the work of Cecez-Kecmanovic and Webb (2000), is relatively recent. This following study may, therefore, be seen as a response to Arbaugh‘s call (2000b:10) for theory-driven empirical research into online learning.

The paper builds on an earlier investigation into students‘ communicative practices (Treleaven & Cecez-Kecmanovic, 2001). Applying the Communicative Model of Collaborative Learning developed by Cecez-Kecmanovic and Webb (2000), that study traced the development and productivity of a collaborative learning space on an electronic bulletin board. A number of methodological and pedagogical implications raised in that study informed subsequent improvements in the web mediated learning design of a related subject that is the focus of investigation here. Specifically, this revised design of discussion forums is examined by evaluating the collaborative learning that more structured discussion enabled.

First, the Communicative Model of Collaborative Learning (CMCL) is discussed in terms of its assumptions, dimensions and classification of linguistic acts. Second, the research site for the empirical study of collaborative learning in the web enhanced undergraduate management subject is outlined. Third, the research method is described. Fourth, applying the CMCL model, a set of student postings on an electronic bulletin board is analysed for the dominant modes of linguistic acts and the flow between these postings. The analysis evaluates the collaborative learning of student postings, the flow between them, and knowledge creation generated. Fifth, these findings are then discussed in terms of their implications for designing web mediated collaborative learning environments. Conclusions are drawn about the value of the CMCL model as a tool for effectively evaluating not only students‘ collaborative learning but also for further improving and testing the design of discussion within electronic forums.

Theoretical framework: Communicative model of collaborative learning

The theoretical framework that shapes Cecez-Kecmanovic and Webb‘s Communicative Model of Collaborative Learning (2000) derives from Habermas‘ Theory of Communicative Action (1984). The CMCL which we apply to our empirical work is therefore based on three assumptions: first, that collaborative learning is enacted and mediated by language; second, that collaborative learning involves processes of social interaction; and third, that acts of communication or language acts function as social interaction mechanisms producing collaborative learning and knowledge co-creation processes. The CMCL framework is summarised in tabular form in Table 1.

Table 1: Communicative model of collaborative learning
(Cecez-Kemanovic & Webb, 2000)

Knowledge domainsSubject matter (1)Norms and rules (2)Personal experiences,
desires and feelings (3)
Dominant orientation
Learning (A)A1 - Linguistic acts about subject matter raised in order to share views and beliefs, to provide arguments and counter-arguments leading to mutual understanding and knowledge creationA2 - Linguistic acts that establish norms and rules regarding interaction and collaboration; cooperative assessment of legitimacy, social acceptability and rightness of individual behaviourA3 - Linguistic acts expressing personal views and feelings about learning process and other learners aimed at sharing experiences and increasing mutual understanding
Achieving ends (B)B1 - Linguistic acts that raise or dispute claims and provide arguments about subject matter, with an intent to frame attention, influence others and achieve goalsB2 - Acts of changing or interpreting norms and rules about the interaction process so as to suit a particular student interest and goals (may be at the expense of others)B3 - Acts expressing personal experiences in a way that influences other learners and instructors so as to help achieve goals (eg. emphasising personal success)
Self represent-
ation and promotion (C)
C1 - Raising or disputing claims and arguments as a performance on a stage that serves personal promotion (often neglecting an ongoing argumentation process)C2 - Raising or disputing claims about norms/rules or their violation in order to attract attention and establish oneself as a distinguished student (eg. a leader, an authority)C3 - Linguistic acts expressing personal experiences and feelings that project an impression of importance in a group or of a key role in a situation (eg. domination)

The CMCL identifies and classifies language acts as constituents of collaborative learning along two dimensions: the dominant orientation of learners and the domain of knowledge. First, the model identifies orientation to learning (manifested as a wish to know, to interact with others to increase mutual understanding and construct knowledge cooperatively); then orientation to achieving ends (manifested by students‘ primary motivation to achieve their goal eg. to get a pass or a good mark or to get the best mark in the class); and orientation to self representation and promotion (manifested by students‘ attempts to impress others by portraying a particular image of self). Second, the model differentiates between language acts that refer to different domains of knowledge, such as those related to subject matter and any substantive issues (theory, application, problem solving, etc.); linguistic acts addressing norms and rules that regulate the conduct of interactions and interpersonal relations in the collaborative learning process; and linguistic acts addressing personal experiences, desires and feelings by which students express themselves and shape both their individual and collective sense of self and of their learning process.

The CMCL thus enables classification of linguistic acts produced in particular learning situations according to the 3x3 scheme (Table 1). Communicative analysis based on this model focuses not only with what a specific linguistic act refers to but also how it contributes (in the flow of linguistic acts in a discussion) to the creation and maintenance of collaborative learning processes. For instance, although a certain linguistic act may be of the same type, eg. disputing (assumed or accepted) norms and rules, what it actually produces in terms of collaborative learning depends on the student‘s orientation. A student oriented to learning may dispute a norm seeking mutual understanding with other students and co-operative resolution of the dispute; on the other hand, a student oriented to achieving a good mark may dispute a norm if it does not suit his/her particular goals, seeking to change it without being much concerned about others; a student oriented to self-representation may dispute a norm for the sake of presenting himself/herself in a particular way, eg. as an important, influential and respected group member. Interpretation of a linguistic act is always within the context of the learning situation and the flow of linguistic acts constituting that learning process.

The research site

This study is located at a ‘new‘ Australian university, created by the amalgamation of three former colleges of advanced education. The empirical research is situated within an innovative, multi-mode delivery of an undergraduate management subject (see Treleaven & Cecez-Kecmanovic, 2001 for further details). The cohort formed an organisation within which groups of students undertook team projects contributing to organisational objectives (after Tyson, 1999). Collaborative learning in workshops was carefully structured to parallel the various components, complexity and challenges of working in organisations. Processes of social interaction shaped by this strategy emphasised collaborative learning.

A web interactive study environment (WISE) provided institutional support for staff and students adopting online teaching and learning strategies (Sheely, Veness & Rankine, 2001). In this online environment, electronic bulletin board forums were established for teams and organisational divisions, as well as forums where messages across the whole organisation were posted. The design development was to create additional forums for each week‘s substantive reading where a range of stimulus questions for threaded discussion was posted by the facilitator and students. Weekly contributions to the bulletin board formed an integral part of the subject assessment (25%) with orientation to collaborative learning being one of several assessment criteria. Students were required to select and annotate their best five contributions throughout the semester according to such criteria. The design of the subject is thereby illustrative of integrating the use of web-based technologies into collaborative student learning (Housego & Freeman, 2000).

Most of the students enrolled in the subject undertake it as a compulsory unit in a Bachelor of Business, Commerce, or Applied Science, though increasingly it is also selected as an elective. The cohort of 500 students each semester is extremely diverse in race and ethnicity. While most are enrolled full-time, many are working full-time, almost all part-time. Some are mature age and gender distribution is fairly balanced. To date, few students have previous experience of web-based learning, though almost all have Internet access where they live, indicating significant uptake of web technologies and social interaction.

In the 13-week semester, the 68 students constituting the ‘Wednesday branch‘ of the organisation in two shifts (a morning and evening class), produced over 1600 postings on their website where the bulletin board set up to enable collaborative learning. It is from one of these weekly forums, Week 5: Communication in Organisations that the threaded discussion on Information overload: a common problem, is presented for analysis. This selection was made first, as symptomatic of the student postings; second, for its inherent interest (hopefully) to those at this conference focusing on the richness of web-mediated communication and, more widely, for its relevance to those of us working in postmodern organisations.

Research method

A two step coding method was developed and applied to the discussion thread (Information overload: a common problem). After reading all messages in the thread, each posting was re-read and a description of ‘what does the student say‘ made to summarise the message. Having reviewed the message in this way, the researcher asked ‘what is the dominant orientation of the student in this posting?‘ and a code (A, B or C) was assigned. Postings were then read for the domain of knowledge by asking ‘what is the main focus of the student‘s attention?‘ with a code (1, 2 or 3) being allocated. This procedure gave each message a coding which was checked against the specific criteria in the relevant box of Table 1. This method was adopted by two researchers, independently and then collaboratively, until a coding was agreed. Indeed, the researchers‘ method of adopting an orientation to learning, mutual understanding and co-operative knowledge co-creation (A1) was highly productive in generating the substantive findings of this study.

As familiarisation with the model takes some time, a later study focusing on issues of coding replication and its validity implications for the CMCL model as an evaluation tool may be valuable.

One researcher brought contextual knowledge of the students having facilitated their learning for a semester; the other did not and recognised both the decontextualising effect on, and independence in, assigning codes. Accordingly, a column for some student context was added to the summary of postings and their analysis presented below in Table 2. Postings have necessarily been edited and ‘scaffolded‘ to retain meaning and flow. To indicate this scaffolding, a range of transcription protocols have been adopted. Words in [brackets] replace or add for purposes of readability, dots ... replace edited out text.

Table 2: CMCL analysis of student postings on information overload thread

ContributorPosting #Topic and Posting contentCodeInterpretive comment
Lesley
Facilitator
#20 initial discussion questionInformation Overload: a common problem
How are you managing your reading and monitoring of the BB? What system have you developed? What practices could you adopt from others to manage it better?
A2Invites students to reflect on and share the norms they have adopted regarding interaction and collaboration on the BB
Kara
Divisional leader of 3 teams, young first year day student with high self-efficacy
#487 (first reply to facilitator‘s discussion question)I have found that time needs to be set aside for the reading of the textbook... I usually [do this] just before completing the [online] quiz, so the information is fresh in my mind. Because my computer won‘t enter the OB site on WebCT, I am required to do all online work at a friend‘s house... Rather than procrastinate until Tuesday night, I complete all WebCT work in one sitting early each week. Other than the inconvenience, this systems keeps me organised and I am required to think ahead each week. With my computer situation, there isn‘t a lot I can alter about the system I am using.A3Expresses personal experiences about the learning process to share with others ways of dealing effectively with the subject‘s online components
Philip
Team leader in Kara‘s division, day student with high class engagement
#490 (in reply to #487)My system of tackling the weekly BB [bulletin board] tasks is very similar to that of Divisional Leader Kara. Though I do not have a specific day or time when I do the work, my routine is the same. Once I have motivated myself to actually sit down and do the work I will read the bulletin board topics prior to reading the chapter. This is so I pay particular note as I‘m reading. Like Kara I try and do the quiz as close after reading the chapter as possible. I always factor in time ...to do this work but inevitably its usually quite late in the week. A friend in another class without fail will do his work the day of his workshop. I should really try and adopt this practice as it would not only assist with my organisation for this subject but with my overall organisation and time management.A3Describes personal experiences and thoughts on learning process in response to another‘s experiences. The posting builds on the shared learning experience by offering potential strategies thereby increasing mutual understanding.
Terry
Team leader in day class, young, quiet reflective style
#538 (in reply to posting #490)I take a standard approach each week. I set aside one afternoon/ evening a week to read the BB and reply and also complete my quiz ...usually a Monday night ...I think the problem with this system is that there are a lot of late repliers on the Tuesday, so when I log on the following Monday, there are quite a number of unread messages. However, due to my classes going till 9pm on a Tuesday, I can‘t quite be bothered to do that. So far, though, my system seems to be running smoothly.B3Shares personal experiences with a primary focus on achievement of goals (getting the work done) rather than seeking to collaborate or increase mutual understanding.
Lily
Team member in evening class, working fulltime, reflective style of contribution, hence quiet in large class
#489 (in reply to facilitator‘s question)Contributions from everyone present us with large amounts of information. This may make us feel that we will miss something if we do not review/read all available info before making contributions. However, we need to recognise that we will not have every bit of information available before we can write up something. We can‘t look at everything. The key decision is: When do I have enough? ... To cope with volumes of contributions, we can develop our own filters (in mind) that screen out less-than-critical messages and prevent an overwhelming amount of ideas from being thrown in. Noting down good ideas and contributions and follow up with our own ideas if we have time. To further improve and control such situation(s), we of course need more time and well organise our work.A1Links shared feelings about the subject matter with arguments, supports views and beliefs about ways to address the subject matter so that mutual understanding and knowledge creation about the subject matter can be developed.
Terry
Second posting in this thread
#539 (in reply to #489)How do you decide what is ‘less-critical‘ information? Would you still not need to read some of the information to determine if it is critical or not?
I would also like to devise a way of focusing on what is important and what is less important.
A1Responds and raises further questions about the subject matter in order to create knowledge and foster mutual understanding.
Aaron
Team member, young day student
#578 (in reply to facilitator‘s question)Unlike most of you I tend to do all the web based activities without any set structure I usually just visit the site whenever I‘m on the net and do some of the work. The reading I do however is usually done just before the workshop each week.C2Dismisses without engaging in previous arguments on norms of interaction (eg read then post) established by previous postings.
Asmet
Member of Terry‘s team, young day student, reflective and quiet in large group settings
#584 (in reply to facilitator‘s question)The rapid pace to which students are expected to learn in today‘s environment is quite staggering. Last week on Channel Seven‘s a current affair program, there was an episode about the side effects that too much homework can have on the family relationship and stresses at home, for children at school. The program found that the longer children spent at home studying did not necessarily improve the quality of their time at school...The conclusion that I have come to, is that there needs to be a balance between schooling and social life. I know that I am most happy when I‘ve completed all the tasks for this subject and can then enjoy a night out with friends... I think a reward system such as this, is very important for your personal health, and helps you maintain the focus when you need it.A2Addresses norms and rules regarding interaction and collaboration by raising issues about the social acceptability and rightness of behaviour.
Saul
Team member in Kara‘s division, mature age, highly experienced in workforce
#597 (in reply to facilitator‘s question)I work two jobs and have a large number of other outside matters requiring my time. Accordingly... I have found that I had to be more organised to get my work done on time and hopefully to a suitable standard... My time management skills have increased by prioritising things I have to do. Prioritising is one of the best skills I have developed. I have also found that I am learning a lot from the course. They say a bit of stress helps, and therefore I must be getting a lot of help in getting my work done. In response to information overload, I heard a story on the radio which supported the TV show previously discussed, namely that too much is harmful. I have found that sometimes I just have to take a few hours off from work and Uni and relax. Although work builds up in this time, the benefits of a break outweigh the short term loss of time. I wonder if anyone else has found this feature important in organising their time and reducing the effects of information overload?B1Raises issues about the subject matter which are primarily concerned with achieving ends. Seeks to influence others by referring to and building on previous arguments concerning ways to get the work done (goals). Instrumental approach to subject matter.
Michaela
Team member in evening class, working fulltime, shy in large group
#636 (in reply to #597)During the few recent weeks before mid sem break until now, I find myself managing all the work and readings needed in OB, particularly the online component... I consult the learning guide first to see what previous work has to be done before reading the relevant chapter...Practices I could adopt in making my study routine more effective is to get all the work done well in advance so I have time to go over it rather than leave it the day before. I‘m a big believer in Saul‘s statement on taking breaks and relaxing to prevent information overload, which is the main reason I don‘t do all my OB homework in one day...B3Shares personal experiences primarily concerning achievement of goals (instrumental approach).
Binh
Team member, mature full-time worker in evening class, confident and inquiring learner in large group
#692 (in reply to # 597)In reference to the points you made Saul, I, too, find the demand of full time study and work quite overwhelming. What has kept me sane ...[is from] a time management course I undertook a few years ago. I was taught when confronted with the lack of time and a demanding work schedule, you should do what is IMPORTANT first and if time permits embark on what is URGENT. Now you may think what is the difference? The difference is that what we perceive as URGENT may not be IMPORTANT. The URGENT things ...you can delay and the consequences of not accomplishing them are not as great as those deemed IMPORTANT... Armed with this way of thinking my normal week includes working a 40 hr week, doing voluntary work for Mission Australia, tutoring, picking up my nephew from school and through it all find time to study. I also apply this theory of IMPORTANT VS URGENT to what I study ... I don‘t know the long term effects of this but for now it‘s working for me. Perhaps there is a better way. If there is, please enlighten me.A1Raises issues and arguments about the subject matter in order to share views and beliefs to facilitate mutual understanding and foster knowledge creation.
Facilitator
Validating reflection
#700 (in reply to #584)A thoughtful contribution Ashley showing that even when relaxing watching TV this question of information overload is related to your everyday life, and quality of life decisions.A2Encouraging norms regarding interaction and collaboration that facilitate application of learning

Analysis of bulletin board postings: The information overload thread

The thread presented in Table 2 has been selected to illustrate first, how the CMCL can be applied to students‘ linguistic acts in order to evaluate the collaborative learning taking place on the bulletin board; second, to examine the facilitator‘s linguistic acts, and third to demonstrate how through the flow of messages, students created and shared knowledge about information overload. Students were asked to reflect on how they managed their reading and monitoring of the BB and to share their system with each other with a view to improving their strategies for managing overload.

Students‘ linguistic acts

Application of the CMCL shows that in this thread, most students‘ postings (Kara, Philip, Lily, Asmet and Binh) demonstrate a dominant orientation to learning (A). Saul and Michaela‘s postings (B) are more orientated to achieving the subject requirements whilst Aaron‘s message (C) dismisses the argumentation process as not applicable to him. On the other hand, Terry‘s two postings (A and B) differ, showing that a student‘s orientation is not fixed and may be influenced by the orientation of the previous posting.

Further application of the CMCL identifies the domain of knowledge in which the student focuses the posting. When the student‘s focus is on the subject matter (1), they consider a broad range of issues related to information overload (Lily, Terry, Saul and Binh). When the student‘s focus is on norms and rules of interaction (2), they think about information overload in terms of the rightness or legitimacy of the decisions they make about it in their own lives (Aaron and Asmet). When the students‘ focus is on personal experiences, desires and feelings (3), they use personal language and examples to illustrate their points of view and strategies for dealing with information overload (Kara, Philip, Terry and Michaela).

By looking at both the dominant orientation and the domain of knowledge, a deeper analysis of their contributions can be made. For example, Lily‘s (A1) dominant orientation to learning and focus on the subject matter leads her to raise the need for effective screening and selection of information whilst recognising that action may need to be taken without having taken account of all information available. In turn, Terry (A1) engages with the question of ‘what is critical‘ raised by Lily. Another approach to the subject matter is taken by Saul (B1) whose dominant orientation here is towards achieving a ‘suitable standard‘ drawing on his knowledge of time management and prioritising. His message is distinguishable from a B2 response in that he seeks to influence others in meeting their goals by referring to and building on their previous arguments about how to get the work done. Terry‘s first posting in this thread (B3) is primarily focused on achieving ends so he shares his personal system of getting the work done. Whereas Kara (A3), starting off the thread, indicates her orientation to learning when she shares the learning process she has established to accommodate technical challenges.

Facilitator‘s linguistic acts

The facilitator‘s postings here have both been coded A2, indicating that they are oriented to learning (A) and are encouraging students to interact and collaborate about their system and practices (2), their strategies for managing information overload. It is noteworthy that the facilitator intended to stimulate a discussion on handling information overload in respect of their BB reading and monitoring, where approximately 100 messages were posted each week, only some of which students were expected to read. However, this intended focus on how electronic communication in organisations is problematic and needs to be managed is not the dominant discourse that took off on the thread. Kara‘s personal experience (A3) interpretation of reading beyond the BB and slippage into the whole course, shaped the primary context for many of the postings.

Thus, information overload was understood by most of the students to refer not only to the many messages posted on the BB but also to the information available to them for the course itself. This ignition of unexpected discursive activity, while in itself unproblematic where there is an orientation to learning generic skills (Gibbs et al., 1994; Falk, 1999) and application of theory to practice, can be disruptive if the facilitator is especially focused on a particular knowledge domain, most especially subject matter. However, in this management subject, content was to some extent de-emphasised over efforts to guide students towards reflection on their own management practices. Experience with alternative outcomes in discussion threads, especially the undigested regurgitation of textbook content on subject matter, is another paper in itself. Furthermore, this learning agenda is consistent with the aims of the subject to prepare students to work in organisations with reflection on their practice informed by theory and to develop a range of graduate skills, qualities and attributes. Thus students were permitted, indeed encouraged, to adopt patterns of BB activity that were orientated to learning and shared their experience (A3); in essence, to learn collaboratively online.

Flow of postings and collaborative learning

The information overload thread on this website comprised 36 postings of which 31 make explicit reference to earlier postings. In itself, this feature is evidence of high levels of social interaction and collaboration. Furthermore, 32 postings respond to the stimulus question, framed towards learning, by reflecting and commenting on how they have or could improve their practices of dealing with information overload. Only 4 postings do not show evidence of this orientation to learning by applying or relating to others‘ ideas, strategies or feelings.

Indeed, most individual postings describe how they manage high levels of information while showing evidence of learning that has been transformative. This transformation manifests in relation to their understanding of the subject matter, the norms they have established regarding interactions and collaboration on the BB, and how they feel about the learning process. Thus there is evidence that students, with one exception, have engaged in individual learning that they have then shared with others through the BB. In this way, knowledge about information overload and managing it has been created by the students together.

Students here come to understand the need to for a system that suits them, to be organised, to prioritise, to select what is critical, to give attention to what is important over what is urgent. In terms of establishing norms, they recognise that there are many different ways of approaching the information overload, that they must take action without ever being in possession of all informat- ion, that they must deal with technological difficulties and find ways to take breaks, all in the context of a balanced life. They come to understand that the dynamics of their feelings about the learning process, to understand the benefits of learning through a different teaching and learning process, and that, as fellow travellers, many of us struggle to put into practice what we know.

New ways forward towards evaluation

Application of the CMCL shows that it is indeed possible to evaluate the type of collaborative learning taking place in web mediated learning environments, especially electronic bulletin boards. The CMCL enables useful and fine analytical distinctions to be made between where the student is coming from (the dominant orientation of the student‘s posting) and where they are focusing their attention (the domain of knowledge). The CMCL also enables the flow of linguistic acts to be traced showing how collaboration and knowledge co-creation is generated within web-mediated environments. Thus the importance of the CMCL framework is not just that it evaluates what has taken place (ie student‘s collaborative learning and their knowledge co-creation) but also how these outcomes were shaped and generated. In doing so, the CMCL enables designers to look at how we create web-mediated collaborative learning environments. This is important because as designers better understand the social interaction processes from within these web-mediated environments, we can design better learning opportunities.

Furthermore, designers do not need to wait until student outcomes have been produced for a post-hoc evaluation only. The CMCL can be deployed at design stage to test and identify the possible responses that their stimulus questions, formulated for electronic bulletin boards, may elicit by coding the stimulus question for its dominant orientation and knowledge domain. Such testing requires the designer/facilitator to make explicit and then align with their operating assumptions and aims in constructing questions, and later moderating and posting. It may be also possible to reduce ambiguities in stimulus questions by asking what responses according to the CMCL codes the designer/facilitator would expect the question to elicit/generate. Each part of the design phase can thereby be tested before its implementation. However, the CMCL model cannot, of course, be predictive of student responses, since they are always context-sensitive as we have seen in this particular thread. Nevertheless, there is sufficient evidence here, and indeed in other linguistic frameworks, to indicate the importance of the orientation to learning in shaping students‘ postings.

This study concludes from analysis of one set of selected student postings on an electronic bulletin board in a diverse group of undergraduate management students that it is indeed possible to evaluate online collaborative learning. The CMCL model developed by Cecez-Kecmanovic and Webb enables evaluation that is both descriptive as well as functional, and significantly, can be applied indicatively to improve the design of web-mediated collaborative learning environments.

Acknowledgments

The author acknowledges the professional support of staff in WISE, CHED and FLU at University of Western Sydney Hawkesbury Campus, and the participation of Blacktown students in Organisational Behaviour (MG204A) 2000-2002; colleagues Dubravka Cecez-Kecmanovic and Carolyn Webb for stimulating curiosity about web based learning, and Karen McMillan for insightful and skilled research assistance. The assistance of a UWS Seed Grant is gratefully acknowledged.

References

Alavi, M. (1994). Computer-mediated collaborative learning: An empirical evaluation. MIS Quarterly, 18, 159-174.

Arbaugh, J.B. (2000a). Virtual classroom versus physical classroom: An exploratory comparison of class discussion patterns and student learning in an asynchronous Internet-based MBA course. Journal of Management Education, 24(2), 207-227.

Arbaugh, J.B. (2000b). How classroom environment and student engagement affect learning in Internet-based MBA courses. Business Communication Quarterly, 63(4), 9-26.

Askov, I. & Simpson, M. (2001). Researching distance education: Penn State‘s Online Adult Education Med Degree on the World Campus. Proceedings AVETRA 2001. [verified 11 Feb 2003 at http://www.avetra.org.au/2001%20conference%20pages/PAPERS%202001/askov.pdf]

Bates, T. (2001). International distance education: Cultural and ethical issues. Distance Education, 22(1), 122-136.

Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32, 347-364.

Bonk, C.J. & Dennen, V. (1999). Teaching on the Web: With a little help from my pedagogical friends. Journal of Computing in Higher Education, 11(1), 3-28.

Boud, D. (1999). Sustainable assessment. Keynote Address to Northumbria Assessment Conference, September.

Cecez-Kecmanovic, D. & Webb, C. (2000). Towards a communicative model of collaborative web-mediated learning. Australian Journal of Educational Technology, 16(1), 73-85. http://www.ascilite.org.au/ajet/ajet16/cecez-kecmanovic.html

Chalk, P. (2000). Webworlds-Web-based modelling environments for learning software engineering. Computer Science Education, 10(1), 39-56.

Chang, C. (2001). A study on the evaluation and effectiveness analysis of Web-based learning portfolio (WBLP). British Journal of Educational Technology, 32(4), 435-458.

Chen, D-G, Liu, C-C, Ou, K-L & Lin, M-S. (2001). Web-learning portfolios: A tool for supporting performance awareness. Innovations in Education and Training International, 38(1), 19-32.

Collings, P. & Pearce, J. (2002). Sharing designer and user perspectives of web site evaluation: A cross-campus collaborative learning experience. British Journal of Educational Technology, 33(3), 267-278.

Collot, M. & Belmore, N. (1996). Electronic language: A new variety. In S. C. Herring (Ed), Computer-mediated communication -- linguistic, social and cross-cultural perspectives (pp. 13-28). Amsterdam: John Benjamins.

Falk, J. (1999). Taskforce on the Characteristics of the UWS Graduate: Report to the UWS Senate. Richmond: UWS.

Freeman, M. (1997). Flexibility in access, interaction and assessment: The case for web-based teaching programs. Australian Journal of Educational Technology, 13(1), 23-39. http://www.ascilite.org.au/ajet/ajet13/freeman.html

Gallini, J. K. & Barron, D. (2002). Participants‘ perceptions of Web-infused environments: A survey of teaching beliefs, learning approaches, and communication. Journal of Research on Technology in Education, 34(2), 139-156.

Gibbs, G., Rust, C., Jenkins, A., & Jaques, D. (1994). Developing students‘ transferable skills. Oxford: Oxford Centre for Staff Development.

Gosling, D. (2000). Using Habermas to evaluate two approaches to negotiated assessment. Assessment & Evaluation in Higher Education, 25(3), 293-304.

Gunawardena, C. N., Nolla, A. C., Wilson, P. L., Lopez-Islas, J. R., Ramirez-Angel, N. & Megchun-Alpizar, R. M. (2001). A cross-cultural study of group process and development in online conferences. Distance Education, 22(1), 85-121.

Gunawardena, C., Carabajal, K. & Lowe, C. A. (2001). Critical analysis of models and methods used to evaluate online learning Networks. Paper presented at the Annual Meeting of the American Educational Research Association, Seattle, 10-14 April.

Habermas, J. (1984). The Theory of Communicative Action - Reason and the Rationalisation of Society (Vol I). Beacon Press, Boston, MA.

Herring, S. C. (Ed) (1996). Computer-mediated communication--linguistic, social and cross-cultural perspectives. Amsterdam: John Benjamins.

Housego, S. & Freeman, M. (2000). Case studies: Integrating the use of web-based learning systems into student learning, Australian Journal of Educational Technology, 16(3), 258-282. http://www.ascilite.org.au/ajet/ajet16/housego.html

Kumpulainen, K., Salovaara, H. & Mutanen, M. (2001). The nature of students‘ sociocognitive activity in handing and processing multimedia-based science material in a small group learning task. Instructional Science, 29(6), 481-515.

Lauzon, A. C. (2000). Distance education and diversity: Are they compatible? The American Journal of Distance Education, 14(2), 61-70.

Mason, R. (1992). Computer Conferencing: The Last Word. Victoria, BC: Beach Holme.

Nakleh, M. B., Donovan, W. J. & Parrill, A. L. (2000). Evaluation of interactive technologies for chemistry Websites: Educational Materials for Organic Chemistry Web site (EMOC). The Journal of Computers in Mathematics and Science Teaching, 19(4), 355-378.

Phillips, R. A. (2002). Learning-centred evaluation of computer-facilitated learning projects in higher education: Outcomes of a CUTSD Staff Development Grant "Staff Development in Evaluation of Technology-based Teaching Development Projects: An Action Inquiry Approach". Committee for University Teaching and Staff Development, Commonwealth of Australia. http://www.tlc.murdoch.edu.au/project/cutsd01.html

Ronteltap, F. & Eurelings, A. (2002). Activity and interaction of students in an electronic learning environment for problem-based learning. Distance Education, 23(1), 11-22.

Rust, C. & Gibbs, G. (Eds) (1997). Improving Student Learning through Course Design. Oxford: Oxford University Press.

Salmon, G. (2000). E-moderating: The key to teaching and learning online. London: Kogan Page.

Sanders, D. W. & Morrison-Shetlar, A. (2001). Student attitudes toward Web-enhanced instruction in an introductory biology course. Journal of Research on Computing in Education, 33(3), 251-262.

Sheely, S., Veness, D. & Rankine, L. (2001). Building the Web Interactive Study Environment: Mainstreaming online teaching and learning at the University of Western Sydney. Australian Journal of Educational Technology, 17(1), 80-95. http://www.ascilite.org.au/ajet/ajet17/sheely.html

Spiller, D. (1998). Using the peer group as a teaching and learning resource: Redefining the role of the teacher. Proceedings of the 1998 Annual Conference of HERDSA, Auckland. [verified 11 Feb 2003] http://www2.auckland.ac.nz/cpd//HERDSA/HTML/TchLearn/spiller.HTM

Stacey, E. and Rice, M. (2002). Evaluating an online learning environment. Australian Journal of Educational Technology, 18(3), 323-340. http://www.ascilite.org.au/ajet/ajet18/stacey.html

Sullivan, P. (2001). Gender differences and the online classroom: male and female college students evaluate their experience. Community College Journal and Practice, 25(10), 805-818.

Topping, K. & Ehly, S. (Eds) (1998). Peer Assisted Learning. London: Lawrence ErlaumAssociates.

Treleaven, L. & Cecez-Kecmanovic, D. (2001). Collaborative learning in a web-mediated environment: A study of communicative practices. Studies in Continuing Education, 23(2), 169-183.

Treleaven, L. (forthcoming). Evaluating web-mediated collaborative learning. In T. Roberts (Ed), Online Collaborative Learning: Theory and Practice. Hershey USA: Idea Group, Inc.

Tu, C. & Corry, M. (2001). A paradigm shift for online community research. Distance Education, 22(2), 245-263.

Tyson, T. (1999). CALO - the Class as a Learning Organisation: the CAO model revisited and revitalised. Human Resource Management/Organisation Behaviour, Working Paper Series, School of Business, Swinburne University of Technology, Melbourne.

Warschauer, M. (1998). Technology and indigenous language revitalization: Analyzing the experience of Hawaii. The Canadian Modern Language Review, 55(1), 139-159.

Weasonforth, D., Biesenbach-Lucas, S. & Meloni, C. (2002). Realizing constructivist objectives through collaborative technologies: threaded discussions. Language, Learning and Technology, 6(3), 58ff.

Weiner, C. (2002). A new alternative: Adolescent students study in cyberspace. Dissertation Abstracts-International Section A: Humanities and Social Science, Vol 63(1-A), 155.

Yates, S. C. (1996). Oral and written linguistic aspects of computer conferencing: A corpus based study. In S. C. Herring (Ed), Computer-mediated communication -- linguistic, social and cross-cultural perspectives (pp. 29-46). Amsterdam: John Benjamins.

This article was nominated for an Outstanding Paper Award at ASCILITE 2002, gaining the additional recognition of publication in AJET (with minor revisions). The reference for the Conference version is:

Treleaven, L. (2002). Evaluating a communicative model for web mediated collaborative learning and design. In A. Williamson, C. Gunn, A. Young and T. Clear (Eds), Winds of Change in the Sea of Learning: Proceedings of the 19th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education, pp643-652. Auckland, New Zealand: UNITEC.

Author: Dr Lesley Treleaven, Director, ISKM (Information Systems Knowledge Management) Research Group, and Senior Lecturer, School of Management, College of Law and Business, University of Western Sydney, Locked Bag 1797, Penrith South DC NSW 1797, Australia. l.treleaven@uws.edu.au

Please cite as: Treleaven, L. (2003). Evaluating a communicative model for web mediated collaborative learning and design. Australian Journal of Educational Technology, 19(1), 100-117. http://www.ascilite.org.au/ajet/ajet19/treleaven.html


[ AJET 19 ] [ AJET home ]
HTML Editor: Roger Atkinson [rjatkinson@bigpond.com]
This URL: http://www.ascilite.org.au/ajet/ajet19/treleaven.html
Created 10 Mar 2003. Last revised 11 Mar 2003.

 
本站仅提供存储服务,所有内容均由用户发布,如发现有害或侵权内容,请点击举报
打开APP,阅读全文并永久保存 查看更多类似文章
猜你喜欢
类似文章
【热】打开小程序,算一算2024你的财运
The Future of Instruction: Teacher as ‘Co-Learner‘
Multiple Intelligences Theory and English Language Teaching
合作学习与协作学习概念辨析:collaborative learning versus cooperative learning
Engagement Theory:A framework for technology-based teaching and learning
e-learning2.0渗透教室
Exploring Instructional Design Factors Prompt...
更多类似文章 >>
生活服务
热点新闻
分享 收藏 导长图 关注 下载文章
绑定账号成功
后续可登录账号畅享VIP特权!
如果VIP功能使用有故障,
可点击这里联系客服!

联系客服