Artha’s E-learning Blog(^.^)

Our group explored the topic of evaluating the effectiveness of e-learning with relevant case studies. Evaluating e-learning initiatives means to measure its value. It is a waste for organisations to have e-learning initiatives if they don’t increase or improve the performance of the organisation and its employees. Therefore it is vital for organisations to spend some of their effort in evaluating their e-learning initiatives so that they can be sure that the e-learning being implemented will have a positive impact in the organisation. Another reason for this is to identify any improvements that can be made to advance the e-learning initiative and to decide on whether the e-learning should be terminated or continued. By concentrating on level three and four of Kirkpatrick’s evaluation model, organisations will be able to discover on whether their e-learning initiatives have changed the behaviour of their employees and whether through these behavioural changes, positive results have been made in the organisations. The case studies that we looked at were: Energy
Australia, Qantas, Citibank, Maxis Mobile and The Australian Army. This review will look at the similarities and differences within these case studies as well as the issues that have arisen for educators or practitioners on the topic of evaluation. There will also be some recommendations for future research in this field.

Energy Australia implemented an e-learning initiative to train their employees the basic fundamentals of OH&S in installing a new electric device. Their evaluation process is evident throughout the four stages of their e-learning development. They used a questionnaire as a method for evaluating how effective the e-learning has been to the employees which included first year apprentices, trainers, compliance officers, members of the Enerserve’s Executive Management, engineers and engineering officer responsible for customer supply. They asked questions such as ‘How well has the program been designed?’ ‘Did it address correct technical barriers?’ and ‘Was the course based between theory and practice?’

 Qantas implemented their e-learning initiative to improve the quality of their service. Before implementing the new service system, Qantas held several big company meetings to inform the employees, employers, customers and stakeholders about it and acquired their opinions. They used online surveys and questionnaires given to customers to evaluate the effectiveness of the new system.

Citibank implemented their e-learning initiative to train their employees all over the world on sales. Their evaluation method was their in-cooperation of developers and target audience in their e-learning system’s authoring team, therefore opinions and feedback are given from two different perspectives. Maxis mobile, with the help of IBM implemented a customised e-learning Web-based training program in the workplace in order for the company to decrease cost and increase training flexibility. Their evaluation method is the calculation of the total cost saving which measured the result of the e-learning initiative.

The Australian Army’s implementation of their e-learning initiative was evaluated by having several interviews with the Australian Army personnel involved in the development and implementation of e-learning, course developers and instructional designers, and instructors and managerial staff involved in the e-learning delivery. During the interview, questions such as, ‘What do you think of computer based learning?’ and ‘What are the advantages and disadvantages?’ were asked to all interviewees.

Through all of these case studies there are some important factors influencing the effectiveness of e-learning implementation that can be seen. They include making sure that the organisational and learning goals are achieved and meeting learners’ needs. One of the similarities between these case studies is the method for evaluating that was used. Even though the questions asked were different, all of the organisations in the case studies used a form of questionnaire, interview or survey to collect feedback on their e-learning initiatives. Therefore, this method is useful and recommended by many organisations and has been proved to be successful. But for a more thorough and accurate evaluation, organisations need to incorporate other methods such as calculating the organisations’ costs to see whether profit has been increased after the e-learning was implemented and Balanced Scorecard (BSC) – (an accounting method of measuring organisational performance) can also be used. Thus, for a successful evaluation process all employees within the organisation need to be involved including top level management. For example, human resource personnel can work together with trainers and accountants to determine the value of the e-learning initiative and the improvements that may be needed.

From this study there are some issues that educators or practitioners may be faced with such as determining the right questions for the survey, questionnaire or interview as they need to address behavioural changes in learners and also the results that are made due to these changes. Another issue that will affect them is in the area of learning styles because the evaluation of e-learning initiative would determine on whether knowledge has been transferred successfully or not. If knowledge has not been transferred it may be due to the learners’ learning style, they might not be suited to the lack of personal interaction that e-learning may not offer and therefore educators and practitioners need to be able to identify these learners and find a solution for them so that knowledge can be transferred successfully, their behaviour may be changed and thus positive result can occur in the organisation. As well as these issues, there are further researches that need to be conducted to determine the best method for evaluating the e-learning initiatives.  In the program design area, research can be made on how to develop a program that would cater every learner’s learning styles.

Although there are still further research to be made to further verify the importance of evaluating the effectiveness of e-learning initiatives and to determine the methods for evaluating both the behavioural changes and the organisational impacts; these case studies have shown the importance of evaluating organisations’ e-learning initiatives so that their e-learning initiatives becomes an important part of their organisation.