Online learning

Robert A. Blackman

EDT 646

Response #11

Evaluation

            When it comes to evaluation of online learning, there seems to be some controversy regarding what information should be included.  R.E. Clark states that one should choose to evaluate either the instruction or the method of delivery while Kozma is of the opinion that the two are interwoven and need to be evaluated together.  Of the two theories, I feel that Kozma’s views of evaluating online learning is more appropriate because it affords the evaluator more in-depth information as to what works and what doesn’t in the  instructional process.  In addition, evaluating both the instruction and the delivery method may yield valuable insights to how students utilize technology as a learning tool.  To properly evaluate the online learning activity that I have set forth in paper #2: Saving the Rain Forest, it will be necessary to evaluate both my instruction and the method of deliver for the purposed unit. 

To create a good formative evaluation I will first need to define the purpose of the evaluation.  This purpose is three-fold; 1) to determine if my instructional design is causing an increase in student achievement in their core curriculum classes, 2) to see if the students can transfer the skills taught to new and different problems, and 3) if tappedin.org helps to facilitates this outcome.  In order to guide my evaluation I would then need to draft an evaluation statement.  This statement provides a mechanism with which I can begin to target the methods and measurement approaches that best align with an evaluation effort (Dabbagh, Ritland, 2005).  To conduct a balanced evaluation that takes into account all factors, I will need to craft activities that address levels 1, 2, and 3 of Kirkpatrick’s model which are respectively reaction, learning, and behavior.

            Since increasing my students’ achievement levels in their core curriculum classes is my first goal, my evaluation must include activities that help uncover this information.  To achieve this, my first activity will be to discuss my design with colleagues to determine if the information that I plan on presenting to the students will be useful.  The feedback that they provide me with will help me to determine if the information will be valuable when planning for my supplementary materials and give me some guidance for the direction I want my students to take in their learning.  I would also ask them to help me design a pre and post test to use in determining their prior knowledge and their achievement after the unit has been delivered.  Furthermore, by making my colleagues aware of my design, I may be able to use data taken from their assessments to further supplement my evaluation. 

            Informal feedback will also be gathered from the students.  This feedback will come in the form of weekly reflections, student postings, and final projects.  In their reflections students are to sum up what new information they have learned and how they might apply this information to a new problem.  Additional data pertaining to the students’ postings and the frequency of their activity will also be collected from the MUD as well as the assessment of their finial project.   Students will also give input to help evaluate the method of delivery.  This information will be gathered by survey and informal interviews regarding any difficulties they might have or suggestions for improvement.  These informal feedback mechanisms provides information for the instructor to review so that they can think about changes for the course or can be used as a data source for more rigorous qualitative analysis (Dabbagh, Ritland, 2005). 

            To achieve improvement in the lesson design, the data from many of these activities will be collected and repeated throughout the instruction.  Using this method will afford me the opportunity to correct unforeseen complications that may have a negative effect on my desired outcomes for student achievement.  Evaluation of the students post test and their final projects will also give me insight to the overall success of my design.

I have added Paper #2 here if you wish to read it.  The paper contains plans for a lesson that I hope incorporate next year.   Any comments would be appreciated.  

Free is GoodI recently attended the MACUL conference in Detroit.  I thought that I would share one of the resources that I found there.