Context and Conditions
During the program internship, EDET 650 – Internship in Educational Technology, taken in the Fall semester 2022 , I assisted with updating instructional material that was originally developed in 2016. I had the opportunity to complete a course evaluation on one of the courses in which I was updating. The organization I was working with did not ultimately address the issues and recommendations, but this was a great experience for me.
Scope
The purpose of the course evaluation was to fulfill a class assignment, however, valid opportunities for course improvement were discovered during the process.
Role
I was working as an intern for the learning and education team at the host organization. As an intern, I reported to the learning and development specialist. In this role, I was solely the instructional designer and I had to rely on others as subject matter experts.
Instructional Design
The course that was evaluated was called Basic Budgeting and Municipal Finance. The summative evaluation revealed that learning objectives or goals were not clear or measurable. Also, the instruction included a student survey post course, but no graded assessment was used to measure the learner’s understanding of the content. I used the methods outlined in the Morrison, Ross and Kemp model to carry out the evaluation (Morrison et al., 2013).
Related Performance Indicator
The course evaluation included a process that assessed the adequacy of the learning. This course evaluation supports AECT Standard 2 – Content pedagogy. Completing the program or course evaluation requires the evaluation of the instructional material to determine if improvements are needed.
Reflection
The internship was a great experience. Completing the course evaluation, I was able to speak with subject matter experts (field service managers) that knew what content was needed for the course and they had great feedback on the old instructional material. After interviewing one of the field service managers, I was documenting my report and realized that I had not been clear in my notes that I wrote and I wasn't sure what she meant on a statement I wrote down. I ended up having to call her back for clarity. This showed me that when interviewing, you must be clear and thorough on the questions, ask questions until you are satisfied with understanding their answer and then make sure to document the details for later review. When I first took a look at the video instruction that includes a verbal explanation, along with written slides, I immediately noticed that the learning objectives were too broad and not measurable. Also, as I took the course that I was evaluating, and I noticed the ungraded assessments, but there were no graded test, quiz or assessment to gage the learners’ comprehension. Therefore, determining what instruction improvements were needed was a tough task. Therefore, my first recommendation was to create more specific learning goals and then also put some measurement in place to be able to evaluate the success of the instruction. Ultimately, the organization chose to maintain the broader objectives and decided not to implement any graded assessments for the learners. The elearning modules were to help elected officials gain an understanding about local goverment. The municipalities paid for these elected officials to take the course and it is important that these individuals take the courses. If my recommendations were implemented, it wold likely decrease the participation. The thought process is that watching the videos and doing the ungraded practice questions is better than having no learning at all. I understood their reasoning, but certainly explained that needed improvements will be tough to gage in the future with no standards to measure. Without having objective data on how well the learner comprehended the content, it is difficult to identify what areas need to be improved. Standard 2 is reflected in this artifact because I used multiple techology tools. I manually entered all data into a spread sheet and then analyzed the data. I documented the report. In this process, through interviews, surveys and reviewing the course myself, I assessed the adequacy of the previously developed learning and attempted to evaluate the instruction and implementation of education material delivered in Articulate Storyline.