PROFESSIONAL LEARNING AND PROGRAM EVALUATION
Georgia Professional Standard 5
Candidates demonstrate the knowledge, skills, and dispositions to conduct needs assessments, develop technology-based professional learning programs, and design and implement regular and rigorous program evaluations to assess effectiveness and impact on student learning.
- Element 5.1 Needs Assessment - Candidates conduct needs assessments to determine school-wide, faculty, grade-level, and subject area strengths and weaknesses to inform the content and delivery of technology- based professional learning programs.
- Element 5.2 Professional Learning - Candidates develop and implement technology-based professional learning that aligns to state and national professional learning standards, integrates technology to support face-to- face and online components, models principles of adult learning, and promotes best practices in teaching, learning, and assessment.
- Element 5.3 Program Evaluation - Candidates design and implement program evaluations to determine the overall effectiveness of professional learning on deepening teacher content knowledge, improving teacher pedagogical skills and/or increasing student learning.
program evaluation
In FRIT-7237 Program Evaluation, I co-evaluated the Positive Behavior Interventions and Support (PBIS) program along with two classmates. Our evaluation was designed to determine the overall effectiveness of the PBIS program. The Evaluation Plan sought to answer two questions: Is the system for teaching the behavioral expectations to students being utilized with fidelity? And, Is data used consistently to inform and improve decision-making?
One of my classmates actually worked in a school setting where the program had already been implemented and was active. Admittedly, it was very hard for me to evaluate a program in which I had no direct involvement or participation. Most of our evaluation considerations came from my classmate. However, I did learn a lot about it by reading the pbis.org website, school internet articles, research presentations, etc. This helped guide my expectations on how the program should function within a school. Each member of our group had a strength in one area of the assignment or another and this is what it took for us to get it done even when parts of the assignment were unclear. I think program evaluation will be one area that I will need more practical exposure. I think if I had a realistic role in evaluation process, my experience would be richer and my understanding would be fuller. |
Technology Program Administration
I decided to evaluate the Educational and Collaborative Technology Center at Georgia Regents University. The Center is located on the Summerville campus at GRU. The best part about completing this evaluation was my interactions with the staff as they were assisting students who were working on multimedia and video projects. Most of the students in the Center at that time were working on projects for Sociology related topics. The footage was very well put together. Every now and then the Center would all of a sudden emerge with bursts of laughter as students humorously worked together to edit video footagaefor a class project. Soon afterwards, the whole Center would engage in the laughter and then everyone would go back to work quietly. This was pretty awesome. I don't think the students knew how much this added to their productivity.
I really wanted to incorporate more student and faculty feedback into the evaluation however this proved to be very challenging. However, my timing may have been inopportune. My evaluation was conducted during the final weeks of classes and so students were very focused on completing their assignments, if not frantic at times, and didn't have much time to devote to non-course related requests. But just having the opportunity to observe them at work really spoke volumes to how important the Center was to them and of the value that the Center places on their success. For a general overview of the assignment, please visit my blog |
Face-TO-FACE STAFF DEVELOPMENT WORKSHOP
A few weeks before the workshop, a senior level member of my team approached me to ask if my site supervisor and I would be able to give a presentation or conduct a workshop for the Office of Institutional Effectiveness and Office of Faculty Development to explain how Desire2Learn (D2L) learning environment tools could be used to assess student learning outcomes and promote student learning. The workshop would be part of a larger effort already in process to establish a system and standard for assessing student learning outcomes across the institution. A Student Learning Outcomes (SLO) Core Team had been formed that consisted of institutional and college-level administrators and select faculty that would head the effort. My site supervisor and I discussed the request and began planning the workshop. We both agreed that this would be a great way to fulfill my Staff Development Workshop Practicum requirement.
I conducted a brief Needs Assessment in order to determine how to approach the instructional piece of the workshop. The Needs Assessment findings revealed that faculty and instructors would need to be trained on the functional and instructional use of the LMS tools used in an online Learning Environment and that exposure to good online course design and teaching principles and strategies would help encourage a pedagogical change in thinking about how to teach online. I had read the Chickering and Ehrmann (1996) article on good teaching some time before and thought that these principles would be a great foundation to lay D2L tool fundamentals upon. Since our institution was a subscriber to the Quality Matters Program, we could also incorporate Quality Matters into the workshop as a foundation for quality course design. This would also give a chance to promote the QM program on our campus.
I believe that communication is key to the success of any project, no matter how large or small, or how much knowledge and skill are available to accomplish the project. There were some elements of the workshop request that weren't communicated very well or that got lost in the multiple transmissions of information from one group to another. When my site supervisor and I were approached with the request, we were asked to show how Desire2Learn could be used to measure student learning objectives. With that being the only thing said at that time, we planned our instruction from faculty teaching perspective as a topic for a faculty development workshop. I knew that the institution had an expectation that faculty who taught online should utilize the tools available in the Learning Management System to enhance the student learning experience. I thought this was a great opportunity to address both the means and the end. However, we soon found out that the participants were more interested in how D2L could be used to assess and manage learning outcomes at the program level. Their interest was in how the D2L Competency and Analytics tool could be used to track student outcomes across courses in a program. So our workshop took a slightly different turn from its intended purpose.
The workshop discussions were very lively and each participant was deeply engaged in the discussions. One of the performance tasks was to
brainstorm a set of activities, strategies, and assessments that could be created using Desire2Learn course tools and related campus technology given a list of learning outcomes. Though they didn't do this exactly, one participant did walk over to the whiteboard and drew a visual that that showed the relationship between program level outcomes, course level objectives, and course activities. This particular person was known to be very detail-oriented and analytical. I was very impressed. Another performance task was that the participants review the Quality Matters Rubric and discuss with the group how the rubric could be applied in concert with applying the Seven Principles for Good Practice. Again, they didn't do the task exactly the way it was designed, however they did discuss how the Quality Matters standard about Learning Objectives would be a very important consideration in the scope of the greater SLO project. Lastly, we had planned for the participants to collaborate to draft a strategic outline for faculty development in online teaching. We figured this could be a more long term activity that we could followup on at a future workshop. Since the Student Learning Outcomes project was already underway, the group was already collaborating with the academic programs to train program administrators and faculty on how to write student learning outcomes and objectives at the program level.
Though the workshop was not planned in vain, we had set out to address a need at a much lower level than what our audience had in mind. The workshop would be more suited for preparing or refreshing faculty for teaching online. One of the errors I made was not fully engaging all vested groups in the Needs Assessment process. I based a lot of what I did on the assumptions I made about what everybody wanted. I saw the workshop as an opportunity to discuss topics that I felt had the greatest need. My focus was on faculty and students at the course level. Their focus was on faculty and students at the program and college level. Both foci would ultimately converge because one serves as the foundation for the other. Maybe I should have known that the workshop would take on a more high-level focus. Maybe we all made assumptions about what the other person or group knew or wanted. Hence, my beliefs about effective communication.
For a general overview of the assignment, please visit my blog.
Note: I could not include the Quality Matters Rubric in my list of artifacts since it is only available to subscribers and cannot be distributed without permission from Quality Matters.
I conducted a brief Needs Assessment in order to determine how to approach the instructional piece of the workshop. The Needs Assessment findings revealed that faculty and instructors would need to be trained on the functional and instructional use of the LMS tools used in an online Learning Environment and that exposure to good online course design and teaching principles and strategies would help encourage a pedagogical change in thinking about how to teach online. I had read the Chickering and Ehrmann (1996) article on good teaching some time before and thought that these principles would be a great foundation to lay D2L tool fundamentals upon. Since our institution was a subscriber to the Quality Matters Program, we could also incorporate Quality Matters into the workshop as a foundation for quality course design. This would also give a chance to promote the QM program on our campus.
I believe that communication is key to the success of any project, no matter how large or small, or how much knowledge and skill are available to accomplish the project. There were some elements of the workshop request that weren't communicated very well or that got lost in the multiple transmissions of information from one group to another. When my site supervisor and I were approached with the request, we were asked to show how Desire2Learn could be used to measure student learning objectives. With that being the only thing said at that time, we planned our instruction from faculty teaching perspective as a topic for a faculty development workshop. I knew that the institution had an expectation that faculty who taught online should utilize the tools available in the Learning Management System to enhance the student learning experience. I thought this was a great opportunity to address both the means and the end. However, we soon found out that the participants were more interested in how D2L could be used to assess and manage learning outcomes at the program level. Their interest was in how the D2L Competency and Analytics tool could be used to track student outcomes across courses in a program. So our workshop took a slightly different turn from its intended purpose.
The workshop discussions were very lively and each participant was deeply engaged in the discussions. One of the performance tasks was to
brainstorm a set of activities, strategies, and assessments that could be created using Desire2Learn course tools and related campus technology given a list of learning outcomes. Though they didn't do this exactly, one participant did walk over to the whiteboard and drew a visual that that showed the relationship between program level outcomes, course level objectives, and course activities. This particular person was known to be very detail-oriented and analytical. I was very impressed. Another performance task was that the participants review the Quality Matters Rubric and discuss with the group how the rubric could be applied in concert with applying the Seven Principles for Good Practice. Again, they didn't do the task exactly the way it was designed, however they did discuss how the Quality Matters standard about Learning Objectives would be a very important consideration in the scope of the greater SLO project. Lastly, we had planned for the participants to collaborate to draft a strategic outline for faculty development in online teaching. We figured this could be a more long term activity that we could followup on at a future workshop. Since the Student Learning Outcomes project was already underway, the group was already collaborating with the academic programs to train program administrators and faculty on how to write student learning outcomes and objectives at the program level.
Though the workshop was not planned in vain, we had set out to address a need at a much lower level than what our audience had in mind. The workshop would be more suited for preparing or refreshing faculty for teaching online. One of the errors I made was not fully engaging all vested groups in the Needs Assessment process. I based a lot of what I did on the assumptions I made about what everybody wanted. I saw the workshop as an opportunity to discuss topics that I felt had the greatest need. My focus was on faculty and students at the course level. Their focus was on faculty and students at the program and college level. Both foci would ultimately converge because one serves as the foundation for the other. Maybe I should have known that the workshop would take on a more high-level focus. Maybe we all made assumptions about what the other person or group knew or wanted. Hence, my beliefs about effective communication.
For a general overview of the assignment, please visit my blog.
Note: I could not include the Quality Matters Rubric in my list of artifacts since it is only available to subscribers and cannot be distributed without permission from Quality Matters.
ONLINE STAFF DEVELOPMENT WORKSHOP
This assignment took a very long time to complete. I totally underestimated the amount of time it was going to take to complete it. Once I got going I started thinking about all the important elements that needed to be in an effective online course module. Every time I thought I had a winner product, I realized some "good design practice" that I had forgotten and had to go back. If I had a dollar for every time I revised this course, I'd be able to pay off all my student loans today! I used Weebly to deliver my online course. I like it ok, but it has some accessibility limitations that needs to be addressed in a future update possibly.
In the early design stages, I had trouble trying to figure out how to incorporate the appropriate web 2.0 tool into my course. I wanted to use a tool that would serve a valuable purpose in the course and not just look like it was thrown in there to meet a course requirement. I was experiencing a creativity block. I had to step outside the status quo mindset of course design and think outside my brain-box a bit. It all came to me after while. I did not include the Quality Matters (QM) performance task in the online course since QM materials and resources are available to subscribers only and cannot be distributed in any way or form without written permission from QM. It served a great purpose in a face-to-face workshop where all the members present were part of the subscribing institution. However, I did include just a brief description of QM as one of the course lessons and created a hyperlink to the QM website for more detailed description of the program and rubric. I spent the most time trying to create the perfect learning objectives. Since this was an online version of my face-to-face, the objectives would need to be tweaked some to accommodate an asynchronous course design. It was a challenge trying to write high-level cognitive objectives that met each element of a "good" objective as defined by the ABCD's of writing objectives including the Gagne and Briggs version and the Mager format. After much deliberation, I spit a few out that was generally pleased with. However, I will definitely be engaging in a more technical analysis of learning objectives in the future. It was long process but it was good for me. It pushed me to think about what I was really trying to convey to my learners and the outcomes expected from them. I truly enjoyed creating this assignment. View the Online Staff Development Workshop |