Summative Evaluation

Summative Evaluation Report for Promotional Video made for “Rebuilding Lives: Finding a Place in the Sun”

Instructional Designer: Micki Archuleta
Date: 30 November 2024

1. Purpose of the Evaluation Session

This evaluation session aims to assess the effectiveness of the promotional video “Rebuilding Lives: Finding a Place in the Sun” as a reusable learning object (RLO). The evaluation aims to determine how well the video conveys its intended message, engages the target audience, and aligns with educational objectives.

2. Evaluation Session User Profiles

Participants will include 20 undergraduate students enrolled in a social work program at at State University. The group will comprise an even number of females and males, aged between 19 and 24, with diverse cultural backgrounds. All participants will have prior coursework in social justice and community rehabilitation.

3. Evaluation Session Location

The evaluation session will be conducted in a university multimedia lab equipped with individual workstations, high-definition monitors, and surround sound systems to ensure optimal viewing conditions.

4. Evaluation Methods

Pre-Viewing Survey: Assess participants’ prior knowledge and attitudes toward community rehabilitation.
Observation: Monitor engagement levels during the video presentation.
Post-Viewing Assessment: Evaluate comprehension and retention of key concepts presented in the video.
Focus Group Discussion: Gather qualitative feedback on the video’s impact and effectiveness.

5. Evaluation Results

Pre-Viewing Survey: Show what percentage of participants had limited knowledge of community rehabilitation initiatives.
Observation: Show what percentage of participants displayed high engagement, indicated by attentive viewing and note-taking.
Post-Viewing Assessment: Show average score, reflecting a significant increase in understanding of the video’s content.
Focus Group Discussion: Participants should report that the video was emotionally compelling and enhanced their awareness of community rehabilitation efforts.

6. Reaction to Evaluation Results

The evaluation results are expected to indicate that the promotional video effectively increased the target audience’s awareness and understanding of community rehabilitation. The expected high engagement levels and positive feedback should suggest that the video’s design and content resonate well with viewers.

7. Action Response and Revisions

Based on the feedback, the following actions should be recommended:

Enhance Accessibility: Add subtitles to accommodate hearing-impaired viewers and non-native English speakers to be optionally turned on.
Provide Supplementary Materials: Develop discussion guides and resource lists for deeper topic exploration.
Expand Distribution Channels: Utilize social media platforms to reach a broader audience and increase the video’s impact.

8. Appendices

Appendix A: Pre- and Post-Viewing Survey Instruments
Appendix B: Observation Checklist
Appendix C: Focus Group Discussion Guide
Appendix D: Detailed Data Analysis

This summative evaluation should demonstrate the promotional video’s effectiveness as an educational tool and provide actionable insights for further enhancement

Appendix A: Pre- and Post-Viewing Survey Instruments

Pre-Viewing Survey

Prior Knowledge Assessment:
- On a scale of 1 to 5, how familiar are you with community rehabilitation initiatives?
  - 1: Not familiar
  - 2: Slightly familiar
  - 3: Moderately familiar
  - 4: Very familiar
  - 5: Extremely familiar
Attitudinal Assessment:
- Indicate your level of agreement with the following statement:
  - “Community rehabilitation is essential for societal development.”
    - Strongly Disagree
    - Disagree
    - Neutral
    - Agree
    - Strongly Agree

Post-Viewing Survey

Comprehension Assessment:
- List three key points you learned from the video about community rehabilitation.
Attitudinal Shift Assessment:
- After watching the video, indicate your level of agreement with the following statement:
  - “I am motivated to support community rehabilitation efforts.”
    - Strongly Disagree
    - Disagree
    - Neutral
    - Agree
    - Strongly Agree
Engagement Assessment:
- How engaging did you find the video content?
  - Not engaging
  - Slightly engaging
  - Moderately engaging
  - Very engaging
  - Extremely engaging
Open-Ended Feedback:
- What aspects of the video did you find most impactful?
- Do you have any suggestions for improvement?

Appendix B: Observation Checklist

Engagement Indicators:
- Attentiveness during the video
- Note-taking behavior
- Non-verbal cues (e.g., nodding, facial expressions)
Disengagement Indicators:
- Signs of distraction (e.g., looking away, using mobile devices)
- Lack of response to key video segments
Technical Observations:
- Any technical issues encountered (e.g., audio/video glitches)
- Clarity and quality of the video presentation

Appendix C: Focus Group Discussion Guide

Opening Question:
- What are your initial thoughts after viewing the “Rebuilding Lives” video?
Content-Specific Questions:
- Which segment of the video resonated most with you, and why?
- How did the video influence your understanding of community rehabilitation?
Design and Delivery Questions:
- How effective were the visuals and narration in conveying the message?
- Did you find the length of the video appropriate?
Impact and Application Questions:
- Has the video inspired you to take any specific actions or seek more information?
- How do you think this video could be used in educational settings?
Closing Question:
- Do you have any additional comments or suggestions regarding the video?

Appendix D: Detailed Data Analysis

Pre- and Post-Viewing Survey Results:
- Prior Knowledge Assessment:
  - Average pre-viewing familiarity score: x.x
  - Average post-viewing familiarity score: x.x
- Attitudinal Assessment:
  - Pre-viewing agreement on the importance of community rehabilitation:
    - % Neutral
    - % Agree
    - % Strongly Agree
  - Post-viewing agreement:
    - % Neutral
    - % Agree
    - % Strongly Agree
- Engagement Assessment:
  - % of participants who rate the video as Very Engaging or Extremely Engaging
Focus Group Themes:
- Emotional Impact:
  - Participants should express that personal stories featured in the video made the content relatable and impactful.
- Educational Value:
  - The video should effectively increase awareness and understanding of community rehabilitation processes.
- Suggestions for Improvement:
  - Participants might suggest including more diverse case studies to represent a broader spectrum of rehabilitation scenarios.
  - Participants might suggest Providing actionable steps for viewers interested in contributing to community rehabilitation efforts.