AI Grading in Peer Reviews: Enhancing Coursera’s Learning Experience with Faster, High-Quality Feedback

In the ever-evolving landscape of online education, Coursera is doubling down on innovation by integrating artificial intelligence (AI) into its peer review experience. AI Grading in Peer Reviews aims to revolutionize grading efficiency and quality by ensuring consistent, timely, and scalable feedback using instructor-created assignment rubrics. This initiative is designed to enhance the learner experience by providing immediate and valuable feedback on text-based submissions while reducing the wait time and short feedback often seen in human-graded peer review assignments.

Coursera is dedicated to incorporating Generative AI (GenAI) into its platform to advance its pedagogical pillars, support learners in building mastery, and provide tailored assistance. AI Grading in Peer Reviews is a key component of this broader initiative. By harnessing the power of GenAI, Coursera aims to create a more personalized and effective educational experience-streamlining the grading process and enabling learners to quickly identify their strengths and areas for improvement with immediate, constructive feedback. This targeted support helps learners achieve a deeper understanding and boosts retention of course material.

Goals of AI Grading in Peer Reviews

The primary objectives of this product feature are multifaceted:

• Improve grading efficiency and quality: By leveraging AI, Coursera simplifies the grading process, making it faster and more accurate.
• Ensure consistent and scalable grading: AI-driven grading ensures that assessment grades are consistent and scalable, adhering to established rubrics while saving humans time.
• Enhance the learning experience: Immediate and valuable feedback from AI elevates the overall course experience for learners and removes blockers for their continued progress.

Monitoring quality and success

To gauge the success and quality of this product feature, several key metrics were monitored during the beta test period:

• Submissions graded: Approximately 300,000 submissions were graded by the AI system.
• First attempt pass rate: The first attempt pass rate stands at 72%, a notable decrease compared to human-graded peer review assignments.
• Feedback thumbs up rate: 90% of learners who responded to the thumbs-up or -down rating expressed ‌satisfaction with the AI feedback from Coach alongside their peer review grade.
• Learners switching back to human grading: Only 7% of learners switched to peer grading. Notably, a majority of these learners (84%) did not receive a passing grade from the AI, prompting manual reviews by our team to validate the AI's accuracy.

Impact on learning metrics during beta test

There are positive signals from critical learning metrics including:

• Greater progress: Course completions within a day of peer review impression increased by 16.7%
• Faster grading: Learners received AI grades within 1 minute of submission on average, compared to 15 hours with human graders (900x faster)
• More feedback: Learners received an average of 45x more feedback in the AI grading group. By developing a prompt adhering to pedagogical best practices developed with the Teaching & Learning team, this product feature delivers an average of 326 characters of feedback.

We also saw signals of increased rigor:

• Across all assignments and learners, AI grades in peer review were 3% lower, on average, than those given by peer graders.
• The first-attempt pass rate for AI-graded assignments was 72% compared to 88% for human-graded peer review assignments.
• The AI grading system is less likely to award perfect scores (100%) and more likely to give 0% when the submission fails to meet any rubric criteria.
• Learners assessed by AI are submitting more attempts, on average, to pass their peer review assignment.

Next Steps

AI Grading in Peer Reviews on Coursera is showing promising results in enhancing the learning experience while improving grading efficiency, quality, and scale. As we continue to monitor and expand the system to support additional submission types, we are optimistic about its potential to remove barriers to progress and provide meaningful feedback on learners' hard work.

This AI Grading initiative is an important part of a broader initiative at Coursera to incorporate GenAI seamlessly and ethically into our product to benefit the learning experience. As Coursera continues to innovate around the best integrations of GenAI, pedagogically-based AI grading and feedback will play a pivotal role in shaping the future of online education. Ultimately, GenAI is another tool Coursera's Product, Engineering, Design, and Teaching & Learning teams are using to make the platform more accessible, efficient, and impactful for learners worldwide.

Note: AI Grading in Peer Reviews is only available in select regions and also excludes any content currently stacking into for-credit degree programs on the Coursera platform.

Public link

Please use the above public link if you want to share this noodl on another website.