APTA CLASSROOM ORCHESTRATION TOOL
WHAT
Educational Technology
WHO
Dr. Vincent Aleven
Dr. LuEttaMae Lawrence
Dr. Octav Popescu
Dr. Jonathan Sewall
Vikrant Bathala
Zimmy Kang
William Huang
Christina Li
WHERE
Human-Computer Interaction Institute - Carnegie Mellon University, Pittsburgh, PA
This dashboard supports teachers in pairing students in a tutee/tutor relationship. By displaying student analytics and system suggestions, this tool gives teachers the power to make informed pairings that benefit student learning and development in a classroom setting.
The motivation for this project was based on observations of teachers creating student tutor/tutee pairs on the fly in classrooms to get struggling students the help they need immediately. This orchestration tool supports teachers in the dynamic pairing practice by suggesting student pairings based on struggle. This is challenging without technology because it requires the teacher to simultaneously manage multiple tasks such as monitoring individual and collaborative progress, finding appropriate pairings, etc.. An overarching goal for this is to help create the classroom of the future that benefits students and teachers by combining teacher and AI strengths. Additionally, the orchestration tool extends the literature on analytics-based teacher tools, in that few such tools have been developed for K-12 and none for dynamic switching between individual and collaborative learning.
The research questions we wanted to explore were:
-
How should the pairing process be shared between the teacher and AI?
-
How can an orchestration tool support teachers while pairing students in real time and monitoring individual and collaborative learning?
WHEN
Summer 2021
_edited.jpg)
METHODS
We used an iterative design and development process sectioned into five “sprints,” or 2-week time spans with specific goals associated with each deadline. We applied a user centered design process through teacher feedback sessions and user testing, which were a crucial part of each sprint's progression. These sessions involved eight different teachers and directly guided the design decisions which were handed off to the developers for implementation using HTML, CSS, JavaScript, and Vue.js.
Design Feedback Sessions
-
9 feedback sessions with 8 different teachers took place for 1 hour each throughout the course of the project
-
The designers screen shared interactive Figma prototypes and asked teachers for their thoughts and suggestions regarding major features and interactions in the dashboard
-
Design decisions and iterations reflected the feedback from the teachers
Design Feedback Sessions
-
3 user testing sessions with 3 different teachers took place for 1 hour each during the 4th and 5th sprint
-
The developers gave each teacher the ability to interact with a working prototype through Zoom’s remote control feature and asked them to think aloud as they made pairs and interacted with the tool
-
The goal was to observe how teachers interact with the dashboard and make and monitor pairs to receive detailed feedback that informed the iterations.
_edited.jpg)
RESULTS
A few key points and ideas arose from the design feedback and user testing sessions:
-
Overall, teachers preferred information to be communicated visually rather than textually, as pairing often happens quickly with little opportunity to read longer explanations
-
Teachers unanimously agreed that pairing students with average knowledge levels was more effective than pairing “best with worst”
-
Although the AI is effective at monitoring students’ academic performance, teachers have a better understanding of personal relationships and individual work habits, reiterating the need for shared control during pairing
PROGRESS
The design went through numerous iterations, exploring different configurations and ways to display features such as tutor/tutee suggestions, student skills, and pair monitoring.
Sprint 2
One of the first iterations explored how to assign tutee/tutor roles and generally structure the dashboard, using two side panels for creating and monitoring pairs.
Sprint 3
This version reorganizes the sidepanels to create more space for individual and pair analytics. Consideration was given to a seating chart, but teachers deemed a grid system more versatile.
Sprint 5
The final design focused on displaying the items which teachers deemed most valuable for use in the classroom, prioritizing individual progress, student skills, and clear system suggestions.
_edited.jpg)
_edited.jpg)
_edited.jpg)
_edited.jpg)
CONCLUSIONS
-
Ultimately the orchestration tool supports a collaboration between AI and teachers, as both bring different types of knowledge when creating informed pairings
-
The APTA project is a stepping stone into the future of classrooms where teachers and AI work in harmony to improve student education
FUTURE DIRECTIONS
-
Continue user testing and classroom testing with teachers as the software evolves and improves
-
Implement more pairing policy options. Currently set to random, but AI suggestions based on knowledge distance between students are in progress
-
Further teacher options such as customizations, options, requests, and notifications
_edited.jpg)