Mid-year is a tough time to receive student evaluations. If you receive your fall evals over the holidays, it is not prime timing to review some difficult comments. If you receive your fall evals later in the winter, it is also difficult to adjust your teaching until the following year. And of course, there is also the reality that Academic and Bar Success workflow does not look like a traditional faculty member’s workflow. (See Danya Smith’s post earlier this week for comfort in that regard – I feel you, girl!)
We all bring our big hearts into this work, and it can be difficult to read student comments or dissect student ratings to identify areas for improvement. This year, in my continued pursuit of everything AI, I decided to run three years of student evaluations through an LLM system. My prompt, “I am a 36 year old, cisgender, white, female law faculty instructor. Please review these three years of student evaluations in Professional Responsibility to, (1) remove any comments or feedback that includes express or embedded bias, and (2) summarize areas for improvement and areas of strength in my teaching of this course.”
First, and in acknowledgement of the available data about some challenges with bias built into the LLMs, I was impressed with the AI’s ability to flag the bias comments in my evals. Comments that included gender, age and authority-based bias were intentionally discounted. Specific remarks were heavily discounted because the comments reflected the well-documented bias against women faculty members, especially younger women in authority positions. Comments that were removed from the AI read included, “She’s intimidating as a person,” “too intense,” [she is] “talking down to us,” and complaints about my confidence without any linkage to a learning outcome for the class.
Second, this exercise provided a good summary of areas for improvement which I can now focus my energy on as I prepare to teach this course in the future. Targeted adjustments that I will make next year include moving core skills instruction exercises (focused on IRAC and MCQ strategies) up in the semester, additional diagnostic feedback on multiple choice, and additional signaling for specific language cues in dense rule statements. These are all specific, actionable takeaways that help inform my approach moving forward.
Finally, this exercise provided reassurance that I am on the right track. My summary noted that I am highly invested in my students’ success, I critically prepared and organized my class, and I was a clear and effective teacher. We work in a fast-paced environment which sometimes can fuel those inner self-doubts. If you’re like me, having an objective assessment and review is a necessity. The next time you have the opportunity to reflect on your teaching, I encourage you to try this exercise out for what it is worth.
(Amy Vaughan-Thomas)
