Students and their research project director at the Alabama School of Mathematics and Science sought to incorporate large language models such as ChatGPT into traditional literary research informed by historical and biographical methods. They queried ChatGPT and other models for assistance in locating sources, revising claims, and improving prose – to mixed results. On the one hand, AI assistants hallucinated up to 40% of their cited sources, with entries so convincing that it took an expert in the field to rule them out. On the other hand, these same assistants provided useful templates and stylistic advice, and they excelled in niche uses such as identifying misremembered sources based on limited information. For research applications, the experience suggested that LLMs are not “force multipliers” in the humanities research process but are instead specialized tools that need vetting by content experts.

Presenters

• Mitch Frye mitchfrye@gmail.com, Alabama School for Math and Science
• Jiro Culbertson chiyo.culbertson@asms.net, Alabama School of Mathematics and Science
• Alton Raburn alton.raburn@asms.net, Alabama School of Mathematics and Science