Developing an AI Infrastructure for Analytics and Institutional Effectiveness

Session Abstract: In this session, we will provide case studies showing how Virginia Tech is using large language models to complement our work across analytics and institutional effectiveness. Using foundational models embedded in a cloud-based infrastructure, we can implement use cases while circumventing concerns about data sharing and privacy. First, we explore a project to classify courses and research outputs into categories of strategic interest. The tools allow rapid and accurate labeling of outputs and allow summarization to understand how outputs precisely contribute to and support areas in our efforts toward reputational excellence. Second, we demonstrate how AI infrastructure can improve the efficiency and potentialities of academic program assessment and administrative assessment processes. Beginning with a synopsis of baseline uses such as comparative assessments, we conclude with efforts toward building Knowledge Bases for Retrieval-Augmented Generation and making the tools available to end users.

Keywords: large language models (LLMs), cloud-based infrastructure, generative AI (GenAI), AI/ML analytics

Contributing Authors: Ben Greenawald – Assistant Director for Faculty Data Science, Office of Analytics & Institutional Effectiveness, Virginia Tech