Assessment and evidence strategy for AI
Building a credible evidence model when a program or team is using or planning to use AI.
Higher Education Assessment, Evaluation, and AI
Learning, evidence, and assessment for the AI era.
Roger S. Taylor, Ph.D., is an educational research scientist and higher education assessment professional. His work focuses on learning outcomes assessment, program evaluation, learning analytics, and how generative AI changes evidence of learning.
About
Roger S. Taylor, Ph.D.
Roger S. Taylor, Ph.D., is an educational research scientist specializing in assessment, program evaluation, learning analytics, and artificial intelligence in education.
Across more than two decades in higher education and applied research, Dr. Taylor has worked with faculty, academic leaders, grant teams, and educational organizations on questions involving learning, assessment, evaluation, and institutional improvement. His background includes academic assessment and analytics in the Office of the Provost at Boston University, accreditation and institutional-effectiveness work supporting NECHE review at Quincy College, doctoral and undergraduate teaching in research methods, and federally funded research supported by the National Science Foundation, the Institute of Education Sciences, and the National Institutes of Health, including senior roles on NSF- and NIH-funded projects.
His current research addresses assessment validity, AI literacy, and comparisons of human and AI judgment and performance. He serves as editor for Intersection, the journal of the Association for the Assessment of Learning in Higher Education, and served as a faculty mentor for the AAC&U Institute on AI, Pedagogy, and the Curriculum in 2025–2026.
Ph.D., Cognitive PsychologyUniversity of Pittsburgh, Learning Research and Development Center
Experience
Higher education assessment and professional leadership
Selected experience
2023–2025
Associate Director of Academic Assessment
Boston University, Office of the Provost
Assessment and analytics initiatives across academic units; a university-wide undergraduate survey on student AI use; senior personnel for assessment on an NIH/NHLBI research-training grant.
2021–2022
Senior Academic and Assessment Analyst
Quincy College
Coordinated a college-wide learning outcomes assessment system; prepared accreditation evidence and self-study materials supporting NECHE review.
2017–2019
Director of Research
Emotuit, educational technology startup
2010–2017
Assistant Professor of Psychology and Human-Computer Interaction
State University of New York at Oswego
2008–2010
IES Advanced Learning Sciences Postdoctoral Research Fellow
Vanderbilt University, Learning Sciences Institute
2005–2007
Postdoctoral Research Fellow
University of Memphis, Institute for Intelligent Systems
Professional leadership
2025–Present
Editor, Intersection: A Journal at the Intersection of Assessment and Learning
Association for the Assessment of Learning in Higher Education (AALHE)
2026–2027
Vice Chair, Intersection Committee
Association for the Assessment of Learning in Higher Education (AALHE)
2025–2026
Faculty Mentor (competitive national selection)
AAC&U Institute on AI, Pedagogy, and the Curriculum
2026–Present
Coordinator and Facilitator, Assessment Design and Validity Working Group
Assessment Institute in Indianapolis GenAI Community of Practice (Affiliate Member, 2025–Present)
Review service. NSF review panel, Research and Evaluation on Education in Science and Engineering program. Ad hoc reviewer for journals including the Journal of Educational Psychology and the Journal of the Learning Sciences.
Selected Work
Current research and recent presentations
Ongoing work, much of it with collaborators across higher education.
- Generative AI as a raterReliability of generative AI in rubric-based assessment, and human-AI comparisons of rating consistency.
- AI-assisted rubric developmentThe Rapid Rubric Iteration Process (RRIP), a generative AI-assisted method for developing, testing, and refining rubrics.
- AI literacy measurementPsychometric development of short forms of the Generative AI Literacy Test, including item response theory and validity analyses.
- Student AI decision-makingAI and Learning Under Constraint (AILUC), an IRB-approved study of how time pressure and assignment value shape student decisions about AI use.
- Faculty emotion and professional identityThe AIEPI-F framework and survey on how faculty respond to generative AI.
- Institutional AI evidenceA university-wide survey of undergraduate AI use and attitudes at Boston University; text analysis of AI policy language across university syllabi; an early empirical study of whether faculty can identify AI-generated student work.
Selected writing and presentations, 2023–2026
2026
- UpcomingTaylor, R. S., & Roberts, C. J. (2026, October). AI and Learning Under Constraint (AILUC): How time pressure and assignment value shape student AI decision-making. AAC&U Conference on AI and Higher Education, Atlanta, GA.
- UpcomingTaylor, R. S., & DiSabito, D. M., Jr. (2026, October). When artificial intelligence (AI) becomes a rater: Evaluating the reliability of generative AI in rubric-based assessment. Assessment Institute in Indianapolis, Indianapolis, IN.
- UpcomingDiSabito, D. M., Rodriguez, J. S., Hansen, L., Taylor, R. S., & Mennella, T. (2026, October). Rapid Rubric Iteration Process (RRIP): Compressing years of rubric refinement into minutes with generative artificial intelligence (GenAI). Assessment Institute in Indianapolis, Indianapolis, IN.
- Taylor, R. S. (2026, August). An overlooked dimension of generative AI in assessment: Faculty emotion and professional identity. Emerging Dialogues, Association for the Assessment of Learning in Higher Education.
2025
- Taylor, R. S., & Eaton, L. (2025, November). Teaching vs. policing: Analyzing generative AI policies in higher ed. NEean Fall Forum, Virtual.
- Taylor, R. S. (2025, September). Evidence-based AI literacy: Moving from assumptions to actionable data. AAC&U Institute on AI, Pedagogy, and the Curriculum, Online.
2024
- Taylor, R. S., & Regan, J. (2024, June). Examination of how ChatGPT changes the process and product of undergraduate student writing. AALHE Conference, Portland, OR.
2023
- Taylor, R. S., & Marzluft, J. E. (2023, June). Assessment in the time of ChatGPT [Poster presentation]. International Assessment in Higher Education Conference, Manchester, England.
Full record: ORCID | Google Scholar
Independent Practice
AI Education Consulting
AI Education Consulting is Dr. Taylor's independent practice. Through it, he contributes assessment, evaluation, and learning analytics expertise to funded projects, grants, pilots, evaluations, and advisory work with colleges and universities, grant teams, and education organizations. The emphasis is on validity, evidence quality, and what available data can and cannot support.
Grant evaluation design
Coherent outcomes, measures, logic models, evaluation plans, and data-collection strategies.
Learning analytics evidence review
Determining what existing data can validly show, where its limits are, and how it can inform improvement.
AI literacy measurement
Instruments, rubrics, performance tasks, and pre/post designs for AI literacy, judgment, and applied capability.
Oral and conversational assessment
Protocols, rubrics, and validity arguments for settings where AI-generated work makes learner capability harder to infer.
Expert review of AI learning tools
Educational review of a tool, pilot, course, or evidence claim, with recommendations for improvement.
Get in touch
Roger welcomes conversations with colleagues in higher education assessment, institutional effectiveness, and research on AI in learning.
Acton, MA 857-241-7434 roger.taylor@aieducation.consulting linkedin.com/in/roger-taylor