New TDAI Publications Advance Evaluation of Joint Human-AI Systems
Two new publications from The Ohio State University’s Translational Data Analytics Institute (TDAI) examine how organizations can evaluate and improve systems in which humans and artificial intelligence work together. Available in TDAI's Knowledge Bank collection, the publications offer practical guidance and research-based recommendations for assessing human-machine teaming.
Short summaries of each publication below:
Joint Activity Testing Implementation Guide: Human-Machine Teaming Handbook
Developed by researchers at OSU, the Joint Activity Testing Implementation Guide: Human-Machine Teaming Handbook introduces Joint Activity Testing (JAT), a methodology for evaluating human and AI performance as an integrated team. The guide outlines a framework for planning, conducting, and analyzing evaluations of human-machine systems in operational environments.
The publication also explores approaches for assessing team performance, resilience, and the impact of increasing levels of operational challenge. In addition, it introduces tools such as the Joint Performance Graph (JPG), which supports the comparison of human-machine performance across different conditions.
Read Full Report: Joint Activity Testing Implementation Guide: Human-Machine Teaming Handbook
Evaluation of Joint Human-Machine Systems: Literature Review & Recommendations
The Evaluation of Joint Human-Machine Systems: Literature Review & Recommendations examines the current landscape of research on human-automation teaming and human-AI collaboration. Drawing on 227 published studies, the report reviews commonly used evaluation methods, measures, and research approaches for assessing joint human-machine systems.
The publication identifies key considerations for researchers and practitioners and discusses emerging frameworks, methodologies, and recommendations for future testing programs and research testbeds focused on human-AI teamwork.
Read Full Report: Evaluation of Joint Human-Machine Systems: Literature Review & Recommendations
Read More Publications on Joint-Human AI Systems: Joint Human-AI Systems Publications | Translational Data Analytics Institute
- Joint Activity Testing Implementation Guide: Human-Machine Teaming Handbook
- Evaluation of Joint Human-Machine Systems: Literature Review & Recommendations
Read the full reports: Joint Human-AI Systems Publications | Translational Data Analytics Institute
- Joint Activity Testing Implementation Guide: Human-Machine Teaming Handbook
- Evaluation of Joint Human-Machine Systems: Literature Review & Recommendations