Artical
Permanent URI for this collection
Browse
Recent Submissions
Item Judging facts, judging norms: Training machine learning models to judge humans requires a modified approach to labeling data(2023) Aparna Balagopalan Gillian K. Hadfield 1 *, David Madras 2,3,5,6,7 2,3 , David H. Yang , Marzyeh Ghassemi 1,2,3 2,4 , Dylan Hadfield-Menell 1 ,Acknowledgments:Wewouldliketo thank D. Simon(Universityof Southern California School of Law), T. Lyon (University of Southern California School of Law), R. Zemel (University of Toronto), E. Creager (University of Toronto), and five anonymous reviewers for their helpful comments and reviews. We also thank the participating annotators for their responses. Funding: We acknowledge the Schwartz Reisman Institute for Technology and Society for funding this research. A.B. was funded in part byan Amazon Science PhD Fellowship at the MIT Science Hub. D.M. was supported by an NSERC Alexander Graham Bell Canada Graduate Scholarship-Doctoral (CGS-D) during a portion of this research. D.H.-M. was funded in part by a gift from the Hirji Wigglesworth Family Foundation and in part by the Bonnie and Marty (1864) Tenenbaum Career Development Chair. G.K.H was funded in part by the Schwartz Reisman Chair in Technology and Society and a CIFAR AI Chair at the Vector Institute. M.G. was funded in part by the Hermann L. F. von Helmholtz Career Development Professorship at MIT, Microsoft Research, a CIFAR AI Chair at the Vector Institute, a CIFAR Azrieli Global Scholar award, and a Canada Research Council Chair. Resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute. This project was approved by the University of Toronto’s Institutional Research Ethics Board (protocol no. 00037283). Author contributions: The research was conceived by D.H.-M. and G.K.H.; the study was designed by D.H.-M., G.K.H., D.M., andM.G.;datacollection wasdonebyA.B.andD.H.Y.;datainterpretationandanalysis wasdone by A.B., D.M., D.H.Y., D.H.-M., G.K.H., and M.G.; and the manuscript was prepared by A.B., D.M., D.H.-M., G.K.H., and M.G. Competing interests: The authors declare that they have no competing interests. Data and materials availability: All data needed to evaluate the conclusions in the paper are present in the paper and/or the Supplementary Materials. Code to reproduce the paper’s main findings can be found at https://doi.org/10.5281/zenodo.7782689.Item Judging facts, judging norms: Training machine learning models to judge humans requires a modified approach to labeling data(2024) Aparna Balagopalan Gillian K. Hadfield 1 *, David Madras 2,3,5,6,7 2,3 , David H. Yang , Marzyeh Ghassemi 1,2As governments and industry turn to increased use of automated decision systems, it becomes essential to con sider how closelysuch systems can reproduce humanjudgment.Weidentifyacorepotentialfailure,findingthat annotators label objects differently depending on whether they are being asked a factual question or a norma tive question. This challenges a natural assumption maintained in many standard machine-learning (ML) data acquisition procedures: that there is no difference between predicting the factual classification of an object and an exercise of judgment about whether an object violates a rule premised on those facts. We find that using factual labels to train models intended for normative judgments introduces a notable measurement error. We show that models trained using factual labels yield significantly different judgments than those trained using normative labels and that the impact of this effect on model performance can exceed that of other factors (e.g., dataset size) that routinely attract attention from ML researchers and practitioners.Item Exploring Pre-Service Teachers' Intention to Use Artificial Intelligence in Education: A Structural Equation Modeling Approach Based on UTAUT2(2024) Juan José Victoria Maldonado, Santiago Alonso García, Alejandro Martínez Menendez and Manuel Enrique Lorenzo MartínAbstract The growing relevance of Artificial Intelligence (AI) in education necessitates a better understanding of its acceptance among future educators. This study investigates the factors influencing pre-service teachers' intention to use AI, employing the UTAUT2 model extended with sociodemographic moderators. A cross-sectional quantitative design was applied to a sample of 908 undergraduate students from Early Childhood and Primary Education programs in Andalusia, Spain. Structural equation modeling results reveal that performance expectancy, effort expectancy, and habitual use are significant predictors of behavioral intention toward AI use. In contrast, the proposed moderating effects of gender and academic year were found to be non-significant. Findings highlight the pivotal role of habitual engagement with AI while questioning the effectiveness of current curricular approaches in promoting its pedagogical use, as academic progression showed no moderating influence. The study emphasizes the need for targeted training and curricular updates to foster meaningful AI integration in teacher education. Index Terms Artificial Intelligence, teacher training, technology integration, UTAUT2, pre-service teachers, moderating variables, structural equation modeling.Item Autonomous Artificial Intelligence vs Artificial Intelligence–Assisted Human Optical Diagnosis of Colorectal Polyps: A Randomized Controlled Trial(2024) Roupen Djinbachian,1,2,* Claire Haumesser,1,* Mahsa Taghiakbari,1,2 Heiko Pohl,3,4 Alan Barkun,5 Sacha Sidani,2 Jeremy Liu Chen Kiow,2 Benoit Panzini,2 Simon Bouchard,2 Erik Deslandres,2 Abla Alj,6 and Daniel von Renteln1,2BACKGROUND & AIMS: Artificial intelligence (AI)–based optical diagnosis systems (CADx) have been developed to allow pathology prediction of colorectal polyps during colo noscopies. However, CADx systems have not yet been vali dated for autonomous performance. Therefore, we conducted a trial comparing autonomous AI to AI-assisted human (AI-H) optical diagnosis. METHODS: We performed a randomized noninferiority trial of patients undergoing elective colonos copies at 1 academic institution. Patients were randomized into (1) autonomous AI-based CADx optical diagnosis of diminutive polyps without human input or (2) diagnosis by endoscopists who performed optical diagnosis of diminutive polyps after seeing the real-time CADx diagnosis. The primary outcome was accuracy in optical diagnosis in both arms using pathology as the gold standard. Secondary outcomes included