Journal articles
Year
2024
Authors
RENO Roberto, MENKVELD ALBERT J., DREBER ANNA, HOLZMEISTER FELIX, HUBER JUERGEN, JOHANNESSON MAGNUS, KIRCHLER MICHAEL, NEUSÜß SEBASTIAN, RAZEN MICHAEL, WEITZEL UTZ, ABAD‐DÍAZ DAVID, ABUDY MENACHEM, ADRIAN TOBIAS, AIT‐SAHALIA YACINE, AKMANSOY OLIVIER, ALCOCK JAMIE T., ALEXEEV VITALI, ALOOSH ARASH, AMATO LIVIA, AMAYA DIEGO, ANGEL JAMES J., AVETIKIAN ALEJANDRO T., BACH AMADEUS, BAIDOO EDWIN, BAKALLI GAETAN, BAO LI, BARBON ANDREA, BASHCHENKO OKSANA, BINDRA PARAMPREET C., BJØNNES GEIR H., BLACK JEFFREY R., BLACK BERNARD S., BOGOEV DIMITAR, CORREA SANTIAGO BOHORQUEZ, BONDARENKO OLEG, BOS CHARLES S., BOSCH‐ROSA CIRIL, BOURI ELIE, BROWNLEES CHRISTIAN
Abstract
In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.
MENKVELD, A.J., DREBER, A., HOLZMEISTER, F., HUBER, J., JOHANNESSON, M., KIRCHLER, M. … RENO, R. (2024). Nonstandard Errors. Journal of Finance, 79(3), pp. 2339-2390.