Abstract

Physical lab experiments have played an instrumental role in sculpting the history of experimental economics, facilitating controlled information conditions, efficient monetary inducements, and exclusive advantages via immediate human interaction and engaging experiences. These unique benefits render in-person lab experiments essential for the future of experimental economics, complementing the growth of online experiments and the emerging AI revolution. We characterize the environments in which it seems particularly important to conduct lab-in-the-lab experiments. Overall, the lab benefits culminate in a comprehensive research procedure that produces precise and enlightening outcomes, ultimately enriching the domain of experimental economics, and potentially extending benefits to the broader realm of social science.

1. Introduction

In experimental economics, the research question ideally drives the choice of the setting — lab, field, online, or a mixture. Recent years, however, have seen a marked shift toward online experiments at the expense of traditional lab settings, as shown by Fréchette et al. 2022. We suspect that many of these research questions might have been more effectively addressed in a lab environment. These might involve complex settings and strategic and social interactions. Nevertheless, this growing preference for online settings has several valid underpinnings. At least three main factors contribute. First, research funding constraints and power calculation requirements lead to a compelling draw toward collecting data at a reduced cost. Second, online experiments can reduce logistical burdens. They minimize the time, material, human resources, and financial commitments associated with creating a sample pool, managing a lab, and conducting the experiment. Third, online platforms provide researchers access to more heterogeneous and representative data, which can partly mitigate concerns about external validity (Snowberg and Yariv, 2021) .

In light of the accelerating digital economy and the generative AI revolution (Bommasani et al., 2022; Acemoglu and Johnson, 2023) , compounded by the shift to remote work in the wake of Covid-19 (Barrero et al., 2021; Hansen et al., 2023), this trend toward online experimentation is expected to intensify. Although the potential of AI in all stages of experimentation is explored in the community (Charness et al., 2023b), we argue that lab experiments retain unique, irreplaceable benefits. Rather than being rendered obsolete by generative AI and other experimental settings, these advantages complement them. One obvious advantage of lab experiments is that the people subjects see are obviously humans, and invisible manipulation would violate no-deception traditions. Calling on the rich history of lab experiments, we champion the continued use of physical lab experiments when they are best suited to the research question. This article advocates for recognizing and preserving the uniqueness of physical lab experiments as a valuable research tool for current and future generations of economists. Calling on the rich history of lab experiments, we champion the continued use of physical lab experiments when they are best suited to the research question and as a valuable research tool for current and future generations of economists.

The rest of the paper is organized as follows. Section 2 addresses the unique advantages of participant engagement and comprehension facilitated by laboratory settings. Section 3 considers the crucial role of maintaining strict information conditions in laboratory experiments. Section 4 discusses the pivotal role of physical procedures and credibility in the context of laboratory experiments. Aside from discussing these benefits, Section 5 presents several specific case studies that illustrate the advantages of in-person lab experiments. Section 6 discusses and attempts to identify the environments and circumstances where lab experiments seem to be the best tool available and concludes.

Read the rest of the paper