New Working Paper Explores Policy Experimentation in China Since 1980

Determining which policies to implement and how to implement them is an essential government task. However, policy learning is complicated by a host of factors, encouraging countries to engage in various policy experiments to help resolve policy uncertainty and to facilitate policy learning. A new paper co-written by Assistant Professor Shaoda Wang analyzes systematic policy experimentation in China since the 1980s, where the government has systematically tried out different policies across regions and often over multiple waves before deciding whether to roll out the policies to the entire nation.

China is an important case study for two reasons. First, the systematic policy experimentation in China is unparalleled in terms of its depth, breadth, and duration. Second, scholars have argued that policy experimentation was a critical mechanism leading to China’s economic rise over the past four decades. Even so, surprisingly little is understood about the characteristics of such policy experimentation, or how the structure of experimentation may affect policy learning and policy outcomes.

The authors focus on two characteristics of policy experimentation to assess whether it provides informative and accurate signals on general policy effectiveness. First, to the extent that policy effects are often heterogeneous across localities, representative selection of experimentation sites is critical to ensure unbiased learning of the policy’s average effects. Second, to the extent that the efforts of the key actors (such as local politicians) can play important roles in shaping policy outcomes, experiments that induce excessive efforts through local political incentives can result in exaggerated signals of policy effectiveness.

Motivated by questions that address these concerns, the authors collect 19,812 government documents on policy experimentation in China between 1980 and 2020 and construct a database of 633 policy experiments initiated by 98 central ministries and commissions. The authors describe their methodology in detail within the paper, but broadly speaking they link the central government document that outlines the overall experimentation guidelines with all corresponding local government documents to record its implementation throughout the country. They measure numerous characteristics of policy experiments, including ex-ante uncertainty about policy effectiveness, career trajectories of central and local politicians involved in the experiment, the bureaucratic structure of the policy-initiating ministries, the degree of differentiation in policy implementation across local governments, and local socioeconomic conditions. 

The authors find the following:

• Policy experimentation sites are substantially positively selected in terms of a locality’s level of economic development, and misaligned incentives across political hierarchies account for much of the observed positive selection. 
• Experimental situation during policy experimentation is unrepresentative: local politicians exert strategic efforts and allocate more resources during experimentation that may exaggerate policy effectiveness, and such strategic efforts are not replicable when the policy eventually rolls out to the rest of the country.
• The positive sample selection and unrepresentative experimental situation are not fully accounted for when the central government evaluates experimentation outcomes, which would bias policy learning and national policies originated from the experiments.

Among its important implications, this research offers insights into the fundamental trade-off facing a central government: structuring political incentives to stimulate politicians’ effort to improve policy outcomes, while making sure that such incentives are not exaggerated during the experimentation phase, so that policy learning remains unbiased. Solutions that improve mechanism design could improve the efficiency of policy learning and, likewise, could be of valuable policy relevance and importance.

This article originally appeared at the Becker Friedman Institute for Economics at the University of Chicago.