Implementation, Uncertainty, and the Policy Sciences

Matthew Potoski, Iowa State University

 

 

Over the last few years, Policy Currents has been reporting discussions around the amount of “progress” realized from implementation studies for understanding public policy.  The exchanges among scholars have been lively, with debates surrounding how far we are from a “Grand Unified Theory” (GUT) of the policy sciences.  The central issues in such debates boil down to how best to describe our own uncertainty about how well we are able to explain both the processes and outcomes of policy implementation. 

 

The desideratum of the policy sciences, it seems to me, is to find ways to map how preferences, as expressed individually and collectively in diverse institutional contexts, produce public policy and policy outcomes.  The more we know about the interaction of rules, preferences, and contexts, the more accurately we can explain and perhaps even predict policy outcomes.

 

We use the term “uncertainty” in this sense to describe our own (the scholars’) perspective about how well we understand the phenomena we study.  For example, our theories look to explain why one government chose more stringent environmental policies and another more lax ones, and we explain our uncertainty about these theories in reference to the degree to which they are “accurate” in some sense.  Thus, we can define “discipline uncertainty” as the overall state of our ignorance/understanding about the phenomena we study.[1] 

 

            Of course, we also use “uncertainty” to describe actors and their decisions in the policy process.  Uncertainty in its various guises undermines the ability of governments to solve complex problems and produce coherent public policy.  Actors may know the outcomes they want to achieve, but may be unsure about what decisions and actions will best achieve those outcomes.  Politicians may pass a law, perhaps even with clearly stated goals, but because they are unable to monitor its implementation, street-level bureaucrats use the law for purposes that are at variance with the goals of the politicians.  Uncertainty, in this sense, is akin to unpredictability, at least from the perspective of the politicians enacting the law.  Uncertainty problems span levels of government, from local governments, to states, to policymaking at the national level.  Uncertainty also spans the policy process, from politicians passing laws and managing relations with agencies, to street-level bureaucrats looking to apply abstract guidelines to complex problems. 

 

            If we are to proceed towards an optimal desideratum for the policy sciences, our theories, models, frameworks, and methods need to better account for the uncertainty inherent across the policy process.  The study of uncertainty I have in mind includes how individuals can be uncertain about their own choices (e.g., Jones 2001).  In this sense, an individual choosing what car to buy must decide the relative importance of the various car attributes (price, gas mileage, safety, color, etc), as well as which alternatives (makes and models) provide the best mix of those attributes. 

 

But the role of uncertainty in the policy process is even more complex than choices in buying a car, once we take into account the context in which decisions take place.  For example, predicting the outcome of collective decision making is more uncertain, in part because we would need to know the individual preferences underlying the collective choice, but also because collective outcomes are prone to strategic agenda manipulation, majority cycling, and so on.

 

In short, there are three sources of uncertainty, although these sources are not mutually exclusive and are all symptoms of transactions costs of some sort inherent in the decision context.  First, individual actors may lack information about the world, thus leaving them uncertain about how they can achieve their desired outcomes.  Second, the institutional context structures the degree of uncertainty.  Collective decisions are more unpredictable (from both scholars’ and practitioners’ perspectives) than are individual decisions (it’s easier to predict an individual’s edict than the outcome of a committee vote).  Different institutional arrangements can change the nature of such uncertainty (consider, for example, how principal agent relations change under divided and unified government).   Third, some policy problems are more technically vexing than others, leaving actors more uncertain about what policy options will achieve desired outcomes.  Considerable expertise is necessary for solving such policy problems.  

 

We are only beginning to understand the role of uncertainty in the policy process from the perspective of practitioners.  If actors in the policy process had perfect information and there were no transaction costs in making and implementing decisions about allocating resources, they could write very detailed instructions describing exactly what actions should be taken and thus ensure the policy achieved the desired outcomes.  However, such conditions are never met in practice.  In real-world settings, actors may sometimes know the outcomes they want to achieve – such as a set of environmental policies that properly balance cleaning the environment with economic growth – but may not know which mix of policies, programs, and enforcement regimes will best achieve them.  Inherent uncertainty and complexities in social interaction exceed the ability of practitioners to predict future events, specify policies for all circumstances, and ensure that actual policy outcomes match specified policy objectives.  Consequently, effective policymaking requires more than just deciding who should get what resources.  It also requires developing strategies for gathering information and for implementing policies to achieve desired outcomes. 

 

            Below I propose a taxonomy of the different types of uncertainty and suggest some of their implications for theories of public policy.  

 

Technical uncertainty

Policymakers do not have the information, expertise, or time to develop and implement even simple policies, let alone complex ones such as air pollution regulation.  Technical uncertainty is higher to the extent actors are unsure about the consequences of their decisions (e.g., how much a policy will reduce pollution and hinder economic growth) and whether the decision is properly defined (e.g., whether the problem is properly framed as an environment vs. economics tradeoff.  See Jones, Talbert, and Potoski 2001).

 

An important implication of this is that rational actors may not arrive at decisions that make themselves better off in an immediate sense, but that help reduce uncertainty over the long run.  This is one reason why people buy “Consumer Reports,” a magazine that provides evaluative and technical information about some of the products people may want to buy.  Doubtless readers get little satisfaction from knowing which refrigerator has the lowest energy consumption; but knowing more about refrigerators will reduce their uncertainty – and help them make better decisions when making their next major appliance purchase.  A “better decision” when purchasing an appliance is one that better matches the preferences of the decision making.  A decision maker who is highly uncertain in this sense will be less likely to select the appliance that best matches his or her preferences. 

 

Some policy programs work like “Consumer Reports.”  Their purpose is not so much to allocate resources among competing claimants – the who gets what and why question – but rather to help policymakers better understand the technical features of the policy problem to enable them to develop programs that are more likely to accomplish their objectives.  In air pollution regulation, for example, ambient monitoring programs provide information to policymakers by sampling outdoor air, assessing the levels of various pollutants, and estimating its sources, causes, and consequences.  Such information can help policymakers set standards for pollution sources for different sources in different regions. 

 

Political Uncertainty

We can think of two types of political uncertainty. Actors can be uncertain about the future political environment, that is, when the future threatens to bring political coalitions with policy preferences hostile to the status quo.  When the political environment is uncertain, politicians look for ways to protect the agreement (Moe 1989).  One way to do this is to ossify political bargains through institutional mechanisms that are difficult to overturn down the road.  Insulating bureaucracies from legislative control, for example, can prevent future coalitions that may have hostile preferences from changing the direction of policy (McCubbins, Noll, and Weingast 1989). 

 

Collective Decision Uncertainty

This type of uncertainty occurs when groups make majority decisions across multiple evaluative dimensions.  In contexts where majority cycles and strategic behavior are possible, individual decision makers may not know how their choices will affect collective outcomes.  Majority decision cycling, log-rolling, and other strategic behaviors make it difficult to predict the outcome of collective decision from members’ preferences alone, particularly when the group members face both attribute and alternative uncertainty about their own choices and those of other members of the group.  Such uncertainty can exist even under perfect information, although imperfect information can certainly exacerbate it.  Thus, collective decision uncertainty is heightened even further as options are assessed along multiple evaluative dimensions, even though for individuals the final decision is reduced through computation or heuristic, to a single dimensional choice. Divided government exacerbates collective decision uncertainty by complicating  the development, implementation, and enforcement of policies (Epstein and O’Halloran 1999).

 

We’ve all had the experience of sitting through a committee meeting, listening to long-winded debates about the merits of various alternatives, only to end up with a committee decision that no one seems to like.  Even if we knew everyone’s preferences, there is no simple way to predict the outcome of committee decisions when intricate, multi-dimensional issues are at stake and the rules governing agenda proposals, amendments, and so on are complex.

 

Principal Agent Uncertainty

“If you want something done right, you have to do it yourself” goes the common saying.  And most of the time this rings true.   But there are times we know what we want to accomplish, but not how to accomplish it.  In such cases, solving problems requires not just deciding what needs to be done and implementing the solution, but also learning what needs to be done.  Sometimes, we find ways to learn about the problem ourselves, although this requires investing time, effort, and resources.  Other times, we rely on others’ expertise to solve hard problems – for example, when hiring a mechanic to fix the menacing “pings” in our car – even though delegating to experts means we risk losing some control over the final outcome.  Will the mechanic fix what we want repaired, and not any more or less? 

 

This is the dilemma confronting elected politicians when developing technically complex policies.  Politicians may know what policy outcomes they want to achieve (e.g., a cleaner environment at minimal economic cost) but may not know how to achieve them (e.g., what environmental regulations will provide the cleanest environment for the least economic cost).  There are no easy solutions to such problems.  On the one hand, the politicians can develop the policy themselves, although without the help of experts, politicians risk developing technically inferior policies.  On the other hand, politicians can delegate policy responsibility to third parties such as air pollution control agencies.  Such delegation capitalizes on the bureaucrats’ expertise, although it means forfeiting some control over what the agencies do.  In such contexts, delegation reflects tradeoff between the technical competence of the bureaucracy's decisions and political control over the bureaucracy's decisions (Bawn 1995).  In other words, politicians can delegate more policy authority to the agency, capitalizing on bureaucrats’ information and expertise, but must concurrently forfeit some political control of the agency's policy choices.  Conversely, politicians can constrict agency autonomy and increase their political control, but must also forgo some of the advantages of the bureaucrats’ time, resources, and expertise.

 

Conclusion

            Clearly, uncertainty has important implications for policy science theories.  And, further progress in the policy sciences, I believe, will require further integrating these and other uncertainty ideas into policy theories, models, and approaches.  This integration will require some adjustments to policy sciences research.

 

            First, understanding the role of uncertainty in public policy leads us to adjust our normative yardsticks for evaluating government performance.  In many cases, the assumed normative standard, not always explicitly stated, is that policy outcomes are “good” to the extent they closely match public preferences.  More uncertainty means that we are less likely to see such strict “democratic accountability.” .  The bureaucratic expertise-responsiveness tradeoff, for example, means that the loss of democratic accountability may be offset by gains in the technical sophistication of policy solutions.  What is needed, then, is more subtle and nuanced standards for evaluating the policy process, ones that offer multiple evaluative dimensions for gauging how well government performance improves public well being. 

 

Second, studying uncertainty in public policy requires analytic tools that better capture uncertainty from the perspective of actors in the policy process.  Consider the following approach for gauging uncertainty about decision outcomes from the perspective of policymakers.  If certainty means that decision makers have more confidence in their estimates of policy outcomes, then uncertainty can mean that their predictions are less accurate.  In other words, the variance of the policy outcome is higher when decision makers are uncertain about the outcomes of their choices.  One empirical approach receiving some early success along these lines is conditional heteroskedastic (CH) modeling (Alvarez and Brehm 1995).  CH techniques model both the conditional mean and the conditional variance of a dependent variable as functions of separate sets of independent variables.  Properly applied, the independent variables affecting the variance can reflect the degree of the policymakers’ uncertainty while the variance in the dependent variable can reflect the consequences of uncertainty for public policy (e.g., O’Toole and Meier 1999).  Such an approach can accurately capture the decision maker’s point of view, where higher uncertainty clouds her judgement about how her decision will affect the policy outcome. I am not claiming that policy sciences have entirely ignored uncertainty, but it seems clear the subject has not received theoretical and empirical scrutiny befitting its importance.  Readjusting normative standards and empirical approaches to better reflect the uncertainty inherent in the policy processes will certainly complicate the study of public policy.  In short, making further progress towards the desideratum of policy sciences requires diverse methods, standards, and approaches for better understanding uncertainty.

 

References

 

Alvarez, R. Michael and John Brehm. 1995. “American Ambivalence Towards Abortion Policy: A Heteroskedastic Probit Method for Assessing Conflicting Values.” American Journal of Political Science 39 (November): 1055-1082.

 

Bawn, Kathleen. 1995. “Political Control versus Expertise: Congressional Choices about Administrative Procedures.” American Political Science Review 89 (March): 62-73.

 

Epstein, David and Sharyn O’Halloran. 1999.  Delegating Powers: A Transaction Cost Politics Approach to Policy Making under Separate Powers. New York: Cambridge University Press.

 

Jones, Bryan D. 2001. Politics and the Architecture of Choice.  Chicago: University of Chicago Press.

 

Jones, Byran, Jeff Talbert and Matthew Potoski. 2001.  “Uncertainty and Political Debate: How the dimensionality of political issues gets reduced in the legislative process.” Paper presented at the 2001 Annual Meeting of the Midwest Political Science Association, Chicago, Illinois.

 

McCubbins, Mathew, Roger Noll and Barry Weingast. 1989. “Structure and Process, Policy and Politics: Administrative Arrangements and the Political Control of Agencies.” Virginia Law Review 75 (March): 431-482.

 

Moe, Terry M. 1989. “The Politics of Bureaucratic Structure.” In John Chubb and Paul Peterson, eds., Can the Government Govern? Washington, DC: Brookings, pp. 267-329.

 

O’Toole, Lawrence J. and Kenneth J. Meier. 1999. “Modeling the Impact of Public Management: Implications of Structural Context.” Journal of Public Administration Research and Theory 9 (October): 505-526.