We start by arguing why policy-making is more complex and more important than ever in nowadays globalized world. We then describe the typical process of policy-making, and finally analyse the key challenges to the process. This analysis is based on desk analysis and the on-going survey of users needs.

Our first consideration is that policy-making is more important and complex than ever. The role of government has substantially changed over the last twenty years. Governments have to re-design their role in areas where they were directly involved in service provision, such as utilities but also education and health. This is not simply a matter of privatisation, or of a linear trend towards smaller government. Indeed, even before the recent financial turmoil and nationalisation of parts of the financial system, government role in the European societies was not simply “diminishing”, but rather being transformed. At the same time, it is increasingly recognized that the emergence of new and complex problems requires government to increasingly collaborate with non-governmental actors in the understanding and in the addressing of these challenges2. As an OECD report states the following:

“Government has a larger role in the OECD countries than two decades ago. But the nature of public policy problems and the methods to deal with them are still undergoing deep change. Governments are moving away from the direct provision of services towards a greater role for private and non-profit entities and increased regulation of markets. Government regulatory reach is also extending in new socio-economic areas. This expansion of regulation reflects the increasing complexity of societies. At the same time, through technological advances, government’s ability to accumulate information in these areas has increased significantly. As government face more new and complex problems that cannot be dealt with easily by direct public service provision, more ambitious policies require more complex interventions and collaboration with non-governmental parties”

This is particularly challenging in our "complex" societies. “Complex” systems are those where “the behaviour of the system as a whole cannot be determined by partitioning it and understanding the behaviour of each of the parts separately, which is the classic strategy of the reductionist physical sciences”. The present challenges governments must face, as described by the OECD, are complex as they are characterised by many non-linear interactions between agents; they emerge from these interactions and are therefore difficult to predict. The financial crisis is probably the foremost example of a complex problem, which proved impossible to predict with traditional decision-making tools.

The job of policy-makers: the policy cycle

Policy-making is typically carried out through a set of activities described as "policy-cycle" (Howard 2005). In this document we propose a new way of implementing policies, by first assessing their impacts in a virtual environment.

While different versions of the cycle are proposed in literature, in this context we adopt a simple version articulated in 5 phases:

1. agenda setting encompasses the basic analysis on the nature and size of problems at stakes are addressed, including the causal relationships between the different factors
2. policy design includes the development of the possible solutions, the analysis of the potential impact of these solutions3, the development and revision of a policy proposal
3. adoption is the cut-off decision on the policy. This is the most delicate and sensitive area, where accountability and representativeness are needed. It is also the area most covered by existing research on e-democracy
4. implementation is often considered the most challenging phase, as it needs to translate the policy objectives in concrete activities, that have to deal with the complexity of the real world. It includes ensuring a broader understanding, the change of behaviour and the active collaboration of all stakeholders.
5. Monitoring and evaluation make use of implementation data to assess whether the policy is being implemented as planned, and is achieving the expected objectives.

The figure below (authors’ elaboration based on Howard 2005 and EC 2009) illustrates the main phases of the policy cycle (in the internal circle) and the typical concrete activities (external circle) that accompany this cycle. In particular, the identified activities are based on the Impact Assessment Guidelines of the European Commission (EC 2009).



Traditionally, the focus about the impact of technology in policy-making has been on the adoption phase, analysing the implications of ICT for direct democracy. In the context of the CROSSOVER project, we adopt a broader conceptual framework that embraces all phases of policy-making.

The traditional tools of policy-making

But what are the methodologies and tools already traditionally adopted in policy-making?

Typically, in the agenda-setting phase, statistics are analysed by government and experts contracted by government in order to understand the problems at stake and the underlying causes of the problems. Survey and consultations, including online, are frequently used to assess the stakeholders’ priorities, and typically analyzed in-house. Linear, general-equilibrium models are used to identify causal relationships between different factors.

Once the problems and its causes are defined, the policy design phase is typically articulated through an ex-ante impact assessment approach. A limited set of policy options are formulated in house with the involvement of experts and stakeholders. For each option, econometric and other simulations are carried out in order to forecast possible sectoral and cross-sectoral impacts. These simulations are typically carried out by general-equilibrium models if the time frame is focused on short and medium term economic impacts of policy implementation. Based on the simulated impact, the best option is submitted for adoption.

The adoption phase is typically carried out by the official authority, either legislative or executive (depending on the type of policy). In some cases, decision is left to citizens through direct democracy, through a referendum or tools such as participatory budgeting; or to stakeholders through self-regulation.

The implementation phase typically is carried out directly by government, using incentives and coercion. It benefits from technology mainly in terms of monitoring and surveillance, in order to manage incentives and coercion, for example through the database used for social security or taxes revenues.

The monitoring and evaluation phase is supported by mathematical simulation studies and analysis of government data, typically carried out in-house or by contractors. Final results are published in report format, and fed back to the agenda setting phase.

The key challenges of policy-makers

Needless to say, the current policy-making process is seldom based on objective evidence and not all views are necessarily represented. Dramatic crisis seem to happen too often, and governments struggle to anticipate and deal with them, as the financial crisis has shown. Citizens feel a sense of mistrust towards government, as shown by the decrease in voters turnout in the elections.

In this section, we analyze and identify the specific challenges of policy-making. The goal is to clearly spell out "what is the problem" that policy-making 2.0 tools can help to solve.

The challenges have been identified on desk-based research of "government failure" in a variety of contexts, and are illustrated by real-life examples.

One first overarching challenge is the emergence of a distributed governance model. The traditional division of “market” and “state” no longer fits a reality where public decision and action is effectively carried out by a plurality of actors. Traditionally, the policy cycle is designed as a set of activities belonging to government, from the agenda setting to the delivery and evaluation. However in recent years it has been increasingly recognized that public governance involves a wide range of stakeholders, who are increasingly involved not only in agenda-setting but in designing the policies, adopting them (through the increasing role of self-regulation), implementing them (through collaboration, voluntary action, corporate social responsibility), and evaluating them (such as in the case of civil society as watchdog of government). As Elinor Ostrom stated in her lecture delivered when receiving the Nobel Prize in Economics4: “A core goal of public policy should be to facilitate the development of institutions that bring out the best in humans. We need to ask how diverse polycentric institutions help or hinder the innovativeness, learning, adapting, trustworthiness, levels of cooperation of participants, and the achievement of more effective, equitable, and sustainable outcomes at multiple scales”. This acknowledgement leads to important implications for the CROSSOVER roadmap: policy-making 2.0 tools are not just tools for government, but for all stakeholders to participate in the policy-making process5.

Detect and understand problems before they become unsolvable

The continuous struggle for evidence-based policy-making can have some important and potentially negative implications in terms of the capacity of prompt identification of problems. Policy-makers have to balance the need for prompt reaction with the need for justified action, by distinguishing signal from noise. Delayed actions are often ineffective; at the same time, short-term evidence can lead to opposite effects. In any case, government have scarce resources and need to prioritize interventions on the most important problems.

For instance the significant underestimation of the risks of the housing bubble in the late 2000s, and the systemic reaction that it would lead to, led to delayed reactions. The detection of the ozone hole was delayed because satellite detection instruments were calibrated to consider as "errors" measurements outside a certain boundary; it turned out that correct low measurement of ozone were assessed as false negative.

Systemic changes do not happen gradually, but become visible only when it's too late to intervene or the cost of intervening is too high. For example, ICT is today recognized as a key driver of productivity and growth, but evidence to prove this became available at a distance of years from the initial investment. In fact the initial lack of correlation between ICT investment and productivity growth was mostly due to incorrect measurement of ICT capital prices and quality. Subsequent methodologies found that computer hardware played an increasing role as a source of economic growth (see inter al. Colecchia and Schreyer 2002, Jorgenson and Stiroh 2000, Oliner and Sichel 2000).

The problem is in this case is therefore twofold: to collect data more rapidly; and to analyze them with a wider variety of models that account for systemic, long term effects and that are able to detect and anticipate weak signals or unexpected wild cards.

Generate high involvement of citizens in policy-making

The involvement of citizens in policy-making remains too often associated with short-termism and populism.

It is difficult to engage citizens in policy discussions in the first place: public policy issues are not generally appealing and interesting as citizens fail to understand the relevance of the issues and to see "what's in it for me". The decline in voters turnout and the lack of trust in politicians reflects this. More importantly, there are innumerable cases where the "right" policies are not adopted because citizens "would not understand" or because it is not politically acceptable.

While the Internet has long promised an opportunity for widespread involvement, e-participation initiatives often struggle to generate participation. Participation is often limited to those that are already interested in politics, rather than involving those that are not.

When participation occurs, online debates tend to focus on eye-catching issues and polarized positions, in part because of the limits of the technology available. It is extremely difficult and time consuming to generate open, large scale and meaningful discussion.

Identify “good ideas” and innovative solutions to long-standing problems

Innovation in policy-making is a slow process. Because of the technical nature of issues at hand, the policy discussion is often limited to restricted circles. Innovative policies tend to be "imported" through "institutional isomorphism". Innovative ideas, from both civil servants and citizens, fail to surface to the top hierarchy and are often blocked for institutional resistance.

Existing instruments for large-scale brainstorming remain limited in usage, and fail to surface the most innovative ideas. Crowdsourcing typically focus on the most “attractive” ideas, rather than the most insightful.

Reduce uncertainty on the possible impacts of policies

When policy options have been developed, simulations are carried out to anticipate the likely impact of policies. The option with the most positive impact is normally the one that is proposed for adoption.

Most existing methodologies and tools for the simulation of policy impacts work decently with well known, linear phenomena. However, they are not effective in times of crisis and fast change, which unfortunately turn out to be exactly the situations where government intervention is most needed.

As an example nowadays the European Central Bank bases its analysis of the EURO Area economy and monetary policy on a derived version of the DSGE model developed by Frank Smet and Raf Wouters in 20036. Smet and Wouters’ model is deeply microfounded, allowing for a rigorous theoretical structure of the model. Moreover in this setting the reduced form parameters are related to deep structural parameters in order to mitigate Lucas’ critique, while the utility of agents can be taken as a measure of welfare in the economics (Phelps ed. 1970).

However, the DSGE models suffer from several shortcomings jeopardizing their ability to predict, let alone to prevent, a global crisis:

1. Agents are assumed to be perfectly rational, having perfect access to information and adapting instantly to new situations in order to maximize their long-run personal advantage
2. So far agents have entered the models as homogeneous representative entities, while it would be a step forward being able to take into account agents heterogeneity
3. Canonical models consider atomistic agents with little or no interactions and thereby are not able to cope with network externalities

But most of all it is the very notion of equilibrium which prevents standard models from dealing with crisis. A stable steady state equilibrium is a condition according to which the behaviour of a dynamical system does not change over time or in which a change in one direction is a mere temporary deviation. This condition is proper of general equilibrium theory, in which a stable steady state is believed to be the norm rather than the exception. When in the canonical model we are out of equilibrium, the situation is seen just as a short lapse before the return to the steady state. This is in sharp contrast with the very notion of crisis, which represents a steady deviation from the equilibrium. Loosely speaking, the crisis phenomenon is not even conceived within the framework of standard models.

All these flaws are not only related to DSGE models, Computational General Equilibrium (CGE) or macro-econometric forecasting models, but generally affect the traditional policy making tools. In this view it would be very important to find new frameworks capable of avoiding those shortcuts. Some of such methodologies and methods already exists, e.g. System Dynamics and other hybrid models, and some governments are using them. Our aim is to push forward in that direction.

We need to move away from the equilibrium paradigm in order to be able to assess other issues: evolutionary dynamics; heterogeneity of technologies and firm; political and legal determinants of social stability; incentive structures; better modelling technological change, innovation diffusion and economic systems (taking into account finance, debt and insurance); interactions between heterogeneous economic agents (firms and households) and central governments; heterogeneous responses to government incentives; economic dependence from the ecosystem.

Trichet, the former head of ECB, clearly put it: “This doesn't mean we have to abandon DSGE...(but)...atomistic rational agents don't capture behaviour during a crisis...rational expectations theory has brought macroeconomics a long way ... but there is a clear case to re-examine the assumptions”

But the need for new policy making tools is not limited to the economic realm: in the future it will become more and more important to anticipate non-linear potentially catastrophic impacts from phenomena such as: climate change (draught and global warming); threshold climate effects such as poles’ sea-ice withdraw, out-gassing from melting permafrost, Indian monsoon, oceans acidification; social instability affecting economic well being (social conflict, anarchy and mass people movements).

Lack of understanding of systemic impact has driven to short term policies which failed in grasping long term, systemic consequences and side effects:

1. An example of this approach might be given by the sovereign debt issue. In fact it is relatively easy for governments under popular pressure to increase expenditure and public debt to cope with short term necessities, such as offsetting the negative impacts brought about by a regional or global crisis. On the other hand it is harder to take into account the long term effect determined by higher interest rates on private investments and consumption through crowding out and fiscal pressure
2. Another example of short-termism are the financial policies pursued in south East Asia at the beginning of the 90s. Many countries, such as Thailand, liberalized their financial markets fostering the inflow of investments aimed at sustaining growth. Unfortunately those capitals triggered a real estate bubble which has been at the roots of the 1997-1998 crisis
3. In 2008 the Central Bank of Iceland yielded liquidity loans for saving banks on the verge of default on the basis of newly-issued, uncovered bonds, i.e. effectively printing fiat money on demand, causing a significant rise in inflation. To cope with this rise in prices, the Iceland Central Bank had to keep very high interest rates thereby leading to an economic bubble
4. According to a great number of economists the financial crisis was triggered by US government policies spanning across two administrations which were intended to ensure citizens’ right but instead determined an unprecedented high number of risky mortgages, as well as the decline in mortgage underwriting standards that ensued. According to the “Financial Crisis Inquiry Commission Report” 7 those policies, together with the deregulation of the financial system, might have been catalyzed the crisis.
5. Other examples can be the bail out of financial institutions: in the short run those actions maintain employment and economic standards, while in the long term they induce moral hazard, keep operating inefficient companies and decrease the trust of economic agents in regulation, which is the funding pillar of our economic system

Ensure long - term thinking

In traditional economics, decisions are utility-maximising. Agents rationally evaluate the consequences of their actions, and take the decision that maximize their utility. However, it is well known that this rationalistic view does not fully capture human nature. We tend to overestimate short-term impact and underestimate the long term (see Saffo). In policy-making, short-termism is a frequent issue. People are reluctant to accept short-term sacrifices for long-term benefits. Politicians have elections typically every 5 years, and often their decisions are taken to maximize the impact “before the elections”. There is also the perception that laypeople are less sensitive to long term consequences, which are instead better understood by experts. Overall, long-term impact is less visible and easier to hide, due to lack of evidence and data, as well as of models to simulate ex-ante alternative policy options. As a result, decisions are too often taken looking at short-term benefits, even though they might bring long term problems.

Climate change is a typical policy area where sub-optimal decisions were taken because the short-term costs were considered to outweigh the long term consequences. The long term impact is not visible, while the short term sacrifices were, even though ICT had an important role in stimulating the debate and catalysing attention of the media on the issue.

Encourage behavioural change and uptake

Once policies are adopted, a key challenge is to make sure that all stakeholders comply with regulations or follow the recommendations. It is well known how the greatest resistance to a policy is not active opposition, but lack of application.

For instance, several programmes to reduce alcohol dependency problems in the UK failed as they excessively relied on positive and negative incentives such as prohibition and taxes, but did not take into account peer-pressure and social relationships. They failed to leverage “the power of networks” (Ormerod 2010). For instance, any policy related to reduction of alcohol consumption through prohibitions and taxes is designed to fail as long as it does not take into account social networks, as binge drinkers typically have friends who also have similar problems. In another classical example (Christakis and Fowler 1997), a large scale longitudinal study showed that the chances of a person becoming obese rose by 57 per cent if he or she had a friend who became obese.

The identification of social networks and the role of peer pressure in changing behaviour is not considered in traditional policy-making tools.

Manage crisis and the “unknown unknown”

The job of policy-makers is increasingly one of crisis management. There is robust evidence that the world is increasingly interconnected, and unstable (also because of climate change). Crises are by definition sudden and unpredictable. Dealing with unpredictability is therefore a key requirement of policy-making, but the present capacity to deal with crises is designed for a world where crises are exceptional, rather than the rule. Donald Rumsfeld, former secretary of state, famously said during the Iraq war that while the US government was capable of dealing with the “known unknown”, the difficulty was the increasing recurrence of “unknown unknown”: those things that we don’t known that we don’t know.

There is evidence that the instability and chaotic natures of our world is increasing, because of its increasing connectedness. Every year, intense climate phenomena throw our cities in disarray, because of snow, flooding, fires. Each crisis seems to find our decision-makers unprepared and unable to deal with it promptly. As Taleb (2007) puts it, we live in the age of "Extremistan": a world of "tipping points" (Schelling 1969) “cascades” and "power laws" (Barabasi 2003) where extreme events are "the new normal". There are many indications of this extreme instability, not only in negative episodes such as the financial crisis but also in positive development, such as the continuous emergence of new players on the market epitomised by Google. The random vulnerability of today’s world is well illustrated by this chart from the EC DG RESEARCH.



Moving from conversations to action

The collaborative action of people is able to achieve seemingly unachievable goals: experiences such as ZooGalaxy and Wikipedia show that mass collaboration can help achieve disruptive innovation. Yet too often web-based collaboration is confined to complaints and discussions, rather than action. As one blogger put it, paraphrasing Marx: “Philosophers have only interpreted the world: the point is to complain about it”8.

For example, the recent Italian elections saw an explosion of activity in social media discussing about the different candidates. This energy then failed to translate into concrete action in the aftermath of the elections.

Detect non-compliance and mis-spending through better transparency

In times of crisis, it is ever more important for governments to ensure that financial resources are well spent and policies are duly implemented. But monitoring is a cost in itself, and a certain margin of inefficiency in resources deployment is somehow “natural”. Yet the cost of this mismanagement is staggering: for instance, in 2010, 7.7% of all Structural Funds money was spent in error or against EU rules9. OECD estimates place the cost of corruption equals 5% of global GDP10. Thereby it would be crucially important to be able to avoid the mismanagement with anticipatory corrective actions.

Understand the impact of policies

Measuring the impact of policies remains a challenge. Ideally, policy-makers would like to have real-time clear evidence on the direct impact of their choice. Instead, the effects of a policy are often delayed in time; the ultimate impact is affected by a multitude of factors in addition to the policy. Timely and robust evaluation remains an unsolvable puzzle.

This is particularly true for research and innovation policy, where the results from investment are naturally expected at years of distance. As Kuhlmann and Meyer-Krahmer (1994) puts it, “the results of evaluations necessarily arrive too late to be incorporated into the policy-making process”.