Printer Friendly

The vestiges and vanguards of policy design in a digital context.

Introduction

The digital era--marked by the proliferation of digital technologies in all facets of society, from dating to shopping, personal health monitoring to political activism--alters the context of policymaking. Accordingly, policy designers--those engaged in "the deliberate and conscious attempt to define policy goals and connect them to instruments or tools expected to realize those objectives" (Howlett 2014)--increasingly find their work couched in a new vocabulary of "digital by default," "agile," "open," "innovation," "Government 2.0" and "government-as-a-platform" (O'Reilly 2011), which are weaved into public sector renewal agendas and political speeches across the globe. Designers' toolboxes have expanded to include a range of new digitally-enabled policy instruments, including open data, big data, crowdsourcing, social media, robotics and artificial intelligence. Altering the very context within which policy design is undertaken and the tools that are applied in that context, the digitization of everyday life presents opportunities, challenges and new expectations for policy designers operating in today's governments.

We argue those on the vanguard of digital era policy design (DEPD) should be aware of the contextual effects that have plagued "analog" (that is, pre digital era) design, and which may endure (or even intensify) in the face of new digital realities. There are clear risks, and early evidence that DEPD proves more heat than fire with rhetoric far outpacing its uptake (Clarke 2014; Clarke and Francoli 2017; Craft 2014). In this case, as society embraces the digital era and is transformed by it, governments may simply lag behind, blending forays into DEPD with an enduring commitment to analog policy design. And while many scholars presume that DEPD will foster a more collaborative, co-productive model of state-to-non-state cooperation, and with it, democratic renewal (see O'Reilly 2011; Noveck 2009, 2015; Margetts and Dunleavy 2013), the role of individual citizens and non-governmental organizations in social problem solving remains unclear in the digital age. We identify fruitful points of departure and sketch out a preliminary research agenda by which public administration students and practitioners can begin to identify, assess, and respond to these implications.

We begin by detailing the contours of DEPD, focusing specifically on the characteristics of the new policy instruments and related public sector renewal agendas through which its activities are being introduced. We argue that while DEPD consists of a varied and ever-evolving set of activities and instruments, it is characterized by four dominant traits: (1) prioritization of the tool of information; (2) heavy reliance on the actions of nonstate actors and resources; (3) iteration and short-term experimentation; and (4) segmentation of policy design to individuals and groups. We detail each of these characteristics below and then consider three key areas of policy design ripe for re-appraisal and ongoing monitoring as DEPD practices are integrated into government.

Characteristics of digital era policy design: an instruments perspective

DEPD is a moving target, comprised of a wide range of activities that evolve and expand as quickly as the technological offerings of tech firms, hackathons and basement-bound teenage coders emerge. We identify four core characteristics of DEPD, derived through an emphasis on the policy instruments the digital age makes possible. Policy instruments are the techniques or means through which states attempt to attain their goals and have long served as a lens for studying and undertaking policy design (Linder and Peters 1989).

In the digital era, governments are experimenting with several instruments for policymaking. For example, the ubiquitous place of web-based technologies in society--a trend fuelled by the growing uptake of mobile technologies and social media in particular--renders it possible for governments to access citizens anywhere, anytime. Social media such as Facebook and Twitter have become mainstream communication tools across most public sector bureaucracies, folded into public awareness campaigns, informational service delivery, transparency initiatives, and engagement exercises (Digital Engagement Team 2011; Government 2.0 Taskforce 2015).

The potential for real-time, anywhere government-to-citizen communication has also raised interest in behavioral economics--more specifically, "nudge" (Behavioural Insights Team 2014; Charney 2014; Thaler 2008) approaches to social problem solving--and for gamification (King et al. 2013; McCall, Koenig, and Kracheel 2013). Here, socially desirable citizen behaviour is incentivized through information cues, reducing the need for direct state intervention via "carrots and sticks." In addition to these "information pushes," governments are also turning to the social web for "information pulls," collecting big data--large-scale, unmediated and unstructured data--describing citizen behavior and preferences, as reflected in their interactions online, in order to inform more evidence based and/or democratically responsive policy development processes and service delivery improvements (Clarke and Margetts 2014; Keen et al. 2012; Manyika et al. 2011; Rutter 2014).

The back-office functions of government are ever-more digitized (email communications versus phone, for example) and outside their walls, online service delivery means that traditional government-citizen interactions increasingly take place in digital spaces (Allen et al. 2001; Brainard and McNutt 2010; Dunleavy et al. 2006; Dutil 2010). With improvements to algorithmic, automated data analysis techniques, governments can engage in more sophisticated analyses of transactional big data long in its possession and new sources of data created by digital services--both can be used to scrutinize the efficiency and effectiveness of government operations. In particular, big data and the digitization of government-citizen interactions create new opportunities to experiment with different policy options and designs through A/B testing, (1) allowing for real-time adjustments to government interventions that were not possible in an age where government-citizen interactions occurred in settings without a traceable digital data trail (such as face to face service delivery), and were less amenable to on-the-fly tweaks.

Governments are also experimenting with web-based crowdsourcing for policy development and service delivery, operating under the assumption that this distributed model will allow for more effective and efficient outcomes than might result from state-centric approaches alone (Aitamurto and Landemore 2015; Farmer 2013; Gao et al. 2011; Noveck 2009). Similarly, open data--the release of machine readable government data, with accompanying reuse licenses--is being vigorously promoted and pursued by governments for economic growth and public sector accountability, but also to faciliate non-governmental service provision, and to enable outsiders to better feed into policy processes (Clarke and Francoli 2014; Davies 2010; Davies and Bawa 2012). In addition, social media are often discussed as powerful platforms for interactive citizen engagement via so-called "digital dialogue" (Coleman and Moss 2011; Mergel 2014; Papadopoulos and Warin 2007).

To what extent are these additions to the policy designer's toolkit "new"? From one perspective, these innovations are digital variants of well-worn, "analog" policy instruments. For example, information campaigns on Twitter represent a digitized version of the information campaigns that governments have long relied on to ensure a well-informed, resilient population. Crowdsourcing is old hat, evident, for example in voluntary citizen reporting of retail price changes and beach erosion (Grabosky 2013), and in public sector consultation, stakeholder engagement and expert evidence gathering (Culver and Howe 2004; Edelenbos and Klijn 2006; Foo et al. 2011). Likewise, A/B testing and experimentation are not new; experimental research is an established, if at times under-exploited, component of traditional evidence-based policymaking (Bozeman and Scott 1992; Margetts 2011). From this perspective, in DEPD, the "digital" serves as adjective to "era" and not "policy design." The era or context in which design takes place is more digital, evident in the undeniable infiltration of digital ICTs on all aspects of human society (Castells 2004), including the public sector. This contextual shift to a digital era therefore affects policy design because it makes possible the digitization of "analog" policy instruments, and justifies their digitization (since a digital era denotes a digital society on which policies act, and to which services are directed). For example, an information campaign previously delivered through radio can now be delivered via social media, or both. But perhaps more importantly, for this policy instrument to be effective, it is likely that it must be delivered via social media since this is increasingly how citizens access information, as has been evidenced in research exploring citizens' information seeking habits in the wake of recent natural disasters (Bird, Ling, and Haynes 2012; Kodrich and Laituri 2011).

What happens when analog policy instruments become digitized? We propose four features that underscore how digital era policy design can be considered to be something "new" or unique that warrants attention for its potential impact on policy making more generally (and for design in particular).

Four characteristics of digital era policy design

First, drawing on Hood's "Tools of Government" scheme (1983), when policy instruments become digitized they tend to be assigned to the category of Information (or Nodality, as originally termed by Hood) rather than the other categories of tools (Authority, Treasure and Organization). In The Tools of Government in the Digital Age (2007) Hood and Margetts predicted the primacy of the tool of Information in contemporary government activities. Indeed, as most aspects of human society have become ever more digitized the scope to produce, collect, interpret and apply data (a subsidiary instrument under the broader umbrella tool of Information) to social problem solving has expanded, thus augmenting the practical value of information as a policy resource. This results from two affordances of digital technologies. First, digital technologies reduce the temporal, geographic, administrative, and in some cases linguistic barriers which might prevent governments and citizens from easily exchanging data. This potential for low-cost data exchange is at the heart of the policy instruments emerging in the digital age, including social media, A/B testing, crowdsourcing, and most obviously, big and open data. Second, the practical value of data as a policy development resource rises in the digital era because of digital technologies' inherent capacity to produce and collect data trails. For instance, an online service transaction, such as a passport application, produces an immense trail of digital data indicating where, when and how an applicant filled in the form, where errors were made, and how these errors varied across different application formats. This data can be captured in real-time, without the error of human intervention in transcribing the data, or the biases arising from asking the applicant what worked and did not work well in the application (as happens with surveys or customer feedback forms, classic examples of analog information tools). Again, digitizing passport applications does not fundamentally alter the policy instrument, but it does alter its characteristics, in this case transforming it into a data-generator that can support the policy process in ways its analog, paper-based variant could not.

In contending that the tool of Information gains prominence in the digital age, we are not suggesting the tools in the categories of Authority, Treasure and Organization are marginalized. Governments still pass regulations, tax and distribute resources, and maintain a battery of organizational resources, including public servants and public infrastructure. And importantly, it is in wielding the tools of Authority, Treasure and Organization that the tool of Information gains and loses its potency in the digital era; digital era Information tools, as wielded by both governments and citizens, are shaped by state regulation of the telecommunications industry, investments in digital infrastructure and broader Internet governance policies (for example, policies related to net neutrality, privacy, intellectual property). Nonetheless, we posit that the other tools of government must increasingly be wielded in tandem with the tool of Information in the digital era to be wielded effectively. If the digital era is characterized by ubiquitous information production, distribution and consumption in online networks, then government's capacity and function in these socio-informational networks (the essence of the tool of Information) necessarily rises in importance relative to its utility in analog contexts (Clarke 2014). This is true whether Information is wielded on its own or when coupled with the other tools of government, a logic that has even driven some to posit that in the digital age the tool of Information is so central that governments, in essence, become their websites (Margetts 2011; Steinberg 2012). And as Marando and Craft (2017) detail, the prominence of information in a digital policy process raises important implications not only for the supply of information as part of policy design, but of information as a resource to be brokered and consumed by political staff, elected officials and citizens (see also Craft and Howlett 2013b).

In addition to favouring the tool of Information, digital-era policy design instruments tend to privilege the participation of non-government actors in government activities. This could involve co-production, in which both governments and nongovernment actors contribute to policy formulation or implementation (Bovaird and Loeffler 2013; Brandsen, Pestoff, and Verschuere 2012; Joshi and Moore 2004). Co-production is most evident in crowdsourcing, digital dialogues and open data initiatives, centered on combining government resources with contributions from non-government actors to either formulate or implement policy. Less obviously, coproduction is at play in digital era policy instruments that rest on "information pushes," such as communications applications and nudge approaches, both of which represent policy instruments that demand some degree of citizen participation to be effective. For example, a social media based information campaign relies on co-production insofar as the information distributed by the state must be consumed and acted upon by the public for the policy goal to be achieved. Nudge, similarly, rests on a government-issued information cue that induces voluntary action by citizens or organizations that supports a government policy objective.

In this sense, digital policy instruments tend to be procedural in that they indirectly affect social outcomes through networks of policy actors, which is different from substantive instruments directly affecting social outcomes through government-centric command and control (Howlett 2011). In some cases, procedural instruments grant government great latitude to manipulate and control the networks of actors they act on (nudge, information campaigns), aligning with the notion of "steering" that has risen in practice and in theories of the policy process from the 1980s onwards (Osborne and Gaebler 1992; Peters 2011). In other cases, procedural instruments have the potential to redistribute agency from governments to non-government actors, through bottom-up decentralized processes, as in open data, crowd-sourcing and digital dialogues. This non-government-centric view of digital instruments has led some scholars and practitioners to posit wholesale public management reforms: the hollow state is no longer cast as "steering" networked social problem solving efforts (Howlett 2000; Klijn 2002; Rhodes 1994), but rather as a "platform" (O'Reilly 2011). The state exists to enable non-government actors to define and solve social problems, whether by providing resources and coordination mechanisms, or by outsourcing government functions to these actors using contracts, public procurement, and privatization (Bracken 2012; Chief Technology Officer 2014; Janssen and Estevez 2013; O'Reilly 2011) (although notions of government as a platform are evolving--see Clarke 2016).

The third defining feature of digital era policy instruments is their tendency to favour small-scale experimentation and iteration. We see this characteristic most obviously in A/B testing and in the use of randomized-controlled trials to design strategic nudges, typically (but not always) delivered through digital communications channels. Equally governments have begun experimenting with the release of alpha and beta versions of websites in order that users can provide feedback on information and service offerings as they are developed (Government Digital Service 2012). A similar approach is invoked in discussions of digital dialogue, which call for governments to involve citizens in regular feedback on policy design as it enfolds. Citizens provide suggestions for edits and tweaks at all stages of design, as opposed to releasing a policy for feedback once it has already largely been designed as has traditionally been the case in government consultations (Coleman and Gotz 2001). Additionally, innovation labs and hubs have emerged as popular new policy design instruments in the digital age, and are typically meant to provide a space (both conceptual and physical) in which policy designers can prototype and pilot novel approaches to policy problems and service delivery methods (Bellafontaine 2013; MindLab n.d.). Whenever these new policy design instruments are raised, they are praised as instances of "agile" or "open source" models of policy design, dubbed as such to contrast them with the long development cycles that have typically characterized government's approach to policy design, and which have tended to lack scope for regularized feedback and on-the-fly improvements. This reflects uptake of so-called "design thinking" in the public sector (Clarke and Craft 2017), and is central to a new "digital government orthodoxy" championed by central digital service units across the globe (Clarke 2017).

Finally, the digitization of policy design facilitates greater opportunities for governments to implement "user-centered" design principles which cater to individual and group preferences and needs. Policy instruments choice theory has long recognized that the target of the policy is an important determinant in instrument selection (Schneider and Ingraham 1990). In a digitized design context governments' instruments choices can profit from a wealth of data, new feedback loops, and relatively low-cost mechanisms for adjusting designs. These can result in design calibration to specific individualized and segmented groups, empowering design fit or congruence with user-centered expectations, and avoiding the drawbacks of "one-size-fits-all" models. Such segmentation can allow governments to better reach target groups, thus strengthening the quality of designs by prioritizing a usercentric lens. For example, with a digital passport application segmentation may allow a government and the service user to benefit from simple automations, such as populating fields in web-enabled forms, or from more elaborate design cues such as offering apps or digital platforms better suited to passport applicants in locations with mobile rather than landline Internet connectivity.

This four-part description brings focus and boundaries to our discussion of this new phase of policy design, illustrating how digital era policy design can be distinguished from previous analog policy design approaches. However, it does not reveal the extent to which policy designers are engaging in the Information heavy, non-state centric, iterative and segmented models of policy design the digital age makes possible. Equally, this does not outline what happens when these new approaches to policy design meet established policy designs already in place. Finally, while we posit that policy design is increasingly a "team sport" in the digital age, involving a broader range of actors, it remains unclear what this will entail for long standing debates on the role afforded to individual citizens and non-governmental organizations in design, and democratic systems more broadly. We turn to these issues next, laying out a research agenda for the study of DEPD.

A research agenda for digital era policy design

Digital era policy design and non-design in practice

Policy design scholarship acknowledges that the various policy instruments, policy aims, and actors engaged in design are embedded in spatial, relational, and cognitive contexts which can be more or less amenable to design, or to its opposite--non-design--a form of policy making which betrays the "gold standard" of instrumentality and rationality prized in ideal policy design (Linder and Peters 1990; Torgersen 1985; Skodvin, Gullberg, and Aakre 2010). Thus, the first area ripe for critical enquiry picks up on this contextual orientation to probe the question: will the digital age and the instruments it makes possible lead to policy design in practice, and if so, how and under what conditions?

Design and non-design in policymaking can be distinguished according to intentionality (that is, a policy maker's willingness to engage in instrumental, rational policymaking) and capacity for policy design (that is, a policymaker's ability to collect, interpret and act on knowledge in rational, instrumental policy design processes) (Howlett and Mukherjee 2014). Intentionality can be undermined by partisan, ideological and faith-based agendas, by which policy designers displace public good considerations and a concern for sober evidence assessment with parochial self-interest and power-seeking decision rationales (Aucoin 2012; Howlett and Mukherjee 2014; Majone 1975). Capacity for policy design rests on cognitive, technical and institutional conditions, including access to, and ability to interpret, high quality information that reduces policymakers' bounded rationality and facilitates evidence-driven, instrumental and logical problem solving (Jones 2003; Simon 1991; Weyland 2006).

To what extent are these conditions at play in DEPD? Early empirical evidence suggests that digitized policy processes do not necessarily live up to the policy design ideal. In particular, governments lack the capacity to integrate new digital policy instruments into their existing toolkits (Clarke 2014, 2018; Clarke and Margetts 2014; Mergel and Desouza 2013), especially big data and sophisticated data analytics (Cook 2014; Decker 2014; Giest, 2017; McAfee and Brynjolfsson 2012). Indeed, a cursory look at the literature on government "policy capacity" or "policy analytical capacity" reveals that governments struggle to even engage in traditional "analog" policy design given low or uneven policy capacity (Painter and Peters 2005; Craft and Howlett 2013a; Tiernan 2011), never mind offering sufficient analytic and technical skills and resources to integrate big data, agile design, experimentation and machine learning into their work.

At the same time, in those cases where policy makers have successfully integrated new digital instruments into policy design processes, longstanding assumptions at play in theories of policy design and non-design require re-appraisal. For example, in what ways might volumes of new data, sophisticated analytical tools, machine learning and collaborative policy design tools (for example, crowdsourcing, citizen science, open data) mitigate long-standing concerns over the bounded rationality of policy designers? Moving forward, digital era design, with its promises of hyper-evidence-based and supremely rational design outcomes provides new scope to appraise theoretical and practical debates on government's policy capacity, and the conditions bolstering or undermining it in everyday policy work (Howlett, Wellstead, and Craft 2017).

In addition, researchers must probe the ways in which design intentionality manifests in the digital age. On the one hand, even where policy designers possess the capacity for DEPD, they may nonetheless pursue non-designs in the face of political decision-makers concerned more with partisan political calculations than they, are with rational, evidence-informed decisionmaking; the reality of electoral cycles and the politics of policy are not erased by big data, crowdsourcing and A/B testing (Clarke, 2014, 2016; Clarke and Craft, 2017). Here, it is worth noting Marando and Craft's (2017) focus on the demand dynamics of DEPD, wherein "demand" for DEPD (versus Digital Era Non-Designs, or traditional analog (non)designs) from political leaders is a crucial determinant of DEPD "on the ground," within policy advisory systems. Alternatively, as the digital trails tracking citizens' online actions, and service and policy successes and failures are made accessible to those outside government (whether via open data initiatives, or at the hand of non-government actors), the ability for governments to rationalize policy designs with little evidence base may wane.

Mixes, bundles, and portfolios in digital era (non)design

The above discussion of policy instruments (digital and otherwise), design and non-design, and design "spaces" underscores that policy design is a nested phenomenon (Howlett 2009). This approach recognizes that policy design typically does not involve goals and means operating at a single level but at varying degrees of specification. Broad policy aims and means--for example, economic growth--include more granular or specific subsidiary policy goals and instruments within them. Further, designs likely do not involve single policy aims or instrument choices, but rather a number of them. For example, a government's economic policy may involve a combination of expenditure reductions and spending, as well as monetary, fiscal, and trade policy goals and instruments. These may be further calibrated or targeted to regional or sectoral policy goals. As such, policy designs are better understood as instrument mixes and policy portfolios with multiple policy instruments, policy goals, and policy designs (and even non-designs) operating concurrently (Jordan and Matt 2014; May 2003; del Rio, Calvo Silvosa, and Gomez 2011). By this logic, policy designs should be disaggregated to highlight their constituent components. This includes the broad policy goals animating a design down to more specific policy objectives, along with broad instrument choice logic, to narrower instrument specification and calibration "on the ground" (Cashore and Howlett 2007; Howlett 2009; May et al. 2005).

From this perspective the propensities for design or non-design are more nuanced, in that the intention and capacity considerations explored above may apply to a range of abstract or specific policy design questions: is it the policy aim or goal, the instrument itself, or the specific on-the-ground programs or instrument calibration(s) that are amenable (or not) to digital design? Moreover, what happens when we start combining digital instruments and designs with analog variants? For example governments may adopt broad "digital by default" policy goals but have various specifications on how to apply them in different policy domains, or calibrate them for policy or programmatic purposes. This could involve different combinations of analog and digital design(s), or their components. Scholars will advance our understanding of DEPD by parsing the conditions that lead to these combinations, along with their compatibility with the design ideal.

Further, while considerable effort has gone into thinking about instrument mix and design portfolio optimization in the analog context, this lens has yet to be applied to DEPD. Early work on mixes and portfolio optimization focused on design integration, asking how the various components of a design work in mutually reinforcing ways, considering consistency of designs (can multiple policy tools complement rather than undermine one another vis a vis a policy goal?), coherence (how well can multiple policy goals logically co-exist?), and congruence (how complementary are multiple goals and instruments?) (Howlett and Rayner 2013; Briassoulis 2005; Stead, Geerlings, and Meijers 2004). Do policy mixes and integration principles apply to DEPD and its contexts in the way they did for analog design? For example, as governments facilitate the digitization of taxation, passport, or immigration programs and services, tensions emerge between privacy and security imperatives and optimization of service or program administration.

Finally, there is the question of how policy design(s) change--either in part or in toto--as policy portfolios evolve, erode or are purposefully adapted. Leveraging neo-institutional theory (Hall and Taylor 1996; Thelen 1999), policy design has grappled with sequencing effects, producing important works that highlight policy design dynamics such as: layering of new design components or designs on existing designs or mixes; wholesale replacement of designs by alternatives; or intentional or unintentional use or stretching of existing designs through drift and conversion2 (Hou and Brewer 2010; Howlett and Mukherjee 2014). Others have adroitly noted that design can involve maintenance or adaptation of existing designs through patching and packaging. Patching involves the conscious use of design methods like layering, conversion and drift to incrementally improve existing policy designs. Packaging involves creating altogether new designs. Current design thinking expects patching to be considerably more common than packaging given that policymakers can rarely "wipe the slate clean" and develop optimal policy mixes from scratch (Howlett and Rayner 2013; Howlett and Mukherjee 2014). Those on the DEPD vanguard are undoubtedly confronting similar challenges already. As new digital policy goals and instruments, and policy designs, come "on-line" how they are combined or used to replace (or extend) existing designs, or their components, is a crucial area of study. Finally, scholars and practitioners seeking optimality across DEPD and policy mixes will be forced to contend with the segmentation effects noted above. User-centred design principles add further consideration to ongoing debates about the creation and management of policy mixes and dynamics, which researchers and practitioners must anticipate and capitalize on.

The role of government and non-government actors in DEPD

As described in the literature to date, digital era policy instruments engage non-government actors in design processes in a number of ways: self-regulating citizens responding to an information cue issued via a digital nudge; citizens and organizations contributing to policy and service delivery through open data and digital dialogue; and iterative policy experimentation and user testing. More fundamentally, since they are linked to the tool of Information, digital era policy design is disposed toward outward-facing solutions and procedural policy instruments relying on social networks as the recipient and source of information. Reflecting this point, the dominant theories of digital era public management (Dunleavy et al. 2006; Noveck, 2009; O'Reilly 2011) depart from Weber's classic bureaucratic model of public administration rooted in government command and control (Dunleavy and Hood 1994; Pollitt and Bouckaert 2011). Instead they emphasize the networks of actors participating in contemporary design processes. These propositions presume that the digital age favours a mutual dependence of resources, authority, and legitimacy embodying the tenets of network, shared and collaborative governance (Bogason and Musso 2006; Phillips 2006; Salamon 2002). But precisely what role are citizens and non-governmental organizations called to play in DEPD, and what are the implications of participatory policy design for democracy writ large?

Some open government rhetoric and strands of the digital government literature focusing on citizen engagement and digital dialogue (see Coleman and Gotz 2001) presume policy design will become more responsive and participatory, realizing a pluralist democratic ideal as citizens are offered "democracy between elections." Yet, in practice, it is often cost-savings and improved program efficiency and effectiveness that drive government's interest in digital era policy design, not a desire to produce policies more democratically responsive per se (Clarke and Margetts 2014; Clarke and Francoli 2014). Thus, policy design may become more participatory, but the links between this shift and broader democratic renewal agendas deserve much more scrutiny in the literature and in government reform claims (but see Clarke, 2014, 2016). Longo's contribution on digital citizen engagement to this Special Issue provides some preliminary answers to the questions raised here, but much more work is required in this area to move beyond the rhetoric of "Digital Democracy" running through the literature to date.

Finally, the roles of government and non-government actors in design work will need re-theorizing as governments grapple with new developments in artificial intelligence, machine learning, automation and robotics. In particular, traditional assumptions about the agency, motivations, intentions and capabilities of designers will demand renewed attention and scrutiny when the agents driving design process are no longer human. In this model, design may become neither government nor nongovernment actor centric. Design research has yet contended with this emerging reality, let alone identified its links to, or departures from, democratic renewal.

Conclusion

Proponents of digital era design rightly emphasize the opportunities that new policy instruments offer governments in the digital age. Framed as a data-driven, supremely instrumental and rational form of social problem solving, DEPD echoes the instrumental rationality and "conscious" manipulation of design(s) emblematic of early design thinking (Linder and Peters 1990; Bason 2014). However, the unbridled enthusiasm of the digital design vanguard would be well served by considerable evidence suggesting that the promise of new instruments, and design optimization, has not always been fulfilled in practice.

Rather than seeing digital era policy instruments as wholesale shifts in the policymaker's toolkit, or representing new categories of tools in their own right, they are often better understood as digitized variants of traditional analog tools: digitized because of the undeniably digital context from which they emerge and to which they are directed. Digital policy instruments prioritize the tool of Information, particularly as the value of data as a policymaking resource expands. Procedural, co-produced policy approaches are likely to be favoured over substantive, government-centric approaches. Iterative, trial-and-error approaches allow for experimentation and innovation, and invite new scope for outsiders to contribute throughout policy cycles. Further, DEPD facilitates granular segmentation as more data and individual and group preferences can be digitally integrated into policy design and its execution. Our four-part definition of digital era policy instruments provides descriptive clarity, or conceptual boundaries, on which future studies of digital era policymaking can draw or which future studies might beneficially contest as new technologies, platforms and related social and economic phenomenon emerge.

In exploring these policy instruments, the practice and study of digital era design can benefit greatly from the more recent and sophisticated design studies reviewed above. These studies address the "nested" approach to design that recognizes the multiple policy components of designs operating at various levels of abstraction. They also acknowledge policy mixes and portfolios sensitive to the role of context, temporality, interaction effects, and propensities for design and non-design. Indeed, this article sought to identify not only what makes DEPD unique but also the challenges and opportunities that analog design theory suggests lay ahead for digital design(ers). The vestiges of analog policy design--the constraints and capability dynamics, the need to deal with the cascading implications of designs operating at various levels of abstraction, and in complex mixes involving multiple concurrent designs--will persist in the digital age. The various empirical and theoretical design contributions we detailed here require careful study to investigate their application in an increasingly digital context.

Notes

(1) A/B testing (sometimes termed bucket testing, or two-sample hypothesis testing) refers to randomized controlled trials in which "A" and "B" represent control and treatment variants, respectively. A/B testing is a prominent component of user experience design in web development. It allows developers to examine which user interfaces are most effective, as evident through the actions that users take on a web page (clicking on the right link versus the wrong one, measuring the amount of time it takes to complete a task, etc.).

(2) Layering occurs when policy mixes or their components are purposefully retained despite shifts in the policy environment. Conversion occurs when policymakers maintain most if not all design mix components but then purposefully apply them to an alternative policy goal(s).

References

Aitamurto, Tanja, and Helene Landemore. 2015. "Five design principles for crowdsourced policymaking: Assessing the case of crowdsourced off-road traffic law in Finland." Journal of Social Media for Organizations 2 (1): 1-20.

Allen, Barbara Ann, Luc Juillet, Gilles Paquet, and Jeffrey Roy. 2001. "E-Governance and government on-line in Canada: Partnerships, people and prospects." Government Information Quarterly 18 (2): 93-104.

Aucoin, Peter 2012. "New political governance in Westminster systems: Impartial public administration and management performance at risk." Governance 25 (2): 177-99.

Auld, Graham, Stephen Renkins, and Benjamin Cashore. 2015. "Transnational private governance between the logics of empowerment and control." Regulation & Governance 9 (2): 108-124.

Bason, Christian, eds. 2014. Design for Policy. Surrey: Grower.

Behavioural Insights Team. 2014. "About Us." Available at: http://www.behaviouralinsights.co.uk/about-us

Bell, Stephen, and Andrew Hind moor. 2009. Rethinking Governance: The Centrality of the State in Modern Society. Cambridge: Cambridge University Press.

Bellafontaine, Teresa. 2013. Innovation Labs: Bridging Think Tanks and Do Tanks. Ottawa: Policy Horizons Canada. Available at: http://www.horizons.gc.ca/eng/content/innovation-labs-bridging-think-tanks-and-do-tanks

Bird, Deanne, Megan Ling, and Katharine Haynes. 2012. "Flooding Facebook: The use of social media during the Queensland and Victorian floods." The Australian journal of Emergency Management 27 (1): 27-33.

Bogason, Peter, and Juliet A. Musso. 2006. "The Democratic Prospects of Network Governance." The American Review of Public Administration 36 (1): 3-18. doi:10.1177/ 0275074005282581.

Bovaird, Tony, and Eric Loeffler. 2013. We're All in This Together: Harnessing User and Community Co-Production of Public Outcomes. Birmingham: Institute of Local Government Studies: University of Birmingham. Available at: http://www.bhamlive3.bham.ac.uk/ Documents/college-social-sciences/government-society/inlogov/publications/chapter-4-bovaird-loeffler.pdf.

Bozeman, Barry, and Patrick Scott. 1992. "Laboratory Experiments in Public Policy and Management." journal of Public Administration Research and Theory 2 (3): 293-313.

Bracken, Mike. 2012. "Why GOV.UK matters: A platform for a digital government." Accessed October 17. Available at: https://gds.blog.gov.uk/2012/10/17/why-gov-uk-matters/

Brainard, Lori A., and John G. McNutt. 2010. "Virtual government-citizen relations: Informational, transactional, or collaborative?" Administration & Society 42 (7): 836-58. doi: 10.1177/ 0095399710386308.

Brandsen, Taco, Victor Pestoff, and Bram Verschuere. 2012. "Co-production as a maturing concept." In New Public Governance, the Third Sector, and Co-Production, edited by Taco Brandsen, Victor Pestoff and Bram Verschuere, 1-12. London: Routledge.

Briassoulis, Helen 2005. "Analysis of policy integration: Conceptual and methodological considerations." In Policy Integration for Complex Environmental Problems: The Example of Mediterranean Desertification, edited by H. Briassoulis. Aldershot: Ashgate.

Cairney, Paul, Kathryn Oliver, and Adam Wellstead. 2016. "To bridge the divide between evidence and policy: Reduce ambiguity as much as uncertainty." Public Administration Review 76: 399-402.

Cashore, Benjamin, and Michael Howlett. 2007. "Punctuating which equilibrium understanding thermostatic policy dynamics in Pacific Northwest forestry." American journal of Political Science 51 (3): 532-51. doi:10.1111/j.1540-5907.2007.00266.x

Castells, Manuel. 2004. The Network Society a Cross-Cultural Perspective. Cheltenham, UK; Northampton, Mass.: Edward Elgar Pub. Available at: http://search.ebscohost.com/ login.aspx?direct=true&scope=site&db=nlebk&db=nlabk&AN=114765

Chancellor of the Duchy of Lancaster. 2007. The Government's Response to The Power of Information: An independent review by Ed Mayo and Tom Steinberg (2007) (No. Cm 7157). London. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/243238/7157.pdf

Charney, Nick. 2014. "Blending Public Sentiment, Data Analytics, Design Thinking and Behavioural Economics." Accessed January 24. Available at: http://www.cpsrenewal.ca/ 2014/01/blending-public-sentiment-data.html

Chief Technology Officer. 2014. "Government as a platform: Services built on a shared core." Accessed September 1. Available at: https://www.gov.uk/service-manual/technology/government-as-a-platform.html

Clarke, Amanda. 2014. Government-Citizen Relations on the Social YJeb: Canada and the United Kingdom, 2006-2013. Oxford, UK: University of Oxford.

--2016. "The Evolution of Government as a Platform." Paper presented at Association for Research on Nonprofit Organizations and Voluntary Action, Washington, DC, November 17-19.

--2017. "Digital Government Units: Origins, orthodoxy and critical considerations for public management theory and practice". Available at: https://papers.ssrn.com/sol3/ papers.cfm?abstract_id=3001188

--. 2018. Opening the Government of Canada: The Federal Bureaucracy in the Digital Age. Vancouver: UBC Press.

Clarke, Amanda, and Jonathan Craft. 2017. "The Twin Faces of Public Sector Design." Paper presented at Toronto Public Policy and Governance Workshop, Toronto, March 23-24.

Clarke, Amanda, and Mary Francoli. 2014. "What's in a name? A comparison of 'open government' definitions across seven Open Government Partnership members." eJournal of eDemocracy and Open Government 6 (3): 248-66.

--. 2017. "Digital Government and Permanent Campaigning." In Permanent Campaigning in Canada, edited by Alex Marland, Anna Esselment, and Thierry Giasson, 241-58. Vancouver: UBC Press.

Clarke, Amanda, and Helen Margetts. 2014. "Governments and citizens getting to know each other? Open, closed, and big data in public management reform." Policy & Internet 6 (4): 393-117. doi:10.1002/1944-2866.PO1377.

Coleman, Stephen, and John Gotz. 2001. Bowling Together: Online Public Engagement in Policy Deliberation. London: Hansard Society.

Coleman, Stephen, and Giles Moss. 2011. "Under construction: The field of online deliberation research." Journal of Information Technology & Politics 9 (1): 1-15. doi:10.1080/ 19331681.2011.635957.

Cook, Thomas D. 2014. '"Big Data' in research on social policy: Point/counterpoint." journal of Policy Analysis and Management 33 (2): 544-47. doi:10.1002/pam.21751.

Craft, Jonathan. 2014. "The promise and paradox of open government in the Harper era." In How Ottawa Spends 2014-2015: The Harper Government--Good to Go?, edited by G. B. Doern and C. Stoney. McGill-Queen's University Press, pp. 209-22. Montreal/Kingston: [Mismatch]

Craft, Jonathan, and Michael Howlett. 2012. "Policy formulation, governance shifts and policy influence: Location and content in policy advisory systems." Journal of Public Policy 32 (2): 79-98.

--. 2013a. "Policy capacity and the ability to adapt to climate change: Canadian and U.S. case studies." Review of Policy Research 30 (1): 1-18.

--. 2013b. "The dual dynamics of policy advisory systems: The impact of externalization and politicization on policy advice." Policy and Society 32 (3): 187-97.

Culver, Keith, and Paul Howe. 2004. "Calling all citizens: The challenges of public consultation." Canadian Public Administration 47 (1): 52-75. doi:10.1111/j.1754-7121.2004. tb01970.x.

Davies, Tim. 2010. The Potential of Open Government Data as a Tool in Democratic Engagement and Reform of Public Services: The Case of Data.gov.uk (MSc Thesis). University of Oxford: Oxford, UK.

Davies, Tim, and Zainab Ashraf Bawa. 2012. "The promises and perils of open government data." The Journal of Community Informatics 8 (2). Available at: http://www.ci-journal.net/ index.php/ciej/article/view/929/955.

Decker, Paul T. 2014. "Presidential address: False choices, policy framing, and the promise of 'Big Data': Presidential Address." Journal of Policy Analysis and Management 33 (2): 252-62. doi:10.1002/pam.21755.

deLeon, Peter 1988. "The contextual burdens of policy design." Policy Studies Journal 17 (2): 297-309.

Del Rio, Pablo. 2014. "On evaluating success in complex policy mixes: The case of renewable energy support schemes." Policy Sciences 47 (3): 267-87.

del Rio, Pablo, Anxo Calvo Silvosa, and Guillermo I. Gomez. 2011. "Policies and design elements for the repowering of wind farms: A qualitative analysis of different options." Energy Policy 39 (4): 1897-908.

Deputy Minister Committee on Policy Innovation. 2014. Deputy Ministers' Committee on Policy Innovation Interim Report to the Clerk (Unpublished Internal Document). Ottawa: Government of Canada.

Digital Engagement Team. 2011. "Introducing the Government Digital Service." Government Digital Service. March 15. Available at https://gds.blog.gov.uk/2011/03/15/introducingthe-government-digital-service/

Digital by Default Service Standard. 2015. March 12. Available at: https://www.gov.uk/service-manual/digital-by-default

Dunleavy, Patrick, and Christopher Hood. 1994. "From old public administration to new public management." Public Money & Management 14 (3): 9-16. doi:10.1080/ 09540969409387823.

Dunleavy, Partick, Helen Margetts, Simon Bastow, and Jane Tinkler. 2005. "New public management is dead: Long live digital-era governance." Journal of Public Administration Research and Theory 16 (3): 467-94.

--. 2006. Digital Era Governance: IT Corporations, the State, and E-Government. Oxford: Oxford University Press.

Dutil, Patrice A. 2010. The Service State: Rhetoric, Reality and Promise. Ottawa: University of Ottawa Press.

Edelenbos, Jurian, and Erik-Hans Klijn. 2006. "Managing stakeholder involvement in decision making: A comparative analysis of six interactive processes in the Netherlands." Journal of Public Administration Research & Theory 16 (3): 417-46.

Farmer, Liz. 2013. "Iceland Writes the World's 1st Crowdsourced Constitution." February. Available at http://www.governing.com/topics/politics/lawmaking/gov-iceland-writes-worlds-1st-crowd-sourced- constitution.html

Fisher, Frank. 1989. "Beyond the rationality project: Policy analysis and the postpositive challenge." Policy Studies Journal 17 (4): 941-54.

Foo, Loke-Min, Darinka Asenova, Stephen Bailey, and John Hood. 2011. "Stakeholder engagement and compliance culture." Public Management Review 13 (5): 707-29.

Freeman, Gary P. 2008. "National styles and policy sectors: Explaining structured variation." Journal of Public Policy 5 (4): 467. doi:10.1017/S0143814X00003287.

Gao, Huiji, Xufei Wang, Geoffrey Barbier, and Huan Liu. 2011. "Promoting coordination for disaster relief--From crowdsourcing to coordination." In Social Computing, Behavioral-Cultural Modeling and Prediction, edited by John Salerno, Shanchieh Jay Yang, Dana Nau, and Sun-Ki Chai. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 197-204. Available at http://www.springerlink.com/index/10.1007/978-3-642-19656-0_29

General Services Administration. 2010. Evangelist for Data.Gov Open Government. August 13. Available at: https://www.fbo.gov/index?s=opportunity&mode=form&id= 29ecfa67711f7d249a5fbf6d341a8055&tab=core&_cview=0

Giest, Sarah. "Big data for policymaking: fad or fasttrack?" Policy Sciences. Online first: 1-16. Available at https://link.springer.com/article/10.1007/sll077-017-9293-l

Global Solution Networks. 2015. Accessed March 12. Available at: http://dontapscott.com/ global-solution-networks/

Government Digital Service. 2012. "Tour of GOV.UK." Available at https://www.gov.uk/tour Government 2.0 Taskforce. 2015. About. Accessed March 13. Available at: http://gov2.net.au/

Grabosky, Peter. 2013. "Beyond responsive regulation: The expanding role of non-state actors in the regulatory process." Regulation & Governance 7 (1): 114-23. doi:10.1111/ j.1748-5991.2012.01147.x.

Hall, Peter A., and Rosemary C. R. Taylor. 1996. "Political science and the three new institutionalisms." Political Studies 44 (5): 936-57. doi:10.1111/j.1467-9248.1996.tb00343.x.

Hood, Christopher. 1983. The Tools of Government. London: Macmillan.

--. 2007. "Intellectual obsolescence and intellectual makeovers: Reflections on the tools of government after two decades." Governance 20 (1): 127-44.

-. 2011. The Blame Game: Spin, Bureaucracy and Self-Preservation in Government. Princeton, NJ: Princeton University Press.

Hood, Christopher C., and Helen Margetts. 2007. The Tools of Government in the Digital Age. Basingstoke: Palgrave Macmillan.

Hou, Yilin, and Gene A. Brewer. 2010. "Substitution and supplementation between co- functional policy instruments: Evidence from state budget stabilization practices." Public Administration Review 70 (6): 914-24.

Howlett, Michael. 2000. "Managing the 'hollow state': Procedural policy instruments and modern governance." Canadian Public Administration 43 (4): 412-31. doi:10.1111/j.1754-7121.2000. tb01152.x.

--. 2009. "Governance modes, policy regimes and operational plans: A multi-level nested model of policy instrument choice and policy design." Policy Sciences 42 (1): 73-89.

--. 2011. Designing Public Policy: Principles and instruments. New York: Routledge.

--. 2014. "From the 'old' to the 'new' policy design: Design thinking beyond markets and collaborative governance." Policy Sciences 47 (3): 187-207.

Howlett, Michael, and Ishani Mukherjee. 2014. "Policy design and non-design: Towards a spectrum of policy formulation types." Politics and Governance 2 (2): 57-71.

Howlett, Michael, and Jeremy Rayner. 2007. "Design principles for policy mixes: Cohesion and coherence in 'New Governance Arrangements'." Policy and Society 26 (4): 1-18.

--. 2013. "Patching vs packaging in policy formulation: Assessing policy portfolio design." Politics and governance 1 (2): 170-82.

Howlett, Michael, Adam Wellstead, and Jonathan Craft. 2017. Policy Work in Canada: Professional Practices and Analytical Capacity. Toronto: University of Toronto Press.

Janssen, Marijn, and Eisa Estevez. 2013. "Lean government and platform-based governance--Doing more with less." Government Information Quarterly 30: S1-S8. doi:10.1016/ j.giq.2012.11.003.

Jones, Bryan D. 2003. "Bounded rationality and political science: Lessons from public administration and public policy." journal of Public Administration Research and Theory 13 (4): 395-412.

Jordan, Andrew, and Elah Matt. 2014. "Designing policies that intentionally stick: Policy feedback in a changing climate." Policy Sciences 47 (43): 227-47.

Jordan, Andrew, Rudiger K. W. Wurzel, and Anthony Zito. 2005. "The rise of 'new' policy instruments in comparative perspective: Has governance eclipsed government?" Political studies 53 (3): 477-96.

Joshi, Anuradha, and Mick Moore. 2004. "Institutionalised co-production: Unorthodox public service delivery in challenging environments." Journal of Development Studies 40 (4): 31-49. doi:l 0.1080/00220380410001673184.

Kay, Adrian. 2007. "Tense layering and synthetic policy paradigms: The politics of health insurance in Australia." Australian journal of Political Science 42 (4): 579-91.

Keen, Justin, Richard Calinescu, Radu Paige, and John Rooksby. 2012. "Big health data: Institutional and technological challenges." Presented at the Internet, Politics, Policy 2012: Big Data, Big Challenges, Oxford, UK. Available at: http://microsites.oii.ox.ac.uk/ ipp2012/sites/microsites.oii.ox.ac.uk.ipp2012/files/Keen%20et%20al%20Big%20Data%20OII. pdf

Khan, Jordan. 2011. "Guideline for digital oblivion." November 22. Available at: http:// restraint.org/politics/2956/guideline-for-digital-oblivion/

King, Dominic, Felix Greaves, Christopher Exeter, and Ara Darzi. 2013. "'Gamification': Influencing health behaviours with games." JRSM 106 (3): 76-78. doi:10.1177/ 0141076813480996.

Klijn, Erik-Hans. 2002. "Governing networks in the hollow state: Contracting out, process management or a combination of the two?" Public Management Review 4 (2): 149-65.

Kodrich, Kris, and Melinda Laituri. 2011. "Making a connection: Social media's key role in the Haiti earthquake." journal of Communication and Computer 8: 624-27.

Lejano, Raul P., and Savita Shankar. 2013. "The contextualist turn and schematics of institutional fit: Theory and a case study from Southern India." Policy Sciences 46 (1): 83-102.

Linder, Stephen H., and B. Guy Peters. 1989. "Instruments of government: Perception and contexts." Journal of Public Policy 9 (1): 35-58.

--. 1990. "Policy formulation and the challenge of conscious design." Evaluation and Program Planning 13: 303-11.

Lindquist, Evert A. 1992. "Public managers and policy communities: Learning to meet new challenges." Canadian Public Administration 35: 127-59.

Majone, Giandomenico. 1975. "On the notion of political feasibility." European Journal of Political Research 3 (2): 259-74.

Manyika, James, Michael Chui, Brad Brown, Jacques Bughin, Richard Dobbs, Charles Roxburgh, and Angela Hung Byers. 2011. Big Data: The Next Frontier for Innovation, Competition, and Productivity. McKinsey Global Institute.

Marando, Dylan, and Jonathan Craft. 2017. "Digital era policy advising: Clouding ministerial perspectives?" Canadian Public Administration 60 (4): 498-516.

March, James G., and Johan P. Olsen. 2006. "The logic of appropriateness." In Handbook of Public Policy, edited by Martin Rein, Michael Moran, and Robert E. Goodin. Oxford: Oxford University Press, pp. 689-708.

Margetts, Helen Zerlina. 2011. "Experiments for public management research." Public Management Review 13 (2): 189-208. doi:10.1080/14719037.2010.532970.

Margetts, Helen, and Patrick Dunleavy. 2013. "The second wave of digital-era governance: A quasi-paradigm for government on the Web." Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 371 (1987): 20120382. doi:10.1098/ rsta.2012.0382.

May, Peter J. 2003. "Policy design and implementation." In Handbook of Public Administration, edited by B. Guy Peters and J. Pierre. Beverly Hills: Sage Publications, pp. 223-33.

May, Peter J., Bryan D. Jones, Betsi E. Beem, E. A. Neff-Sharum, and Melissa K. Poague. 2005. "Policy coherence and component-driven policymaking: Arctic policy in Canada and the United States." Policy Studies Journal 33 (1): 37-63.

McAfee, Andrew, and Erik Brynjolfsson. 2012. "Big Data: The management revolution." Harvard Business Review 90 (10): 60-68.

McCall, Rod, Vincent Koenig, and Martin Kracheel. 2013. "Using gamification and metaphor to design a mobility platform for commuters." International Journal of Mobile Human Computer Interaction 5 (1): 1-15. doi:10.4018/jmhci.2013010101.

Meijer, Albert, Kees Boersma, and Pieter Wagenaar, eds. 2009. ICTs, Citizens and Governance: After the Hype! Amsterdam: IOS Press.

Mergel, Ines. 2014. "Opening government: Designing open innovation processes to collaborate with external problem solvers." Social Science Computer Review. 33 (5): 599-612. doi: 10.1177/0894439314560851.

Mergel, Ines, and Kevin C. Desouza. 2013. "Implementing open innovation in the public sector: The case of challenge.gov." Public Administration Review 73 (6): 882-90. doi:10.2307/ 42003136.

Miller, Laura, and Andy Williamson. 2008. Digital Dialogues--Third Phase Report: August 2007-August 2008. London: Hansard Society. Available at: http://www.hansardsociety. org.uk/wp-content/uploads/2012/10/Digital-Dialogues-Third-Phase-Report-2008.pdf

MindLab. n.d. About MindLab. Available at: http://mind-lab.dk/en/om-mindlab/

Noveck, Beth Simone. 2009. Wiki Government: How Technology Can Make Government Better, Democracy Stronger, and Citizens More Powerful. Washington, DC: Brookings Institution Press.

--. 2015. Smart Citizens, Smarter State: The Technologies of Expertise and the Future of Governing. Cambridge, Mass.: Harvard University Press.

Open Data Institute. 2012. "Open Data Institute: Business Plan 2012-2017." Available at: http://theodi.org/about-us

Open Government Partnership. 2012. "National "Open Government Camp" in Denmark." September 18. Available at: http://www.opengovpartnership.org/blog/blog-editor/ 2012/09/17/national-open-government-camp-denmark

O'Reilly, Tim. 2011. "Government as a platform." Innovations 6: 13-40.

Organization for Economic Co-operation and Development. 2015. "Public Sector Innovation and e-Government." Available at: http://www.oecd.org/internet/public-innovation/

Orszag, Peter R. 2009. "Open government directive." Accessed December 8. Available at: http://www.whitehouse.gov/open/documents/open-government-directive

Osborne, David, and Ted Gaebler. 1992. Reinventing Government: How the Entrepreneurial Spirit Is Transforming the Public Sector. New York, NY: Plume.

Osborne, Stephen P. 2006. "The new public governance?" Public Management Review 8 (3): 377-87.

Painter, Martin, and Jon Pierre. 2005 "Unpacking policy capacity: Issues and themes." In Challenges to State Policy Capacity: Global Trends and Comparative Perspectives, edited by M. Painter and J. Pierre. New York: Palgrave Macmillan, pp. 1-18.

Papadopoulos, Yannis, and Philippe Warin. 2007. "Are innovative, participatory and deliberative procedures in policy making democratic and effective?" European journal of Political Research 46 (4): 445-72. doi:10.1111/j.1475-6765.2007.00696.x.

Pestoff, Victor Alexis, Taco Brandsen, and Bram Verschuere. 2012. New Public Governance, the Third Sector and Co-production. New York: Routledge.

Peters, Guy B. 2011. "Steering, rowing, drifting, or sinking? Changing patterns of governance." Urban Research & Practice 4 (1): 5-12. doi:10.1080/17535069.2011.550493.

--. 2014. "Is governance for everybody?" Policy and Society 33: 301-306

Phillips, Susan D. 2006. "The intersection of governance and citizenship in Canada: Not quite the third way." IRPP Policy Matters 7 (4): 2-33.

Pollitt, Christopher, and Geert Bouckaert. 2011. Public Management Reform: A Comparative Analysis: New Public Management, Governance, and the Neo-Weberian State. 3rd ed. Oxford, New York: Oxford University Press.

Phillips, Susan D., and Michael Orsini. 2002. Mapping the Links: Citizen Involvement in Policy Processes. Ottawa: Canadian Policy Research Networks. Available at: http://cprn.org/ documents/ACFRJK8po.PDF

Pierre, Jon. 2005. Debating Governance. Oxford: Oxford University Press.

Premfors, Reune. 1981. "National policy styles and higher education in France, Sweden and the United Kingdom." European journal of Education 16(16): 253-62.

Public Accounts Committee. 2012. Implementing the Transparency Agenda. London: House of Commons. Available at: http://www.publications.parliament.uk/pa/cm201213/cmselect/cmpubacc/102/10202.htm

Rhodes, R. A. W. 1994. "The hollowing out of the state: The changing nature of the public service in Britain." The Political Quarterly 65 (2): 138-51.

Richardson, Jeremy, Gunnel Gustaffson, and Grant Jordan. 1982. "The concept of policy style." In Policy Styles in Western Europe. Abingdon: Routledge, pp. 1-16.

Roberts, Nancy C. 2011. "Beyond smokestacks and silos: Open-source, web-enabled coordination in organizations and networks" Public Administration Review 71 (5): 677-93. doi: 10.1111/j. 1540-6210.2011.02406.x.

Rocco, Philip, and Chloe Thurston. 2014. "From metaphors to measures: Observable indicators of gradual institutional change." journal of Public Policy 34 (1): 35-62.

Rutter, Tamsin. 2014. "Want to win the general election? First fix the civil service--John Healey." The Guardian, April 24. Available at: http://www.theguardian.com/public-leaders-network/2014/apr/24/governup- john-healey-civil-service- reform?CMP=new_1194

Salamon, Lester, eds. 2002. The Tools of Government: A Guide to the New Governance. Oxford; New York: Oxford University Press.

Schneider, Anne, and Helen Ingrahm. 1990. "The behavior assumptions of policy tools." journal of Politics 52 (2): 511-29.

Sidney, Mara. 2007. "Policy formulation: Design and tools." In Handbook of Public Policy Analysis: Theory, Politics, and Methods, edited by F. Fischer, G. J. Miller, and M. Sidney. Boca Raton: CRC Press, pp. 79-87.

Simon, Herbert. A. 1991. "Bounded rationality and organizational learning." Organization Science 1 (2): 125-34.

Skodvin, Tora, Anne Therese Gullberg, and Stine Aakre. 2010. "Target-group influence and political feasibility: The case of climate policy design in Europe." Journal of European Public Policy 17 (6): 854-73.

Snijders, Chris, Uwe Matzat, and Ulf-Dietrich Reips. 2012. "'Big Data': Big gaps of knowledge in the field of Internet." International Journal of Internet Science 7: 1-5.

Stead, Dominic, Geerlings, Harry, and Meijers, Evert. 2004. Policy Integration in Practice: The Integration of Land Use Planning, Transport and Environmental Policy-Making in Denmark, England and Germany. Delft: Delft University Press.

Steinberg, Tom. 2012. "Governments don't have websites: Governments are websites." Accessed March 13. Available at: https://www.mysociety.org/2012/07/18/governments-dont-have-websites-governments-are- websites/

Thaler, Richard H. 2008. Nudge: Improving Decisions About Health, Wealth, and Happiness. New Haven: Yale University Press.

Thelen, Kathleen. 1999. "Historical institutionalism in comparative politics." Annual Revieiv of Political Science 2 (1): 369-404. doi:10.1146/annurev.polisci.2.1.369.

Thelen, Kathleen 2004. How Institutions Evolve: The Political Economy of Skills in Germany, Britain, the United States, and Japan. Cambridge: Cambridge University Press.

Tiernan, Anne. 2011. "Advising Australian federal governments: Assessing the evolving capacity and role of the Australian public service." Australian Journal of Public Administration 70 (4): 335-6.

Torgersen, Douglas. 1985. "Contextual orientation in policy analysis: The contribution of Harold D. Lasswell." Policy Sciences 18 (3): 240-52.

Weyland, Kurt G. 2006. Bounded Rationality and Policy Diffusion: Social Sector Reform in Latin America. Princeton, NJ: Princeton University Press.

Amanda Clarke is Assistant Professor, School of Public Policy and Administration, Carleton University, Ottawa, Ontario. Jonathan Craft is Assistant Professor, Department of Political Science, School of Public Policy and Governance, University of Toronto.
COPYRIGHT 2017 Institute of Public Administration of Canada
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2017 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Clarke, Amanda; Craft, Jonathan
Publication:Canadian Public Administration
Geographic Code:1CONT
Date:Dec 1, 2017
Words:9114
Previous Article:Understanding governance in the digital era: An agenda for public administration research in Canada.
Next Article:Digital era policy advising: Clouding ministerial perspectives?
Topics:

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters