Printer Friendly

Assessing the performance of community systems for children.

An effective health system promotes and improves the health of a population. Services for children should be organized at the community level to fulfill community-wide goals of delivering an array of health services that permit children to develop into healthy, productive, well-functioning adults. From a community perspective, this includes services that are provided by personal medical care providers such as physicians, hospitals, and managed care organizations, and by public health departments and other community providers. To determine whether community systems for children are organized to allocate resources efficiently in improving children's health, methods are needed to capture system indicators of system performance, and to stimulate improvement at the system level. As new organizational forms and financing arrangements evolve in the U.S. health system, measuring systems quality for children is becoming more important.

Significant work is under way nationally to define, measure, and improve the quality of healthcare for children, yet much of the focus has been on provider practices and health plan indicators of quality. Measuring the performance of the health system from the perspective of children's health brings new challenges. This is because the focus on the systems' contribution to population health is a relatively new research area; because community systems have different goals and functions than providers and health plans; and because of the complexity of the current child health system. Community systems of health services for children are structurally fragmented and organizationally complex, and they lack formal hierarchies of accountability due to their broad scope.

The purpose of this article is to begin to define the essential components of community systems and to determine the means to accomplish the measurement of community system performance. We present concepts of community systems and system quality for children that can help direct evaluations of system effectiveness. We describe key dimensions and features of systems necessary to provide continuums of care for children and families. The framework that we propose is an adaptation of the clinical quality-accountability approach, and is largely based on Donabedian's structure, process, and outcome approach to evaluating quality in healthcare. In describing key dimensions and features of systems necessary to provide continuums of healthcare for children, we draw from Aday's notions of community continuums of care, and from Starfield's notions of community-oriented primary care systems. We apply these concepts to the components of community systems for children (including health programs, networks, and systems) and describe the structural, organizational, financing, and accountability dimensions that must be measured. In our discussion of ways in which system performance can be measured and improved, we draw from the Institute of Medicine model for community health improvement and performance monitoring. We discuss possible approaches to using quality measures in characterizing system performance and in implementing quality improvement.

Finally, while it appears that measuring system performance can increase the effectiveness of services for families and the broader community, there are conceptual, organizational, and methodological challenges requiring further research and development. The proposed research agenda focuses on what needs to be done to develop community system performance measurement. Our approach builds on frameworks and techniques developed to measure quality of care at the provider and the health plan level, recognizing that further work is necessary to adapt these techniques for community systems. The agenda focuses on ways of measuring children's health and on the conceptual work that is needed to refine a model of community system performance. The agenda also describes the research needed to measure system effectiveness, to develop community practice guidelines, to develop measures of system quality, and to design effective quality improvement processes in communities.


While traditional morbidity and mortality statistics are improving for children, problems such as child abuse, drug abuse, behavioral and developmental disorders, and the consequences of family and neighborhood violence are "new morbidities" for children (Haggerty, Roghmann, and Pless 1975; Sells and Blum 1996; Singh and Yu 1996; Schorr 1988). Research shows that these complex health conditions tend to cluster in a relatively small proportion of children (Starfield 1992a) and often require a broad array of medical as well as wrap-around services. For example, a child who has been abused and placed in foster care may have multiple medical and mental health problems requiring services from health, mental health, and child welfare systems (Halfon, Mendonca, and Berkowitz 1995; Dubowitz et al. 1992).

Substantial evidence exists of the effectiveness of systems designed to address specific health problems or specific subgroups of children. For example, the implementation of regionalized perinatal service networks has reduced infant mortality (Paneth 1990; Paneth et al. 1982; Brown 1988); multidisciplinary early intervention programs targeted to high-risk families show that continuums of care can improve children's and families' functioning cost-effectively (Olds et al 1993; Farran 1992; Barnett and Escobar 1992); and mental health demonstration projects have demonstrated that the organization of services can improve health and functional outcomes in targeted populations (Salzer and Bickman 1997; Schlenger et al 1992; Rosenblatt and Attkisson 1992a, b). For the general population of children as well as for adults, consensus that organized systems of healthcare delivery can improve health outcomes continues to fuel health system changes, such as the rapid expansion of managed care and the consolidation of provider networks.


Community systems that function well are considered to be particularly important for children because of the broad determinants of their health and because of the multiple providers and institutions involved in their care (Schorr 1988). Children's health is affected by biological, social, and economic conditions (Schor 1995; Schor and Menaghan 1995; Palfrey 1994; Jameson and Wehr 1993; Garbarino 1992). Models of health determinants identify multifactorial "pathways" to individual health outcomes based on the confluence of individual, family, and environmental factors (Evans and Stoddart 1994; Halfon et al. 1997). Although children's environments can pose health risks, the timely provision of medical and other health services can respond to these health risks and "new morbidities" (whether lead exposure, child neglect, or prematurity) and can lead to better physical and developmental outcomes.

Children who are generally healthy as well as children with complex conditions often must have access to multiple providers and institutions, in both the private and the public sector, for health promotion and treatment services. This is in part because the unique vulnerabilities of children, the failure of markets to meet the unique healthcare needs of children, and the public's interest in promoting their welfare has led to the evolution of a fragmented system of programs and financing for education and health (Halfon and Hochstein 1997). Public programs that target the "exceptional child," for example, developed incrementally, resulting in multiple layers of programs designed to fill gaps in coverage or need that were unmet in the private sector (Schlesinger and Eisenberg 1990). These programs are poorly integrated and often overlap in ways that duplicate services. There are programs for personal health services, such as immunizations, as well as for community and population-based services, such as family preservation and other child welfare services, early intervention, and health promotion (e.g., accident prevention) (Halfon, Inkelas, Wood 1995; Schlesinger and Eisenberg 1990). Failure of one component in this service "continuum" reduces the effectiveness of other components and undermines the system's capacity to promote and improve children's health.

The response to lead exposure in a community typifies this fragmentation of related services. Typically, the primary care provider assesses a child's lead level and refers the child to treatment when necessary; community programs such as the Women, Infants, and Children Supplemental Food Program (WIC) conduct family education on the prevention of lead poisoning; and public health and housing agencies are responsible for home assessment and abatement activities. When one or more of these activities does not take place, the resources allocated to the activities become less effective in achieving the desired health outcome.

Ideally, community systems should be organized to respond to the determinants of children's health by incorporating the range of health and related services that many children require. A community system perspective for children must encompass the variety of programs, networks, and provider systems within the community that deliver these services.

Focus on Families. Community systems are unique in their responsibility for families as well as for children's overall health and function. The primary product of a community's services should be child and family health and well-being. In thinking about how systems operate and how they could function better, the goal is to guide systems toward developing the resources and the functions necessary for families to receive quality care. A family perspective is useful in examining community systems because, although structurally there are multiple programs, networks, and systems serving families within a community, the family ideally should encounter a seamless system of care that is responsive to and supportive of their healthcare needs. Moreover, because families are the only observers with a lens on each system component, their input regarding the quality and effectiveness of the system should be highly valued.

Research on health systems frequently takes a "top-down" approach in identifying what is effective; the alternative is to begin with what families need and encounter in the community. Lessons learned from the development of community systems for children with special healthcare needs suggests that a family-centered approach to the organization and evaluation of care is essential, and it has been used as an organizing principle for the design of systems for this population (Brewer et al. 1989).

Along these lines, principles have been developed by a number of groups to guide the development of community systems for children. For example, the Maternal and Child Health Bureau (MCHB) has developed a set of global measures designed to help communities track progress in system development (1995). These measures are based on five principles, developed for children with special health needs by a national collaborative group ("Communities Can"), that focus on the ability of families (1) to gain access to the system; (2) to receive high-quality services; and (3) to have access to services, with involvement in the services they receive. In addition, these measures focus (4) on the system coordinating care as well as (5) on integrating system functions (MCHB 1995). Concepts of family-centeredness also have been used by groups, such as "New England Serve," to design evaluations of managed care for children with special healthcare needs, with an emphasis on family surveys of experiences and satisfaction with care (New England Serve 1997).

Defining Community Systems for Children

Because the concept of community systems has been described differently depending on the disciplinary perspective, there is no single definition of a community system for children. For example, for perinatal services, the need for regionalized tertiary care may require a broad definition of community. For early intervention services, a broad definition of community may be appropriate to public planners so that accessibility is assured, even as families and specific programs focus on the services and organization within much smaller neighborhood boundaries. We offer working definitions for community systems and their components to guide the conceptual development of key areas of system performance.

The Institute of Medicine (IOM) defines a community as individuals with shared geography and shared affinity, who organize to address commonly held concerns collectively (IOM 1997; Labonte 1988; Patrick and Wickizer 1995). Shortell et al. (1993) offer a definition of an organized health system as "a network of organizations that provides or arranges to provide a coordinated continuum of services to a defined population and is willing to be held clinically and fiscally accountable for the outcomes and the health status of the population served." A system is generally defined as an array of service components that are bounded in some way, and that operate using a set of internal rules or incentives.

One difficulty in defining community systems for children stems from the multiple and conflicting boundaries used by agencies for service planning. Another difficulty is that a community system for children is actually an aggregation of programs, networks, and mission-specific subsystems with differing objectives. If the structural ties implicit in these definitions existed for community systems for children, the approach to measuring system quality would still be technically difficult but would not be as conceptually complex as it currently is. Understanding the components of care so that quality measures can be developed is more straightforward in clinical care, where rules can be operationalized in protocols and practice guidelines. In contrast, few formal rules dictate how community systems operate. A third difficulty is that definitions of health systems often do not encompass the full scope of health services for children, which includes medical services as well as health-related services provided outside of health plans (Halfon and Hochstein 1997; Halfon et al. 1996). Many community services have been developed for particular problems of children that are not well accounted for in the regular health system, and for the children most vulnerable to poor outcomes.

Services for children have been organized within communities in several different ways. A service delivery program is an organized array of multidisciplinary services that targets a particular condition or outcome. One example of service delivery programs is the early intervention programs for low-birthweight infants that combine healthcare with parent education and with family support services such as the Infant Health and Development Program, or IHDP (1990). Another example is the "one-stop-shopping" prenatal/perinatal care centers that provide (or co-locate) prenatal care, nutrition education, counseling, and other social and income support for mothers and infants (Brown 1988).

A service delivery network is a regionalized array of providers, organized to assure access to specific services to the entire community. One example of service delivery networks is a regional perinatal network of intensive care nurseries linked with prenatal and postnatal transportation services. A perinatal network assigns mothers/infants to one of three neonatal facility levels based on their need for basic care, intermediate services, or intensive specialty care (Guyer et al. 1995).

Finally, a service delivery system refers to an organized system of care that provides a continuum of services in a particular area. Multiple service programs and networks are organized (to varying degrees) to achieve a broad mission such as improving the health, education, or safety of children. For example, a child protective services system can be organized to provide a continuum of preventive, family maintenance, and foster care services designed to protect children's well-being. Similarly, the child health system can be described as a collection of delivery systems that are related by the services they provide but that do not necessarily function as a continuum for a particular child or family. The child health system actually is composed of different types of care (preventive, treatment, tertiary, and rehabilitative) with varying degrees of coordination. "Subsystems" may be set up within a service delivery system to focus on a particular group of children; an example includes the Title V Children with Special Health Needs (CSHN) agencies that organize networks of specialty and rehabilitative services for children with complex chronic health conditions. This underscores the need to coordinate services across a continuum of intensity based on the child's need.

Thus the building blocks for a functional community system for children includes service delivery programs, service delivery networks, and service delivery systems. It is their characteristics and relationships that, taken together, contribute to its overall performance.


System Objectives: Effectiveness and Efficiency

Measuring the performance of specific programs, or the quality of specific processes of care, differs somewhat from measuring performance of community systems. An important distinction between measuring quality of health services and the performance of community systems is the added focus on efficiency as well as effectiveness. These concepts are developed more fully by Aday et al. (1993) as objectives of any health system.

Effectiveness is measured by the outputs produced by particular delivery programs, networks, or systems. For example, how many children are brought up to date with immunizations given the resources placed in public health immunization services? Do school-based clinics established to provide access to services reach their target population? Efficiency is measured by (1) ways in which resources are allocated across or within programs, networks, or systems to achieve population goals; and (2) whether those resources are used in the best possible way to achieve the desired outcome (Aday et al. 1993). Are the resources invested in public health immunization services targeting the appropriate population of children, and would using those resources for incentives in the private delivery systems achieve better outcomes? Could resource pooling across mental health, health, and early intervention programs produce better case management services for families? Thus, system-level measures indicate whether resources are allocated for children's services in the community in their "best possible use" to produce positive child health outcomes.

System Attributes: Structure, Process, and Outcomes

A framework that is used frequently in health services research to measure quality of care was developed by Donabedian (1982). The Donabedian framework uses three types of domains - structural, process, and outcome measures - to describe resource investments and to represent functional relationships between invested resources and health outcomes. Resources such as providers, hospitals, and programs and their relationships constitute the structure of the delivery system. Characteristics of system structure can be expanded to include population needs and personal resources (Aday 1993). The delivery of clinical services represents the process of care. Both intermediate outcomes (the provision of appropriate services to the population) and health outcomes (health status, function) represent the health system outcomes.

This paradigm is useful for describing quality within a particular array of services. For example, in evaluating the quality of immunization delivery in a practice, one can identify structural measures (provider training, office computer systems, mechanisms to obtain and exchange immunization records), process measures (children 0 to 5 years screened for immunization status, active recall system, education provided to parents), and outcomes (age-appropriate immunization levels of children in the practice) that describe overall quality at the provider level. It is also useful for evaluating the quality of programs and of subsystems that focus on specific problems. To evaluate the quality of regional perinatal service networks, structural measures could include regionalization (geographic coverage, time/distance to facilities); process measures could include how well mothers were referred to appropriate settings and the rates and timing of in utero transfers; and outcomes could include infant morbidity and mortality.

Thus, for the individual programs, subsystems, and networks that are components of community systems for children, the Donabedian framework is useful for developing specific indicators. It can guide the design and organization of evaluations of quality when both the outcome and the infrastructure that affects that outcome can be clearly identified.

Examples: Evaluating Systems for Specific Populations and Community-Oriented Services. Application of the Donabedian framework to evaluate more complex systems demonstrates that the structure, process, and outcomes can be useful in conceptualizing systems-level quality as well. Aday has elaborated a perspective for evaluating the quality of care provided to vulnerable populations, such as high-risk mothers and families with problems of child abuse and neglect (1993). This population-based perspective describes the arrays of services most relevant for particular needs. Aday describes normative attributes for systems that are defined less by service (e.g., primary care) than by population (e.g., people with chronic disabling conditions). For example, Aday describes how one might go about measuring the quality of the system encountered by high-risk infants and mothers. System attributes include the availability of neonatal intensive care units. Process attributes include the timing and content of prenatal care services. Outcomes that can be used to gauge system performance include birth outcomes, and infant morbidity and mortality.

Structure, process, and outcome indicators also have been used to elaborate key functions and an evaluative framework for primary care (Starfield 1992b). Structural components of the system include the range of services,and the personnel, organization, continuity/information systems, accessibility, financing, and governance. Process components include the provision of care (e.g., diagnosis and disease management) and the receipt of care (e.g., utilization and satisfaction). Outcome components are the attributes of health status, including perceived well-being, activity, disease, and resilience.

Starfield's framework for evaluating primary care systems provides important principles for assessing community systems. In addition to identifying key components of the health system, Starfield defines four attributes of primary care that describe its function and quality: first-contact care, longitudinality, coordination, and comprehensiveness. These are normative attributes of primary care (IOM 1978; Parker 1976) for which there is at least some empirical evidence that supports their value. These attributes are incorporated in the Maternal and Child Health Bureau's definition of primary care for children and adolescents (Johansen, Starfield, and Harlow 1994; Grason and Wigton 1995). A community can assess the quality of its primary care system by measuring the system's performance for each attribute within the structure, process, and outcome framework. For example, longitudinality could be assessed through the measurement of provider transiency in a community (structure), longevity of patient-provider relationships (process), and improved patient outcomes (or patient satisfaction) for individuals with a stable patient-provider relationship (outcome).

By expanding its focus to contributions of multiple (and autonomous) providers and institutions in achieving a population objective, this framework moves closer to evaluating the quality of a community system.

System Functions: Community Practice Guidelines and Accountability

While structure and process measures are useful as screening mechanisms for system performance (APHA 1996), the complexity of relationships in a community system are not easily captured in such measures. Community systems for children are multilayered arrays of networks and programs that have direct, indirect, and interactive effects on child health outcomes. The functional relationships across local programs and networks must be included in evaluating the quality of community systems. For community system measurement it may be necessary to adapt the approach taken in evaluating clinical care, which captures functions across providers,

Clinical practice guidelines are defined by the IOM as "systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances" (Field and Lohr 1990, 1992). A practice guideline includes an expected patient outcome, one or more clinical indicators of whether this outcome is achieved, and a protocol for achieving the outcome (and indicator) (Merritt et al. 1997). Identification of responsibility for specific elements of the protocol is also essential in practice guidelines. Practice guidelines are used to convey key elements of appropriate care, and they also can be applied to reduce practice variation, thereby reducing health costs and/or increasing the effectiveness of care (Merritt et al. 1997). In general, practice guidelines include "critical pathways" (key elements of protocols) that proscribe the timing and sequencing of interventions to improve the quality of care (Coffey 1992; Merritt 1997).

Clinical practice guidelines may have a counterpart in system function. System-level (population-based) outcomes and the indicators used to measure success in achieving those outcomes can be used descriptively by a community (as part of a needs assessment or planning effort). The system-level "protocol" associated with a given outcome, which might include a specific strategy/plan for achieving that outcome as well as designated accountability for particular aspects of the strategy/plan, could evolve into a continuous quality improvement process. The practice guideline concept is particularly useful for understanding community systems quality since the protocol can incorporate the structural and process features across systems and networks that are involved in producing a particular outcome. These multiple functions must be incorporated into a system-level evaluation of overall performance.

Critical pathways are important for community systems not only because they represent key decisions, but because they represent hand-offs from one responsible group to another. For example, in a critical pathway of pediatric cardiac surgery, important hand-offs occur in patients' transitions from the operating team to the intensive care unit team. It is where these transitions occur that quality of care is easily compromised (e.g., when a failure in coordination leads to the wrong concentration of medicines given). Similar transitions also occur in community systems; for example, a child in foster care receives healthcare from a healthcare system that needs to transmit important information about the child's mental health status to the child welfare worker who is making placement decisions based on this information. Delays or interruptions in this information transfer would indicate low quality if operationalized as a performance measure.

Because of the scope and complexity of community systems, it may be necessary to simplify the measurement process by focusing on essential functional relationships. We also recognize that approaches to systems analysis in business management and in other fields also involve the notion of critical pathways (Harry et al. 1990). Quality improvement techniques developed in these fields may provide lessons in conceptualizing key system pathways and in designing methods to measure these pathways.

We must also recognize that the lack of common objectives and the historic autonomy of system components complicate performance measurement for children's services. Organized health systems, school systems, and developmental service systems (among others that serve children) are administratively distinct, although many of the divisions are imposed more by financing streams and tradition than by function. The division of roles across institutions and institutional autonomy means that no institution by itself is accountable for the health status of children in the community (Hughes et al. 1996; Nerenz 1996). Linkages among institutions are so important because they help to instill functionality and responsibility. While child outcomes are influenced by the characteristics of community systems, it is difficult to attribute complete responsibility for health outcomes or even health planning to particular institutions. As a result, an array of institutions share responsibility for assessing and improving the performance of community systems for children. At the same time, because community systems for children are composed of autonomous subsystems, accountability for system efficiency is shared, and it is often shared between private and public institutions. Specifying accountability, part of the performance measurement process, is an ongoing challenge.


In addition to understanding how system performance measurement can capture functional relationships, it is necessary to identify the system functions that are important for children. Recognizing that community systems operate based on structural features and functional relationships, we now describe several dimensions of systems quality for children that can guide the evaluation of these systems. Community systems can be represented by key dimensions of structure/availability, organization, financing, and accountability [ILLUSTRATION FOR FIGURE 1 OMITTED].

Although this is not a formal typology, the dimensions are useful in thinking about quality measurement in a complex system of programs and networks. These dimensions of quality are consistent with Starfield's primary care model (1992b), as well as with the population-based focus of Aday (1993). Within each of these dimensions, specific attributes describe the particular organization and function of the system. The attributes in each dimension are drawn from the literature on children's healthcare, including standards of care for specific groups of children (e.g., Child Welfare League of America 1988); studies of access to care (Halfon, Inkelas, and Wood 1995; Dutton 1986); and principles for systems developed by federal and state agencies that serve children (Green 1994; Select Panel for the Promotion of Child Health 1981). In evaluating a community system, each attribute within these dimensions of quality would be examined. Once operationalized as quality measures or incorporated within practice guidelines as indicators of critical pathways, each attribute could be examined, and performance targets could be established in communities.

Measures of Community System Structure and Availability

Measures of system structure and availability include the characteristics of providers and their tools, resources, and organizational settings. This dimension of community systems encompasses resources across delivery systems and networks. Key attributes include the scope of services, service capacity, and accessibility.

Scope. Scope refers to the breadth and depth of the different types of services provided in the community. The array of services, ranging from primary care to early intervention, mental health, and family support services, among many others, defines scope. For the community overall, scope refers to the continuum of services for children that is available locally (Aday 1993). Comprehensive services for abusing families, for example, would include medical, mental health, and family support services (e.g., parenting, job, literacy, and home management skills training), all of which would be adequately staffed by trained personnel (Child Welfare League of America 1988). Not only must specific services be offered, but also the adequate training of providers who deliver these services, to achieve the desired health outcome (Jameson and Wehr 1993). For individual children and families, scope refers to the comprehensiveness of the services for which the child is eligible.

Accessibility. Accessibility refers to the location, distribution, and capacity for services within the community (Penchansky and Thomas 1981). For the community overall, aspects of accessibility include co-location of services, placement of "centers of excellence" in the community, transportation arrangements, and community outreach to identify and serve those eligible for services. For children and families, accessibility refers to the presence or absence of barriers to care, whether geographical, financial, or cultural (Halfon, Inkelas, and Wood 1995; Dutton 1986; Penchansky and Thomas 1981).

Measures of Community System Organization

The organization dimension focuses on system-level function. Organization refers to the relationships between the different delivery systems, networks, and providers in the community system. Some of these relationships may be hierarchical, such as from agency to health plan (Ireys 1994; Durch 1994). Other relationships are cross-sectoral rather than hierarchical, such as relationships between programs and networks, or between programs and systems. An example that illustrates cross-sectoral relationships for a specific population is treatment for asthma; communication between a school system, health plans, and providers can help to organize the type and location of the different types of services that children with asthma may need. Memoranda of understanding (MOUs) that divide responsibilities among agencies or programs serving the same population of children are an example of system organization.

Coordination. Coordination refers to the ways in which programs, providers, and networks exchange information, coordinate activities, and share resources for the care of individuals. Moving children and families effectively through a system requires coordination to ensure that they have access to appropriate services (Kahn and Kamerman 1992). To be successful, this process also requires communication about the needs, services, and plans for the children and families. Coordination is a key dimension for clinical care as well as for system function. While coordination at the individual level might involve a patient's physical transition from one responsible hospital unit to another, coordination at the systems level may involve a change in responsibility for rehabilitation and treatment services from the hospital to the school system. The necessary information flows may be achieved through hand-held "passports" or through comprehensive, accessible, and secure data systems that permit tracking.

Integration. Service integration describes the sharing of resources and infrastructure within and across agencies and institutions based on the shared goal of improving health outcomes (Melaville and Blank 1991). Integration is different from care coordination because it focuses on relationships designed to improve system efficiencies, as opposed to relationships designed to help individual families through the system. The level of coordination within a community system can range from the absence of communication across programs or systems, to the full integration of services with coordinated application procedures, program consolidation, and pooled funding (Kahn and Kamerman 1992).

Measures of Community System Financing

The financing dimension of community systems refers to the amount and allocation of resources within the community for health and wrap-around services. Important attributes of financing that are relevant to systems performance are the adequacy, allocation, flexibility, and longevity of funding. Adequacy refers to the ability of funding levels in the community (including federal, state, and local sources) to match the scope and the need for services. Allocation refers to the distribution of funds across different services and providers in the system. Examples of related performance measures are the allocation of expenditures for chronically ill children between preventive and tertiary health services, or allocations between foster care and family support-child abuse prevention services.

Flexibility refers to the level of restrictions on local funding disbursement (referred to as decategorization when it occurs at the agency level). The degree to which a community uses flexible funding to increase the range of services provided and system efficiency can be evaluated (Hughes et al. 1997; Newacheck et al. 1995). Longevity of funding is another potential attribute of system quality, referring to the stability of the funding base and the successful pursuit of additional funding sources for sustaining or developing necessary services.

Measuring Accountability

Another crucial element in this framework is the designation of accountability in community systems. Accountability includes not only the responsibility for such critical aspects of performance as financing, organization, service delivery, and quality monitoring, but also the responsibility for leadership and policy development. Systems with accountability have built-in incentives for quality and for continual improvement (Langley et al. 1996). In a fully integrated community system, the clear designation of accountability might make centralized monitoring and quality improvement possible. However, few hierarchies of accountability exist for the current programs, networks, and systems that serve children in communities. In most community systems, accountability is in place for specific components of the system, but no single entity is responsible for all aspects of the system or for the functionality of cross-sector relationships and coordination.

However, this does not mean that accountability for system function cannot take place or that it does not take place. One area in which there has been systems accountability is within the Title V programs, for Children with Special Health Needs, in which financing, organization, and quality measurement have been co-located. New England Serve has demonstrated an ability to provide some of this functionality. System accountability also has been addressed in demonstration projects for severely emotionally disturbed children that have assessed system characteristics, adjusted functional relationships, and measured system outcomes (Froelich 1994; Cecil G. Sheps Center for Health Services Research; Rosenblatt and Attkisson 1992a).

State and local Maternal and Child Health programs may be appropriate organizations to assume responsibility for assuring accountability through ongoing needs assessments and other activities. The federal Maternal and Child Health Bureau has identified normative attributes of community systems for children and recently identified a set of key assessment and assurance functions (Grason and Guyer 1995a, 1995b). At a minimum, these agencies may provide the infrastructure for quality assurance in communities through ongoing analysis and dissemination of data on system-level attributes. In some communities, despite the absence of hierarchies of responsibility, coalitions may be more successful at profiling system functions and pressuring institutions within the system toward effectiveness and efficiency as community goals (Nolan and Knapp 1996). Finally, even where systemwide accountability is not yet in place, key leaders and agency heads can be held responsible for deficiencies (as well as for the positive aspects of system functioning). An important purpose of measuring quality in community systems is to identify system objectives for which designated agencies or individuals, or both, can be held accountable.

Broadly defined, accountability also includes the monitoring activities under way in a community (IOM 1997). For example, does the community have a population-based child health profile? And what health indicators are included in it? Some community health reports include process measures that are useful for assessing the delivery system (e.g., indicators of system integration-continuity of care, indicators of the quality and appropriateness of healthcare) (Halfon et al. 1997). Both deficit and capacity measures of community systems can be incorporated into these report cards to highlight the importance of positive as well as negative system characteristics (Halfon et al. 1997).


The development of performance measures may follow a process similar to that used to develop and implement clinical practice guidelines for physicians and health plans. The quality-accountability continuum put forth by the Agency for Health Care Policy and Research describes key steps in the process of developing and implementing quality measurement in clinical care. In Table I, the stages of quality measurement are described along with the processes within each stage. (The additional research that is necessary to support each stage is outlined in our research agenda).

The first stage in the process is conducting or synthesizing the existing research on quality systems for children. This step includes compiling research on critical system pathways that influence health outcomes. Evidence would be drawn from research on system effectiveness (e.g., how service organization in a system affects health outcomes; how organizational changes in systems affect health outcomes). There is a growing literature on measuring the performance of agencies, organizations, and specific systems in terms of their activities and their impact (Beinecke et al. 1997; Osborne and Gaebler 1992; Wholey and Hatry 1992; Plsek 1991; Hatry et al. 1990; General Accounting Office 1994, 1995, 1996; Congressional Budget Office 1993). A project undertaken in California to produce a statewide index of health, the California Health Index, used the field model (Evans and Stoddart 1994) to elaborate the multifactorial pathways that influence health outcomes (Halfon et al. 1997). Pathways leading to specific health outcomes for children and adolescents, among other groups, were outlined drawing from the empirical literature, while key indicators that contributed to multiple health outcomes also were identified. Several communities have explicitly identified community pathways that contribute to the specific child outcomes in their regularly published community health profiles (Weitzman and Doniger 1994).
Table 1: Stages and Process of Community System Performance
Measurement and Monitoring

Stages Process

Outcomes and * Identify relevant system elements from
Effectiveness Research (1) conducting new research and (2) the
 review and synthesis of existing

 * Use system dimensions to organize
 empirical research and review.

 * Identify critical pathways for system

Community Practice * Integrate empirical evidence with
 normative standards.

Guidelines * Formalize guidelines using expert

 * Validate guidelines.

Quality Measurement * Operationalize critical pathways in
 community practice guidelines into
 quality indicators.

Quality Improvement * Customize community practice guidelines
 and indicators to the local setting.

 * Conduct a baseline assessment.

 * Develop a quality improvement strategy.

 * Implement the quality improvement
 strategy and reassess it.

The second stage is the development of practice guidelines. Clinical practice guidelines are based on evidence from controlled trials or alternatively from expert panels that synthesize the empirical basis of specific processes of care (Brook 1986). For community practice guidelines, the evidence base used to generate these guidelines may be integrated with professional consensus or normative standards where specific evidence is lacking or unobtainable. For example, principles for perinatal health networks and for the provision of healthcare to children in foster care have been developed by professional organizations and can be adapted as community practice guidelines.

In the quality measurement stage, community practice guidelines are "deconstructed" into indicators of key elements. For these guidelines to be useful as community diagnostic and improvement tools, specific indicators (representing critical elements in each guideline) will need to be defined operationally. Table 2 identifies examples of measurable indicators for system-level indicators that might be used for children with chronic illness. Because community practice guidelines integrate structural and functional characteristics, the indicators come from several system dimensions: structure/availability (can all families identify a primary care provider/medical home for their child?); organization (do plans exist for coordination of clinical and community-based health promotion?); financing (is continuity of primary care maintained for children regardless of payer status or change?); and accountability (are heads of key agencies accountable for system function? are there systemwide efforts to ensure that children receive periodic health assessments?). The process of developing the indicators should reflect selection criteria such as relevance to many communities, validity and reliability, and resource cost.

Finally, the quality improvement stage is an iterative process in which community practice guidelines are implemented, indicators are developed, baseline performance is reassessed, and an improvement strategy is put in place [ILLUSTRATION FOR FIGURE 2 OMITTED]. A 1997 Institute of Medicine report examined ways in which communities can use performance monitoring to improve the health of their populations. It is an ongoing, community-based process of applying indicators on a regular basis to assess system effectiveness (IOM 1997). While community performance monitoring is often seen as an activity of community-based coalitions and advocacy organizations, such monitoring could also be conducted as part of local agencies' community needs assessments.

A number of recent efforts have been made to understand what makes community-based monitoring effective, so that this process can be more widely adopted. The Public Health Practice Office in the Centers for Disease Control and Prevention (CDC) has recently sponsored a detailed analysis to examine the content and conceptual basis of effective community report cards and to explore which of the characteristics of community organization are associated with successful development of these report cards (Fielding, Halfon, and Sutherland 1997).

A key question at this stage lies in selecting the agency or organization that will be responsible for overseeing the quality improvement process. Some quality improvement efforts are best undertaken through community-based monitoring, while others may be undertaken most successfully through collaboration among the involved subsystems, programs, and networks. For example, for some system functions, such as integration of case management and information systems, agency coordination and accountability are key; because of the technical complexity, monitoring may be most effective when it is performed by an interagency task force rather than by a community-based coalition. For other issues, community coalitions can be effective in comparing the local system to standard community practice guidelines, planning improvement strategies, applying pressure to the institutions involved, and monitoring the quality improvement activities undertaken. For example, the prevention of children's injuries from automobiles may be most effectively accomplished when a broad-based community coalition mobilizes agencies and families through community-wide education and publicity, and tracks accident reduction through a community report card with neighborhood-level accident rates. Some level of"strategic alignment" (Shortell, Gillies, and Anderson 1994) is needed for a community system to coordinate activities and to engage in quality improvement. Defining and delegating responsibility across system components remains an unresolved challenge to meaningful quality improvement in systems.


The proposed research agenda acknowledges that the area of community system performance assessment is in the early stages of conceptual and technical development. Reaching agreement on the first steps of a research agenda is important to move this area of work forward. The proposed research agenda also recognizes the need to have short-term as well as long-term objectives within each area so that the field of system measurement can develop a set of strengths to guarantee advancement. It is also possible that nascent efforts at community performance measurement may serve to clarify the conceptual framework as well as to clarify lines of accountability - not unlike what has occurred over the past two decades in clinical quality improvement.

The evolution of community practice guidelines is likely to be incremental. If achieved, the following research agenda will enable each step of the quality-accountability continuum to reach completion in the ultimate development of community practice guidelines. The process begins with identifying the health production goals of community systems and with analyzing more comprehensively the status and the effectiveness of efforts already under way to define and measure community health. We have identified five general areas in which research can contribute greatly to the performance measurement of community health systems. These areas include (1) refining the conceptual framework; (2) examining the empirical base for community systems assessment; (3) identifying critical pathways and developing quality measurement indicators for potential community practice guides; (4) studying how the process of community quality monitoring can be made successful; and (5) identifying ways to create information systems that can support the performance monitoring process.

Refining the Conceptual Framework for Community Systems Performance Measurement

The framework proposed in this article represents a first step based on adaptation of a framework to measure clinical quality. The framework that we propose adapted the clinical quality-accountability framework and combined it with the community health measurement and improvement approaches of the Institute of Medicine and RAND. More work needs to be done to refine a conceptual framework for community systems performance measurement. Although community systems for children are complex, it is important to have a framework for evaluation that is clear and compelling. Without a framework that is accessible to agencies, providers, and community coalitions, it will be difficult to assemble the types of community-based initiatives in quality improvement that appear to be effective in achieving systems change.

Measuring Community Health. Some work already has been done on measuring community health. This work should be built upon to help community systems identify the key outcomes that will be indicators of a well-functioning community system. For example, in a child health community report card, what would be the indicators of a well-functioning perinatal regional delivery system, and what indicators could measure whether children with special health needs were receiving appropriate rehabilitation services in school? In addition to key outcome indicators, communities need to develop measures of the process of service delivery in their programs, networks, and systems.

Examining the Empirical Base for Community Systems Performance Assessment

It will be necessary to establish empirical support for critical pathway models and for the application of community practice guidelines. A stronger research base will encourage the development of community practice guidelines and their acceptance in communities. This research would focus on understanding whether and how service organization and organizational change, among other system-level characteristics, affect children's health outcomes as well as the quality of care they receive. Identifying the associations between system attributes and actual outcomes for children will enable communities to adopt a meaningful performance measurement in community systems for children. For example, what is the effect of linking primary care with mental health services in detecting and treating debilitating health conditions in children? How do referral and transportation protocols by delivery hospitals affect perinatal outcomes?

Identifying Critical Pathways and Developing Community Practice Guidelines

The approach taken by Aday in identifying system attributes relevant to specific populations provides an important template for future work. Aday's continuums suggest the importance of the structural components of community systems as well as the importance of developing relationships necessary to link services into a well-functioning approach. The literature on systems integration and lessons from the integration efforts undertaken in mental health and other domains can be used to begin to describe system-level critical pathways. An expert consensus process will be needed to develop this. And once the critical elements of "community system practice guidelines" are identified, the next step will be to put these practice guidelines into operation for purposes of measurement. Short-term efforts should focus on compiling evidence on the role of the more easily identified critical pathways in community systems. Long-term efforts should focus on elucidating the causal pathways associated with particular health outcomes (e.g., those prioritized by communities) and on tracking those pathways back to system functions. For example, community practice guidelines for children with asthma might spell out the service delivery components that need to be in place to guarantee a full continuum of prevention, treatment, and rehabilitation services for these children.

In order to determine how well communities are performing in these dimensions and attributes, the issue of standards needs to be addressed. It is necessary not only to define the content of practice guidelines but also to determine normatively how well communities should perform. A well-defined expert consensus process could be undertaken in the short term to establish practice guidelines and to generate normative standards for performance. Longer-term efforts should focus on validating those guidelines beyond the expert consensus process.

Moreover, communities differ in many ways including the complexity of their systems, the attributes of their governance, and the resources at hand. Because of significant differences in populations and systems organization across communities, substantial work will be needed in the longer term to customize practice guidelines for their use in diverse systems. Over time, communities may evolve toward more common standards of system organization and performance.

Evaluating Indicators for Use in Performance Monitoring

The evaluation of performance indicators likely will follow a process parallel to the one established for assessing indicators of health plan/provider performance. Once a range of indicators is developed, the indicators will need to be evaluated for use in performance monitoring. Important issues that need to be studied include how many of the indicators can be supported by current data, how resource-intensive the data collection on these attributes will be, and how these indicators should be prioritized given these constraints. The National Commission for Quality Assurance (NCQA) provides a model for making these decisions. Desirable features of quality measures adopted for the Health Plan Employer Data and Information Set (HEDIS) include relevance (meaningfulness, clinical importance, financial importance, variation in or substandard care, ability to control, potential for improvement, cost-effectiveness); scientific validity (reliable, well specified, accurate, reproducible, valid); and feasibility (reasonable cost for data collection, confidentiality, logistical feasibility) (NCQA 1996).

Although some features (such as comparability) may be more difficult than others to achieve in measuring community systems, more work needs to be done to determine which of these features can in fact be realistically achieved for community performance measures. An approach that rates each potential indicator according to NCOA criteria would be one way of identifying those systems indicators that could be tentatively used and those requiring additional validation (Milstein 1997). In the short term, specific indicators can be developed in each of the four system dimensions presented in Table 2. Long-term efforts will focus on validating such indicators, elaborating additional indicators that support community practice guidelines, and refining a set of indicators that can be standardized across communities.

Quality Improvement. Research and development in the ways in which quality improvement can function in community systems is a critical phase of the research agenda. Quality improvement activities include monitoring, analysis, and implementation of corrective alterations in the system in an iterative manner. Some work is being done on ways in which community coalitions can effectively implement this type of quality improvement process (Nolan and Knapp 1996). Guidelines from PATCH and APEXPH for public health departments to use in carrying out their mission of assessment, assurance, and policy development provide an example that could be replicated for community performance measurement (IOM 1997; National Association of County Health Officers 1993). Because community coalitions may not be the appropriate entity to direct and implement all quality improvement efforts, however, public as well as private institutions in the systems will need to recognize their respective roles in continually refining and reassessing the technical elements of the system.

Finally, just as with clinical practice guidelines, we need to form a better understanding of the ways in which the measuring of system performance can affect the delivery of care, and ultimately health outcomes. Thus, some process to analyze the effectiveness of community systems performance measurement, however such measurement has been implemented across communities, is an important final step.

Developing Information Sources

It is important to improve the process of compiling and disseminating both information on children's health and important indicators of system performance. A cornerstone of the infrastructure necessary for monitoring quality of community systems is the availability of data. Part of the research task is to determine the data that need to be collected for purposes of community systems monitoring, and to find ways of collecting, analyzing, and disseminating these data. This is a particularly important part of the research agenda because of the continuing changes in healthcare delivery systems for children, and the potential effects that these changes can have on the availability of continual and comparable data on children's health and services. Research and development in the area of information systems is also important to facilitate the exchange of information across all of the components of community systems for individual children. Adequate data systems constructed within communities are necessary both to evaluate systems quality and to improve the coordination of care for the individual child.

Table 2: Examples of System-Level Indicators for Children with Chronic Illness

System Dimension and Sample Indicators


* Geographic distribution of primary care physicians is in place.

* Families know the child's primary care physician.

* The number of appropriately trained, credentialed pediatric specialists is appropriate.

* Multidisciplinary disease management teams are regionalized.


* Procedures axe established for coordination of health promotion among appropriate systems (primary care physician, schools, public health, CBOs, social services).

* Each child has a coordinated care plan.

* The family is involved in development and implementation of the care plan.

* Family satisfaction with access and quality of care is assessed and disseminated.


* Continuity of care is maintained regardless of payer status, changes.

* Funding is blended and flexible.

* Funds are allocated efficiently among prevention, tertiary, and rehabilitation services.


* Primary care physicians know their patient population.

* Providers act to ensure health exams according to periodicity guidelines.

* Monitoring of key system components and processes is in place and data are collected and disseminated.

* Key agency heads are held accountable for system function.


Aday, L. A., C. E. Begley, D. R. Lairson, and C. H. Slater. 1993. Evaluating the Medical Care System: Effectiveness, Efficiency, and Equity. Chicago: Health Administration Press.

Aday, L. A. 1993. At Risk in America: The Health and Health Care Needs of Vulnerable Populations in the United States. San Francisco:Jossey-Bass.

American Public Health Association. 1996. Planning for Community-Oriented Health Systems. Baltimore, MD: United Book Press, Inc.

Annie E. Casey Foundation. 1996. KIDS COUNT Data Book: State Profiles of Child Well-Being. Baltimore, MD: Annie E. Casey Foundation.

Barnett, W. S., and C. M. Escobar. 1992. "Economic Costs and Benefits of Early Intervention." In Handbook of Early Childhood Intervention, edited by S.J. Meisels and J. P. Shonkoff. New York: Cambridge University Press.

Beinecke, R. H.,J. Leavey, P. Mosley, and M. A. Matava. 1997. "Development and Description of Two New Instruments for Measuring the Impact of Coordinated Social Services for Children." Child Welfare League of America Journal 76 (3): 379-91.

Brewer, E.J, Jr., M. McPherson, P. R. Magrab, and V. L. Hutchins. 1989. "Family-Centered, Community-based, Coordinated Care for Children with Special Health Care Needs." Pediatrics 83 (6): 1055-60.

Brook, R. H. 1986. "A Method for the Detailed Assessment of Appropriateness of Medical Technologies." International Journal of Technology Assessment in Health Care 2 (1): 53-63.

Brown, S.S., ed. 1988. Prenatal Care: Reaching Mothers, Reaching Infants. Washington, DC: Institute of Medicine/National Academy Press.

Child Welfare League of America. 1988. Standards for Health Care Services for Children in Out-of-Home Care. Washington, DC: CWLA.

Coffey, R.J. 1992. "An Introduction to Crucial Paths." Quality Management in Health Care 1 (1): 45-54.

Congressional Budget Office. 1993. Using Performance Measures in the Federal Budget Process. Washington, DC: CBO. July.

Donabedian, A. 1982. Explorations in Quality Assessment and Monitoring. Vol. I: The Definition of Quality and Approaches to Its Assessment. Chicago: Health Administration Press.

Durch, J. S., ed. 1994. Protecting and Improving Quality of Career Children Under Health Care Rearm: Working Highlights. Committee on Maternal and Child Health Under Health Care Reform. Washington, DC: National Research Council, Institute of Medicine.

Dutton, D. B. 1986. "Financial, Organizational, and Professional Factors Affecting Health Care Utilization." Social Science and Medicine 23: 721-35.

Evans, R. G., and G. L. Stoddart. 1994. "Producing Health, Consuming Health Care." In Why Are Some People Healthy and Others Not?, edited by R. G. Evans, M. L. Barer, and T. R. Marmor, pp. 27-64. New York.

Farran, D.C. 1992. "Effects of Intervention with Disadvantaged and Disabled Children: A Decade Review." In Handbook of Early Childhood Intervention, edited by S.J. Meisels and J. P. Shonkoff. New York: Cambridge University Press.

Field, M.J., and K. N. Lohr, eds. 1990. Clinical Practice Guidelines: Directions for a New Program. Washington, DC: National Academy Press.

-----. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: National Academy Press.

Fielding, J., N. Halfon, and C. Sutherland. 1997. "Characteristics of Community Report Cards: United States, 1996." Mortality and Morbidity Weekly Report 46 (28): 647-66.

Garbarino, J. 1990. "The Human Ecology of Early Risk." In Handbook of Early Childhood Intervention, edited by S.J. Meisels and J. P. Shonkoff. New York: Cambridge University Press.

General Accounting Office. 1994. Managing for Results: State Experiences Provide Insights for Federal Management Reforms. Pub. No. GAO/GGD-95-22. Washington, DC: Government Printing Office.

-----. 1995. Block Grants: Issues in Designing Accountability Provisions. Pub. No. GAO/AIMD-95-226. Washington, DC: Government Printing Office.

-----. 1996. Executive Guide: Effectively Implementing the Government Performance and Results Act. Pub. No. GAO/GGD-96-118. Washington, DC: Government Printing Office.

Grason, H. A., and B. Guyer, eds. 1995b. Assessing and Developing Primary Career Children: Reforms in Health Systems. Arlington, VA: National Center for Education in Maternal and Child Health.

-----. 1995a. MCH Quality Systems Functions Framework. Arlington, VA: Association of Maternal and Child Health Programs.

Grason, H. A., and A. Wigton. 1995. Review of the Literature and Measurement Strategies Related to Key Principles in the Development of Systems of Care for Children and Youth: Health Systems Research, Inc. March.

Green, M., ed. 1994. Bright Futures: National Guidelines for Health Supervision of Infants, Children, and Adolescents. Arlington, VA: National Center for Education in Maternal and Child Health.

Guyer, B., M. Shediac, C. Cassady, and H. Grason. 1995. "Health Care Systems and Primary Care for Children and Adolescents: Overview and Context." In Assessing and Developing Primary Care for Children, edited by H. Grason and B. Guyer, pp. 9-22. Arlington, VA: National Center for Education in Maternal and Child Health.

Haggerty, R. J., K. J. Roghmann, and I. B. Pless. 1975. Child Health and the Community. New York: Wiley.

Halfon, N., P. Evener, N. Sastry, and Others. 1997. California Health Report. DRU1592-TCWF. Santa Monica, CA: RAND.

Halfon, N., and M. Hochstein. 1997. "Developing a System of Care for All: What the Needs of Vulnerable Children Tell Us." In Health Care for Children: What's Right, What's Wrong, What's Next, edited by R. E. K. Stein, pp. 303-38. New York: United Hospital Fund.

Halfon, N., M. Inkelas, and D. Wood. 1995. "Nonfinancial Barriers to Care for Children and Youth." Annual Review of Public Health 16: 447-72.

Halfon, N., M. Inkelas, D. L. Wood, and M. A. Schuster. 1996. "Refinancing and Restructuring the U.S. Child Health System." In Changing the U.S. Health Care System: Key Issues in Health Services, Policy, and Management, edited by R. M. Andersen, T. H. Rice, and G. F. Kominski, 227-54. San Francisco: Jossey-Bass Publishers.

Halfon, N., A. Mendonca, and G. Berkowitz. 1995. "Health Status of Children in Foster Care: The Experience of the Center for the Vulnerable Child." Archives of Pediatric and Adolescent Medicine 149 (3): 386-92.

Halfon, N., P. Newacheck, D. Hughes, and C. Brindis. 1998. "Community Health Monitoring: Taking the Pulse of America's Children." Submitted to the Maternal and Child Health Journal.

Hatry, H. P., J. R. Fountain, J. M. Sullivan, and L. Kremer. 1990. Service Efforts and Accomplishments Reporting: An Overview. Norwalk, CT: Governmental Accounting Standards Board.

Hughes, D., N. Halfon, C. Brindis, and P. Newacheck. 1996. "Improving Children's Access to Health Care: The Role of Decategorization." Bulletin of the New York Academy of Medicine: A Journal of Urban Health 73 (2): 127-54.

The Infant Health and Development Program. 1990. "Enhancing the Outcomes of Low-Birth-Weight, Premature Infants: A Multisite, Randomized Trial." Journal of the American Medical Association 263 (24): 3035-42.

Institute of Medicine. 1978. A Manpower Policy for Primary Health Care. IOM Pub. No. 78-02. Washington, DC: National Academy of Science.

-----. 1997. Improving Health in the Community.' A Role for Performance Monitoring, edited by J. S. Durch, L. A. Bally, and M. A. Soto. Washington, DC: National Academy Press.

Ireys, H. 1994. "Children with Special Health Care Needs: Evaluating Their Needs and Relevant Service Structures." Background paper commissioned by the Institute of Medicine. Washington, DC. November.

Jameson, E.J., and E. Wehr. 1993. "Drafting National Health Care Reform Legislation to Protect the Health Interests of Children." Stanford Law and Policy Review 5: 152-76.

Johansen, A., B. Starfield, and J. Harlow. 1994. Analysis of the Concept of Primary Care for Children and Adolescents: A Policy Research Brief. Baltimore, MD: The Johns Hopkins University Child and Adolescent Health Policy Center.

Kahn, A. J., and S. B. Kamerman. 1992. Integrating Services Integration: An Overview of Initiatives, Issues, and Possibilities. National Center for Children in Poverty, Columbia University School of Public Health.

Labonte, R. 1988. "Health Promotion: From Concepts to Strategies." Health Care Management Forum I (3): 24-30.

Langley, G., K. Nolan, T. Nolan, C. Nomlan, and L. Provost. 1996. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. San Francisco: Jossey-Bass, Inc.

Maternal and Child Health Bureau, Health Resources and Services Administration, Department of Health and Human Services. 1995. A Preliminary Strategy for Assessing Community Systems of Career Women, Children, and Adolescents. Washington, DC: MCHB. 31 October.

McGlynn, E. A., C. Damberg, E. A. Kerr, and R. H. Brook. 1996. Community Health Data systems: A Guidebook for Decision Makers. DRU- 1194-HF. Santa Monica, CA: RAND.

McGlynn, E. A., N. Halfon, and A. Leibowitz. 1995. "Assessing the Quality of Care for Children: Prospects Under Health Reform." Archives of Pediatric and Adolescent Medicine.

Melaville, A. I., and M.J. Blank. 1991. "What It Takes: Structuring Interagency Partnerships to Connect Children and Families with Comprehensive Services." Washington, DC: Education and Human Services Consortium.

Merritt, T. A., D. Palmer, D. A. Bergman, and P. H. Shiono. 1997. "Clinical Practice Guidelines in Pediatric and Newborn Medicine: Implications for Their Use in Practice." Pediatrics 99 (1): 100-14.

National Association of County Health Officials. 1993. APEXPH: Assessment Protocol for Excellence in Public Health. Washington, DC: NACHO.

National Commission for Quality Assurance. 1996. HEDIS 3.0. Washington, DC: NCQA.

Nerenz, D. 1996. "Who Has Responsibility for a Population's Health?" Milbank Quarterly 74 (1): 43-49.

New England Serve. 1997. Assessing the Quality of Managed Career Children with Special Health Care Needs: Summary of a Regional Survey of Families, Primary Care Providers, and Managed Care Organizations in New England. Boston: New England Serve.

Newacheck, P. W., D.C. Hughes, C. Brindis, and N. Halfon. 1995. "Decategorizing Health Services: Interim Findings from the Robert Wood Johnson Foundation's Child Health Initiative." Health Affairs 14 (3): 232-42.

Nolan, T. W, and M. Knapp. 1996. "Community-Wide Health Improvement: Lessons from the IHI-GOAL/QPC Learning Collaborative." The Quality Letter for Healthcare Leaders 8 (1): 13-20.

Olds, D. L., C. R. Henderson, Jr., C. Phelps, and Others. 1993. "Effect of Prenatal and Infancy Nurse Home Visitation on Government Spending." Medical Care 31 (2): 155-74.

Osborne, D., and T. Gaebler. 1992. Reinventing Government: How the Entrepreneurial Spirit Is Transforming the Public Sector. Reading, MA: Addison-Wesley.

Palfrey, J. 1994. Community Child Health: An Action Plan for Today. Westport, CT: Praeger Publishers.

Paneth, N.,J. L. Kiely, S. Wallenstein, and Others. 1982. "Newborn Intensive Care and Neonatal Mortality in Low-Birthweight Infants: A Population Study." The New England Journal of Medicine 307 (2) 149-55.

Paneth, N. 1990. "Technology at Birth." American Journal of Public Health 80 (7): 791-92.

Parker, A.,J. Walsh, and M. Coon. 1976. "A Normative Approach to the Definition of Primary Health Care." Milbank Memorial Fund Quarterly 54 (4): 415-38.

Patrick, D. L., and T. M. Wickizer. 1995. "Community and Health." In Society and Health, edited by B.C. Amick, S. Levine, A. R. Tarlov, and D. Walsh, pp. 4692. New York: Oxford University Press.

Penchansky, R, and J. W. Thomas. 1981. "Consumer Satisfaction." Medical Care 19 (1): 127-40.

Plsek, P. E. 1991. "A Primer on Quality Improvement Tools." In Curing Health Care: New Strategies for Quality Improvement, a Report on the National Demonstration Project on Quality Improvement in Health Care, edited by D. M. Berwick, A. B. Godfrey, and J. Roessner, pp. 177-219. San Francisco: Jossey-Bass Publishers.

Rosenblatt, A., and C. C. Attkisson. 1992a. "Integrating Systems of Care in California for Youth with Severe Emotional Disturbance. I: A Descriptive Overview of the California AB377 Evaluation Project." Journal of Child and Family Studies 1 (1): 93-113.

-----. 1992b. "Integrating Systems of Care in California for Youth with Severe Emotional Disturbance. II: Initial Group Home Expenditure and Utilization Findings from the California AB377 Evaluation Project." Journal of Child and Family Studies 1 (3): 263-86.

Salzer, M., and L. Bickman. 1997. "Delivering Effective Children's Services in the Community: Reconsidering the Benefits of System Interventions." Applied and Preventive Psychology 6 (1): 1-13.

Schlenger, W. E, R. M. Etheridge, D.J. Hansen, D. W. Fairbank, and J. Onken. 1992. "Evaluation of State Efforts to Improve Systems of Care for Children and Adolescents with Severe Emotional Disturbances: The CASSP Initial Cohort Study." The Journal of Mental Health Administration 19 (2): 131-42.

Schlesinger, M., and L. Eisenberg. 1990. "Little People in a Big Policy World: Lasting Questions and New Directions in Health Policy for Children." In Children in a Changing Health System: Assessments and Proposals for Reform, edited by M. Schlesinger, and L. Eisenberg. Baltimore, MD: Johns Hopkins University Press.

Schor, E. L., and E.G. Menaghan. 1995. "Family Pathways to Child Health." In Society and Health, edited by B. Amick, S. Levine, A. Tarlov, and D. Walsh, pp. 18-45. New York: Oxford University Press.

Schor, E. L. 1995. "Developing Communality: Family-Centered Programs to Improve Children's Health and Well-Being." Bulletin of the New York Academy of Medicine 72 (2): 413-42.

Schorr, L. B. 1988. Within Our Reach: Breaking the Cycle of Disadvantage. New York: Doubleday.

Select Panel for the Promotion of Child Health. 1981. Better Health for Our Children: A National Strategy. Washington, DC: U.S. Department of Health and Human Services.

Sells, C. W., and R. W. Blum 1996. "Morbidity and Mortality Among U.S. Adolescents: An Overview of Data and Trends." American Journal of Public Health 86 (4): 513-19.

Shortell, S. M., R. R. Gillies, D. A. Anderson,J. B. Mitchell, and K. L. Morgan. 1993. "Creating Organized Delivery Systems: The Barriers and Facilitators." Hospital and Health Services Administration 38 (4): 447-66.

Shortell, S. M., R. R. Gillies, and D. A. Anderson. 1994. "The New World of Managed Care: Creating Organized Delivery Systems." Health Affairs 13 (5): 46-64.

Singh, G. K, and S. M. Yu. 1996. "U.S. Childhood Mortality, 1950 Through 1993: Trends and Socioeconomic Differentials." American Journal of Public Health 86 (4): 505-12.

Starfield, B. 19923. "Child and Adolescent Health Measures." The Future of Children 2 (2): 25-39.

-----. 1992b. Primary Care: Concept, Evaluation, and Policy. New York: Oxford University Press.

Weitzman, M., and A. Doniger. 1994. "Pathways to a Coordinated System of Health Care and Human Services for Children and Families." Rochester, NY. August.

Wholey, J. S., and H. P. Hatry. 1992. "The Case for Performance Monitoring." Public Administration Review 52 (6): 604-10.

Supported by grants to Dr. Halfon and Dr. DuPlessis from the Interdisciplinary Maternal and Child Health Training Program (MCJ-069-500-01-4), and to Dr. Halfon from the National Center for Infant and Early Childhood Health Policy (MCU-069385-02-1).

Address corresspondence to Helen M. DuPlessis, M.D., M.P.H., Associate Medical Director, L. A. Care Health Plan, 3530 Wilshire Blvd., 9th Floor, Los Angeles, CA 90010. Dr. DuPlessis is also Assistant Professor, UCLA Schools of Medicine and Public Health. Moira Inkelas, M.P.H. is a Doctoral Candidate, RAND Graduate School, and Neal Halfon, M.D., M.P.H. is Professor, UCLA Schools of Medicine and Public Health. This article, submitted to Health Services Research on July 8, 1997, was revised and accepted for publication on March 5, 1998.
COPYRIGHT 1998 Health Research and Educational Trust
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1998 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Improving the Quality of Healthcare for Children: An Agenda for Research
Author:DuPlessis, Helen M.; Inkelas, Moira; Halfon, Neal
Publication:Health Services Research
Date:Oct 1, 1998
Previous Article:Improving the quality of care for children in health systems.
Next Article:Getting the incentives right for children.

Terms of use | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters