Printer Friendly

Web-PPGIS usability and public engagement: a case study in Canmore, Alberta, Canada.


Over the past decade or so, public engagement has increasingly been an important theme in the urban planning process (Talen 1999, Kingston et al. 2000, Kessler 2004, Kingston 2007). This assertion is based on the premise that public engagement in the process can lead to a more sustainable, legitimate, democratic, and effective plan. Public meeting is one of the most popular methods of public participation. The method requires that the meetings are held in a certain place during a fixed period of time. This limits the number of people who can be involved in a decision-making/planning process. Therefore, there is a need for developing tools that can enable and support new ways to involve the citizens in the decision-making process (Krek 2005). In the past, various tools (a three-dimensional cardboard scale model, poster, kiosk, etc.) have been used to facilitate public participation (Rambaldi and Callosa 2000, Berner 2001). Since the later 1990s, the high-powered computer, the low-cost desktop GIS, and decision support software have been used for supporting community collaboration and public participation in urban and community planning processes (Craig and Elwood 1998, Klosterman 1999, Talen 1999). This has been developed into a broad area of research, generally referred to as public participatory GIS (PPGIS). However, traditional GIS has been criticized as an elite technology (Pickles 1995), which is operated mainly by a small group of scholars, GIS technocrats, and planners because of high operation costs, complex design, and great learning barriers. A little progress has been made to encourage the general public to join in community-based GIS projects (Chua and Wong 2002).

In recent years, the appearance of the Internet and improved WWW technologies provide opportunities for PPGIS researchers. This has speeded up the incorporation of PPGIS into the WWW technologies (Kingston et al. 2000, Kessler 2004, Simao et al. 2009). This type of system often is referred to as Web-based PPGIS (Web-PPGIS). Web-PPGIS overcomes many problems

caused by the traditional GIS and conventional public participation methods (Kingston et al. 2000, Chua and Wong 2002, Kessler 2004). For example, people can join the public participation process at any time and at any place that has a computer and Internet service. The complexity of GIS and spatial analysis is hidden from the user. A Web-PPGIS enables people to express their views by posting comments in a relatively anonymous and nonconfrontational manner. It also supports two-way to multiway flows of information.

Most Web-PPGIS research and projects have focused on making Web-PPGIS available and accessible to the general public to stimulate more informed participation and decision making (Sieber 2006, Kingston 2007). At the same time, the rapid technical progress in the area of developing Web-PPGIS has raised some questions regarding the evaluation of Web-PPGIS technology. One concern is related to the usability of Web-PPGIS. When an increasing number of laypeople obtain access to a Web-PPGIS, it is important to raise the issue of how usable the system is for a wide range of potential users. Web-PPGIS practitioners need not only upload a Web-PPGIS to a Web site, but also must design it in an effective, efficient, and satisfying way for users to perform specific tasks (ISO 1998). If the system usability is unsatisfactory, this could cause issues such as wasting users' time, making them worry and frustrated, and eventually discouraging their engagement in the public participatory process. This leads to the concept of degree of public engagement that, in this study, is referred to as the degree of public participants' interactions with the Web site holding a Web-PPGIS and other participants in the online public participatory decision-making process against a set of clearly defined goals.

An important objective of Web-PPGIS projects is to use the technology to engage grassroots public members in the decision-making process. Thus, we suggest that empirical studies are needed to (1) evaluate the usability of a Web-PPGIS and the degree of public engagement quantitatively and (2) explore the Web-PPGIS usability as the determinant of the degree of public engagement. These two research objectives will be investigated by using a Web-PPGIS: ArgooMap (Rinner 2001, Kessler 2004) for tackling a multicriteria site selection problem, which involves public participants determining the "best" location for a new parking facility in downtown Canmore, Alberta, Canada.

The paper is organized as follows. The following section provides a brief review of the system usability evaluation and public engagement, including the levels and degree of public engagement. Then the paper demonstrates how to collect the system usability and the degree of public engagement metrics for different users in a real-world public participatory planning situation. The next section provides the results and analysis on the relationships between the system usability and the degree of public engagement. Finally, discussion and conclusions are presented.

Usability Evaluation and Public Engagement

Usability evaluation refers to the process of systematically collecting data on how people use the system for a particular task in a particular environment (Preece et al. 2002). Two main types of methods are used to evaluate usability (Banati et al. 2006). First, the inspection methods (e.g., the heuristic evaluation, cognitive walk-through, and walk-through inspection) mainly involve system developers or experts to do the test following a set of schemes. Second, the user testing methods involve a set of procedures for collecting data when users interact with the system to perform prespecified tasks. Then the system usability can be quantified in terms of users' performance and satisfaction during the interaction with the system (Butler 1996). No general rule exists for how usability measures should be chosen or aggregated (ISO 1998). However, it is suggested that at least one measure of effectiveness, efficiency, and satisfaction be provided (ISO 1998). In addition, how to choose the measures and decide the details of each measure depends on "the objectives of parties involved in the measurement" (ISO 1998, p. 10).

There is some usability research performed in the context of Web-PPGIS. Haklay and Tobon (2003) discuss the association between human-computer interaction (HCI) and usability evaluation. They argue that software characteristics such as ease of use and user friendliness are more elusive than one may expect, so only appropriate testing can verify whether the system design is successful in meeting users' needs. Sidlar and Rinner (2007) provide a case study focusing on different aspects of the usability of a Web-PPGIS: Argumentation Map (Rinner 2001, Kessler 2004). Although considerable progress has been made in advancing research about the system usability (Haklay and Tobon 2003, Sidlar and Rinner 2007, Haklay and Zafiri 2008, Ingensand and Golay 2010), there are no studies on measuring various aspects of usability systematically and quantitatively.

The phrase public engagement in the urban planning context refers to a process of bringing together citizens, community nonprofit organizations, businesses, and government to solve planning problems that affect everybody's life (Rabinovitch 2004). It often is used haphazardly in PPGIS research and confused with the notion of public participation. Public engagement suggests a degree of personal choice, commitment, and willingness (Rabinovitch 2004). It also implies an interweaving of responsibility, action, and a degree of control. PPGIS practitioners tend to use the ladders of public participation as the conceptual framework to guide public participation (Arnstein 1969, Weidemann and Femers 1993, IAP2 2004). In PPGIS literature, the term public engagement often is used interchangeably with the notion of public participation when it refers to high levels of public participation activities (see Table 1).

PPGIS practitioners often assume that public participants would be empowered if they engaged in high levels of public participation activities. The assumption is based on the premise that the higher the level of public participation, the more control for the decision outcomes. Therefore, a key focus of Web-PPGIS research has been on enhancing the level of public engagement by providing access to the relevant tools, data, and information to enable more informed engagement and decision making. However, the assumption is questionable.


After adopting the new technologies to facilitate public participation, the feedback from local community members sometimes is quite frustrating. Lee (2000) demonstrates that about 34 percent of all users visit the Web site holding GIS on a monthly or occasional basis. Hopkins et al. (2004) find that some participants often just give up in the middle of the participation process because of great learning barriers and complicated interfaces. Sidlar and Rinner (2007) and Ingensand and Golay (2010) report that the project ends up having a few number of participants. Although the general public has the opportunity to contribute and exert its influence on the project, it seems that many people are reluctant to become engaged. This is an issue long ignored by Web-PPGIS professionals. This evidence suggests that providing an improved access to the systems and relevant data no longer is sufficient to enhance the degree of public engagement in the participatory decision-making process.

May (2007) introduces a model called Triangle of Engagement that can be used to explain the relationships among the level of public engagement, the degree of public engagement, and the prevalence (the number of participants engaged) (see Figure 1).

In the model, the level of public engagement increases from the base to the apex. Each level of public engagement corresponds to a different degree of public engagement that costs the participant varying amounts of time and energy (May 2007). The higher up the triangle, the higher the degree of public engagement and the less the prevalence. However, the May (2007) model descriptively discloses the relationships among the prevalence, level, and degree of public engagement rather than quantitatively. It does not demonstrate how to measure the degree of public engagement quantitatively. It falls short of explaining why some participants could commit to a higher degree of engagement than others while they have joined the same level of public participation.



Study Area

Canmore, Alberta, Canada, is located in the Canadian Rocky Mountains (Kananaskis Country), approximately 20 kilometers east of Banff and 100 kilometers west of Calgary (see http:// The town is an administration and business center for residents and employees of the Banff National Park, Kananaskis Country, and the Bow Valley. It has a population of approximately 16,000. Canmore is a transitional town experiencing changes because of tourism expansion. As a result of growing population pressures and changes in the nature and intensity of economic activities, issues of land-use planning have become increasingly important (Town of Canmore 2007).

Evaluating Sites for Building a Parking Facility in Canmore, Alberta, Canada

Two groups of people (local residents and tourists) contribute to the demand for parking facilities in downtown Canmore. The tourists can be further categorized into the day-visit tourists and stay tourists depending on whether they stay overnight in the town. The day-visit tourists usually stay in Canmore for just few hours. They get off Highway 1 by Benchlands Trail Overpass, drive to downtown Canmore for a meal, gas, or other short-time activities, and then head to other places without staying overnight. The stay tourists spend at least one night in Canmore. The number of hotel room units is used to quantify the demand from the stay tourists. In Canmore, the private vehicles are the dominant transportation mode for local residents. However, the local government does not collect vehicle ownership statistics. Therefore, the 2006 Canmore population data are used as an approximate measure for the demand for parking from the local residents. The Local Delivery Units (LDUs) (the smallest postal delivery zones, see Figure 2) are employed for describing the spatial distribution of population and stay tourists. The centroids of each LDU and location of Benchlands Trail Overpass are used as demand points. There are 410 LDUs in Canmore.

The Planning Department of Canmore has preselected four candidate sites for a new parking facility in downtown Canmore (see Figure 2). However, the department welcomes any suggestions or recommendations from local residents regarding potential locations for constructing the parking facility. Nevertheless, only the four candidate sites will be evaluated and ranked quantitatively.

A set of criteria including (1) weighted average distance to local residents, (2) weighted maximum distance to local residents, (3) weighted average distance to stay tourists, (4) weighted maximum distance to stay tourists, (5) distance to Benchlands Trail Overpass, (6) distance to Main Street, (7) the number of people living within 100 meters of a candidate site, (8) the size of a candidate site, and (9) the cost of land acquisition are employed for evaluating the suitability of a candidate site. Except for the size of a candidate site, all evaluation criteria are to be minimized.


ArgooMap is based on the concept of an Argumentation Map that provides a foundation for Web-PPGIS tools design and development (Keiler 2004, Rinner et al. 2008, Boroushaki and Malczewski 2010). In the Argumentation Map model, argumentation elements and geographic reference objects are regarded as independent entities (Rinner 2006). The relationships between a user-initiated discussion and the discussion-related place on a map are specified. In addition, the model includes user-defined graphic reference objects and supports the many-to-many relationship between any kinds of objects.

The design of ArgooMap (second version of Argumentation Map) is an AJAX-based implementation with Google Map interface. ArgooMap was customized by Boroushaki and Malczewski (2010). The customized ArgooMap consists of three main sections: (1) registration and log in, (2) main map, and (3) questionnaire about the user's characteristics (see http://www. The registration and log-in section is composed of four different pages: "Log In," "User Registration," "Terms and Conditions," and "About ParticipatoryGIS." The main map section of the system contains two Web pages: "Tutorial" and "Main Decision Map." The Tutorial contains two parts. The first part describes the goal and objectives of the parking site selection problem, provides a detailed description of the address of each candidate site and photos of the sites, and explains the evaluation criteria. The second part provides screen shots and instructions on how to join the online public participatory decision-making process. The Main Decision Map contains a multicriteria decision analysis (MCDA) module. The user can give his or her preference regarding the importance of each criterion by choosing one of the following terms: none, very low, low, medium, high, and very high (Chen and Hwang 1992). In addition, the participants have to select a linguistic quantifier (Boroushaki and Malczewski 2010) to indicate how the attribute data and their preferences will be aggregated to provide the final scores and rankings of the alternative sites. In the Main Decision Map element, the users can use the "group decision" function to explore the decision outcome based on group preferences. The function generates rankings and suitability scores for the alternative sites based on the fuzzy majority procedure (Pasi and Yager 2006). In the Main Decision Map, the users can explore the area using the zoom-in/zoom-out function or shift the background map to a satellite image or a map-satellite image hybrid module. The attributes of each alternative can be retrieved by clicking one of the four alternatives. The users can read existing comments and initiate a discussion or reply to an existing thread by turning on the "discussion board."

Recording Public Participants' Move

To evaluate the usability of ArgooMap and measure the degree of public engagement in the online public participatory decision-making process, public participants' every move on the Web site was recorded using UsaProxy (Atterer et al. 2007). UsaProxy uses an HTTP proxy approach that works with the current server and browser setups. It employs JavaScript technology that stands behind the client-side monitoring, so the activities of users on the Web site are not affected (Atterer et al. 2007). By using the software, it is possible to collect highly detailed and useful log data about the actual usage of the Web site and the ArgooMap system. Actions such as moving the mouse pointer, scrolling a page, or filling out a form in a specific order, etc., were automatically recorded into a log file stored on the server where UsaProxy and ArgooMap were installed. In addition, the events, such as opening a Web site, logging in the system, and clicking a button, were directly written into the log file. This data recorded in the log file are useful for evaluating the usability and measuring the degree of public engagement.

An example of the log data generated by UsaProxy during the public participatory decision-making process is shown in Figure 3. The log output displays events such as mouse move (pointer position changed), mouse over (the pointer was moved over a DIV HTML element or something similar), focus (the cursor was moved into an input field), etc. Users' IPs, the time of the events, and the coordinates of the mouse pointer are recorded along with the events.


Generating Usability and Degree of Public Engagement Metrics

The design of ArgooMap is based on the "walk up and use" principle. In other words, the system is supposed to be used by first-time users who do not need any training before being able to effectively use the system. Therefore, the metrics employed for evaluating the usability of ArgooMap are the "walk up and use" measures suggested by ISO (1998) as well as measures proposed in previous research/projects (Nielsen 1993, Haklay and Tobon 2003, Sidlar and Rinner 2007). The measures of usability include effectiveness, efficiency, and satisfaction.

Effectiveness refers to the "accuracy and completeness with which users can achieve their goals" (ISO 1998, p. 19). In this study, the effectiveness is measured by the number of major tasks completed successfully on the first attempt (ISO 1998). The major tasks involve using three groups of functions available in ArgooMap system: mapping, deliberation/argumentation, and MCDA. The first major task is to use mapping functions to explore the study area. The functions include zooming (zoom-in and zoom-out) and background view change (map view, satellite image view, and map-satellite hybrid view). The second major task is to use the deliberation/argumentation functions to communicate with other participants regarding the existing four candidate sites, other possible candidate sites, or other concerns related to the parking site selection project. The functions are reading comments from previous users and initiating a new georeferenced discussion or replying to an existing georeferenced discussion. The third major task is to use MCDA functions to resolve the site selection problem. The functions used in the third task include site attribute inquiry, group decision-making outcomes inquiry, and identification of the best site using an ordered weighed averaging (OWA) module (Yager 1996, Boroushaki and Malczewski 2010).

Efficiency refers to the system's ability to fulfill the level of effectiveness while taking a certain amount of resources (ISO 1998, Sidler and Rinner 2007). It is measured by the time needed to perform a prespecified task on the first attempt. As mentioned, users were asked to perform three major tasks using a number of functions. For mapping functions, the time spent on zooming or background view change depends on the objects (size, color, contrast, etc.) that they are exploring, so the time used to zoom or change the background view on the first attempt cannot be used to measure efficiency. In terms of deliberation/argumentation functions, the length of comments posted by the users and their typing speed are quite different. The time used to read comments posted by other users depends on available comments and the length of comments. In addition, only a limited number of users posted or read comments. Thus, the time used to post or read a comment on the first attempt cannot be used to measure efficiency. Every user performed MCDA and the time spent on identifying the best site using the MCDA module is comparable. Therefore, the time used to identify the best site on the first attempt using the MCDA module is chosen to measure the system efficiency in this study.

Satisfaction is defined as "freedom from discomfort, and their attitudes towards the use of the system" (ISO 1998, p. 19). Satisfaction is a response of the users when interacting with the product. In the questionnaire section, the users were asked to rate their overall experience with ArgooMap on a six-point scale ranging from zero to five (with zero the lowest score and five the highest score).

Peterson (2008, p. 5) describes user engagement in the context of the Web analytic as "an estimate of the degree and depth of visitor interaction on the site against a clearly defined set of goals." In this study, there are two main goals: to promote the participants' interaction with the Web site and to encourage the participants to interact with each other from the beginning to the end of the online public participatory decision-making process. They are encouraged to remain on the Web site, visit the Web site frequently, if possible, view every page, and join the discussion by posting new comments and reading existing comments. Accordingly, the degree of public engagement is measured by the following metrics: (1) the total time of stay on the Web site, (2) the number of total visits, (3) the number of page views, (4) the number of comments posted, and (5) the number of times to read comments.

Results and Analysis

The ArgooMap system and relevant data were uploaded to for use from October 1, 2008, to December 30, 2008. Canmore residents were invited to identify their concerns, ideas, suggestions, or preferences for the candidate sites and evaluation criteria for locating a new parking facility in the downtown area. The ArgooMap Web site and the parking site selection project were advertised in the local community newspaper--Rocky Mountain Outlook. The advertisement also was displayed on the Web site of the Department of Local Economic Development and the Department of Planning and Engineering. Fifty-eight participants joined the online public participatory decision-making process.


The Usability Metrics.

The descriptive statistics for the usability metrics are summarized in Table 2. For the number of major tasks completed successfully on the first attempt, the data shows that most of the participants performed at least two major tasks. The average time to perform a task on the first attempt is 131 seconds, but the standard deviation of 74 seconds indicates that the participants spent a wide range of time finishing the task. The average subjective satisfaction level of using the ArgooMap is fairly high (slightly greater than three).

The Public Engagement Metrics.

Descriptive statistics of the degree of public engagement metrics (the total time of stay on the Web site, the total number of visits, the number of page views, the number of comments posted, and the number of times to read comments) are presented in Table 3. The total time spent on the Web site varies from 325 seconds (or about five minutes) to about 80 minutes, with an average of 18 minutes. The number of visits ranges from 1 to 3, and 62 percent of the participants visited the Web site once. The number of page views varies from 5 to 12, with the mean value of 7. In terms of the number of comments posted, 69 percent of the participants did not post a single comment. When it comes to the number of times the users read comments, more than half of the participants did not read comments and suggestions from others.

The Relationships between Usability and the Degree of Public Engagement

A summary of Spearman's correlation coefficients p (Spearman 1904) for the usability metrics and the degree of public engagement metrics is shown in Table 4. The correlation results provide detailed relationships among system usability metrics and the degree of public engagement measures. It demonstrates that different aspects of system usability have very diverse effects on various aspects of the degree of public engagement.

A longer time of stay on the Web site could maintain user interest in the site (Bucklin and Sismeiro 2003) and give users more time to consider and make a decision. As shown in Table 4, the total time of stay on the Web site has a significant relationship with effectiveness ([rho] = 0.377, p < 0.01). However, the measure has statistically insignificant relationships with the efficiency and satisfaction metrics. These results are inconsistent with the findings of Langerak et al. (2003). In their research, satisfaction with the Web site (used to facilitate member-member and member-organizer interactions in their study) has positive effects on the member participation, which is quantified by visit frequency and duration. The correlation results show that the system effectiveness is a main determinant for users' visit duration on the Web site holding ArgooMap. The higher the system effectiveness that users obtained from interacting with the Web site, the longer they would stay on the Web site. This finding partly supports research conclusions by Danaher et al. (2006). They argue that Web site usability is one factor that has been shown to be correlated to Web site likability and the length and the depth of the visit.

The total number of visits is significantly correlated with two usability metrics: satisfaction ([rho] = 0.312,p < 0.05) and efficiency ([rho] = -0.365, p < 0.01), but it is insignificantly correlated with the system effectiveness (see Table 4). The correlation results suggest users' satisfaction and system efficiency can be two major factors that determine the participants' total number of visits. These findings are consistent with previous Web site usability research (McKinney et al. 2002, Langerak et al. 2003, Pearson et al. 2007). These studies suggest that the higher the level of users' satisfaction with browsing the Web site, the larger the number of return visits. In the context of WebGIS usability, Lee (2000) suggests that the Web site usability has a direct effect on the likelihood of visiting the Web site. Nielsen (2003) argues that if the customer finds the site too difficult to use, there will be no return visits. Brandtzasg and Heim (2008) point out that low usability of online community Web sites is one of the main reasons why users decrease their participation frequency over time until they completely cease visiting the sites. The research findings at least partly support those conclusions by demonstrating that the system efficiency and users' satisfaction have a significant influence on the number of total visits.

The number of page views is a term that often is used in Web-based marketing and advertising for predicting revenue. This measure is used here because the online public participatory decision-making process is completely Web-based so that the users are expected to go through all the pages to obtain the relevant information for resolving the parking site selection problem. As shown in Table 4, the number of page views is significantly correlated with the system efficiency measure ([rho] = -0.296, p < 0.05). However, the system effectiveness and users' satisfaction have no significant relationships with the number of page views. The correlation between the number of page views and the system efficiency indicates that the system efficiency is a major source influencing the number of page views. The higher the system efficiency (less time spent on performing the task on first attempt), the greater the number of pages the participants would view. Nielsen (1998) notes that because of its growing attention on Web site usability, has experienced a 28 percent average increase in page views each year and an increase of 15 percent per year in earnings per page view. Farrell-Vinay (2008, p. 300) points out that "the website page views increase when the usability has been improved." Thurow and Musica (2009, p. 95) suggest that "the page views increase after improving the usability of the website" in the context of Web searching. This study partly supports the previous research conclusions by showing one aspect of usability (the system efficiency) has a significant relationship with the number of page views.

Different ideas, suggestions, and discussions regarding the parking site selection problem are vital to the success of the project. This is because the participants may have the knowledge that planners don't have; the participants can exchange their views to get fresh thoughts, understand each other's point of view, and compromise. Therefore, both reading existing comments and posting new comments (initiating a new thread or replying to an existing topic) have been encouraged. As shown in Table 4, statistically significant correlations have been found between the number of comments posted and the system effectiveness ([rho] = 0.489, p < 0.01) and the system efficiency ([rho] = -0.342, p < 0.01). The number of times to read comments also is highly correlated with the system effectiveness ([rho] = 0.731, p < 0.01) and the system efficiency ([rho] = -0.413,p < 0.01). The correlation coefficients indicate that the system effectiveness and efficiency can be considered as two major sources having an impact on the degree that the participants interact with each other. The higher the system effectiveness and efficiency obtained by the participants, the greater the number of comments they would read and post. However, the number of comments posted and the number of times to read comments are insignificantly correlated with users' satisfaction. Kim (2000) and Preece (2000) have independently presented a set of design principles and strategies for building social interaction Web sites that focus on users and their interactions. Their strategies overlap with each other on the design principle: the Web design for usability. Specifically, the principle focuses on improving usability so that users can interact and perform tasks easily and effectively. Although the principle is advanced in the area of Web designing for social interaction, it also is applicable in the area of Web-PPGIS where user interaction is a vital element for a successful application. The results of correlation analysis emphasize the importance of the principle by showing that two aspects of system usability (effectiveness and efficiency) significantly affect user interactions.


Considerable advances have been made in the development of Web-PPGIS applications over the past decade or so. However, there are still many issues surrounding the systems' usability. This paper has focused on examining the system usability as a determinant of the degree of public engagement. The relationships between the three aspects of the system usability (effectiveness, efficiency, and satisfaction) and the five metrics that quantify the degree of public engagement have been examined individually. At least one aspect of the system usability has a significant effect on each of the five degrees of public engagement measures.

The results of this research show that the system effectiveness has a strong influence on the users' duration on the Web site and interactions with each other. This suggests that Web-PPGIS designers should focus on improving the system features, such as navigating the Web site, locating desired documents, and enhancing content; design functions that work with standard Web browsers; and choose the right resolutions to attract users to stay longer on the Web site and interact more with others. The study has shown that the system efficiency has a significant impact on users' number of visits, number of page views, and interactions with others. These findings indicate that Web-PPGIS designers should advance and highlight certain features, such as using fewer buttons and clicks to get to the destination page, speeding up page loads, reducing steps in the process, and reducing the amount of information to be filled out so users visit the Web sites more frequently, view more pages, and interact more often with other participants. The level of satisfaction is significantly correlated with the number of visits. This finding indicates that Web-PPGIS designers should modify and enhance some features--such as making the task more obvious and intuitive so it is more easily completed by users--to attract users to the Web site more often.

The Web-PPGIS infrastructures (the Internet, discussion forum, GIS, decision support tools, etc.) alone are insufficient to retain and promote the degree of public engagement. The correlations between the system usability and the degree of public engagement show that an effort in improving Web-PPGIS usability is justified if a goal of the system applications is to enhance the degree of public engagement. The correlations also indicate that Web-PPGIS employed to ensure public participation in the project can itself become an obstacle for certain participants to effectively engage in the decision-making process. Notwithstanding all the advantages brought by using Web-PPGIS, the system can be used as a complementary tool rather than a replacement for the traditional public participation methods (such as public meetings, poster demonstrations, etc.). Some participants have great difficulties using this type of system for participatory planning. As a result, those participants may not be convinced that Web-PPGIS tools are "better" than the conventional methods and this can lead to a further division with respect to public participation using Web-PPGIS in the future. Therefore, we suggest that usability testing approaches (user-testing methods) should be integrated into the system design process so that Web-PPGIS designers can detect the usability problems and make changes to the system prior to the public participatory planning process.


This research was supported by the GEOIDE Network (Project: HSS-DSS-17) of the Networks of Centers of Excellence. The authors would like to thank Gary Buxton, senior manager of the Planning and Engineering Department of the town of Canmore, for his ongoing support in data preparation, criteria selection, and preliminary testing for ArgooMap application. Also, we would like to thank Carsten Kessler, Claus Rinner, and Soheil Boroushaki, who contribute greatly to ArgooMap development and customization. We acknowledge all participants in the case study for their contribution to this research project. We wish to thank anonymous reviewers for their valuable comments on an earlier draft of this manuscript.


Arnstein, S. R. 1969. A ladder of citizen participation. Journal of the American Institute of Planners 35(4): 216-24.

Atterer, R., A. Schmidt, and M. Wnuk. 2007. A proxy-based infrastructure for Web application sharing and remote collaboration on Web pages. Proceedings of the 11th International Conference on Human-Computer Interaction, Rio de Janeiro, Brazil, September 2007, 74-87.

Banati, H., P. Bedi, and P. S. Grover. 2006. Evaluating Web usability from the user's perspective. Journal of Computer Science 2(4): 314-17.

Berner, M. 2001. Citizen participation in local government budgeting. Popular Government Spring: 23-30.

Boroushaki, S., and J. Malczewski. 2010. Participatory: A WebGIS-based collaborative multicriteria decision analysis. URISA Journal 22(1): 23-32.

Brandtzasg, P. B., and J. Heim. 2008. User loyalty and online communities: Why members of online communities are not faithful. Proceedings of 2nd International Conference on Intelligent Technologies for Interactive Entertainment, Cancun, Mexico, January 2008.

Bucklin, R. E., and C. Sismeiro. 2003. A model of Web site browsing behavior estimated on clickstream data. Journal of Marketing Research 40(3): 249-67.

Butler, K. A. 1996. Usability engineering turns 10. Interactions 3(1): 59-75.

Chen, S. J., and C. L. Hwang. 1992. Fuzzy multiple attribute decision making: Methods and applications. Berlin: Springer-Verlag.

Chua, Y. L., and S. Wong. 2002. Data intermediation and beyond: How the Web modifies the dissemination of GIS information. Proceedings of the 2rd Annual PPGIS Conference, New Brunswick, New Jersey, July 2002.

Craig, W. J., and S. A. Elwood. 1998. How and why community groups use maps and geographic information. Cartography and Geographic Information Systems 25(2): 95-104.

Danaher, P. J., G. W. Mullarkey, and S. Essegaier. 2006. Factors affecting Web site visit duration: A cross-domain analysis. Journal of Marketing Research 43(2): 182-94.

Farrell-Vinay, P. 2008. Manage software testing. Boca Raton: Taylor and Francis.

Haklay, M., and C. Tobon. 2003. Usability evaluation and PPGIS: Towards a user-centered design approach. International Journal of Geographical Information Science 17(6): 577-92.

Haklay M., and A. Zafiri. 2008. Usability engineering for GIS: Learning from a screenshot. The Cartographic Journal 45(2): 87-97.

Hopkins, L. D., M. Twidale, and V. G. Pallathucheril. 2004. Interface devices and public participation. Proceedings of the 3rd Annual PPGIS Conference, Madison, Wisconsin, July 2004, 71-83.

IAP2 (International Association for Public Participation). 2004. Public participation for decision makers, training/decision-fact-sheets.pdf, October 6, 2009.

Ingensand, J., and F. Golay. 2010. Task-oriented usability evaluation of a Web-GIS for a real world community. URISA Journal 21(1).

ISO (International Organization for Standardization) 9241-11. 1998. Ergonomic requirements for office work with visual display terminals--part 11: Guidance on usability, ISO.

Kessler, C. 2004. Design and implementation of argumentation maps. Diploma Thesis, Westfalische Wilhelms Universitaty, Munster, Germany,, October 9, 2009.

Kim, A. J. 2000. Community building on the Web. Berkeley: Peachpit Press.

Kingston, R., S. Carver, A. Evans, and I. Turton. 2000. Web-based public participation geographical information systems: An aid to local environmental decision making. Computers, Environment and Urban Systems 24(2): 109-25.

Kingston, R. 2007. Public participation in local policy decision making: The role of Web-based mapping. The Cartographic Journal 44(2): 138-44.

Klosterman, R. E. 1999. The what if? collaborative planning support system. Environment and Planning B: Planning and Design 26(3): 393-408.

Krek, A. 2005. Rational ignorance of the citizens in public participatory planning. Proceedings of CORP and Geomultimedia Conference, Vienna, Austria, April 2005.

Langerak, F., P. C. Verhoef, P. W. J. Verlegh, and K. Valck. 2004. The effect of members' satisfaction with a virtual community on member participation. ERIM Report Series Research in Management, Erasmus Research Institute of Management, Erasmus Universiteit, Rotterdam, Netherlands.

Lee, A. T. 2000. Web site usability, usefulness, and visit frequency. Proceedings of Human Factors and Ergonomics Society Annual Meeting, San Diego, California, July-August, 2000, 1, 404-7.

May, J. 2007. The triangle of engagement: An unusual way of looking at the usual suspects. Public Money and Management 27(1): 69-75.

McKinney, V., K. Yoon, and F. Zahedi. 2002. The measurement of Web-customer satisfaction: An expectation and disconfirmation approach. Information Systems Research 13(3): 296-315.

Nielsen, J. 1993. Usability engineering. Boston: Academic Press.

Nielsen, J. 1998. Why Yahoo is good (but may get worse). Http://, October 10, 2009.

Nielsen, J. 2003. Usability 101: Introduction to usability. Http://, April 26, 2009.

Pasi, G., and R. R. Yager. 2006. Modeling the concept of majority opinion in group decision-making. Information Sciences 176(4): 390-414.

Pearson, J. M., A. Pearson, and D. Green. 2007. Determining the importance of key criteria in Web usability. Journal of Management Research News 30(11): 816-28.

Peterson, E. T. 2008. Measuring the immeasurable: Visitor engagement, research and analysis from Web analytics demystified, the Web analytics thought leaders. Http:// Web_Analytics_Demystified_and_NextStage_Global_-_Measuring_the_Im- measurable_-_Visitor_Engagement.pdf, October 1, 2009.

Pickles, J. 1995. Representations in an electronic age: Geography, GIS, and democracy. In J. Pickles, Ed., Ground truth. New York: Guilford Press, 1-30.

Preece, J. 2000. Online communities: Designing usability, supporting sociability. England: John Wiley and Sons.

Preece, J., R. Yvonne, and S. Helen. 2002. Interaction design: Beyond human-computer interaction. New York: John Wiley and Sons.

Rabinovitch, J. 2004. Transforming community practice: [re] moving the margins. Unpublished Doctoral Dissertation, Union Institute and University, papers2005/rabinovitch/contents.htm, September 21, 2009.

Rambaldi, G., and J. Callosa. 2000. Manual on participatory 3-dimensional modeling for natural resources management. Essentials of protected areas management in the Philippines. Vol. 7, NIPAP, PAWB-DENR, Philippines.

Rinner, C. 2001. Argumentation maps--GIS-based discussion support for online planning. Environment and Planning B: Planning and Design 28(6): 847-63.

Rinner, C. 2006. Argumentation mapping in collaborative spatial decision-making. In S. Dragicevic and S. Balram, Eds., Collaborative GIS. Hershey: Idea Group Publishing, 85-102.

Rinner, C., C. Kessler, and S. Andrulis. 2008. The use of Web 2.0 concepts to support deliberation in spatial decision-making. Computers, Environment and Urban Systems 32(5): 386-95.

Sidlar, C., and C. Rinner. 2007. Analyzing the usability of an argumentation map as a participatory spatial decision support tool. URISA Journal 19(1): 47-55.

Sieber, R. E. 2006. Public participation and geographic information systems: A literature review and framework. Annals of the American Association of Geographers 96(3): 491-507.

Simao, A., P. J. Densham, and M. Haklay. 2009. Web-based GIS for collaborative planning and public participation: An application to the strategic planning of wind farm sites. Journal of Environmental Management 90(6): 2,027-40.

Spearman, C. 1904. The proof and measurement of association between two things. The American Journal of Psychology 15: 72-101.

Talen, E. 1999. Constructing neighbourhoods from the bottom up: The case for resident-generated GIS. Environment and Planning B: Planning and Design 26(4): 533-54.

Thurow, S., and N. Musica. 2009. When search meets Web usability. Berkeley: New Riders.

Town of Canmore. 2007. The municipal development plan. Http:// business-development/ business-development.html, October 21, 2009.

Weidemann, P. M., and S. Femers. 1993. Public participation in waste management decision making: Analysis and management of conflicts. Journal of Hazardous Materials 33(3): 355-68.

Yager, R. R. 1996. Quantifier guided aggregation using OWA operators. International Journal of Intelligent Systems 11(1): 49-73.

Corresponding Address:

Yunliang Meng, Ph.D.

Department of Geography

The University of Western Ontario

1151 Richmond Street

London, Ontario, Canada, N6A 5C2

Jacek Malcewski, Professor

Department of Geography

The University of Western Ontario

1151 Richmond Street

London, Ontario, Canada, N6A 5C2
Table 1. Ladders of public participation

Arnstein (1969)    Weidemann and       IAP2 (2004)
                   Femers (1993)

Citizen Power      * Public            Empower
* Citizen control  participation in    * Implement what
* Delegated power  final decision      the public decides
* Partnership      * Public            Collaborate         Increasing
                   participation in    * To identified     Public
Tokenism           assessing risks,    preferred solution  Participa-
* Placation        and recommending    with the public     tion of
* Consultation     solutions           Involve             Citizen
* Informing        * Public            * To work directly  Control
                   participation in    with the public
Nonparticipation   defining interests  Consult
* Therapy          and actors and      * To obtain public
* Manipulation     determining agenda  feedback
                   * Public right to   Inform
                   object              * To keep people
                   * Informing the     informed
                   * Public right to

Source: Schlossberg and Shuford 2003, LAP2 2004

Table 2. Descriptive statistics for the usability metrics

Usability Metrics                Minimum   Maximum   Mean     Standard

Effectiveness: Number of major   1         3         2.43     0.60
tasks completed successfully
on the first attempt

Efficiency: Time to perform a    26        375       131.07   74.36
task on the first attempt (in

Satisfaction: A scale from 0     0         5         3.02     1.33
to 5

Table 3. Descriptive statistics for the degree of
public engagement metrics

Public Engagement Metrics           Minimum  Maximum  Mean  Standard

Total time of stay on the Web site  329      4765     1100  882.91
Total number of visits              1        3        1.43  0.59
Number of page views                5        12       6.95  1.57
Number of comments posted           0        8        0.74  1.45
Number of times to read comments    0        6        1.62  1.99

Table 4. Correlations between the usability and the degree of public
engagement metrics

                Total Time of Stay   Total Number of   Number of Page
                on the Web Site      Visits            Views

Effectiveness   0.377 **             0.225             0.249
Efficiency      -0.082               -0.365 **         -0.296 *
Satisfaction    0.151                0.312 *           0.136

                Number of         Number of Times
                Comments Posted   to Read Comments

Effectiveness   0.489 **          0.731 **
Efficiency      -0.342 **         -0.413 **
Satisfaction    0.247             0.217

Note: ** Correlation is signification at p < 0.01 (two-tailed test).

* Correlation is signification at p < 0.05 (two-tailed test).
COPYRIGHT 2010 Urban and Regional Information Systems Association (URISA)
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2010 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Meng, Yunliang; Malczewski, Jacek
Publication:URISA Journal
Article Type:Case study
Geographic Code:1CANA
Date:Jan 1, 2010
Previous Article:Task-oriented usability evaluation of a WebGIS for a real-world community.
Next Article:Geospatial education at U.S. community colleges: background, challenges, and opportunities.

Terms of use | Privacy policy | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters