Printer Friendly

A Delphi study into the audit challenges of it governance in the Australian public sector.

I. INTRODUCTION

Information Technology (IT) has become an indispensable element for success in the contemporary business world, due to the increase dependency on IT to support, sustain, and drive the growth of the business [1]. IT is not just critical to the private sector, but has also become integral to the public sector in delivering efficient and cost-effective services to the public [2]. Therefore, in order to sustain service provision the public sector need to undertake suitable governance of the IT environment and processes to address the intensifying focus on government accountability by taxpayers and the Parliament [3]. The centrality of IT to the capability and capacity needed for sustaining normal services makes research on the means to provide effective Information Technology Governance (ITG) in the public sector of critical importance [4, 5].

ITG consists of structures, processes, and relational operational mechanisms working together as one entity to ensure that IT and business objectives are aligned [6]. The cornerstone of ITG is the provision of an acceptable level of assurance that an organisation's strategic objectives are not jeopardised by IT failures [7]. A conventional, or rather, inevitable approach for measuring this acceptable level of assurance includes thorough audit and quality assessment. Audit is a discipline born of the need to assess the degree of conformation with standard practice [8] and addresses a wide range of assurance and consulting services through the utilisation of methodologies and frameworks. Audit activities in such contexts seek to provide a credible level of assurance of rigour in ITG by systematically examining controls efficiency, identify key risk areas, advise about possible IT failures, as well as offer suggestions on how to improve current practices [9]. In this case, IT/IS auditors are responsible for the assessment of the structures, processes, and relational mechanisms competency, in what we refer to in this paper as ITG audit.

This research focuses on providing insight into the range of ITG audit challenges currently facing public sector organisations and likely future challenges. The focus of this practice-oriented research is relatively new and less researched in academic literature where work has focused on ITG implementation [10-12], factors influencing implementation [2, 11, 13] or on audit frameworks generally [14-18]. However, little research is available on the challenges organisations encounter in seeking to have effective audit capability, and which ones might be given more priority than others. The aim of this research is to contribute to an enhanced understanding of the challenges of implementing effective ITG and to provide guidance on ITG audit challenges.

II. BACKGROUND

A. Revisiting IT governance and Audit

ITG, at a basic level, is a subset discipline of Corporate Governance focused on information technology initiated by compliance requirements, such as Sarbanes-Oxley (SOX) in the USA, Basel II in Europe, and CLERP 9 in Australia [19]. Ideally, ITG bridges the gap between IT initiatives and business goals [20].

Sound implementation of ITG assists organisations in achieving critical success factors, protect investment in IT through the efficient and effective use of IT resources [21], and contributes directly to high business performance [22]. On the other hand, organisations with ineffective ITG may risk loss from decreasing value of IT investments [23] due to project failures [24, 25], or the inability to utilise organisational resources effectively [26].

Several frameworks exist to assist organisations in ITG implementation and assessment [4], such as the Control Objectives for Information and Related Technologies (COBIT) framework, has emerged as the most widely accepted ITG compliance and audit tool. Currently the latest version of COBIT 5, divides IT into five domains (Evaluate, Direct and Monitor; Align, Plan and Organise; Build, Acquire and Implement; Deliver, Service and Support; and Monitor, Evaluate and Assess) which are broken into 37 high-level processes and 300 detailed IT controls covering aspects of IT management and governance [27]. Due to COBIT's 5 massive size, it has been suggested that a considerable amount of work is needed to establish practical methods to utilise this framework as an audit tool [28].

The traditional perceptions of the audit function as being a corporate watchdog have changed dramatically over the years to be perceived as a tool for monitoring compliance [29]; to operate in a complex environment due to the extensive utilising of IT; and to present an advisory function that provides management with assurance that controls governing IT are adequate. Two types of audit exist in the public sector: financial-related audits and performance audits [30]. ITG audit is considered as part of the latter--often referred to as value-for-money audits--which provide an independent assessment of the efficient and effective use of public resources to improve public accountability and facilitate decision-making [31].

A new role of auditors within ITG provides great development opportunities to IS/IT auditors [32], resulting in better value delivery to organisations [33], and providing management with an independent assessment of the effect of IT decisions on the business [34]. Ultimately, success with measuring the value from IT as a result of ITG will come only when all stakeholders (IT and business) work together with a common understanding through engaging IS/IT auditors in ITG initiatives [35].

B. Public Sector ITG

The Public and Private sectors can be defined by the level of government or market influence on ownership and control as displayed in Figure 1. Public sector entities have a specific obligation to provide services to all citizens through the utilisation of tax payers' money while maintaining the highest levels of integrity and ethical values [36]. A distinguish characteristic of the public sector in Australia seems to be the way it falls behind the private sector when it comes to IT investment, due to challenging budgetary constraints [37]. Public sector entities are under increasing pressure to exhibit transparency and accountability in using taxpayers' money to deliver outcomes at the same time as operating under greater budgetary constraints, higher complex regulatory requirements, and struggling to attract staff when compared to the private sector [25]. In consequence, the need to govern public and private sector entities in a different way is becoming a necessity.

In Australia, the public sector regularly relies on IT to deliver quality and accessible services to the community, such as E-Government services. Given the fact that the Commonwealth Government of Australia spent $5.19 billion in 2010-11 on IT [39], the effective governance of IT becomes crucial to public sector organisations for achieving full optimisation of IT investments [2].

While a deal of literature exists that has examined ITG in the private sector [4, 40], little research has been completed on this practice within public sector organisations [2, 41] despite the recognition of the value of effective ITG to the success of these organisations [42]. A factor that has been noted is that ITG in the public sector is more complex to that in the private sector because of differences in environmental factors, organisation-environment transactions, and internal structures and processes [43, 44]. Consequently, it is more important to establish control over IT in the public arena [45]. Further, getting decision-makers to recognise the value of aligning IT initiatives and business objectives has been a common challenge across the terrain of the Australian public or private sector. However, a "one size fits all" approach for ITG is not practical due to profound differences between the two sectors and a common mistake in future research would certainly be the failure of addressing these differences [46].

C. Research Aim and Scope

The aim of this research is to provide insight into the range of IT governance audit challenges currently facing Queensland public sector organisations and likely future challenges. The Queensland Public sector was chosen as our research participant because its organisational structure and public sector objectives are not substantially different to other jurisdictions within Australia. Further, it is likely that their public sector objects will significantly correspond to other public sector jurisdictions globally other than different cultural aspects may have an influence. Cultural influences though highly important are outside the scope of this research. To address this goal, this research is built around these key questions:

1. What are the significant challenges and why they are important?

2. What are the perceived top ten IT governance audit challenges in the Queensland public sector?

The achievement of this aim is likely to offer a range of benefits to both the auditor and the audited organisation. Prioritising those challenges provides public sector organisations an opportunity to focus on critical concerns and identifies common challenges across the sector. Additionally, ITG audit challenges tailored for a specific sector are likely to be better accepted and more relevant. Lastly, public sector organisations will be able to identify ITG audit challenges quick-wins.

As this research is categorised as applied research, the research scope will be narrowed down on multiple aspects in order to maintain a sufficient level of internal validity [47]. In the first instance, the focus of this research is restricted to the public sector to control the contingencies resulting from differences in sectors. The scope was also reduced in geographic terms and took into account the size of organisations within the public sector. To avoid cultural differences between regions worldwide and contingencies related to the size of the organisations, it was decided to only focus on Queensland public sector organisations with headcounts ranging from 100 to more than 1000 employees. The final scope reduction focuses on the different types of audit. Financial related audit will be discarded as this research focuses only on ITG audit as part of performance audit.

III. RESEARCH METHODOLOGY

This research has an exploratory focus as research in this domain is in its early stages in Australia and there has been little research material developed. The combination of research methods (critical review of literature, informal small group discussions, and Delphi research) constitutes a triangulation of data collection sources in order to gain a complete understanding of audit challenges in ITG.

The first step in this research was aimed at exploring the research domain of ITG/audit through a detailed literature research, focused on identifying an initial list of challenges and issues that organisations might encounter. To complement the initial list, informal discussions were organised involving IS/IT audit and ITG experts to gather feedback on the applicability to Queensland public sector.

After the first exploration, the Delphi research methodology was considered an appropriate research design for this type of exploratory research as it lends itself well to the creation of understanding and theory building on complex issues [48]. To ensure the quality and accuracy of data, special attention was given to selecting qualified panel experts. It is forecasted that 10 to 15 participants may be adequate for a focused Delphi [49]. Based on these considerations, an expert panel was assembled of 28 professionals who are all experienced in Queensland public sector organisations. From the initial group, 24 experts continued to be in the second round (14% drop off rate), 20 experts continued to be the third round (28.5% drop off rate) and 16 experts were involved in the full research effort (total 42.8% drop off rate).

Using the Delphi method, the experts were required to complete an email survey consisting of a three-round questionnaire instrument. Similar to the recent Delphi research work of De Haes and Van Grembergen [50], the first round began with a preceded initial list of ITG audit challenges and respondents were only asked to validate its suitability to the public sector. In the second round, respondents were asked to rate on a 5-point scale, for each of the revised ITG audit challenges, the "perceived impact" (0 = no impact, 5 = high impact) and the "perceived effort to address" (0 = no effort, 5 = high effort). Then, they were asked to take the previous attributes of impact, effort to address, and personal experience into account in order to provide their perception of the top-10 ITG audit challenges.

In the third and final round, the panellists were asked to re-evaluate their round two ratings, taking the group averages into consideration. The goal of this round was primarily to come to a greater consensus within the group. At the end of this round, the degree of consensus between the experts was measured leveraging Kendall's W coefficient scale, specifically for the question on the top-10 ITG audit challenges. The level of consensus reached in this research was 0.49, which is considered moderate and provides a fair degree of confidence in the results [51]. Based on this result and the fact that the top-10 challenges only slightly differed between the rounds, it was decided that no more iterations is required.

In this type of research, the issue of "inadequate preoperational explication of constructs threat" presented itself as an obstacle, which in simple terms indicates that different people often have different understandings of the same concept [47]. A good example is the use of the audit", and "audit" in general. Although they are clearly distinguished in literature, many organisations and practitioners are using these terminologies interchangeably or refer to one of the other terminologies. To solve this, a clear definition was provided (based on literature) in the questionnaire. The questionnaire was also pilot-tested on five experts (practitioners and academics) for ambiguities and vagueness prior to administering to panel members.

IV. RESULTS AND INTERPRETATIONS

Different challenges and issues were identified based on literature research and discussion with informants. The need to group the challenges into logical categories emerged as an important aspect of the research. Consequently, three categories were created, namely, Internal (N), External (E), and Organisational (O) audit challenges. Each category contains challenges that attributes to the category's label (e.g., challenges in the internal audit category represent challenges associated with internal audit, etc.). Based on the findings of this step, an initial list of ITG audit challenges was composed, as shown in Table 1.

The results of the Delphi three-round survey are discussed following.

A. Round 1--Validating the Initial list

Respondents in this round were asked to validate this general list of challenges to make it more oriented toward the public sector. All data was structured and analysed resulting in an extended list of challenges, as illustrated in Figure A-1. Based on this round, an updated list of challenges was used as basis to start up rounds 2 and 3.

B. Round 2 and 3--Evaluating ITG Audit challenges

As illustrated in Figure A-1, the research demonstrated that, according to the panel of experts, some of the identified challenges have higher impact or require more effort to address compared to others. However, the dominance of organisational challenges is clear as they occupy four out of the top five ranks for impact (Figure 2) and required effort (Figure 3). This falls in line with previous research that highlighted the lack of board-level understanding and support when it comes to ITG [1, 61, 62]. This also emphasises the effect of organisational changes and various committees on ITG [63-65], and stresses the importance of auditors experience to the success of the ITG audit program [54, 55].

Since numerous ITG definitions pinpoint the prime responsibility of the board of directors in ITG [66, 67], it is no surprise that these results reveal that challenges relating to the board of directors are amongst the top ranked challenges for impact on ITG audit, which is confirmed by also being ranked relatively high on the required effort. This can be attributed to the fact that making the board of directors more knowledgeable about ITG and associated audit activities is not easy to achieve [68]. Potentially, the results of this research raise questions on how public sector can increase the board's involvement in practice.

Identifying "quick wins" is a general priority, defined as an audit challenge that is considerably high in impact and generally requires minimal effort that can be implemented in a short period of time, or requires reduced resources in a timely manner and cost effective. The main quick wins are "insufficient skills and competencies to undertake effective ITG audits," "inadequate evaluation and testing of the effectiveness of IT governance," and "failure of an audit team to appropriately apply required substantive auditing procedures." Looking closely at the previous challenges shows that they all belong to the internal audit category, and focus on the audit team involvement in ITG audit. Notwithstanding, respondents considered these challenges to be easy to address. This result is also supported by earlier research which identified the crucial need for auditors training [69], and continuous knowledge development as technology and standards change [70] to build the essential expertise [54].

An understanding of the audited organisation business, IT strategy, and the ITG structures should be obtained by the auditor prior to conducting an audit [71]. In the past, IS/IT auditors often focused on mere compliance and have repeatedly utilised long lists of weaknesses instead of providing positive assurance to the organisation [72]. This is changing due to ITG initiatives driving the implementation of effective management structures and controls. Thus, creating opportunities for IS/IT auditors to become providers of assurance to management [73].

Averaged responses for impact and effort for internal, external and organisational challenges (see Figure 4), indicate that organisational and internal audit challenges are in general perceived as having a higher impact on the public sector than external audit challenges. However, it appears that internal and external audit challenges are perceived as being easier to address compared to organisational audit challenges. Although in many cases internal and organisational challenges are closely related. A good example here is the "lack of executive support for, resource allocation to and understanding of extensive ITG audit", which is a crucial element to address the "insufficient skills and competencies" challenge through the provision of training, but the latter is perceived as easier to address compared to the former challenge.

Figure 4 also shows that external are perceived as requiring less effort to address compared to organisational challenges, probably because some of the implemented solutions in the public sector for the latter are considered ineffective, e.g. ineffective audit committees [74]. Contrary to the implemented solutions for external challenges which are perceived to have a more useful result, e.g. communication and coordination between IT executive/senior management and external audit [3, 75].

Next, key factors (background, role, and years of experience) derived from the Delphi analysis will be assessed to develop a better understanding of perceptions of the challenges for these factors (see Table 2).

As displayed in Figure 5 and consistent with the entire group analysis, officers with less than 10 years of experience and an audit background rated "lack of executive management ITG ownership and accountability" as the highest challenge on impact, while the same challenge was rated second-highest by managers with more than 10 years of experience and a mix discipline background. In addition, the same group (officers with less than 10 years of experience and an audit background) considered the "discovery may be slow or non-existent if information is masked" challenge to require more effort than the opposing group (managers with more than 10 years of experience and a mix discipline background). This leads to the assumption that less experienced junior staff from the audit discipline place more emphasis on the role of the board and organisational culture for the success of ITG audit in the public sector.

Furthermore, managers with less than 10 years of experience rated "difficulty to recruit and retain experienced ITG auditors" as the highest challenge on effort whilst the opposing group (officers with more than 10 years of experience) did not perceive it to have a high impact. This illustrates the expectation gap between different levels, roles and background within public sector organisations as experienced staff perceive recruiting and training of auditors important while less experienced managers think it is difficult. The same group (managers with less than 10 years of experience and an audit background) considered "organisational changes impacting roles, responsibilities and stability of the ITG model" to require an application of a substantial amount of effort. Possibly because senior staff from an audit background realise the difficulty of obtaining buy-in from newly appointed management and/or appreciate the effort required to establish a new effective committees.

The rating of impact and required effort by managers with more than 10 years of experience and a mix discipline background (see Figure 5) indicates that there should be greater appreciation of risk in conducting ITG audit in the public sector. Contrary to ratings by officers with less than 10 years of experience and an audit background of low impact and effort. In other words, experienced decision- makers would prefer to focus on risk-based audit while junior auditors would rather perform the traditional one- size-fit-all controls testing (compliance) audit instead.

The largest exception based on responses for impact and effort to address for all the internal, external and organisational challenges is the rating of officers. They perceived internal challenges to require more effort to address than external ones, which is contrary to the entire group average. There seems to be a direct correlation between their perception and the fact that they were the only subgroup to highlight the "lack of developed methodologies and tools" challenge as a priority (rated high on both impact and required effort). Potentially, this would raise questions on what measures can public sector managers from any discipline introduce to assist staff overcome the challenge they stumble upon relating to methodologies in practice.

Another outcome of the Delphi research is the ranking list of ITG audit challenges, specifically for the Queensland public sector. The respondents were asked to build up this ranking list in terms of top-10 challenges, taking the attributes of impact and effort to address into account, together with their professional experience. Figure A-1 shows the top 10 resulting from this ranking exercise, including the mean and total ranking score.

Figure A-1 brings it all together, plotting the previous results on two axes. The vertical axe measures the "perceived effort to address" while the horizontal axe addresses the "perceived impact." The challenges in the grey shape are the ones identified as being the top-10 ITG audit challenges for the Queensland public sector.

The majority of the challenges have high impact and are difficult to implement, apart from "limited knowledge within the audit team of emerging risk exposures related specifically to the audited organisation" and "Repetition of audit activity in place of identification of systemic control failures". Those two external audit challenges were rated low on impact and effort to address, yet appeared in the top-10 list. A possible explanation is that, just as in literature, there is a growing focus on risk-based audit approach and recognising differences in the nature of business instead of the traditional one-size-fit-all controls testing (compliance) approach [58, 76]. In addition, the scope of audit has expanded to include the evaluation of the effectiveness of governance processes [77]. To that end, auditors are increasingly finding it necessary to understand the unique risks associated with each different organisation being audited [78, 79].

An expected challenge to score high on both impact and effort to address is the "lack of developed methodologies and tools" as the need for methodologies and frameworks that enable executives to govern and manage the enterprise's use of IT in an effective and efficient manner has been identified since the early days of ITG. Many methodologies and frameworks have been developed in recent years to assist and evaluate the implementation of ITG. From an auditor's perspective, COBIT has a strong emphasis on monitoring and enables the assessment of existing ITG processes and structures [15, 80]. However, one of COBIT's disadvantages is that practitioners need a lot of knowledge of the framework to be able to conduct successful ITG performance assessments [81, 82]. Perhaps the previous proposed solution for auditors training should focus on this identified gap of specific ITG frameworks training.

V. CONCLUSION

This paper addresses the key questions identified in the Research Aim and Scope section. Regarding the first research question, this practice-oriented research reveals that Queensland public sector organisations are facing a wide range of internal, external and, organisational challenges in auditing ITG. The research identifies a list of 30 ITG audit challenges at level of executive/senior business and IT management. The results demonstrate that some of the identified challenges are regarded as having higher impact and/or easier to address than others. Examples of challenges that are perceived to have a high impact are insufficient skills and competencies, and inadequate evaluation and testing of the effectiveness of ITG controls. Other challenges are perceived as to have a fairly high impact but not easy to address. A good example is the lack of developed methodologies. Finally, some challenges are perceived as not having a high impact while others are perceived as not easy to address in the context of ITG audit, such as slow or non-existent discovery if information is masked, inconsistent, or made unavailable by the audited organisation. These challenges are less likely to come across in the conducting of ITG audit.

The research also assessed key factors derived from the Delphi analysis based on background, role, and years of experience to explore and highlight the effect of these factors on the perception of ITG audit challenges. It was observed that different key factors have different and sometime conflicting opinions. For instance, managers with less experience than audit experts tend to drive the audit towards a risk based approach, while the latter have a tendency to execute the traditional compliance audit.

This paper also brought up a list of top ten ITG audit challenges, specifically for the Queensland public sector in an effort to answer the second research question. This suggests that, in performing ITG audit within a public sector organisation, these challenges may play an important role in preventing a successful outcome (inhibiting factors). Of course, they should be supplemented with other challenges as required by the specific environment of the organisation, to create a specific set or subset of ITG audit challenges.

VI. LIMITATIONS AND FURTHER RESEARCH

It should be noted that the identified ITG audit challenges list is not exhaustive and the challenges at operational level are not addressed in this research. The research captured senior audit and IT managers' perception on the impact of the challenges on public sector organisations, yet did not seek a justification for their opinion. Perhaps this could be explored in future research.

The research measured the perceived effort to address the identified ITG audit challenges; however, it is out of the scope of this research to examine the consequences of not suitably addressing these challenges, to identify opinion on appropriate solutions or to investigate the scope of the resource requirements needed. Although, the authors acknowledge the importance of these aspects, yet they opt to address them in future research.

While this research is focused on the Queensland public sector, it can be expected that many conclusions might apply to other jurisdictions within Australia as well. Further research, focusing on other jurisdictions could support that assumption. Such research could also address the impact of other contingencies. It might for example be that organisations operating in Europe have very different views on what ITG audit challenges exist if compared to organisations operating in Australia.

VII. APPENDIX

Rank   Index   ITG audit challenge      Impact   Required    Total
                                                  Effort    ranking
                                                             score

1       E2     Limited knowledge         3.7       3.3        32
                 within the audit
                 team of emerging
                 risk exposures
                 related specifically
                 to the audited
                 organisation.
2       N1     Insufficient skills       4.2       3.3        33
                 and competencies to
                 undertake effective
                 IT governance
                 audits.
3       O7     Lack of executive         4.3       3.9        42
                 management IT
                 governance ownership
                 and accountability
                 for when audit
                 commitments are not
                 fulfilled
4       O2     Tendencv to focus on      4.2       3.4        47
                 mere compliance with
                 legislation rather
                 than quality.
5       N2     Inadequate                3.9       3.1        48
                 evaluation and
                 testing of the
                 effectiveness of IT
                 governance controls
                 with the purpose of
                 providing a "value-
                 added" service to
                 the organisation by
                 the audit team
6       O1     Difficulty to             4.0       3.9        54
                 recruit and retain
                 experienced IT
                 governance auditors
                 in the public sector
7       E10    Lack of focus in or       3.4       2.3        58
                 repetition of audit
                 activity place of
                 Identification of
                 systemic control
                 failures.
8       E3     Audited public            3.8       3.4        60
                 sector organisation
                 lack of necessary
                 skills or displaying
                 retinence to co-
                 operate.
9       N10    Inadequate                3.9       3.4        62
                 appreciation of risk
                 management in the
                 application of
                 controls or in
                 considering IT
                 governance control
                 of weakness
10      N3     Lack of developed         3.9        35        70
                 methodologies and
                 tools to keep pace
                 wick changes
                 occurring in the
                 auditing and
                 technology field.
11      O3     Lack of executive         3.8       3.4        73
                 support for,
                 resource allocation
                 to and understanding
                 of extensive IT
                 governance audit
                 programs
12      E5     Weak auditee and          3.3       2.7        75
                 auditor relationship
                 in the public sector
13      O4     Reduced influence         3.7       3.4        80
                 audit committees and
                 ill-established
                 internal audit
                 units.
14      N4     Lack of or                3.6       2.8        84
                 inadequate unders
                 tending of the
                 business contest to
                 determine what
                 aspects of audit
                 best fit the
                 relevant
                 organisation.
15      E7     Insufficient              3.4       3.2        91
                 evidence of IT
                 Governance
                 implementation
                 (methodology
                 practices and
                 processes).
16      O9     Public                    3.9       3.8        91
                 administration
                 tendency to deny
                 conceal systemic IT
                 governance problems
                 which prevents
                 identification and
                 remediation.
17      E6     Expectation gap           3.6       3.3        93
                 between public
                 sector perceptions
                 of audit and actual
                 audit practices.
18      N6     Failure of an audit       3.8       2.5        92
                 team to
                 appropriately apply
                 required
                 substanstive
                 auditing procedures,
                 planning processes
                 and reporting
                 findings to the
                 appropriate level.
19      O10    Organisation changes      4.0       3.6        101
                 impacting roles
                 responsibilities and
                 stability of the IT
                 governance model
                 both internally and
                 externally driven.
20      E9     Discovery may be          3.4       3.6        104
                 slow or non-
                 existent if
                 information is
                 masked,
                 inconsistent,
                 unusable or made
                 unavailable by the
                 audited
                 organisation.
21      N9     Lack of specific          3.1       3.0        106
                 legislative or
                 mandatory framework
                 to ensure a
                 consistent audit
                 approach.
22      E10    Inconsistent              3.1       3.0        108
                 execution of audit
                 methodology access
                 public sector
                 organisations.
23      N5     Poor training             3.1       2.7        118
                 arrangements for
                 public sector
                 auditors.
24      O6     Perceived low value       3.3       3.0        119
                 of IT governance
                 audits in comparison
                 to other IT audits.
25      E4     Pressure to               3.1       2.7        127
                 prematurely sign-
                 off on audit reports
                 whilst not following
                 specific legislative
                 requirements.
26      O8     Lack of                   3.6       3.1        128
                 communication
                 between business
                 units responding
                 separately to audit
                 recommendations
                 leading to gaps and
                 duplication in
                 compliance
                 activities.
27      N7     Poor scope                3.3       3.1        131
                 management due to
                 cross-agency service
                 models resulting in
                 imbalanced or
                 incomplete
                 perspective.
28      E8     IT Governance             3.3       2.8        132
                 assessment could be
                 subjective or bias
                 towards "more
                 positive" findings.
29      N8     Subsequent lack of        3.3       2.5        134
                 objectivity in the
                 conduct of audit due
                 to familiarity with
                 internal staff or
                 fear of management
                 weaknesses.
30      O5     Loss of continuity        2.9       2.8        145
                 caudle (audit cycle)
                 due to mandatory
                 audit rotation.


References

[1] S. Posthumus, R. V. Solms, and M. King, "The board and IT governance: The what, who and how," South African Journal of Business Management, vol. 41, no. 3, 2010, pp. 23-32.

[2] S. Ali, and P. Green, "IT governance mechanisms in public sector organisations: An Australian context," Journal of Global Information Management, vol. 15, no. 4, 2007, pp. 41-63.

[3] P. Barrett, "Evaluation and Performance auditing: sharing the common ground," Australasian Evaluation Society--International Conference, 2001, pp. 1-34.

[4] P. Weill, and J. W. Ross, "IT governance: How top performers manage IT decision rights for superior results," Harvard Business School Press, 2004

[5] S. Woods, "Governing IT in the public sector," IT News Africa, 13 August 2010, available at: http://www.itnewsafrica.com/2010/08/ governing-it-in-the-public-sector/, retrieved on 9 April 2012.

[6] S. D. Haes, and W. V. Grembergen, "IT Governance Structures, Processes and Relational Mechanisms: Achieving IT/Business Alignment in a Major Belgian Financial Group," 38th Annual Hawaii International Conference on System Sciences, 0-7695-2268-8, 2005, pp. 237b.

[7] M. Spremic, "Standards and Frameworks for Information System Security Auditing and Assurance," World Congress on Engineering, 978-988-18210-6-5, 2011, pp. 514-519.

[8] A. Cornwell, "Auditing: is there a need for great new ideas?," Managerial Auditing Journal, vol. 10, no. 1, 1995, pp. 4-6.

[9] M. Spremic, "IT governance mechanisms in managing IT business value," WSEAS Transactions on Information Science and Applications, vol. 6, no. 6, 2009, pp. 906-915.

[10] M. Kooper, R. Maes, and E. Lindgreen, "On the governance of information: Introducing a new concept of governance to support the management of information," International Journal of Information Management, vol. 31, no. 3, 2010, pp. 195-200.

[11] A. Prasad, J. Heales, and P. Green, "Towards a deeper understanding of information technology governance effectiveness: A capabilities-based approach," International Conference on Information Systems (ICIS), 2009, pp. 1-19.

[12] T. Coleman, and A. T. Chatfield, "Promises And Successful Practice In IT Governance: A Survey Of Australian Senior IT Managers," 15th Pacific Asia Conference on Information Systems (PACIS), 978-1-86435-644-1, 2011, pp. 1-15.

[13] M. F. I. Othman, et al. "Barriers to information technology governance adoption: a preliminary empirical investigation," 15th International Business Information Management Association Conference, 2011, pp. 1771-1787.

[14] A., Chaudhuri, "Enabling Effective IT Governance: Leveraging ISO/IEC 38500: 2008 and COBIT to Achieve Business-IT Alignment," EDPACS, vol. 44, no. 2, 2011, pp. 1-18.

[15] J. Ribeiro, and R. Gomes, "IT governance using COBIT implemented in a high public educational institution: a case study," 3rd International Conference on European Computing Conference (ECC), 978-960-474-088-8/1790-5117, 2009, pp. 41-52.

[16] J. R. Ruiz, "COBIT as a Tool for IT Governance: between Auditing and IT Governance," The European Journal for the Informatics Professional, vol. 9, no. 1, 2008, pp. 40-43.

[17] M. Simonsson, and P. Johnson, "Assessment of IT Governance-A Prioritization of Cobit," Proceedings of the Conference on Systems Engineering Research, 2006, pp. 1-10.

[18] S. H. Bakry, and A. Alfantookh, "IT-governance practices: COBIT," Applied Computing and Informatics, vol. 5, no. 2, 2006, pp. 53-61.

[19] T. Dahlberg, and H. Kivijarvi. "An integrated framework for IT governance and the development and validation of an assessment instrument," 39th Hawaii International Conference on System Sciences (HICSS), 0-7695-2507-5, 2006, pp. 1-10.

[20] N. Ranken, "Communicating an IT system change: eight tips for success," Strategic Communication Management, vol. 11, no. 4, 2007, pp. 16-19.

[21] J. Callahan, C. Bastos, and D. Keyes, "The evolution of IT Governance at NB Power," Idea Group Publishing, 2004

[22] R. M. Melnicoff, S. G. Shearer, and D. K. Goyal, "Is there a smarter way to approach IT governance?," Accenture Outlook Journal, vol. 7, no. 1, 2005, pp. 80-87.

[23] P. Weill, "Don't just lead, govern: How top-performing firms govern IT," MIS Quarterly Executive, vol. 3, no. 1, 2004, pp. 1-17.

[24] W. Brown, and F. Nasuti, "What ERP systems can tell us about Sarbanes-Oxley," Information Management and Computer Security, vol. 13, no. 4, 2005, pp. 311-327.

[25] L. H. Crawford, and J. Helm, "Government and governance: The value of project management in the public sector," Project Management Journal, vol. 40, no. 1, 2009, pp. 73-87.

[26] D. Sharma, M. Stone, and Y. Ekinci, "IT governance and project management: A qualitative study," Journal of Database Marketing & Customer Strategy Management, vol. 16, no. 1, 2009, pp. 29-50.

[27] ISACA, "COBIT 5: A Business Framework for the Governance and Management of Enterprise IT," IT Governance Institute, available at: http://www.isaca.org/COBIT, retrieved on 19 April 2012.

[28] S. Buckby, P. Best, and J. Stewart, "The current state of information technology governance literature," Information Science Reference (IGI Global), 2008

[29] T. J. Menk, "Internal auditing: key to helping your operations and bottom line," Alpern Rosenthal, 2008, available at: http://www.alpern.com/internal-audit-bottom-line-article.php, retrieved on 18 February 2012.

[30] R. Malan, "Internal auditing in government," Internal Auditor, vol. 48, no. 3, 1991, pp. 90-95.

[31] M. Dittenhofer, "Performance auditing in governments," Managerial Auditing Journal, vol. 16, no. 8, 2001, pp. 438-442.

[32] M. J. Ramos, "How to comply with Sarbanes-Oxley section 404: assessing the effectiveness of internal control," John Wiley & Sons, Inc., 2006

[33] M. Majdalawieh, and I. Zaghloul, "Paradigm shift in information systems auditing," Managerial Auditing Journal, vol. 24, no. 4, 2009, pp. 352-367.

[34] F. Gallegos, "IT auditor careers: IT governance provides new roles and opportunities," Information Systems Control Journal, vol. 3, no. 2003, pp. 40-43.

[35] IT Governance Institute, "IT Governance Implementation Guide: Using COBIT and Val IT," ISACA, 2007

[36] S. Fleming, and M. McNamee, "The ethics of corporate governance in public sector organizations," Public Management Review, vol. 7, no. 1, 2005, pp. 135-144.

[37] T. Sethibe, J. Campbell, and C. McDonald, "IT Governance in Public and Private Sector Organisations: Examining the Differences and Defining Future Research Directions," 18th Australasian Conference on Information Systems, 2007, pp. 833-843.

[38] J. Campbell, C. McDonald, and T. Sethibe, "Public and private sector IT governance: Identifying contextual differences," Australasian Journal of Information Systems, vol. 16, no. 2, 2009, pp. 5-18.

[39] Department of Finance and Deregulation, Australian Government ICTExpenditure 2008-09-2010-11 Report, 2012.

[40] S. Ali, and P. Green, "Effective Information Technology Governance Mechanisms in Public Sectors: An Australian Case," Tenth Pacific Asia Conference on Information Systems, 2006, pp. 1070-1089.

[41] L. Gerke, and G. Ridley, "Towards an abbreviated COBIT framework for use in an Australian State Public Sector," 17th Australasian Conference on Information Systems, 2006, pp. 1-10.

[42] G. Vinten, "Public Sector Corporate Governance-the Turnbull Report," Credit Control, vol. 23, no. 1, 2002, pp. 27-30.

[43] K. Hansen, "Open to the public," Australian CPA, vol. 72, no. 7, 2002, pp. 38-39.

[44] Q. Liu, and G. Ridley, "IT Control in the Australian public sector: an international comparison," Europpean Conference on Information Systems, 2005, pp. 1-12.

[45] S. Beaumaster, "Local government IT implementation issues: a challenge for public administration," 35th Hawaii International Conference on System Sciences (HICSS), 0-7695-1435-9, 2002, pp. 1725-1734

[46] A. Khalfan, and T. G. Gough, "Comparative analysis between the public and private sectors on the IS/IT outsourcing practices in a developing country: a field study," Logistics Information Management, vol. 15, no. 3, 2002, pp. 212-222.

[47] T. Cook, and D. Campbell, "Quasi-experimentation: design and analysis issues for field settings," Rand McNally, 1979

[48] C. Okoli, and S. D. Pawlowski, "The Delphi method as a research tool: an example, design considerations and applications," Information & Management, vol. 42, no. 1, 2004, pp. 15-29.

[49] E. T. Powell, "Quick tips collecting group data: Delphi technique," University of Wisconsin, available at: http://www.uwex.edu/ces/ pdande/resources/pdf/Tipsheet4.pdf, retrieved on 27 July 2011.

[50] S. D. Haes, and W. V. Grembergen, "An exploratory study into the design of an IT governance minimum baseline through Delphi research," The Communications of the Association for Information Systems, vol. 22, no. 24, 2008, pp. 443-458.

[51] R. C. Schmidt, "Managing Delphi Surveys Using Nonparametric Statistical Techniques," Decision Sciences, vol. 28, no. 3, 1997, pp. 763-774.

[52] T. H. Lee, and A. M. Ali, "Audit Challenges in Malaysia Today," Accountants Today, vol. 21, no. 10, 2008, pp. 24-26.

[53] J. Guthrie, "Critical Issues in Public Sector Auditing," Managerial Auditing Journal, vol. 7, no. 4, 1992, pp. 27-32.

[54] D. Stoel, D. Havelka, and J. W. Merhout, "An analysis of attributes that impact information technology audit quality: A study of IT and financial audit practitioners," International Journal of Accounting Information Systems, vol. 13, no. 1, 2012, pp. 60-79.

[55] J. W. Merhout, and D. Havelka, "Information technology auditing: A value-added IT governance partnership between IT management and audit," Communications of the Association for Information Systems, vol. 23, no. 1, 2008, pp. 464-482.

[56] R. B. Raaum, and R. Campbell, "Challenges in Performance Auditing: How a State Auditor with Intriguing New Performance Auditing Authority is Meeting Them," The Journal of Government Financial Management, vol. 55, no. 4, 2006, pp. 26-30.

[57] R. Filipek, "IT Audit Skills Found Lacking," Internal Auditor, vol. 64, no. 3, 2007, pp. 15-16.

[58] A. G. Koutoupis, and A. Tsamis, "Risk based internal auditing within Greek banks: a case study approach," Journal of Management and Governance, vol. 13, no. 1, 2009, pp. 101-130.

[59] C. H. L. Grand, "Performing the IT General Controls Audit," EDPACS, vol. 45, no. 1, 2012, pp. 1-13.

[60] J. V. Carcello, R. H. Hermanson, and N. T. McGrath, "Audit quality attributes: The perceptions of partners, preparers, and financial statement users," Auditing, vol. 11, no. 1, 1992, pp. 1-15.

[61] S. Buckby, P. J. Best, and J. D. Stewart. "The Role of Boards in Reviewing Information Technology Governance (ITG) as part of organizational control environment assessments," IT Governance International Conference, 1877314498, 2005, pp. 1-14.

[62] C. Howard, and R. S. Purdie, "Governance issues for public sector boards," Australian Journal of Public Administration, vol. 64, no. 3, 2005, pp. 56-68.

[63] R. Nolan, and F. W. McFarlan, "Information technology and the board of directors," Harvard Business Review, vol. 83, no. 10, 2005, pp. 96.

[64] A. Prasad, J. Heales, and P. Green, "A capabilities-based approach to obtaining a deeper understanding of information technology governance effectiveness: Evidence from IT steering committees," International Journal of Accounting Information Systems, vol. 11, no. 3, 2010, pp. 214-232.

[65] R. Huang, R. W. Zmud, and R. L. Price, "Influencing the effectiveness of IT governance practices through steering committees and communication policies," European Journal of Information Systems, vol. 19, no. 3, 2010, pp. 288-302.

[66] IT Governance Institute, "Board Briefing on IT governance 2nd Edition," ITGI, 2003, available at: http://www.isaca.org, retrieved on 22 January 2012.

[67] L. Trautman, and K. A. Price, "The Board's Responsibility for Information Technology Governance," John Marshall Journal of Computer & Information Law, vol. 29, no. 2011, pp. 313.

[68] S. D. Haes, and W. V. Grembergen, "An exploratory study into IT governance implementations and its impact on business/IT alignment," Information Systems Management, vol. 26, no. 2, 2009, pp. 123-137.

[69] M. Axelsen, et al, "Examining The Role Of IS Audit In The Public Sector," Pacific Asia Conference on Information Systems, 9781864356441, 2011, pp. 1-15.

[70] M. B. Curtis, et al., "Auditors' Training and Proficiency in Information Systems: A Research Synthesis," Journal of Information Systems, vol. 23, no. 1, 2009, pp. 79-96.

[71] ISACA, "IS Auditing Guideline," Information Systems Audit and Control Association, available at: http://www.isaca.org/, retrieved on 20 February 2012.

[72] R. Lawton, "Transitioning IT From a Compliance to a Value-driven Enterprise Using COBIT," Information Systems Control Journal, vol. 6, no. 2007, pp. 43.

[73] G. Hardy, "The role of the IT Auditor in IT Governance," Information Systems Control Journal, vol. 1, no. 2008, pp. 1-2.

[74] V. D. D. Nest, C. Thornhill, and J. D. Jager, "Audit committees and accountability in the South African public sector," Journal of Public Administration, vol. 43, no. 4, 2008, pp. 545-558.

[75] J. Stewart, and N. Subramaniam, "Internal audit independence and objectivity: emerging research opportunities," Managerial Auditing Journal, vol. 25, no. 4, 2010, pp. 328-360.

[76] A. Kanellou, and C. Spathis, "Auditing in enterprise system environment: a synthesis," Journal of Enterprise Information Management, vol. 24, no. 6, 2011, pp. 494-519.

[77] P. Burnaby, and S. Hass, "A summary of the global Common Body of Knowledge 2006 (CBOK) study in internal auditing," Managerial Auditing Journal, vol. 24, no. 9, 2009, pp. 813-834.

[78] J. F. Brazel, and C. P. Agoglia, "An Examination of Auditor Planning Judgements in a Complex Accounting Information System Environment*," Contemporary Accounting Research, vol. 24, no. 4, 2007, pp. 1059-1083.

[79] J. E. Hunton, A. M. Wright, and S. Wright, "Are financial auditors overconfident in their ability to assess risks associated with enterprise resource planning systems," Journal of Information Systems, vol. 18, no. 2, 2004, pp. 7-28.

[80] F. Lin, L. Guan, and W. Fang, "Critical Factors Affecting the Evaluation of Information Control Systems with the COBIT Framework," Emerging Markets Finance and Trade, vol. 46, no. 1, 2010, pp. 42-55.

[81] M. Simonsson, P. Johnson, and H. Wijkstrom. "Model-based IT governance maturity assessments with COBIT," European Conference on Information Systems (ECIS), 2007, pp. 1276-1287.

[82] D. Radovanovic, et al, "IT audit in accordance with Cobit standard," MIPRO, 978-1-4244-7763-0, 2010, pp. 1137-1141.

Loai Al Omari

Information Security Institute Queensland University of Technology Brisbane, Australia

loai.alomari@student.qut.edu.au

Dr Paul Barnes

Information Security Institute Queensland University of Technology Brisbane, Australia

p.barnes@qut.edu.au

Dr Grant Pitman

Information Security Institute Queensland University of Technology Brisbane, Australia

grant.pitman@qut.edu.au

Table 1: Initial list of ITG audit challenges

                            Name                     Cross-reference
                                                     from literature

Internal   Lack of necessary skills and              [52, 53]
  Audit      competencies to undertake effective
             ITG audits.
           Audit team's inadequate evaluation        [52, 54]
             and testing of the effectiveness of
             ITG controls.
           Lack of developed methodologies and       [53-55]
             tools to keep pace with changes
             occurring in the auditing field.
           Lack of or inconsistent rules to          [53, 54]
             determine what aspects of audit best
             fit the relevant organisation.
           Poor training arrangements for            [53, 56]
             public sector auditors.
           Failure of an audit team to               [52, 54]
             appropriately apply required
             substantive auditing procedures and
             planning processes.
External   Inconsistent execution of audit           [55]
  Audit      methodology across public sector
             organisations.
           Limited knowledge within the audit        [54, 57, 58]
             team of emerging risk exposures
             related specifically to the audited
             organisation.
           Audited public sector organisation        [53, 55]
             lack of necessary skills or some
             reticence to co-operate.
           Pressure to prematurely sign-off on       [54, 55]
             audit reports whilst not following
             specific legislative requirements.
           Weak auditee and auditor                  [53, 55]
             relationship in the public sector.
           Expectation gap between public            [52-54]
             sector perceptions of audit and
             actual audit practices.

Table 2: Respondents' key factors

Key factor                Number of respondents

Background              Audit      Mix (IT, IS, etc.)
                          9                7
Role                   Manager          Officer
                          8                8
Years of experience   < 10 years       > 10 years
                          7                9
COPYRIGHT 2013 College of Information Technology, Universiti Tenaga Nasional
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2013 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Omari, Loai Al; Barnes, Paul; Pitman, Grant
Publication:Electronic Journal of Computer Science and Information Technology (eJCSIT)
Article Type:Report
Geographic Code:8AUST
Date:Jan 1, 2013
Words:7642
Previous Article:Developing a framework to improve and enhance IT services at one Malaysian private university.
Next Article:ICT in telemedicine: conquering privacy and security issues in health care services.
Topics:

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters