Beating IT Risks

254	Appendix 1

•Will the initiative deliver a positive net present value and / or achieve other ‘investment hurdle’ criteria?

Organizational impact

•Is the customer and stakeholder impact understood?

•Are related business process changes defined?

•Do the target organizations have capacity to undertake the required change?

•Are internal and external communications effective in raising awareness of the initiative?

Management and delivery

•Are all impacted organizations involved in this initiative?

•Has the project been set up properly?

•Are enough capable resources currently assigned to the project and can future requirements be met?

•Are suitable processes, methods and approaches being applied and are these consistent with policies and guidelines?

•Are roles, responsibilities and reporting lines clear?

•Are the plans for the requirements and architecture phase clear and actionable?

Key review questions to answer: Completion of requirements and architecture stage

Business support

•Are the business case assumptions still valid and is the initiative being actively supported?

•Have business and user requirements been defined at a detailed level and agreed with executive sponsors and representative users?

•Is the basis for acceptance of the solution agreed?

•Is the project on target to deliver planned benefits?

Solution and delivery choices

•Has the exploration of solution options crystallized an optimum technical solution and delivery approach?

•Has the selected solution option been confirmed as technically feasible and a good fit with the business requirements and service level requirements?

•Have the vendor(s) and internal / external service provider(s) with products and service capability to contribute been engaged in solution design?

•Are committed delivery timeframes acceptable and achievable?

Appendix 1	255

•Is the technical complexity of the solution and its dependencies with other initiatives manageable?

•Are non-functional requirements defined at a detailed level?

•Is the approach to assuring and testing functional and non-functional requirements defined and agreed?

•Is the system design compliant with technology architectures and standards?

Commercial arrangements

•Have acceptable contracts been prepared with vendors and service providers including comprehensive statements of work?

•Does the contracting approach leverage your buying power and existing contractual terms?

•Will the solution as scoped for delivery support the achievement of the tangible business benefits initially proposed?

•In light of further detail on total project cost estimates, will the initiative still meet investment and funding criteria?

•Are actual costs incurred in line with original estimates?

Organizational impact

•Has the approach to implementation been developed and agreed with the impacted organizations?

•Have related business process changes been designed?

•Are internal and external stakeholders aware of and involved in the initiative?

Management and delivery

•Are the organizations and people involved in this initiative working effectively together?

•Are project composition and resource levels sufficient to meet current and future project requirements?

•Are suitable processes, methods and approaches bedded down and are these consistent with policies, guidelines and other relevant standards?

•Are the plans for the remainder of the project clear and actionable?

•Are approaches to managing dependencies with other initiatives working?

•Are project processes meeting probity and audit requirements?

Key review questions to answer: Build mid-point

Business support

•Are the business case assumptions still valid and is the initiative being actively supported?

256	Appendix 1

•Have all changes to scope and / or approach been justified and accepted?

•Have users been involved in validating prototypes or designing usability features?

Solution fitness-for-purpose

•Is the technical solution design comprehensive and does it confirm the solution as a good fit with the business requirements?

•Is the system design compliant with defined technology architectures and standards?

•Is the technical complexity of the solution contained and being tightly managed?

•Are approaches to testing agreed and detailed?

•Do test strategies and plans set out what testing needs to be done to ensure acceptance criteria are met?

•Do all solution design elements (data and information management, hardware, networks, software, business processes) fit together coherently?

Delivery arrangements

•Are vendors and service providers delivering in line with agreed statements of work?

•Are interim milestones being achieved with progress in line with plans?

•Are internal activities being completed to the required level of quality?

•Are implementation plans, including handover activities, defined and agreed?

Commercial arrangements

•Will the solution being delivered support the achievement of the tangible business benefits initially proposed?

•Are vendors meeting their contractual obligations and receiving payments in line with the achievement of interim milestones?

•In light of further detail on total project cost estimates, will the initiative still meet investment and funding criteria?

•Are actual costs incurred in line with original estimates?

Organizational impact

•Has the implementation and benefits realization plan been developed and agreed with the impacted organizations?

•Do internal and external stakeholders remain involved in and supportive of the initiative?

Appendix 1	257

Management and delivery

•Are the organizations involved in this initiative working effectively together?

•Are project composition and resource levels sufficient to meet current and future project requirements?

•Are suitable processes, methods and approaches delivering results and are these consistent with policies, guidelines and other relevant standards?

•Are the plans for the remainder of the project clear and actionable?

•Are approaches to managing dependencies with other initiatives working?

•Are project processes meeting probity and audit requirements?

Key review questions to answer: Testing, acceptance and implementation mid-point

Business support

•Are user representatives actively involved in testing and validating the solution?

•Are the business case assumptions still valid and is the initiative being actively supported?

•Have all changes to scope and / or approach been justified and accepted?

Solution fitness-for-purpose

•Have all solution elements been delivered and assembled in line with the plan?

•Are tests being executed in line with the test plan?

•Is the number and significance of defects being discovered acceptable?

•Has the system compliance with defined technology architectures and standards been assured?

•Are non-functional elements of the solution being tested and assured appropriately (security, resilience, capacity, performance, operability, scalability, maintainability)?

Delivery arrangements

•Have vendors and service providers delivered products and services in line with agreed statements of work and expectations of quality?

•Are test incidents being effectively investigated and defects resolved rapidly?

•Are delivery milestones being achieved?

•Are internal activities being completed to the required level of quality and timeliness?

•Are conversion, implementation and cut-over activities being planned and carried out effectively?

258	Appendix 1

Commercial arrangements

•Will the solution being delivered support the achievement of the tangible business benefits initially proposed?

•Are vendors meeting their contractual obligations and receiving payments in line with the achievement of milestones?

•In light of further detail on total project cost estimates, will the initiative still meet investment and funding criteria?

•Are actual costs incurred in line with original estimates?

Organizational impact

•Are the target organizations ready for implementation?

•Have the change management activities been planned and agreed?

Management and delivery

•Are the organizations involved in this initiative working effectively together?

•Are project composition and resource levels sufficient to meet current and future project requirements?

•Are processes, methods and approaches delivering results and are these consistent with policies, guidelines and other relevant standards?

•Are the plans for conclusion of the project clear and actionable?

•Are approaches to managing dependencies with other initiatives working?

•Are project processes meeting probity and audit requirements?

Key review questions to answer: Post-implementation

Business results

•Is user take-up of the solution in line with expectations?

•Are the system users satisfied with the functionality?

•Are business benefits being realized from utilizing the system?

Solution fitness-for-purpose

•Has the solution met the required service levels since implementation?

•Have post-implementation support and maintenance issues been resolved quickly and effectively?

Operations and support arrangements

•Have vendors and service providers completed handover of products and services in line with agreed statements of work and expectations of quality?

Appendix 1	259

•Have products and solutions been effectively transitioned under support and maintenance (and warranty) cover?

Commercial outcomes

•Is it credible to expect the delivered solution to support the achievement of the business benefits initially proposed?

•Did vendors meet their contractual obligations and receive payments in line with the achievement of milestones?

•Did actual costs align with original estimates and are any variances explained?

Organizational impact

• Have the change management activities been executed in line with the plan?

Management and delivery

•Has the project and delivery structure been disbanded with appropriate contribution to knowledge bases?

•Have project resources been redeployed?

•Were the selected processes, methods and approaches effective in supporting and enabling the delivery of results?

•Is there any feedback or learning to be incorporated into other initiatives, or into wider policies and guidelines?

•Did project processes meet probity and audit requirements?

260	Appendix 1

References	261

References

Aalders, R. (2001). The IT Outsourcing Guide. Chichester, UK: John Wiley & Sons.

ABC (2000a). High price of communication failure. ABC Online, www.abc.net.au, 5 May. ABC (2000b). Love bug reaches Pentagon, ABC Online, www.abc.net.au, 6 May.

ABC (2000c). Three suspected in release of love bug, ABC Online, www.abc.net.au, 9 May.

ABC (2003). Customs computers stolen from Sydney Airport, ABC Online, www.abc.net.au, 5 September.

Abe, J., Sakamura, K. and Aiso, H. (1979). An analysis of software project failure. Proceedings of 4th International Conference on Software Engineering, September.

Addison, T. and Vallabh, S. (2002). Controlling software project risks – an empirical study of methods used by experienced project managers. Proceedings of SAICSIT 2002, 128– 140.

AFR (2000). Hacker in credit card extortion attempt. Chavez, P., Australian Financial Review, 15 December.

AFR (2004a). Cadbury’s Australian profit slides again. Evans, S., Australian Financial Review, 2 June.

AFR (2004b). SCO rides out the Mydoom storm. Lebihan, R., Australian Financial Review, 4 February.

AFR (2004c). Unisys loses appeal in RACV case. Connors, E., Australian Financial Review, 18 May.

AFR (2004d). Customs cargo system faces further delays. Connors, E., Australian Financial Review, 18 May.

AFR (2004e). ANZ cancels network project. Crowe, D., Australian Financial Review, 7 April.

AFR (2004f ). Contracts $750 million over budget. Woodhead, B., Australian Financial Review, 15 January.

AFR (2004g). Jetstar rush: website cracks. Harcourt, T. and Skulley, M., Australian Financial Review, 27 February.

Age (2004). $26m speed payout. Gray, D., Ketchell, M. and Silkstone, D., Melbourne, 15 May.

Allinson, C. (2001). Information systems audit trails in legal proceedings as evidence.

Computers and Security, 20(5), 409–421.

ANAO (2000). Business continuity management: keeping the wheels in motion. Better Practice – a Guide to Effective Control. Canberra, Australia: Australian National Audit Office, January.

262	References

ANAO (2001). Implementation of whole-of-government information technology infrastructure consolidation and outsourcing initiative. Audit Report No. 9, Canberra, Australia: Australian National Audit Office.

ANAO (2003a). Business continuity management and emergency management in centrelink. Audit Report No. 9 2003 –04. Performance audit. Canberra, Australia: Australian National Audit Office.

ANAO (2003b). Business continuity management follow-on audit. Audit Report No. 53 2002 –03. Business support process audit. Canberra, Australia: Australian National Audit Office.

ANAO (2003c). Survey of fraud control arrangements in Australian public service agencies. Audit Report No. 14 2003–04. Performance audit. Canberra, Australia: Australian National Audit Office.

ANAO (2004). Intellectual property policies and practices in Commonwealth agencies. Audit Report No. 25 2003–04. Performance audit. Canberra, Australia: Australian National Audit Office.

AS/NZS 4360 (1999). Risk Management. Strathfield, Standards Association of Australia. ASX, 2003. Principles of good corporate governance and best practice recommendations.

Sydney, Australia: Australian Stock Exchange Corporate Governance Council, March. Aubert, B., Rivard, S. and Patry, M. (2004). A transaction cost model of IT outsourcing.

Information and Management, 41, 921–932.

Auditor-General of Victoria (2003). Results of special reviews and 30 June 2002 financial statement audits: Implementation of RMIT University’s Academic Management System No. 4. Melbourne, Australia: Government Printer for the State of Victoria, February.

The Australian (2004). Why does this website cost $5.3 M?. Riley, J., 16 March. Bandyopadhyay, K., Mykytyn, P. and Mykytyn, K. (1999). A framework for integrated risk

management in information technology. Management Decision, 37(5), 437–444. Barki, H., Rivard, S. and Talbot, J. (2001). An integrative contingency model of software

project risk management. Journal of Management Information Systems, 17(4), 37–69, Spring.

Barros, M., Werner, C. and Travassos, G. (2004). Supporting risks in software project management. Journal of Systems and Software, 70, 21–35.

Beard, J. W. and Sumner, M. (2004). Seeking strategic advantage in the post-net era: Viewing ERP systems from the resource-based perspective. Journal of Strategic Information Systems, in press.

Bell, C. (2000). E-corruption: exploiting emerging technology corruptly in the NSW public sector. Sydney, Australia: NSW Independent Commission Against Corruption. Intelligence Assessment.

Bergeron, F., Raymond, L. and Rivard, S. (2003). Ideal patterns of strategic alignment and business performance. Information and Management.

Besnard, D. and Arief, B. (2004). Computer security impaired by legitimate users. Computers and Security, 23, 253–264.

BIS (2002). Operational Risk Data Collection Exercise – 2002. Basel: Basel Committee on Banking Supervision, Bank for International Settlements, 4 June.

BIS (2003a). The 2002 Loss Data Collection Exercise for Operational Risk: Summary of the Data Collected. Basel: Basel Committee on Banking Supervision, Bank for International Settlements, March.

BIS (2003b). Trends in risk integration and aggregation. Basel: Basel Committee on Banking Supervision, Bank for International Settlements, August.

References	263

BIS (2003c). Overview of the New Basel Capital Accord. Consultative document. issued for comment by 31 July 2003. Basel: Basel Committee on Banking Supervision, April.

Broadbent, M. and Weill, P. (1997). Management by maxim: How business and IT managers can create IT infrastructures. Sloan Management Review, 77–92, Spring.

Broadbent, M., Weill, P. and St Clair, D. (1999). The implications of information technology infrastructure for business process redesign. MIS Quarterly, 23(2), 159–182, June.

Burden, K. (2004). The voice of reason – finding middle ground in IT contracts. Computer Law and Security Report, 20(3).

Caelli, W. (2002). Trusted or trustworthy: The search for a new paradigm for computer and network security. Invited paper to the IFIP/SEC2002 Conference, Cairo, Egypt, 413–420, May.

Carr, N. G. (2003). IT doesn’t matter. Harvard Business Review, May.

CCCA (2000). Information Technology Infrastructure Library: Service Support, Version 1.01 (CD version). Stationery Office for Central Computer and Communications Agency.

Chang, K.-C., Jackson, J. and Grover, V. (2003). E-commerce and corporate strategy: An executive perspective. Information and Management, 40, 663–675.

Charette, R. (1996). The mechanics of managing IT risk. Journal of Information Technology, 11, 373–378.

CIO (2003). Cigna’s self-inflicted wounds. Bass, A., 15 March.

Clemons, E. (1991). Strategic investments in information technology. Communications of the ACM, 34(1).

COSO (2003). Enterprise Risk Management Framework. Committee of Sponsoring Organisations of the Treadway Commission. www.coso.org

CSI/FBI (2003). Computer Crime and Security Survey. Computer Security Institute. Cuneo, E. (2003). Accepting the risk. Banking Systems and Technology, September. Dhillon, G. and Backhouse, J. (1996). Risks in the use of information technology within

organizations. International Journal of Information Management, 16(1), 65–74. Dhillon, G. and Moores, S. (2001). Computer crimes: Theorizing about the enemy within.

Computers and Security, 20(8), 715–723.

Drummond, H. (1996). The politics of risk: Trials and tribulations of the Taurus project.

Journal of Information Technology, 11, 347–357.

EMA (2002). Business Continuity Planning and Disaster Recovery. Business-Government Task Force on Critical Infrastructure. Emergency Management Australia, 19 March.

Ewusi-Mensah, K. (1997). Critical issues in abandoned information systems development projects. Communications of the ACM, 40(9), 74–80, September.

Farrell, D., Terwilliger, T. and Webb, A. (2003). Getting IT spending right this time.

McKinsey Quarterly, 2.

Financial Times (2000). Glitch halts share trading in London. Boland, V., Mackintosh, J., and Van Duyn, A. 6 April.

Fitzgibbon, M. (1998). Burnt but Back on Track. Sydney, Australia: Bankstown City Council.

GAO (2002). Coast Guard’s Vessel Identification System. GAO-02-477, Washington, USA: US General Accounting Office.

GAO (2003). High Risk Series: An Update. GAO-03-119, Washington, USA: US General Accounting Office, January.

GAO (2004a). Testimony before the Subcommittee on Technology, Information Policy, Intergovernmental Relations and the Census, House Committee on Government Reform. GAO-04-478T, Washington, USA: US General Accounting Office, 16 March.