Beating IT Risks

214	Infrastructure

The features of a utility model for IT – in which you ‘plug in’ and consume what you need and pay only for what you use – can be partially achieved today and is clearly flagged by major vendors as a direction for tomorrow.

Supporting technology directions include: autonomic and self-healing computers (they manage and fix themselves when something goes wrong), grid computing79 (delivering virtually infinite capacity on demand), Web services (network-hosted and -invoked application components), and storage virtualization (allowing effective shared use of a large pool of storage resources).

From a risk management perspective, there are a few things to understand:

•Pooling and sharing IT infrastructure does increase your dependencies on others – the extent to which you are willing to expand your ‘circle of trust’ should carefully consider a range of IT risks including in particular information asset and IT service continuity; and

•The extent to which your legacy IT infrastructure will be able to ‘plug into’ the new IT infrastructure may be constrained – as a consequence the costs and risks of changing across must be considered.

We move now from the evolutionary perspective to the here-and-now.

Moving towards ‘set and forget’

Most business managers are not so enamoured of their IT installations that they could be called ‘box huggers’. They are quite happy with the IT infrastructure being out of sight and – if it is working – out of mind. Their preferred risk management strategy we’ll call the ‘man and a dog’ approach. The man is there to feed the dog and the dog is there to bite the man if he tries to touch any of the controls!

This ‘lights-off’ operating nirvana is elusive – even for those who chase it – and ‘set and forget’ a completely inappropriate management stance for enterprises reliant on their systems.

There is a wide range of capability that must be in place to actively manage, operate and support IT infrastructure. Much of the spending in this space is really risk-related and couching it in these terms is useful.

Table 9.1 draws upon the Information Technology Infrastructure Library (the most widely accepted approach to IT service management) practices and the risk mitigation or avoidance outcomes associated with each.

Other necessary practices that might be more familiar relate to basic housekeeping, administration, operations, tuning and monitoring of systems. The mundane task of archiving data from a server is seen as important only if it isn’t

79 Connecting multiple distributed computers and harnessing their combined power.

Moving towards ‘set and forget’	215

Table 9.1—ITIL service management components (UK OGC, 2001)

Why do?	To manage risks of . . .
Service level management	Delivering a service that falls short of requirements
	Failing to act on issues of under-performance
Financial management	Misallocating IT costs and over-spending
IT service continuity management	Loss of service beyond the maximum tolerable
	outage (refer to further details in Chapter 5) and
	inability to respond in times of crisis
Availability management	Unacceptable downtime and unreliability
Capacity management	Running out of headroom to cater for tomorrow’s
	business volumes
	Paying for storage, processing and connectivity
	capacity that you don’t need
Security management	Losing or compromising valuable information assets
Incident management	Impacting the business when service disruptions
	occur
Configuration management	Losing control of valuable IT assets
	High-cost and inefficient systems management
Problem management	Repeated occurrences of system failures that haven’t
	had the root cause addressed
Change management	Defects in production systems that cause costly
	errors and impact users’ productivity
Release management	Loss of integrity and consistency in operational
	systems

done – the disk will fill up and the server will be unable to store any more. The integrity of back-up copies of critical data becomes important only when you need to bring it back!

Now if you have obtained assurance and are confident that those in charge of the IT infrastructure from day to day have all this in hand, you may be able to take a management-by-exception stance.

Over and above the IT infrastructure service management responsibilities, when things go wrong and fall out of the ordinary it will be necessary for ‘escalation’ to occur. The business managers at risk of IT failures must be clearly ‘wired in’ and be ready to call the shots on restoration and remediation priorities. Bear in mind, if customers need to be told bad news it shouldn’t be coming from the IT department and ditto for the press!

216	Infrastructure

De-risking infrastructure transformation

If the trouble and warning signs from day to day are overwhelming and a major overhaul of the IT infrastructure is required, how can this initiative be de-risked?

We won’t repeat here the project-related guidance of Chapter 4 – we’ll focus on some of the top infrastructure-specific challenges.

Set direction

Even IT infrastructure projects require some form of governance. Notwithstanding the technical nature of the work being undertaken, it is important for the business goals and objectives to be established and any ‘no go’ zones clearly set out.

We have found the use of ‘guiding principles’ set at the commencement of infrastructure-related initiatives to be important.

Typically these address the following:

•Priorities of time, cost and quality of the solution – for example, must be done by X, must be done for less than Y, must not introduce high-severity defects and incidents;

•Extent of business impact during change that is acceptable;

•Expected durability of the solution expressed in terms of business growth and /or expansion;

•Stance on accommodating or freezing changes during the course of the transformation project;

•Acceptance or prohibition of publicity around the initiative;

•Service provider rules of engagement;

•Financial transparency and opportunity to control and/or audit;

•Stakeholder consultation requirements; and

•Consistency or alignment with a defined IT strategy and end-state architecture.

These principles form a frame of reference for the transformation team.

A step at a time and fall-back ready

In the GCHQ case study described at the end of this chapter, a major IT transformation and relocation was described as ‘a bit like rebuilding an aircraft carrier from the keel up, whilst it is at sea and flying its aircraft operationally’.

In this project it was simply too risky to attempt to move systems, people and data in a single jump. A step-by-step approach helped reduce risk.

Health check	217

However, as timescales were typically tight, there was generally not the luxury of organizing this as a serial project. Multiple tasks needed to run in parallel and be synchronized with precision.

Each implementation of major IT infrastructure change was supported with a fall-back or back-out procedure that had been tested and rehearsed.

A useful analogy from rock-climbing: beginners are told to move only one hold at a time. Only once the new grip is secure will the three-point hold be released.

Health check

This section assists you consider at a macro-level the company’s IT infrastructure risk health, for input into the IT risk portfolio assessment.

Is this important to your business?

•You spend significant amounts of money on IT infrastructure assets.

•Performance and reliability of systems is important to your business and your customers.

•You have a complex IT infrastructure installation with multiple network links to business partners and the Internet.

If you agree with two or more of the three statements, then IT infrastructure risks are important to your business

Are you doing the right things?

•Your critical IT infrastructure components are all current and supported versions.

•Capacity management plans are kept in place for all critical platforms.

•Your disaster recovery plans and incident response plans are maintained and tested regularly.

•Incidents and problems are resolved in accordance with a repeatable process.

•Your IT infrastructure is well documented.

If you agree with these statements then you are doing the right things in your organization. If you disagree with three or more of the five statements, then there is significant room for improvement in your IT infrastructure risk management capability.

218	Infrastructure

Do you have a good track record?

•You have avoided significant commercial losses from IT infrastructure failure.

•Your systems run reliably and at the required level of performance.

•You consistently meet the business service level requirements for systems performance.

If you agree with these statements, then you have a good track record of managing IT infrastructure risks. If you disagree with one or more of the three statements, then there is evidence of the need to improve IT risk management capability.

Case study: GCHQ1

Designing and managing Europe’s most complex IT relocation

GCHQ is responsible for signals intelligence and information assurance in the UK, working closely with the UK’s other intelligence agencies (commonly known as MI5 and MI6), the Ministry of Defence and law enforcement authorities. Signals intelligence supports government decision-making in national security, military operations and law enforcement. Information assurance defends government communication and information systems from eavesdroppers, hackers and other threats and helps protect the UK infrastructure (such as the power, water and communications systems) from interference. GCHQ is a 24/7 operation and is supported by one of the most advanced IT facilities in the world. The UK’s national security depends on GCHQ’s operational integrity and continuous availability.

GCHQ wanted to consolidate its operations from 50 buildings spread over two sites and move its 4500 people to a single, purpose-built, hightech facility. The key benefit sought from this move was a change in the way the organization worked – to become more joined-up and flexible. To realize this benefit GCHQ needed both to alter its business processes and to develop new systems that could be integrated with the existing ‘legacy’ processes and systems. These changes had to be made whilst moving to the £337 million new headquarters, known locally as the ‘Doughnut’ due to its shape, which is big enough to contain the Albert Hall inside the space at the heart of the immense building.

The complexity of the relocation was increased by the scale of the IT infrastructure – over 5000 miles of communications cabling and 1850 miles

Case study: GCHQ1	219

of fibre optics run through the building and the fully equipped computer rooms cover an area equivalent to approximately three football pitches. Over 100 discrete systems had to be moved to the new building – each larger than the IT facilities of most UK corporations, including probably the largest supercomputer cluster outside the USA. The relocation and transformation programme comprised over 60 highly interdependent IT, relocation and change projects with a combined value exceeding £300 million. GCHQ’s design authority described this complex programme as being ‘a bit like rebuilding an aircraft carrier from the keel up, whilst it is at sea and flying its aircraft operationally’.

The contract to manage the requirements of this relocation implemented a systems engineering approach to designing and building complex systems. This methodology applied to designing, building and operating the ‘whole system’ and covered every aspect of the business, including people, operations, technical infrastructure, support and, importantly, how the overall system will evolve. Combining with GCHQ’s body of knowledge a highly capable and integrated team was formed to be able to deal with the immense complexity involved.

A key element of the programme was to develop an approach that addressed the complexity and scale of the problem:

•Step-by-step approach to minimizing risk: Business continuity dictated that the systems, people and data could not be moved in a single go. Instead, the team developed a step-by-step approach that involved the re-engineering of the legacy systems, integrating these with new systems while addressing business issues, and finally moving the people.

•Overcoming the complexity of the program: Meticulous planning, issue management and dynamic testing all helped to make this complex task achievable. For example, to assure business integrity during enhancements to existing IT applications, the team implemented ‘roll-back’ systems. These ensured that if a change was applied to an existing service and it did not work as planned, then the service could quickly revert to its previous stable state, without any interruption to GCHQ’s operations.

•Simulation models: A major part of the approach was to use computer simulations – for trade-off analysis, performance analysis, reviewing behaviour, failure mode analysis and scenario testing – to help decide on a safe course of action to support the business under all foreseeable circumstances.

The programme met the aggressive schedule within budget, delivered the benefits sought and maintained the continuity of GCHQ’s critical intelligence

220	Infrastructure

operations throughout the move. The UK’s National Audit Office Report of July 2003 commented, ‘Other government departments might learn lessons from the way that GCHQ developed its program management arrangements for the hybrid change program.’

Printed with permission of PA Consulting Group and GCHQ

Note: 1 Formally Government Communications Headquarters but known only as GCHQ

10 Strategic and emergent

Degrees of failure

Although the strategic vision was wonderful: Margaret Hodge, Minister for Lifelong Learning and Higher Education, said: ‘I want the UK to become the leading producer of quality on-line higher education. UK eUniversity Worldwide is key to making that vision a reality. This joint venture creates new opportunities for many students.’ (Sun, 2001)

It has been a bad week for information technology in the public sector. The worst casualty is the Government’s flagship e-university, hailed by ministers as the Open University for the 21st century. It is to be quietly dismantled after embarrassing performance failures . . . The UKeU, launched amid fanfares at the height of the dotcom boom in 2000, aimed to market and deliver British university degrees worldwide through the internet

. . . But £35 million later, it recruited only 900 students. Now the Higher Education Funding Council for England says that it will ‘scale down and transfer’ its work. In other words, the plug has been pulled.

(The Times, 2004)

Enquirers looking for information about the eUniversity are now given the sad message: ‘Enquiries should now be directed to the Higher Education Funding Council for England.’ (ukeu.com, 2004). And the IT assets have been put up for sale. (Public Technology, 2004)

Or a competitor with a technology edge?

While it can be argued that the emergence of the Y2K problem as a business issue was the best example of an emergent risk, a more specific example is helpful. The irrecoverable loss of market share by Barnes & Noble to Amazon was a nightmare scenario for the boards and executives of mature, blue-chip companies. Their safe, established positions could be not simply eroded, but savaged by a company with a technology edge. Barnes & Noble was the timehonoured leader in the book retailing segment across North America. Yet it was

222	Strategic and emergent

unprepared, both in strategic terms and in terms of its technology development, for the Internet customer.

The message was clear and strong for many industries with fragmented markets, products that could be digitized or global opportunities. The responses ranged from prescient to comical.

This chapter opens with a discussion of the impacts on the business of IT strategic failure. Next follows an examination of the role of IT in business change and the contribution to shareholder value. We close with an analysis of the much-touted link between IT and strategic agility and responsiveness.

The impact of IT failing to support the execution of your business strategy

Just as the organization’s strategy is not announced in neon lights over the entrance to corporate headquarters, the support that IT delivers to the strategy is not immediately apparent. While neither strategy nor the IT support role is highly visible to outsiders, they may not even be clear within the organization. In even a moderate-sized organization, the role and contribution of IT are often hidden by the complexity of the systems, their usage and their interconnections.

Does business strategy drive IT or vice versa? Or are they developed together? Even informed executives within an organization may not immediately agree (Clemons, 1991). For most organizations, the business strategy clearly comes first and IT needs to deliver systems, applications, tools and techniques that support and enable. However, IT may have been neglected to such an extent that it is unable to respond. Alternatively IT may have gone down a path of technical challenges that diverge from business goals or needs.

Graceful degradation

The slow, continued decline in IT capability that arises from failure to renew, refresh, retrain and reinvigorate is not apparent on a day-by-day or week-by- week scale. Even in an annual review, critics may receive harsh retaliation. External benchmarking can become unacceptably painful. On a daily basis there is no imperative for change, but if this continues indefinitely, the organization ends up with a reduced capability to envisage IT benefits or to deliver them. This is the ‘boiled frog’ syndrome80 of reality denial.

Of course the opposite is no better – would we want our dilemmas to be otherwise? – the overreaching of an organization, beyond the level of its IT

80 How to boil a frog! Place in cold water under a gentle heat; supposedly the frog does not perceive the incremental change. (No animals were harmed in the writing of this book.)

The impact of IT failing to support the execution of your business strategy

223

capability, leads to the myth of Icarus.81 Many organizations have ‘bet the farm’ with technology developments, only to find themselves at best spending many years in rebuilding their assets.

The life experience of the professional person anywhere in the world includes several such aspects: spending enough time with the children, keeping the weight and waistline in order, balancing work and home. Nothing is critical today, but after ten years it can be too late.

Choose a grail, any grail

IT is one area of the organization that is at extreme risk of ‘mantra-itis’ or being driven by a fad, slogan or management holy grail. Sometimes this sounds quite respectable, such as when trying to ‘align the IT with the organization’ (Strassman, 1997). From the 1970s’ slogan of ‘better MIS with better decision-making’ to today’s ‘agile computing’, there is much risk in chasing a slogan over following a real goal that has relevance to the business.

It is going noticeably wrong when this goal is espoused by IT, but the rest of the organization is unaware. The likely suspect when talking about IT supporting the business strategy is the goal of aligning IT with the organization.82 Almost every CIO will endorse this, claiming to have already achieved it, or to be pursuing the path. Obviously the contrary is untenable: ‘No, we don’t bother about aligning with the business, we’re just making sure we’re the best IT shop around!’ But a reasoned assessment shows that concern with strategy is critical, but a direct mapping is generally very high risk.

What is needed for most organizations is the ability to respond, in a reasonable time and at reasonable cost, to strategic initiatives as they emerge. Business strategy can change much more quickly than IT. A complete IT refresh and rebuild over a period of ten years would be extremely demanding for many organizations, yet corresponding changes to strategy are feasible. Flexible IT infrastructure is much more valuable (and lower risk) than close tracking of each strategic move.

The goal that is often foisted upon IT is one of reducing its own costs consistently each year. An arbitrary percentage, such as the convenient round figure of 10%, needs to be achieved – whatever the strategic imperatives. While this can be realistic for utility inputs, such as server capacity and bandwidth, when outputs are constant, it is seldom capability-building or enhancing. It may bring about its own variety of graceful degradation, as the IT infrastructure is patched up so that more important development projects can proceed.

81According to legend, Icarus flew so close to the sun that the wax attaching his wings melted, leading to his death fall.

82Bergeron et al. (2003) in an innovative study illustrate methods for analysing and interpreting IT alignment with the business.