Leslie Willcocks

Professor Emeritus

London School of Economics and Political Science

An excellent history of methodologies and software development from 1915 to 2010 recently fell into my hands as Editor-in-Chief of the Journal of Information Technology. Likely, if passing review, it will be published in mid-2024. The authors provide a lengthy, two-part account, and indeed there is much to discuss. Part 1 reviews pre-computer information systems development (ISD) methodologies from 1915, through to methodologies created for early computer-based information systems development in the 1980s. Part Two begins in the later ISD methodological era from 1980 onward, and the early post-ISD methodological period from 1990 through the first decade of the Agile era around 2010. In this second era, the scope and purposes of information systems undergo a significant expansion with the spread of on-line terminals, PCs, computer networks, database management systems, and the Internet and related technologies. This section brings ISD methodology history into this millennium and the end of the first decade of the Agile era to show more recent developments, as methodologies have adapted to accelerated platform growth.

Of course, a lot has happened since 2010! This led me to thinking about how software development and digital technologies have gone global, calling into existence new, multi-national ways of managing the emergent market, technology and labour opportunities and challenges. My new co-authored book (see Willcocks, et al. (2023), Transformation in Global Outsourcing: Towards Digital Sourcing Of IT and Business Services (Palgrave, 2023)) puts the date at around 2016 when markets, global sourcing practices and development approaches started to change decisively as digital technologies and services kicked in with a vengeance. And in 2023/2024 we are still witnessing the transformation under way.

However, just as managers, we still have much to learn from history—one being why things have become as they are, and a second being: under what circumstances some things work and some things do not. Four insights emerged in my struggle to makes sense of experiences with software development, IT projects and outsourcing arrangements in the 1980–2023 period. I believe these observations will continue to have abiding relevance in today’s era of digital transformation projects. Let me give some detail.

From technology to business projects

Early on, in what the Americans called the 1970s–1980s ‘Wild West’ days of software development (in Britain the practice was referred to more euphemistically as a ‘craft’), we ran into the question—is it an IT or a business project? This seems amazingly naïve now, but the question did lead to a foundational understanding. It was all very well to treat a technology project as a specialist area when the system was self-contained, and did not have to perform anywhere much except within the IT function, or deep somewhere in a back office. One could throw it over the wall to internal or external IT specialists, who would get a detailed requirement statement, write a detailed contract in terms of time, budget and scope, and delivery could occur without much involvement of anyone else. These were the early days when I was working largely with internal accounting systems.

But where IT projects increasingly penetrated organisational functioning; impacted front office processes; where technology promised business innovation, this ‘specialist focus’ approach became much less useful. The maturity of the technology also mattered. If the technology was stable, had been used in the organisation before, and the developers had the requisite skills, then fine. But powerful new technologies have come on stream ever faster, technologies were increasingly less stable, had not been used in the organisation before, or in this new way, and the skills sets were not always available. A new ‘business/user focus’ was required.

The principal concern in taking a business/user focus is: what business benefits can be gained. The operating style is teamworking. Performance is about delivering a shared goal. Leadership is provided by business executives and users. Contrast that with the specialist focus on professional excellence dependent on individual specialists governed by a detailed service contract with the leadership by technology specialists. The business/user approach needed more integration mechanisms to support collaboration across a multi-functional team composed of users, business stakeholder managers, technical experts, and external resources as needed.

If you follow the logic, you can see the roots of why software and IT projects have become business projects, why ‘outsourcing’ is more suitable with stable technologies and commodity activities (lower risk), why advisory intermediaries became important to the global sourcing marketplace (to strengthen collaboration processes), and why contemporary business partnering eco-systems and their multinational management styles have become almost routine (optimising integration and managing complexity).

From agile methods to business agility

The 1980s saw the growth of structured methodologies to deal with the greater strains being experienced in software development efforts. Projects were scaling bigger, talent was often in short supply, knowledge progressed and needed to be imbedded in repeatable management actions, and more robust ways of organising. It is common to focus on the weaknesses of structured methodologies but, applied correctly in the right circumstances, I found them frequently very productive indeed.

But by the 1990s accelerating emerging technologies and dynamic business contexts globally were forcing managers to think much more about adaptiveness and agility. I well remember a presentation by a Ford Europe executive in 1995, where he said that for all projects the whole organisation had moved to a ‘time box philosophy’. I had seen this used experimentally in software development activities in the early 1990s. What shocked me was the radicalness of the Ford approach. In the future there would be no large-scale IT or business projects. With the quickening pace of business, and difficulties in concentrating resources for a long period, time and a workable business solution became the vital ingredients in the quality/cost/time trade-offs project managers had to make. Ford Europe stipulated that a project had to be completed within 90 days or less. Focus on core functionality and avoid bells and whistles; in IT ‘80 percent systems are OK’. Meanwhile iterative prototyping with heavy user, business stakeholder (and external supplier) involvement was the key operating mode. If the project could not be done in 90 days or less it would not start. Break larger projects down into digestible chunks, but deliver along a planned trajectory that ensures a consistent technology and organisation architecture—resulting in a town rather than a series of ‘Portakabins’ as the executive put it. To push the metaphors even further, my colleague at Templeton College David Feeny mused—"ah, Dolphins, not Whales”. And it’s a phrase about projects that has stuck with me to this day.

After 1995 agile methodologies for IT and business development programs became increasingly popular. Technological assistance can reduce the time box to as little as 30 days, depending, of course, on size of project. Mistakes are easily made. For example, I have seen organisations rush to deliver something but in ways that seriously jeopardised quality and subsequent performance. However, applied correctly the benefits have far outweighed the challenges, and one can see here the bedrock for key management practices today for innovating and delivering fast. But the contemporary search is for more than an agile methodology, rather it is for business agility. This arises due to very dynamic business and global contexts, rising interconnectedness, increasing complexity, and higher uncertainty, resulting in increasing risk, and the need for greater and faster adaptiveness. Global sourcing intermediaries, on this view, are one way of lubricating such business agility on the global stage.

From Outsourcing to Collaborative Innovation

On our strict definition outsourcing is the handing over of activities, assets and/or people to third party management for required result. This is the specialist focus mentioned above. Relying on third party management is inherently risky, so a key question becomes: how do you mitigate the risks? Over many years Mary Lacity, Sara Cullen and I found that IT and business process outsourcing works most effectively for well understood, stable, commodity type activities, on 3–5 year contracts, following a ‘horses for courses’ multi-vendor approach. This is not currently what many activities involve, and software development has been most frequently an area where outsourcing. Strictly defined, really was not a good model to follow. But there are many ways to leverage the external business services market, including buying-in resources to work under internal management, building long-term partnering relationships with key suppliers, Build-Operate-Transfer models, and business partnering eco-systems as described in this book. At the base of all these is the business/user approach detailed above. This also runs through the contemporary development of global business eco-systems. As CEO of an advisory business in such a technology network, Andy Hilliard writes:

“Multi-partner, multinational management is fast becoming the new norm, as businesses work to future proof operations through the creation of worldwide tech value networks, whose members can be called upon to provide the right skills at the right time. It's getting to the point where organisations that don't embrace multinational management models will be left in the dust”.

The other point to make is that transactional outsourcing has never been a great vehicle for technical, let alone business innovation. During the 1990–2014 period client expectations in terms of innovation, suppliers working on and getting rewarded for business outcomes, and suppliers becoming strategic partners, were mostly aspirational, unless they changed the management and organisational model. This meant at a minimum more team working and additional capabilities required within both client and supplier. Studying organisations that achieved collaborative innovation while working with external partners we found nine keys to high performance¹:

Launching the relationship

• • Assign a great (client/supplier) leadership pair
• • Focus on business and strategic benefits beyond cost efficiencies
• • Drive strong transition, transformation and change management capabilities

Keeping the relationship on target

• • Adopt a partnering approach to governance
• • Align the retained organisation, outsourced processes and provider staff
• • Resolve issues together and conflicts fairly

Exploring new frontiers

• • Use technology as enabler and accelerator of performance
• • Deploy domain expertise and analytics
• • Prioritise and incent innovation

Simplicity is Complex

The global context, business sectors, technology, organisations, systems and processes—all have become increasingly complex, and in some ways less understandable places and phenomena. How to act, react, progress? One answer is to apply Ashby’s Law of Requisite Variety. This states that to survive competitively, the degree of environmental complexity an organisation faces must be matched by a corresponding degree of internal complexity or ‘variety’—in its ability to model, create options and take actions that can regulate the impacts of environmental factors and events. In our studies of automation and digital transformation, in organisations with major legacy issues—in particular structure, processes, skills, technology, systems, data - the hidden, but serious problem has become how to achieve the right balance between variety and simplification. Simplicity emerges as complex. Albert Einstein highlighted the problem when he said:

“Make everything as simple as possible, but no simpler.”

Just focusing on software development, simplicity has always been a difficult but very desirable goal. Consider these statements from experienced practitioners:

“User interface is the process of shifting from chaotic complexity to elegant simplicity.

“There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult’.

And that’s just software development. Considering the environment in which organisations now swim, we can borrow from the aspirations of great software developers and begin to see the importance of the role of governance and advisory intermediaries in business eco-systems in controlling complexity rather than creating it, of dealing with a complex environment by reducing the challenges for clients, and creating for client companies the illusion of simplicity.

Towards Business Eco-Systems

All this leads to contemporary developments in what is being called business eco-systems, with technology value networks another term for eco-systems based on technology products and technical services. Given the emergent nature of business eco-systems, Ernst and Young provide some clarity in identifying seven relevant models:

Symbiotic—typical of technology platform companies, as orchestration and value depend on the core platform. For example, the SAP platform leverages additional technology and service providers to create further value for all participants.

Marketplace—an orchestrator coordinates the marketplace and brand and others pay fees to participate e.g., Amazon, Uber, Google.

Scaling—competitors in the same business work together to create and orchestrate scale using agreed risk and reward sharing rules, e.g., airline alliances for booking global travel.

Accretive—each entity has a portion of the overall customer value proposition that combined is worth substantially more than the sum of its parts. Ernst and Young have such a relationship with P&G and help to leverage P&G data / intellectual by building software and act as market channel to other manufacturers wanting to innovate supply and manufacturing processes.

Cooperative—competitors cooperate to create higher customer value. For example, P&G own odour elimination products, while Clorox sells Glad trash bags. Agreeing to sell trash bags containing an odour eliminator proved a resounding success for both parties.

Value chain—There are many examples of value chains world-wide. Value chains sit between original suppliers and end consumers of goods and services and are characterised by many participants across many different role types. Such a value chain becomes an eco-system when an orchestrator exists to build efficiencies for the participants.

Integrator—An orchestrator brings together a bespoke set of ecosystem participants to create a fully integrated end-to-end solution for customers, and offers full commercial responsibility for the workings of the solution delivered to the client. The ecosystem integrator rapidly assembles and integrates the solution, creating a quasi-bespoke value proposition combined with a contract of convenience.

This classification helps organisations clarify what their goals are, what their relationships should be with other parties, and what governance mechanisms and processes would be optimal. For software development much depends on whether the cooperating organisations are endeavouring to develop software products or software services for the commercial market, or for internal use.

Emerging Trends for Software Development

The interesting question is: which of these models could best facilitate delivery on emerging trends in software development. As at end of 2023 these trends are multiple and include:

• • AI demand will grow as it becomes more affordable and accessible. This includes generative AI, and lower-level automation
• • There will be increasing convergence between the physical and the digital propelled by technologies such as augmented reality, virtual reality, mixed reality and the immersive internet.
• • DevSecOps are very much on the cards to expand exponentially Cyber resilience has to become a standard requirement.
• • As an open-source technology used widely by developers in cloud computing environments Kubernetes will gain massively increased usage
• • Web 3.0 usage will rise where it adopts emerging technologies like machine learning and blockchain, in order to enhance the user experience
• • Blockchain developments will move beyond the finance industry—a slow train but it is coming
• • Sustainable technology objectives will become increasingly central in software development activities.
• • Quantum computing is set to convert into tangible payoffs by 2025.
• • 5G technology will arrive as software developers experience its speed and ability to handle large volumes of data easily.
• • Progressive web apps. market share will increase where they succeed in integrating the best of mobile and web application technologies giving users engaging, reliable and fast experiences.
• • Internet of Things applications will grow where experienced developers can create robust, scalable and secure apps, thus releasing their potential.
• • The market for low-code/no-code development platforms will increase and their wide spread use will accelerate software development.
• • Predictive analytics will move more centrally into the software development arena, By integrating analytics into other business applications, businesses will be able to mitigate risk, optimise operations and anticipate customer behaviour.

This is not an exhaustive list, but one can see that it will be very difficult even in very large organisations for in-house operations to exploit the opportunities just these technological developments can create. Not surprisingly I would expect increasing usage of external partnering, and the rise of even more technology focused business eco-systems. However, this can greatly increase the complexity and transaction costs if these new ways of organising, working cooperating and competing are not managed effectively.

Next month I will review a book Synergia by Andy Hilliard, that establishes the importance of, and lays down how to manage, technology value networks as business eco-systems.

¹ Lacity, M. and Willcocks, L. (2015) Nine Keys To World Class Business Process Outsourcing. (Bloomsbury, London)