Powering Ahead: The Need to Reform UK Energy Markets

Our Time to Zero In series makes the case for an inclusive transition to net zero that focuses on people, fairness, technology, markets and communities.

The role of energy in the basic functioning of society is often overlooked – but the crises currently enveloping the market are a reminder of its fundamental role in ensuring that we have access to food and water, our homes are comfortable and well lit, our businesses can function, and we can move around freely.

From the opening of the world’s first coal-fired power station in 1882 to the pioneering deployment of offshore wind, the UK has led the way in terms of innovation in energy technology and markets.

But while there have been huge changes in the production and use of energy during the past 100 years, many of the fundamentals have been remarkably consistent. Energy demand has been characterised by steady growth. This growing demand has been met primarily through fossil fuels, which have been used either to generate electricity or directly applied in transport, heating and industry. And there has been minimal incentive for consumers to flex their demand because the supply of fossil fuels has been plentiful and could be readily adjusted to match natural fluctuations.

That approach broadly worked when governments had two clear objectives: first, there must be security of supply – the lights must stay on; second, costs must be kept down – energy access must be affordable for all.

But in recent years, a third objective has become an increasing – and even defining – priority: cutting greenhouse-gas emissions. Only by decarbonising our energy systems can we hope to keep global temperature rise down to a manageable level.

That transition to a zero-carbon energy system means fundamental changes in the way we produce and use energy:

1. Electrification: there will be a hugely expanded role for the electricity sector, as it becomes the energy source for much of our transport and heating.

2. Generation technologies: the technologies that produce electricity must change as we stop burning fossil fuels.

3. Storage: we need new ways of storing energy as we move away from being able to store it in the form of gas and coal.

4. Active demand: in response to a less predictable supply, we must enable and incentivise consumers to flex their demand.

Our electricity market is not well placed to deliver these changes. The generation market is designed around fossil-fuel technologies that cannot deliver a decarbonised energy system, and our attempts to integrate new technologies have so far involved shoehorning them into a market that is not designed for them. Further, the design of our retail market has led to instability and a race-to-the-bottom on price, without providing incentives for consumers to invest in decarbonisation or the demand flexibility that our future system desperately needs. And the current energy crisis, with more than 20 suppliers going bust in recent months, has exposed profound problems of design and regulation in the retail market.

In both the generation and retail markets, the response has been an ever-expanding role for government and a shrinking role for the market in making decisions – leading to accelerated decarbonisation but also centralisation, distortions and inefficiencies.

We now face a choice: either to stay on the path towards a market that is more and more centralised, or to deliver a net-zero power system in a way that unleashes the power of the market, incentivises flexibility, and empowers and protects consumers, alongside a strategic role for government in setting overall direction.

If we are to take the latter path, tinkering won’t do: a fundamental reorienting of the UK’s energy markets is required to deliver the net-zero transition in a secure and cost-efficient manner.

History

It is important to look at the history of electricity market development in order to understand the context of the current design and to assess its suitability for the future.

The initial expansion of the UK electricity grid in the 19th and early 20th centuries was dominated by localised private generation and supply at a range of voltages and frequencies. This quickly became inefficient as a patchwork of incoherent systems developed, and it became widely recognised that a degree of standardisation and centralisation was required.

The first landmark electricity legislation came in the 1926 Electricity Act and established the Central Electricity Board (CEB), which would build and operate the National Grid. This enabled electricity transmission infrastructure to develop in a more coherent and connected way, and contributed to the rapid electrification of a significant proportion of our energy use. Electricity consumption consequently increased more than sixfold in just 20 years between 1926 and 1946.

The next key piece of legislation was the 1947 Electricity Act, which effectively nationalised the supply of electricity in Great Britain, bringing more than 500 organisations under state control. The British Electricity Authority (BEA) was established to own and operate all generation and transmission facilities with 15 regional boards acquiring supplies from the BEA and distributing power to customers. This nationalised structure of energy markets remained in place for more than 40 years until the liberalisation and privatisation of energy markets by the Thatcher government.

While the era of nationalised electricity saw a huge increase in generating capacity, it also resulted in huge inefficiencies – characterised by vastly inflated capacity margins, poor technology choices and producer capture (when the system was run in the interests of producers rather than consumers).

The 1989 Electricity Act marked a decisive change: shifting ownership away from inflexible and bureaucratic state control and turning to markets to deliver efficient outcomes. While this was presented as a shift to the free market, in fact it impacted differently on different parts of the market. In electricity generation, competition rapidly increased as both ownership and the technology mix diversified, particularly as a result of the “dash for gas”. In energy retail, the market liberalised more slowly, with the Central Electricity Generating Board (CEGB) being replaced by a small group of “vertically integrated” suppliers who owned both generating capacity and the consumer relationship. The transmission and distribution system remained (for good reason) a heavily regulated monopoly with the National Grid owning and controlling the national transmission system, and the 14 distribution networks owned and controlled by six operators, a framework that still exists today.

Liberalisation had marked success – providing more choice for consumers, more diversity in the supply market and significant new investment.

But before long, market failures became evident: under-investment in decarbonisation, concerns about security of supply, and worries that the retail market was not offering value for consumers.

Consequently, the past decade has been marked by increasing state intervention to tackle these problems: new regimes to provide support for low-carbon energy, a capacity market to ensure security of supply and a retail price cap to limit costs to consumers.

The outcome of our approach to date has three major implications:

First, that we still have a market designed around the old technologies and approaches – centralised, high-carbon generation that can respond to demand – and a retail market where competition is based purely on the price of the energy supplied rather than on a more sophisticated service offer.

Second, that our approach is inconsistent. In the generation market, we have de-risked investment in low-carbon generation, to the point that incentives for renewable generators are simply to produce as much electricity as they can, with little regard to the impact on the overall system. But in retail, we’ve taken the opposite approach – incentivising retailers to take excessive levels of risk by encouraging competition based solely on short-term pricing.

Third, that energy markets are set on a path back towards central planning – a system where all generation is dependent on centralised government contracts for a large proportion of its revenue, and where competition in the retail market is undermined by administrative decision-making on prices.

Figure 1

Fluctuations between the free-market approach and a centrally planned approach

Source: TBI

Figure 1 (above) shows how governments have alternated between free-market and centrally planned approaches to UK electricity markets. We’re currently heading back towards central planning, but this time it’s an involuntary direction of travel, rather than one dictated by strategy or ideology.

The Challenge Ahead

The UK government has recently pledged a net-zero electricity system by 2035. That might not seem much of a challenge compared to some of the other transitions we need to make for net zero – after all, we have cut electricity emissions by 70 per cent since 1990. But our achievements to date, while significant, are dwarfed by the scale of the task ahead.

Rapid Low-Carbon Deployment

Between 2020 and 2040, we need to more than double the amount of electricity we produce to power the electrification of transport and heating.

If we are to do that and stay on track for net zero, we need to more than treble the amount of low-carbon electricity we produce.

Between 2010 and 2020, the UK added about 100 terawatt-hours (TWh) of zero-carbon electricity generation – an average of 10TWh of additional generation per year.

Between 2020 and 2040, we need to add roughly 450TWh – or 22.5TWh per year – more than twice as much.

To put this into context, it is roughly equivalent to adding a 1.2 gigawatt (GW) offshore wind farm – the largest ever built – every ten weeks for the next 20 years.

Building a Flexible System to Keep the Lights on and Costs Down

Deploying that volume of low-carbon generating capacity is a huge challenge – but it is not the only issue we need to tackle.

Security of supply is always the first challenge of any energy system. The old cliché that the first task of energy policy is “to keep the lights on” will soon extend to “keep the heating on” – and the implications of failing to meet that task are enormous, given the real risks involved in failing to do so.

The unpredictability of a future energy-generation mix dominated by weather-dependent renewables makes security of supply an even more pressing concern. Hugely increased deployment of renewables means there will be times when we have far more power than we need and, crucially, times when we have far less (Figure 4). There will also be a shift in the demand profile, with significantly more electricity needed in winter as we electrify our heating.

Figure 4

UK wind and gas generation in the first three months of 2021

Source: https://reports.electricinsights.co.uk/q1-2021/when-the-wind-goes-gas-fills-in-the-gap/

Figure 4 (above) shows the current reliance on gas when weather conditions are unfavourable for renewable power. The two lines are virtually a mirror image, and an 11-day wind lull in early March left the system heavily reliant on gas – something that we will have to tackle to deliver a net-zero power system.

To ensure the lights stay on (and the heat pumps continue to run), and at a reasonable cost, we need to build a more flexible system that can maximise the use of green energy when it is available, store the inevitable excess so that it is ready to be deployed during periods of low wind and sun, and maintain secure supplies even during extended periods of low wind.

One aspect of achieving that involves putting incentives in place to build technologies that can deliver flexible generation, such as hydrogen-fired power plants or gas plants combined with carbon capture and storage. But a strategy that focuses only on these technologies, without also incentivising flexibility in demand, will result in vast amounts of capacity that is almost never used – with some plants deployed only once every two or three years during an unusual spell of cold weather and low wind.

An efficient system requires us to move to a market where generation doesn’t just flex to meet demand, but where demand is flexible too. More flexible demand can help deliver a low-cost energy system in two key ways.

First, by reducing the new-build capacity required: if we can cut the level of peak demand (for example, by incentivising people to charge their electric vehicles (EVs) during periods of low demand) then we don’t need to build as much power generation.

And second, by reducing the need for large amounts of additional storage: the use of new technology can hugely increase flexibility – for example, research has shown that 11 million EVs in the UK by 2030 could provide 16GW of daily flexible capacity to the grid, equivalent to around one-third of current peak demand.

The Technology Solutions Exist

These are profound challenges, requiring a transformation in our energy system.

But the good news is that we have a set of technologies and approaches that can enable us to deal with these challenges:

• Renewables such as wind and solar can provide cheap electricity and are ready to deploy rapidly at scale.

• Zero-carbon flexible generation, such as hydrogen-fuelled power stations and gas power, combined with carbon capture and storage can provide flexible generation.

• Batteries – both on the grid and in our electric vehicles – can provide short-term storage capacity.

• Nuclear power can provide steady baseload generation.

• Localised pricing can send clearer price signals.

• New models of retail – such as providing energy as a service rather than a commodity (see the detailed explainer of this model below) – can strengthen the consumer proposition.

However, while these solutions exist, our market is not designed to enable them.

Before considering market reforms, it is crucial that there is clear understanding of the difference between the historic and future objectives of market design.

Generation: A Market Designed for the Technologies We Have, Not the Technologies We Need

Existing Market Design

The generation market we have today was designed around a set of technologies that share three key characteristics:

First, they have relatively low capital costs – they are cheap to build. Second, they have high running costs – they need a lot of costly fuel. Third, they are responsive – able to reliably increase output in response to demand by the unabated burning of fossil fuels.

The wholesale market is dominated by these technologies – in particular gas-fired power stations. These have a natural “hedge” in that rises in gas prices cause electricity prices to rise with them – evidenced by the correlation between gas and electricity prices in recent years (Figure 5).

This model could continue to work in a world where the burning of fossil fuels did not lead to climate change. But the need to cut emissions has created a need to pursue a different set of technologies. The problem this creates is that the generation technologies we need for net zero have an entirely different set of characteristics. They have high capital costs and low running costs (meaning they are expensive to build but relatively cheap to run as they have low or no fuel costs), and are less predictable and responsive to demand.

Figure 6 shows the difference in project cost profile for these two sets of technologies. In short, the generation market was designed to deliver projects with a cost profile like that on the left, but the projects of the future will have a cost profile like that on the right, suggesting a fundamental departure from a market where the gas price effectively sets the market price.

In the current arrangement, the characteristics of supply – dominated by gas – allow the market to flex to meet highly variable demand. But low-carbon technologies are inherently less flexible. Hence the job of matching supply to demand over time will need to be shared by both the supply and demand sides of the energy market of the future.

In addition, the GB electricity market operates on a national basis, with prices set at the national level. This lack of local pricing means that the incentive of generators to build in the “right” place – i.e., the place most helpful to managing the grid in a low-cost way – is limited, and the incentives to build grid infrastructure in pinch-points are inadequate.

To manage the early phase of the transition from existing to new technologies, government has introduced new interventions such as the Contract for Difference (CfD) – a new support regime for renewable generation launched in 2013.

The CfD has achieved a great deal in terms of the deployment of low-cost renewables. But it did not change the underlying fundamentals of the market. It took them as read and added a separate instrument. Generators with a CfD in effect operate outside the market, with renewable generators incentivised to provide as much electricity as they can at all times, but with little incentive to increase their reliability, own the problem of storage in order to flex their supply or locate in places that provide value to the grid.

Retail: A Market Fixated on Price Control Rather Than Innovation

Existing Market Design

The key objective of the retail market is ensuring affordable access to energy for all, and rightly so. However, the success of the government’s approach in pursuit of that objective is up for debate. The approach over the past decade has focused on controlling prices through intense competition and hard regulation. Competition was instigated by the 2013 Retail Market Review, which removed complexity in the market, thereby attracting challenger brands to take on the “Big Six”. This was followed in 2019 by the introduction of the Retail Price Cap, which limits the costs that providers can charge their household customers and is updated at six-month intervals.

Together, these moves have incentivised suppliers to compete hard for customers based on price – the most effective model in a market where the product is undifferentiated (one unit of electricity is identical to any other unit regardless of the supplier). Suppliers, therefore, focus on customer service and financial innovation, and have no incentive to invest in the business-model innovation that the system desperately needs.

By some metrics, this approach has been successful – in particular, the number of household-energy suppliers has shot up in recent years, which could indicate a healthy market (Figure 7).

Figure 7 (above) shows how the number of household-energy suppliers rose rapidly in the early 2010s before falling off slightly in recent years. Unsurprisingly, this has been accompanied by a fall in the market share of the legacy energy suppliers from near 100 per cent to less than 60 per cent in just ten years.

But this has created two significant problems:

First, since there is little else to compete on, this model encourages high-risk hedging strategies to enable lower prices for consumers. Companies are incentivised to avoid entering long-term, well-hedged contracts for supply, which tend to cost more in the short term, and instead seek to lower their costs by gambling on prices in short-term spot markets. This model works well if spot prices are low – but if they rise quickly, and suppliers can’t pass those costs on to their customers, those without deep pockets will fail. We have seen this happen in a limited way in recent years, and much more rapidly as costs of wholesale gas have risen quickly in recent weeks (Figure 8). The result is millions of customers being transferred to a new supplier, which can be a stressful experience for some and comes with a bill increase for many.

Figure 8 (above) shows how initial market liberalisation was followed by consolidation as the Price Cap (trailed in 2018 and introduced on 1 January 2019) impacted business models. The recent energy crisis has seen a mass exit from the market.

Second, it has meant that profitability of energy suppliers is low (Figure 9). While tight margins are a good thing for customers in the short term, they make it more difficult for suppliers to make investments in customer relationships and new business models.

Figure 9 (above) shows how the largest suppliers are operating at a negative margin leaving little room for business-model innovation. These problems discourage the kind of innovation we have seen in markets such as broadband and mobile phones, with different types and levels of service provided based on consumer choice.

Suppliers struggle to build long-term relationships with their customers, and have little incentive or opportunity to develop new models of more sophisticated service provision, such as providing “energy as a service”.

Alongside these new problems, there are two longstanding issues: low consumer satisfaction and the absence of meaningful incentives for flexible demand.

Consumers’ satisfaction with their energy suppliers has been low for many years. The Price Cap, which might have been expected to increase satisfaction, has had no such effect and appears to have constrained competition.

Figure 10 (above) shows that the proportion of customers who think they get a fair deal from their supplier has seen little change over the past five years, hovering around 50 to 60 per cent. Meanwhile, the number of customers switching supplier increased substantially between 2016 and 2019 but has levelled off since the introduction of the Price Cap.

And while technology improvements have introduced innovative consumer offerings such as Time-of-Use tariffs (TOUs), the regulatory approach continues to focus on near-term cost control rather than incentivising the innovation that will deliver longer-term cost savings.

So, in generation, we have a market whose fundamentals are designed around fossil-fuel technologies that cannot deliver decarbonisation, and in retail, we have a market largely comprising unprofitable suppliers providing a commodity-based service to unsatisfied consumers who have no incentive to flex their demand.

An Ever-Growing Role for Government

Government has recognised these problems, and its response has been to intervene to seek to correct them. That approach has had significant successes – in particular, the growth of renewable electricity capacity on the system. But the government’s role has grown piecemeal to the point that it is now responsible for defining an ever-growing number of market fundamentals.

The three key fundamentals are: (a) the amount of low-carbon capacity on the system (defined by the government through the time and size of auctions of CfD contracts); (b) the total amount of reliable capacity on the system (defined through the capacity market); and (c) the price that can be charged to household customers (defined through the Price Cap).

And that has led to contrasting incentives across the system with low-carbon generators deeply insulated from market price risk through the CfD, whereas retailers are left exposed to increasingly volatile wholesale prices while facing a cap on their revenues, which in turn increases exposure further.

The bad news is that the combination of an increased role for government in the market, and the need to build huge amounts of new capacity in the next two decades, means we are on a path to massively increased centralisation of decision-making in the energy market – taking it away from market participants and putting it in the hand of the government, system operator and regulator.

In the generation market, around 7 per cent of electricity is provided by plants on CfD contracts whose incentive is simply to generate as much electricity as they can, when they can. But by 2035, that proportion will rise to as much as 90 per cent.

Figure 11 shows that almost all new generation capacity entering the market is on a central contract and has minimal incentive to respond to demand. As old assets come offline, without action, the market will be increasingly dominated by assets that do not respond to supply and demand signals.

This will have huge impacts – and two are most problematic:

First, electricity generation will become less and less responsive to demand. Generators will simply continue to respond to the incentives they are given that are, on the current CfD model, to generate as much as they can at all times. Without reform, we will have large amounts of excess power at some times, and too little at others.

Second, that the wholesale market will become deeply unstable with low (or even negative) prices much of the time and very steep peaks in times of scarcity. That will both affect the wholesale markets and make the capacity market price very unpredictable and probably very high, as providers of reliable capacity have fewer and fewer running hours in the year, and government becomes increasingly concerned about security of supply in a system with very high levels of renewables.

If the past is a guide to the future, the government response to these impacts will be more and more tweaks to the market – for example through attempts to create stronger incentives to respond to demand in new CfD contracts. But such attempts will not change the fundamental problems.

So, we find ourselves faced with two big challenges, which our current approach cannot address:

First, that the burden of matching supply and demand – historically borne solely by the supply side – now needs to be shared between suppliers and consumers.

Second, that we need to redesign the consumer market to introduce flexibility incentives and enable competition based on a deeper service-offering to consumers, moving away from a simple commodity-based model to one that enables and empowers consumers to profit from the value their flexibility can provide to the market.

The UK’s energy markets are creaking under the pressure imposed by fundamental changes to how we produce, transport and use energy. And, without action, they are on a path to ever more centralisation and higher costs as the market frameworks we have are fundamentally incapable of adapting to the new system we need.

That is a large and complex set of problems – so what should the government do to address them?

Commit to Change

First, the government should acknowledge the scale of the challenge and commit to wholesale reform of energy markets.

To date, government has argued that a low-carbon system can be a low-cost system as a result of falling technology costs and increased flexibility. That flexibility will only happen under a radically different market structure. But government has not taken the step of committing to changing the market structure to deliver the flexibility that is so essential.

A large number of market commentators have called for significant market changes, but the government has so far focused on small tweaks to existing systems. But there is no scenario where we get to 2050 – or even 2035 – with the structures we have now, without requiring a massively expanded role for government and the inefficiencies that come with that.

So, the choice is either to embrace change now or to wait a few years until many of the downsides are embedded before action is taken. Government should take the first choice and set out its intention to fundamentally reform the electricity wholesale market to make it fit for purpose in a net-zero economy.

Set Out a Vision

The four key challenges for the future energy system are increased electrification, decarbonisation, storage and responsive demand. Any one of these would represent a significant shift to a system that has remained relatively unchanged since the completion of the National Grid in 1933. Taken together, they represent a radical overhaul of the generation and supply of electricity, and this overhaul must take place while ensuring the lights stay on and consumer bills remain affordable.

The first step in dealing with this complexity is having absolute clarity of vision. Government must be clear on the desired outcomes from the market-reform process. Flexibility, innovation and fairness should be central to this vision, ensuring that a future market maximises use of renewable energy, encourages (rather than inhibits) innovation in service models, and allocates costs to those that incur them while protecting the vulnerable.

This vision should be formed through engagement with consumers, with those who wish to enter the market and with incumbents. It should look beyond energy to learn lessons from other markets.

Define the Balance Between Strategic Planning and Markets

The level of centralised planning in energy markets has risen inexorably in the past decade and, as noted above, without action we are on a path to a system where almost all key decisions are made centrally.

Some have responded to this by saying that the government should simply get out of the way – for example, by setting a uniform price for carbon and letting the market deliver against it.

But this approach has limitations. It may not be politically deliverable and it ignores the fact that a degree of central planning is required to deliver a transformation on this scale in just 30 years. For example, the transition of our heating from fossil gas to a combination of low-carbon technologies will require coordination at the national and local level, given the potential economies of scale.

A comprehensive market-reform programme should consider how best we strike the balance between government intervention and liberalised markets – with government setting strategic direction and ensuring effective coordination and incentives, while enabling the market to drive innovation in both technologies and business models.

Incentivise the Right Outcomes

The problems in our energy markets fundamentally stem from a misalignment of incentives with the outcomes we need. Retailers are not incentivised to develop business models that enable long-term, technologically enabled relationships with customers. And renewable generators – an ever-growing proportion of capacity – are not incentivised to provide energy when we need it. A process of comprehensive market reform needs to ensure that incentives across the system are aligned with the outcomes we need to deliver. On the example of renewable generators, that would mean working out how better to expose them to market signals that will drive the development of the system flexibility that we urgently need.

Embrace Institutional Change

The nature of decision-making and the responsibilities for effective delivery are as important as the structure of the market. Currently, the picture is confused with government, Ofgem, the National Grid and distribution network operators having overlapping roles. A market-reform process should engage with this issue, defining the roles required in a net-zero power system and changing the institutional frameworks to ensure those roles are effectively discharged.

The Time Is Now

Over the past decade, the UK electricity sector has set the standard for speed of decarbonisation. But in doing so, the needs of the system have quickly outgrown the market framework put forward in the 1990s. With increased demand on the electricity system and accelerated deployment of low-carbon tech, the electricity market becomes the foundation and key enabler of the next stage of decarbonisation in the UK. If the UK wishes to continue leading the way on decarbonisation, the wholesale process of market reform needs to start now.

Lead Image: Getty Images

Charts created with Highcharts unless otherwise credited.