The importance of by elections to Liberal Democrats
(This blog has been updated to include the gain of Richmond Park on 1st December 2016)
The Liberal Democrats claim to have thrown the party’s kitchen sink at the Witney by election. Buoyed by the strong showing there they will now throw the whole liberal house at the Richmond Park by election, which is much more promising territory. Victories and strong showings in parliamentary by elections used to be strongly associated with the Liberal Democrats and its predecessor parties. Spectacular gains such as Crosby, Bermondsey and Brent East gave the party a major boost, often rejuvenating flagging political fortunes. But the party last gained a seat in a by election over a decade ago.
Long established Liberal Democrat members will have fond memories of the exhilaration of a by election win, particularly when the campaigns used to get extensive daily TV and newspaper coverage. But with the exception of the false hope brought about by the retention of Eastleigh in 2013, there has been nothing to cheer since the party went into government in 2010. The Witney campaign may be the sign that the party is rediscovering its lost by election magic.
By elections are far more important to the Liberal Democrats than they are to either Labour or the Conservatives. A spectacular by election performance gives the party the oxygen of positive publicity denied to it by the media for most of the electoral cycle. They have given the party victories on the back of strong campaigns against unpopular government policies such as the poll tax in Ribble Valley in 1991 or VAT rises, as in Christchurch in 1993. The victory in Brent East in 2003 showed that the party was right to oppose the Iraq war, giving Tony Blair his first electoral bloody nose. It may well be that the spectre of Brexit will propel the party to victory in Richmond Park. Previous by election wins have given a jolt to party ratings in the opinion polls. The party needs that by election shock therapy more than ever at the moment.
While by election successes are part of the history of the party and a contributor to its perception as a winner by the media, they also leave a strong internal impression. A win obviously leads to a massive morale boost, leading to more people joining and more members becoming active. Success breeds success. The feel good factor is often temporary but a succession of wins such as during the 1990s can build the momentum for sustained success in general elections. Many by elections have been false dawns, damned by that overused post by election phrase, “if this swing were repeated at a general election…”, which of course it can’t be. By elections enable the Liberal Democrats to assemble all its Davids to take sling shots at Goliath. In a general election the concentration of effort cannot be replicated. By election wins have led to false expectations of a party that has fewer people on the ground, less national money and hardly any media friends compared to both the other major parties of government.
By elections assemble people for a common endeavour. The Liberal family comes together, rather like at party conferences but with an immediate practical electoral purpose. Like all family gatherings they can be happy and sad. In the 2010 Parliament they felt like a procession of funerals. In the 2015 Parliament I think that the party, smashed and left for dead in the general election, can quickly recover and by election success will be part of the journey. They can make politics enjoyable again. New members can learn some old campaign tricks and our professional campaign staff will trial new ones. Local council by election wins are already having a positive effect, with over 20 gains since May 2016.
Since 1981 the SDP has gained 4 MPs in by elections, the Liberal Party also 4 and their successor Liberal Democrats have gained 11 prior to Richmond Park. The Liberals have held one and the Lib Dems two. There have also been several near misses, the most recent being Bromley and Chislehurst, lost by just 633 votes in 2006.
I’ve helped in 27 by elections, with Richmond Park now my 28th. My first, by pure chance was in my home seat of Cynon Valley in May 1984. In a safe Labour seat, it doesn’t feature in anybody’s history of significant by elections. But as a 17 year old I was standing in my school mock election as the SDP candidate. I turned up at the SDP campaign office in Aberdare to ask for some posters and a rosette. I was asked if I wanted to try some real campaigning. I was given a clipboard (a piece of wood and a bulldog clip with a pencil attached by string, nothing flash) and told to go and ask the people on the list for whom they intended to vote. This was the beginning and the end of my canvassing training. But I enjoyed it and I’ve now being doing it for 32 years. I’ve trained dozens of new canvassers in my time, hopefully without throwing them straight into the deep end. I also won my school election and then had to wait 9 years before I won a real one!
Here’s my own list of the most significant by elections for the Liberal Democrats and their predecessor parties after the formation of the SDP and the Alliance with the Liberal Party in 1981:
Warrington (16th July 1981)
The first by election since the formation of the SDP and contested by Roy Jenkins, the grandee of the Gang of Four. The Liberal Party had secured only 9% of the vote in the 1979 general election. Having agreed broad terms of a national pact in the previous month, the Liberals stood down, giving Jenkins a free run. The result was an electoral earthquake. Jenkins won 42% of the vote with Labour holding on with just 48%, down from the 62% in the general election. The result propelled the SDP-Liberal Alliance to giddy heights in the opinion polls. Jenkins himself said that though it was his first defeat in an election, it was by far his biggest triumph.
Croydon North West (22nd October 1981)
The Conservative MP Robert Taylor had died before the result of Warrington and the Liberal Party already had a candidate in place, William Henry (known as Bill) Pitt. Bill Pitt had fought the three previous general elections and neither he nor the local Liberal association were willing to stand aside for a new SDP candidate. But Pitt became the beneficiary of the SDP’s national popularity and standing as the candidate of the Liberal Party “with SDP support” he gained the seat with 40% of the vote and a majority of 3,254 over the Tory. In all subsequent by elections until 1988 the SDP and the Liberals fought joint campaigns as the “Alliance”. Pitt proved that by elections could be won by candidates who were not household names. He lost in the 1983 general election by 4,092 votes and by 1996 had joined Labour.
Crosby (26th November 1981)
With Bill Pitt contesting Croydon NW it would fall to the SDP to contest Crosby. The seat was one of the most prosperous in the north west and had been safely Conservative on various boundaries since 1918. The Liberals had been third in 1979 with 15%. The Crosby by election became the vehicle for a triumphant return to Parliament by Shirley Williams. She became the SDP’s first elected MP, winning with 49% of the vote. But the triumph was short lived as Williams lost the seat (on different boundaries) in the subsequent general election.
Glasgow Hillhead (25th March 1982)
After Shirley Williams’s return to Parliament the SDP’s priority was to find Roy Jenkins a berth. Hillhead was in many ways tailor made for him, being the intellectual middle class heart of Glasgow. It was the last remaining toe hold of the Conservatives in the city. On a huge turnout of 76% Jenkins gained the seat, returning to the Commons he had left in 1977 to become President of the European Commission. He was then elected as the first Leader of the SDP. He held the seat in 1983 but lost in 1987 to the odious George Galloway. One of the young SDP volunteers in the by election was Glasgow student Charles Kennedy.
Mitcham and Morden (3rd June 1982)
After Hillhead it seemed that the SDP-Liberal Alliance was on course for a breakthrough at the next election. But just a week later General Leopoldo Galtieri of Argentina shattered that hope. The Falklands War transformed the public impression of Margaret Thatcher and the electoral fortunes of the Conservative Party. Bruce Douglas-Mann must be one of the unluckiest by election candidates. The incumbent Labour MP, he had defected to the SDP but unlike all the other defectors resigned his seat to fight a by election under his new colours. It must have seemed a safe gamble but by polling day the Alliance national opinion poll rating had slumped and the Conservative Angela Rumbold gained the seat, a rare example of a governing party gaining a seat at a by election. Prior to Mitcham it looked as though the Alliance would indeed “break the mould” of British politics as both Labour and the Tories were deeply unpopular. After Mitcham, the progress of the Alliance would be much more of a hard slog.
Bermondsey (24th February 1983)
One of the bitterest by election contests of all time, Bermondsey was a contest between the Liberal Simon Hughes and Peter Tatchell, then seen as a representative of the Labour hard left. The local Labour party split, with the former MP Bob Mellish (Chief Whip under Harold Wilson) backing the “Real Bermondsey Labour” candidate and former Southwark Council Leader John O’Grady. Tatchell was known to be gay and O’Grady used this against him. It is current Labour folklore that the Liberals exploited Tatchell’s sexuality but the homophobia came mainly from O’Grady, with one of his campaign ditties saying that not only was Tatchell an Aussie but he wore his trousers back to front. The result was a 44.2% swing, the biggest by election swing in history, propelling Hughes into the Commons and giving the Alliance a pre general election boost. I met Peter Tatchell several times while I was an MP, sharing several platforms on gay rights. In 2007 we spoke on the same side in an Oxford Union debate, massively defeating the motion that “this house believes that marriage should be between men and women only”, six years before I led for the Liberal Democrats on the legislative stages of gay equal marriage!
Portsmouth South (14th June 1984)
To some, an unexpected gain for the SDP but demonstrating the strength of a candidate well rooted in the constituency, Mike Hancock was a long serving Labour then SDP councillor. Labour’s candidate was a CND supporter, an unpopular stance in the home of the Royal Navy. Hancock’s victory was a boost for the continued independence of the SDP, which had been reduced to just six seats in the 1983 election. He suffered two narrow defeats by just over 200 votes in the 1987 and 1992 elections before returning in 1997. Rather like Bob Russell in Colchester, Hancock was one of the few former Labour working class members of the Lib Dem Parliamentary party, often giving Nick Clegg a hard time at group meetings. The Tory he defeated in 1997, David Martin (uncle of Chris from Coldplay) went on to be defeated by me in Bristol West in 2005.
Brecon and Radnor (4th July 1985)
A huge rural constituency, the largest in England and Wales. The campaign took Liberal Richard Livsey from third in 1983 to first in the by election in a tight three way contest, which saw the incumbent Conservatives drop to third place. On the back of major gains in the May county council elections it returned the Alliance briefly to the heady days of early 1982. But it was another false dawn. For me the by election was my first taste of victory. I had just sat my A levels and the constituency was next to my home of Cynon Valley. Apart from the usual leaflet delivery I also got the chance to tour Radnorshire villages in the broadcast car. Loudspeaker vehicles used to be a normal feature of election campaigns but disappeared (at least for the main parties) by the 1990s. I was selected after an audition conducted by the agent who decided I had the most pleasing Welsh accent. I said what I liked in English but stuck to the script in Welsh. “Good Morning Knighton, Prynhawn da Trefyclawdd” was followed by what passed for the campaign soundbites of the day. It was all very “hi di hi” for politics but great fun making people jump when bellowed at while going about their shopping.
Bootle (24th May 1990)
I include this not as an example of party triumph but as a vital contest in the battle for party survival. After the 1987 general election disappointment (23% of the vote but a joint tally of only 22 MPs) the SDP and Liberal Party merged to form what became known as the Liberal Democrats. A minority of activists and serving and former MPs of both predecessor parties refused to accept the merger and continued to fight elections as the Liberal Party or SDP. Prior to legislation on party names and ballot papers in 2000, nothing could be done to stop them. The upshot was that by elections in the first two years of the 1987-92 Parliament were often contested by three liberal factions. By elections that would have been won with a united candidate were instead thrown away while the Liberal Democrats struggled to establish themselves as the main third party of British politics. Bootle was a rock solid safe Labour seat and the official Liberal Democrat candidate John Cunningham got only 3,179 votes, 8.9% of the total. But the unofficial Liberals and SDP were beaten to just 474 and 155 votes each, with the “SDP” having the added humiliation of being beaten by the Monster Raving Loony Party. After Bootle the Lib Dems were free to make their own luck, though local rumps of liberals and social democrats remained in a handful of councils. The winning Labour MP died from a heart attack just 57 days later, necessitating a second by election.
Eastbourne (18th October 1990)
The by election was caused by the murder of Conservative MP Ian Gow by a car bomb planted by the IRA. But the Conservatives did not receive a sympathy vote. The party was unpopular, seeing huge reversals in the May council elections as a protest against the poll tax. By the autumn the party in the Commons was bitterly divided over Europe. The result was the first by election win for the Liberal Democrats and David Bellotti became the first elected Lib Dem MP, taking just over 50% of the vote. The by election was a massive boost for Paddy Ashdown and the party went on to win two more by elections at Ribble Valley (7th March 1991, Mike Carr probably putting the nail in the coffin of the poll tax) and Kincardine and Deeside (7th November 1991, won by Nicol Stephen) giving the party enough fuel to survive the 1992 general election with 20 Liberal Democrat MPs. Sadly, the three by election winners were not among them.
Newbury (6th May 1993)
The first by election of the 1992 Parliament was a huge defeat for the Conservatives, shorn of their reputation for economic competence after “black Wednesday” and the pound crashing out of the European Exchange Mechanism in September 1992. David Rendel won a massive 37,590 votes, 65% of those cast. This is still the record vote tally for the party, though Winchester (see below) surpassed it in percentage terms. Newbury was the first of a string of by election losses to the Lib Dems and Labour that robbed John Major’s government of its Commons majority. The victory and the scale of it was all the more remarkable as it occurred on a normal May polling day for the county council elections. Liberal Democrat by election wins are usually the result of extraordinary levels of commitment by activists from all over the country. But Lib Dem activists were involved in battles for council seats all over England and Wales. I won my own Cabot ward seat on Avon County Council on the same day but went to Newbury twice. The party made large council gains on the same day. The subsequent three by election wins in Christchurch (29th July 1993, won by Diana Maddock), Eastleigh (9th June 1994, won by David Chidgey) and Littleborough and Saddleworth (27th July 1995, won by Chris Davies) gave the Liberal Democrats bursts of good publicity and continued electoral credibility. They were also important in establishing the importance of tactical voting to defeat the Conservatives, vital in seeing the party win 46 seats in the 1997 general election.
Winchester (20th November 1997)
Mark Oaten had scraped home to victory by just two votes in the 1997 general election, becoming the 46th Liberal Democrat MP elected at that election. The defeated Tory, Gerry Malone, over turned the result with a court challenge. It was a big mistake. The Tories had crashed to a terrible defeat and the new Blair government was in a long honeymoon. The public were in no mood to help out the Tories. Of all the by election campaigns I’ve helped in, Winchester was the most friendly to canvass and easiest to hand out orange diamond window posters. Mark was re-elected with 68% of the votes cast, a Lib Dem record. Next door Romsey (4th May 2000) was the only other Lib Dem win of the 1997 Parliament, with Sandra Gidley joining Oaten and Chidgey in a strong Liberal Democrat corner of Hampshire.
Brent East (18th September 2003)
While previous Alliance and Lib Dem by election triumphs had mainly been at the expense of the Conservatives (Bermondsey and the SDP’s Rosie Barnes in Greenwich being the exceptions) Brent East was a major victory against Labour. It was the first by election loss for Tony Blair’s government and a vindication of Charles Kennedy’s decision to oppose the Iraq war. Sarah Teather went on to become one of the party’s star performers in the media and held the seat in 2005 and its successor (with a massive boundary change) Brent Central in 2010.
Leicester South (15th July 2004)
This by election has its place in Liberal Democrat history as being the only seat so far to elect a non-white Lib Dem MP. Parmjit Singh Gill gained the seat from Labour on a 21% swing, in similar circumstances to Brent East. He did not establish enough of an incumbency advantage to hold the seat at the general election nine months later. On the same day as the by election the Lib Dems came within 460 votes of winning the safe Labour seat of Birmingham Hodge Hill. This was a repeat of the disappointment in May 1986 when the Liberal Alliance candidate Elizabeth Shields won Ryedale in Yorkshire by Chris Walmsley fell short in West Derbyshire by just 100 votes.
Dunfermline and West Fife (9th February 2006)
This was the last by election gain for the Liberal Democrats for a decade and was won in extraordinary and not very promising circumstances. Charles Kennedy had lost the confidence of his Parliamentary colleagues and had resigned as Leader in the first week of January 2006, before the return of Parliament from Christmas recess. The Labour MP for Dunfermline died on the same day. MPs dutifully trooped north to help in the by election, partly to raise our morale and show we were still in business. I recall being freezing cold walking round a snow bound housing estate delivering leaflets with Nick Clegg and Ed Davey as we chatted about how the party could re-establish itself as a serious outfit in the eyes of the public. Ironically, Charles Kennedy himself helped rescue the situation. To his supreme credit, he turned out to help in the election. I walked down the high street with him, surrounded by journalists and cameramen. It was clear that he was still hugely popular with the Scottish public. Willie Rennie pulled off a most unlikely by election win by a party without a leader.
Oldham East and Saddleworth (13th January 2011)
I include this by election as it was the first electoral signal that the Lib Dem decision to form a coalition government with the Conservatives would cause a haemorrhage in support in Labour facing constituencies and among left leaning voters. The reception on the doorsteps from many previous Lib Dem voters was as icy as the winter weather. The election shows the often personal unfairness of politics. The Lib Dem candidate Elwyn Watkins had fallen short by just 103 votes in the 2010 general election. The court overturned the result as Labour MP Phil Woolas was guilty of “knowingly making false statements” in campaign leaflets. In normal circumstances Watkins would have secured the result he had deserved in the general election but Labour held the seat with a majority of 3,558. The Lib Dem vote did in fact increase by 0.3% but this was as a result of huge tactical voting by Conservatives.
Eastleigh (28th February 2013)
A rare by election in a seat held by the party or its predecessors, the last two in this survey being Cheadle in 2005 and Truro in 1987. Triggered by the resignation of Chris Huhne, pleading guilty to motor insurance fraud, the by election was critical to the fortunes of the Liberal Democrats. Support for the party in council elections and other parliamentary by elections had plummeted. Party HQ and nervous MPs clung to the hope that well entrenched incumbent MPs would withstand the fall in support elsewhere. Eastleigh would put this theory to the test. The party made a huge effort, supporting one of the most successful local party machines that had built an overwhelming domination of council seats. Local councillor Mike Thornton came out on top of a tight three way split, with UKIP pushing the Conservatives into third place. MPs and party strategists heaved a sigh of relief. But two years later no amount of local goodwill and incumbency advantage could withstand the tidal wave that swept all but 8 defending Lib Dem MPs out of office, including Thornton. Eastleigh stabilised the party at the mid-point of the coalition but ultimately it gave false hope.
Witney (20th October 2016)
This was the first by election since the days prior to the 2010 coalition formation when the party came together to fight a major campaign, hoping for a strong advance rather than avoiding embarrassment. The shock of the 2015 general election reversal and then the referendum vote to leave the European Union had resulted in an influx of new members into the Liberal Democrats. The Witney by election saw many of these “newbies” join by election stalwarts in trying to pull off a spectacular result in one of the safest Tory seats in the country. For those of us who are by election veterans it indeed felt as positive as the old days and it was a big learning experience for new members. In the end the excellent local candidate Liz Leffman increased the party’s vote share by 23.4% and jumped to second from the miserable fourth place secured in 2015. Witney showed that the voters were once again prepared to switch to the party in large numbers. It was also a major morale boost to the party, showing that the party could, in the right circumstances, win again and even enjoy by election campaigns!
Richmond Park (1st December 2016)
The contest was triggered by Conservative MP Zac Goldsmith resigning after the government announced it was proceeding with a third runway at Heathrow. Goldsmith must have felt confident. He was honouring a campaign promise to resign and fight as an independent and he was cushioned by a 23,000 majority over the Lib Dems in May 2015. But his gamble ended in disaster. The constituency had voted about 70% to remain in the EU, whereas Goldsmith was one of the most ardent Brexit supporters. His 23,000 majority was also flattering and not a real reflection of the strong Liberal Democrat support in the seat going back decades, including three victories in 1997, 2001 and 2005. Buoyed by the huge swing in Witney, and smelling Brexit blood in the water, the party poured in resources. Hundreds of members turned up every day (I went 4 times) to knock on doors and deliver leaflets. While Goldsmith wanted the election to be fought on the single issue of Heathrow the Liberal Democrats focused on resisting a hard Brexit and calling for a new referendum on the eventual deal with the EU. Three parties decided not to field a candidate. The Conservatives and UKIP stood aside to help Goldsmith. The Green Party pulled out to help the Lib Dems in a “progressive alliance”. Labour insisted on standing and no doubt would have preferred a Goldsmith win in order to stymie a Lib Dem revival. The result was a Lib Dem triumph, with Sarah Olney defeating Goldsmith by 20,510 to 18,638 votes and Labour crashing to 1,515 and a lost deposit. Olney was one of the thousands of new members to have joined the party since 2015. Her win will restore the morale of campaign veterans, excite her fellow “newbies” and no doubt lead to a rise in support for the party. It also plots a path back to success for the party, as the clear voice for the 48% who voted to Remain in the EU and probably for a few more who now have doubts about the sense of a hard Brexit.
And finally this is my own by election roll of honour, 28 campaigns, including Richmond Park:
May 1984 Cynon Valley (Lab hold, SDP 2nd)
July 1985 Brecon & Radnor (Liberal gain from Tory)
February 1987 Greenwich (SDP gain from Labour)
February 1989 Pontypridd (Labour hold, Social and Liberal Democrats 4th, continuing SDP 5th)
March 1991 Ribble Valley (Liberal Democrat gain from Conservative)
May 1991 Monmouth (Labour gain from Con, LD 3rd)
May 1993 Newbury (Lib Dem gain from Con)
July 1993 Christchurch (Lib Dem gain from Con)
February 1995 Islwyn (Labour hold, LD 3rd)
July 1995 Littleborough and Saddleworth (Lib Dem gain from Con, Lab 2nd)
November 1997 Winchester (Lib Dem gain from Con after GE result void)
May 2000 Romsey (Lib Dem gain from Con)
November 2001 Ipswich (Labour hold, LD 3rd)
September 2003 Brent East (Lib Dem gain from Labour)
July 2004 Leicester South (Lib Dem gain from Labour)
July 2004 Birmingham Hodge Hill (Labour hold, LD 2nd)
September 2004 Hartlepool (Labour hold, LD 2nd)
July 2005 Cheadle (Lib Dem hold, Con 2nd)
February 2006 Dunfermline and West Fife (Lib Dem gain from Labour)
June 2006 Bromley and Chislehurst (Con hold, LD 2nd)
July 2007 Ealing Southall (Labour hold, LD 3rd)
May 2008 Crewe and Nantwich (Con gain from Labour, LD 3rd)
June 2008 Henley (Con hold, LD 2nd)
July 2009 Norwich North (Con gain from Labour, LD 3rd)
January 2011 Oldham East and Saddleworth (Labour hold, LD 2nd)
February 2013 Eastleigh (Lib Dem hold, UKIP 2nd)
October 2016 Witney (Con Hold, LD 2nd)
December 2016 Richmond Park (Lib Dem gain from Con)