Corporation's brilliant boffins have nowhere left to hide from prying management

As the nation settles down to watch the funeral of Queen Elizabeth II today, some strange-looking boxes will be recording every moment. Just as they have for every channel for over a decade, making the clips available to around 30,000 programme makers in the British TV community, instantly, on demand. This also powers the National Television Archive at the British Film Institute.

What’s unusual about this system is it should never have existed. What’s even more remarkable is that the BBC created it. It does the job more cheaply than commercial systems can, and has saved the licence fee payer a great deal of money over the years.

It’s called Redux, and this is not the kind of public sector technology story we are familiar with. At cultural quangos, and the BBC is the biggest cultural quango of them all, waste and failure have been all too common. The fact Redux works so well is down to its inventor, the storied head of R&D, Brandon Butterworth, who first put the BBC on the internet in 1994 and repeated the trick with The Telegraph the same year.

Yet in the mid-2000s, the BBC’s top brass had earmarked his post for abolition, along with the closure of Kingswood Warren, the home of BBC R&D. At this country house in Surrey, rather like Bletchley Park, engineers cycled to work over the Epsom Downs for breakfast, and it’s where so much broadcast technology was invented.

Somehow, the R&D team had managed to preserve its eccentric brilliance despite corporate changes, as it was far from prying management. Butterworth was behind many of the more recent digital initiatives.

Redux was a stealth R&D experiment to create a computer system that made archive TV content available as cheaply as possible, at a time when TV archives still used tape. Around the same time, the BBC was preoccupied by its Digital Media Initiative, a vast multiyear effort to do the very same job that Butterworth’s boffins were doing for almost nothing, with no operational budget.

DMI had all the hallmarks of a made-in-W1A horrorshow. There was a delivery group, a steering group, and a “deployment and change group”, all managed by committees and sub committees – who all met on different schedules. Yet nobody seemed to be in charge, the National Audit Office later found, after a whistleblower had disclosed the internal chaos.

By the time Tony Hall became Director General in 2013, cancelling DMI was one of his easier decisions: it had burned through £100m, with nothing to show for it.

“Brandon saved their bacon, and saved the BFI’s bacon too,” says one source familiar with Redux and BBC internal politics. The repercussions of DMI rumbled on for years, and also hastened the end of the BBC Trust. This quango was supposed to defend the licence fee payer’s interests, but had gone native, and a cheerleader for a project it regarded as a techno-utopian holy grail. After DMI failed, no one could trust the Trust any more.

Butterworth’s team also devised a way of storing TV content on disk drives stacked into vertical “beer crates” that used DC power, which ran cooler. Cheap sheet metal was ordered to enclose the crates.

Dead hard drives are left where they are until an entire column is removed. No management consultant on earth would have endorsed such a technical architecture, and few economists would have backed Butterworth’s proposition that the BBC could run a bespoke system cheaper than one that used Amazon’s cloud services, for example. Yet the boffins proved them all wrong.

This is really a corporate parable, too: how much can senior management trust their own R&D teams? It’s the world satirised by Scott Adams’ Dilbert, where pointy-haired bosses, who are easily seduced by slick consultants and their management fads, do battle with the engineers, who do their best to ignore their absurd directives and keep the company running.

Over the years I’ve studied three ambitious BBC technology projects where in each case, a small but highly expert technical team saved the management from their own vanity. One was BBC News Online, a classic “skunkworks” project which launched in 1997 for a fraction of the budget of the BBC’s existing online news portal. It was a smash hit, which did the job at the third of the cost of similar commercial broadcast operations like CBS. Other parts of the BBC, embarrassed by this success, then worked hard to sabotage it.

The original iPlayer was another venture that, like DMI, had been announced with a fanfare, but had dragged on for years with nothing to show for it. At one stage, some 700 middle management weighed in at meetings. A new CTO, Erik Huggers, slimmed this down to 15, and with Anthony Rose decided iPlayer should do one thing, streaming, well. Within months it launched and was a huge success.

Redux is to be retired, the BBC announced in May. I asked the Corporation if this meant the useful technology would be abandoned too, but it declined to elaborate. With Kingswood Warren now a housing development, the BBC’s R&D is dispersed far and wide. Some went to Salford, where the glass walls designed to foster “open collaboration” leave the engineers nowhere to hide and create brilliant work without management interference. Which leaves us to ask: so who’ll save the BBC’s management from itself now?

Andrew Orlowski is on Twitter at @andreworlowski.

We rely on advertising to help fund our award-winning journalism.

We urge you to turn off your ad blocker for The Telegraph website so that you can continue to access our quality content in the future.

Thank you for your support.

Visit our adblocking instructions page.