Don't Look Back in Anger: Wildman Whitehouse and the Great Failure of 1858

Speaker 1: All right, after this lunch break We now have on deck Lily Ryan and she's going to give us the world's only telegraph-driven conference talk.

Speaker 2: Hey, folks. Thank you for coming to my talk. Um yeah, I saw I can promise you a lot of drama and backstabbing and poetry. and useful lessons. But before I do that, as Marcus alluded to, I want to talk a bit about these slide changes. Just for fun and because it's a bit on theme, I've got two telegraph keys. One makes the slides go forward Does it? Ooh. Well makes him go back. And um I'm gonna use this to drive the talk. And I figured I might as well make the telegraph key drive the talk um figuratively as well as literally. So there we go. Moving right along.

Speaker 2: My name is Lily Ryan And my career so far has existed at various points on the spectrum between historian and hacker. And I really like to talk about all of the weird things that went down in the past so that we can learn something from them. Because I think that being able to reflect on the mistakes and the failures of the past helps us to build better teams and do better work in the future. And the story I'm going to tell you today is one of the biggest failures of tech history. And I want to tell you a bit about how Reflection worked for that project. I think that successful reflection comes in two halves. Firstly, we want to understand how we got to where we are. And then

Speaker 2: we want to apply those lessons to where we're going next. The problem is when a team only does the first half of this, which is what can often happen, we think about what we did We don't take any lessons from it, and then we do it again. And you can get away with this for quite a long time if things are going well But when things go wrong, applying these lessons to future actions is quite often the only thing that can pull you out of a bad cycle. And it's usually when we fail that our projects are thrust right into the spotlight, which is the worst time. This is when we would really rather hide. On the other hand, succeeding at something usually feels awesome. One of my favorite

Speaker 2: things about the release of a successful project is the go live celebrations. And it doesn't matter much to me if this comes in the form of an office party, or if a few of the teams slink off to a bar after work, or if The team is even actually allowed to celebrate it all, instead of waiting by the build pipeline lighting candles and praying that things stay green. Probably one of the best go -live parties on record happened in 1858, after the completion of the first transatlantic telegraph cable. To understand why this one telegraph cable was such a big deal, I need to explain what a telegraph is, because it's hardware that's so old that it is still not vulnerable to meltdown or specta.

Speaker 2: So, most people here probably probably haven't used one. That's a picture of a telegraph receiver. These are telegraph keys. Normally you would only have one of them at the one end, but I've got one for for back and one for forward. Basically, telegraphs were the first internet. In the Victorian era, people started to send messages to each other along copper wire using basic electrical pulses. Mostly they would use Morse code for this. This was a system of long and short pulses that corresponded to the letters of the Roman alphabet. So you'd write your message down and you'd take it to the telegraph office and an operator would key it in and someone at the other end would listen to that sequence of dots and dashes and write it down back into the language.

Speaker 2: and send it on to the person who was meant to receive it. It was better than normal mail, because electricity can travel much faster than trains or horses, so telegraph messages got to their recipients much faster. And by eighteen fifty-eight both North America And Britain had nationwide telegraph systems, and they'd had them for a few decades. This was how most people did business on on either side of the Atlantic Ocean. But To send a message from Britain to North America, or vice versa, during this same time, would take several weeks, because you would have to write it down and put it on a boat and sail it across the Atlantic Ocean. And this was why when the world heard that there was now going to be a telegraph cable that would connect the two continents, they went absolutely wild for it.

Speaker 2: The 1858 Telegraph Cables completion was one of the most spectacular GoLife projects. in history. Go life parties in history, sorry. On both sides of the Atlantic, people fired cannon and they let off fireworks, they danced in the streets, and They wrote reams and reams of some of the worst poetry that I've ever heard. Here is some of it. 'Tis done, the angry sea consents, The nations stand no more apart. With clasped hands, the continents feel the throbbing of each other's hearts. Speed, speed the cable, let it run, a loving girdle round the earth, Till all the nations neath the sun

Speaker 2: shall be as brothers of one half. Believe me, there is heaps more where that came from. People really, really, really loved this cable. They loved it so much that they were still celebrating about it three weeks later. But the weeks of party hit a snag when this brand new transatlantic telegraph cable went completely dead. So these days, when a big release goes bad, we usually have some strategies in place. We don't always assume that things are going to work. We hope they will. But we will we we'll plan for the worst. We might have thought of a way to roll the changes back if we need to. We do a soft launch just to make sure things are really working.

Speaker 2: But the telegraph guys hadn't thought of anything except success. There was no fallback plan when this failed. There was just lots and lots and lots of rage. We all like to feel like our projects are going to change the world. Often they do We change people's lives all the time by rolling out new stuff to make sure that the people who trust us with their business can do what they need to do. But the Transatlantic Telegraph Project changed the world in a really huge and history-making way. This kind of makes all of the fireworks understandable from a Victorian perspective, in my opinion. I don't find the poetry understandable, but we can put that part aside.

Speaker 2: But I also hope that it will help you understand just how dramatic things got when this amazing cable totally died. And when your stakeholder literally the entire population of two continents, including the Queen of England, who, because she was the Queen of England, wanted to be the first person to use this thing, you really feel the heat when things go wrong. It's always a hard time when a project fails. And a lot of us have experienced this at one time or another. And if you haven't yet, believe me, you almost certainly will at some point. No matter how excellent the team is or how good the idea is, not everything is going to go smoothly. It's just life. But failures are also very valuable.

Speaker 2: For me, they've been the places where I've learned the most about myself and gotten the most valuable feedback about my work as a team player, my strengths under pressure, the ability of all of us to bounce back when things go wrong And I think that reflecting on your work is often richer after you've had a failure. Asking yourself and your team, what did we learn gives us really useful fuel for moving on. Unfortunately for the transatlantic telegraph cable team, 1858 was only 15 years after the first computer program was written. And it was forty-eight years before the first computer was built. It was a time when the phrase agile retrospective would only ever be used to describe someone looking over their shoulder for bad guys as they sprinted down an alleyway.

Speaker 2: They really didn't have any models to learn from. But this doesn't mean that we can't still learn something from this project which was one of the biggest tech delivery projects in history because in many ways it forms the cornerstone of every modern delivery project. And the key person we're going to learn this lesson from is Dr. Edward Orange Wildman Whitehouse, this guy So in addition to having the best name and the wildest facial hair of his entire team, he was also the chief electrician of the Transatlantic Telegraph Cable Project. This was basically their tech lead. He was also the main architect of its demise. White House had a history as a doctor and a scientist

Speaker 2: and an inventor. When I was doing research for this, I pulled up a bunch of patents that he'd had for a new type of roll escape. But in term uh one of the other patents that he had released was for telegraph invention. He dabbled a bit in telegraphy as part of his side projects and patented some deviations on telegraph technology, which When you look at them today was what he'd done was basically he had forked other people's repos and he had changed maybe a single line and then he'd put all of that out as his work. Anyway, the fact that he had all of those green squares on GitHub had impressed Cyrus Field, who was the American businessman who was providing all of the money for this transatlantic telegraph cable endeavor.

Speaker 2: So despite the fact that White House had no actual experience with actually running or implementing telegraph systems, he got appointed as the chief electrician of the Atlantic Telegraph Company. The main problem with the transatlantic telegraph cable was that nobody was actually sure if it could actually be done. I mean, we'd had telegraph technology for a little while , but we'd never really had something that went underwater for quite such a long way, especially no continuous cable that had to be unbroken about three thousand kilometers long. Felt about as impossible as, you know, you think of building a space elevator to the moon. And the other problem was that scientists at

Speaker 2: the time were still discovering some of the basic laws of physics that we take for granted today. Electric currents were being used for a lot of stuff in 1858, but people didn't really understand that much about how they worked And this meant that a lot of the ideas that people had about how to actually build this cable to run this entire way and put an electric pulse across this such a distance, these were just guesses. And the only way to prove or disprove the theory was to build the thing, which was going to be very expensive. Fortunately for everybody, Cyrus Field had absolutely no idea that this was supposed to be impossible. He just kinda hired some people to do it. One of the other people that Field hired to be on the board of his Atlantic Telegraph Company was this guy.

Speaker 2: He was a young man called William Thompson. He had fewer cool names and fewer credentials than White House, but he was also a keen physicist with an interest in electricity and telegraphy, and he had some interesting new ideas about how electrical currents worked And Field had discovered Thompson after Thompson had written a bunch of proposals about how to build the telegraph cable. And Field liked his spirit, so he hired the guy. And when White House met Thompson, it was rivalry at first sight. They hated each other. White House was used to being listened to. And Thompson was younger than him and questioning his authority. And Thompson had his own measure of respect with the board. Anne Thompson's facial hair wasn't nearly as good as his.

Speaker 2: Although later in life he would grow a beard that was so magnificent that it would make a hipster weep to see it. It was amazing. The main point of contention between the two of them about how to build the cables was literally in the construction of the cables themselves and how they should be designed so that they could hold up to the distance they had to cover. and the situation they had to cover to and still carry a signal. So Whitehouse argued that the copper that was used to conduct the signals should have a very small diameter But that the voltage that was used to carry the signal across should be very strong, kind of like yelling the current across the entire ocean. He also decided that the cable should be very heavy so that it would sink to the ocean floor and it wouldn't move around too much.

Speaker 2: Thompson disagreed with White House about every single one of these points But Whitehouse was confident that he was right, and he was more senior, and he refused to hear any of Thompson's ideas. He was the tech lead, and Thompson was just this junior guy. And there also wasn't much time for Thompson to persuade somebody as stubborn as White House that there were other points of view. Field and the board also wanted this project to be completed as quickly as possible because they'd gotten their investors and most of the people in Britain and America really excited about the project. Everybody wanted it delivered yesterday. So they rushed out the cable, using White House's design. And they stuck two halves of it on two ships.

Speaker 2: One of these ships was going to spool the cable out behind it from Newfoundland in the Americas, and the other ship was going to do the same from Western Ireland. And the idea was that they would meet in the middle, join up the cable, and you would get a working transcontinental telegraph cable. This picture shows a bit about how they were loading the cables onto the ships. They basically made a little bridge to roll them along because they were so heavy, they weighed about an imperial tonne per mile. They couldn't just be carried on. They had to feed them onto the boats and then feed them back off out at sea. Unfortunately, White House's health wasn't that great when this project was going on. This meant that it was up to Thompson to actually get on the boat that was going to carry the cable out to sea and spool it across the ocean.

Speaker 2: So the younger man had to oversee the project while White House remained on shore and waited And the first attempt out to sea was a total disaster. After about three days, the incredibly expensive cable snapped under its own weight and sank to the bottom of the ocean. which I might point out it was actually designed to do. So Field went and made his excuses to the investors and he got some more money to make a replacement cable. And this is what you can see them doing in this picture. They're covering the cables with this layer of a couple of layers of what they call Gutta Percha, which is tree sap that could be molded soft and then it hardened like plastic. And this was before plastic had been invented, so the Victorians were using Guta

Speaker 2: Percha for everything, to the point where the trees that it came from nearly went extinct. Um and then plastic was invented and the tree did not go extinct, but a lot of other things did. Anyway, after about a year of very expensive cable making The new cable was finally ready, and so they loaded it onto the ships same way along a bridge, and set out again. The idea the second time around was that the two ships would start it together in the middle of the ocean and then go towards the coast from there. And it took them three more tries, meeting in the middle, and spooling it out, sailing out again, and every single time the cables snapped and sank. It was getting more and more expensive. And in one of their attempts, the cable was almost completely destroyed by a passing whale, which you can see here.

Speaker 2: This is what I like to call the original fail whale. But by the fourth attempt, they had made it the whole way across without the cable breaking. and it was connected to the local telegraph networks at either end, and it was declared ready for business. And this was about the point where the entire Western world went completely wild and they kicked off that go live party that we talked about earlier. This was also where Wildman Whitehouse sealed his own fate. He and Thompson had argued about how the cable should be built, and now they thought about how it should be operated. As I said earlier, Whitehouse had designed this system to generate a very strong electrical current along the entire length of the line.

Speaker 2: And Thompson didn't think that this was the greatest idea and had managed to convince him to put a few brakes on the voltage. The system wasn't getting nearly as much power as it was capable of generating, but in terms of physics and stuff, this is actually kind of what you want. But as the weeks went on, the demand for the telegraph cable grew and grew and letters were piling up at both ends of the line waiting to be sent through. You know, Morse code. You can only send one at a time. It was not you know, there was one line. So in order to make it faster, so he thought, Whitehouse secretly increased the voltage as high as he could make it. Someone's making a noise like they know that's a bad idea. Because Whitehouse's expectation was that the stronger the current, the faster and clearer the messages would get to the other end of the line.

Speaker 2: The reality was that this amount of voltage completely fried the extremely expensive cables. He had overwhelmed them with power, which meant that they short-circuited into the sea. and some of the nearby sharks had boiled fish for dinner. So the upshot of White House's secret tinkering with the system in prod, mind you, meant that this famous cable was completely dead within three weeks of going live. And the world was very angry. And so was Cyrus Field, and so was the entire board of the Atlantic Telegraph Company, because they literally had one job. It was in the name, and they had managed to do it. This is, to put it in modern terms, about as big a disaster as if the entire internet

Speaker 2: had gone down. And they didn't know if they were ever going to be able to get it back When a project fails this spectacularly and when this much money is involved, there's usually a lot of public head scratching and soul searching about what went wrong and where. Questions get asked, and as you would probably expect, the best companies will take some time to step back and reflect on what's happened to see what lessons they can learn to make it better. And while most of the board of the Atlantic Telegraph Company were doing precisely this, the one man who wasn't was Edward Orange Wildman Whitehouse. He had skipped right across reflection and into blame, which is a pit that it is unfortunately very easy to fall into.

Speaker 2: He didn't want to reflect, because he knew whose fault it was. Everybody's except for him And the public was so angry with the failure of this cable that it demanded and it got a public inquiry into the whole project. This is one of the first project retrospectives to take place in front of two entire continents. During this inquiry, William Thompson testified to the reams of his advice that had been ignored by the chief electrician. And others stepped in to testify to the fact that they had seen the chief electrician messing with the voltage on the cable in the weeks after its completion, which was completely against what the board had decided to do and against all known scientific advice.

Speaker 2: Getting feedback like this is always a bit of a blow, especially when you thought that you were doing a good job. Getting feedback like this publicly is even harder. It is natural to feel defensive when this happens. And honestly, public feedback about specific individuals is generally not a great idea. But that's what happened in this case And it's not usually a great idea, because it often leads to situations like what followed after this. Whitehouse hadn't entered into the inquiry with an open mind, but honestly, he hadn't entered into the entire project with an open mind. He was a self-righteous Victorian era gentleman scientist, and a doctor, and a member of the Royal Society, and he was used to being treated as though that meant he could do whatever he wanted.

Speaker 2: Unfortunately for White House, there was no software development community in 1858 to suggest to him how he might tactfully handle this kind of situation. So he did what angry, self-righteous, Victorian-era gentleman scientists would do. He published a pamphlet about the whole affair. I gosp! And he sent it to all the papers and he had them publish it too. And the pamphlet was called, it was kind of understated, it was called The Atlantic Telegraph. But a better title might have been Everyone Is Wrong Except Me, the Greatest Scientist in the World. I could describe what was in it, but I'm going to let Whitehouse speak for himself here because it's better.

Speaker 2: The charges levelled against my ignorant unsuspicion are three Three of the most derogatory and detrimental, not merely to the fame of a public, but to the character of a private man. In the one case, however, error is but human in the other, disgrace From the first, my advice and wishes as projector had been disregarded and overruled, and as an officer I had constantly been thwarted and obstructed in my operations. I do not shrink, therefore, from the avowal that, accustomed to such treatment, and aware of the incompetence and division of counsel existing on the board, I determine to do my best on my own responsibility

Speaker 2: To save the Enterprise from destruction. Like it was Star Trek, yeah I make no appeal at misericordium. I seek for no sympathy on scientific grounds. Sufficient for me that I have been identified And from the first, with that great prodigy of this age, which may become a new starting point in history until the end of time. A great responsibility rests upon those who have in any way contributed to the failure of this enterprise, but for my own part I can safely say that neither zeal, labour, caution, nor anxiety was wanting upon the part of Edward Orange Wildman Whitehouse.

Speaker 2: And then he signed it at the bottom, so you knew it was him. So it may surprise you to learn that publishing this defensive pamphlet did not really convince anyone that the failure of the telegraph cable was not somehow White House's fault. At any rate, it convinced lots of people that whether or not the cable had been ruined because of White House's failure to listen to anyone else, they sure as hell did not want to work with him on any other project. After this, he was unasked to be chief electrician of the Transatlantic Telegraph Cable Company, which was fine by him, 'cause he didn't want to work with them anyway And he folded his arms and he stormed off and went to his room or something.

Speaker 2: Unfortunately I've seen many projects, even in this day and age, end this way. And this is a real shame, because in eighteen fifty eight we knew that this was a really bad way to run a team. It was kind of obvious. There are a few things I want to highlight out of this telegraph cable affair that I think are really important. Firstly, open-mindedness among your team is crucial to success, but it's also crucial to good failure. Constructive reflection can't happen if there are members of your team who aren't open to being challenged or to having their minds changed, even by people who are considered junior to them. Secondly, if the project gets to the point that things are really bad, feedback should be given sensitively

Speaker 2: and with tact, and most importantly, privately. People are naturally going to get defensive if they feel like they're being attacked, and especially if they feel like they're having their value challenged in public. Thirdly, in order to be effective, retrospectives should generally follow the prime directive, which is here Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand. Having something like this on the wall, if you're starting a formal retrospective, creates a safe space for reflection Where people can openly discuss failures without fear of being blamed for them.

Speaker 2: They can discuss them as the failure itself. It's also a good reminder for everyone in the retro that blame is not appropriate. I often print it out and put it on the wall before I start with a retro. That's just me. I think it can help. Finally, there's no room for hero behavior in a strong team. The story of the first transatlantic telegraph cable might have been very different if the project didn't include Wildman Whitehouse. And while there were definitely problems with the way that the inquiry after the failure of the cable was handled, they may not have even needed one if they had appointed someone else to be chief electrician in the first place. One of the strongest findings of this inquiry was that Whitehouse 's behavior was a huge problem, most

Speaker 2: particularly his inability to listen to the advice of others, and his tendency to run off on his own and make crucial changes to the project. without telling anyone else, especially while it was live. And occasionally you will get people who want to be seen as heroes. This is because our culture often romanticizes the idea of someone who swoops in and saves the world all by themselves. And this is all really cool If you're Batman. But if you've ever watched a Batman movie, you'll see how many innocent people get their legs broken, or you know, they were in a smash building crushing their parked car when Batman's chasing after a bad guy. And the same thing happens with heroes on projects. They might get some personal glory when it's all said and done, but the team members who ended up with those broken legs and smashed cars

Speaker 2: are not going to remember that person fondly, and they probably won't want to work with them again. All in all, hero behavior makes for a terrible team environment. And we all know that unhappy teams don't do good work. And here is a quick recap of all of that in case you missed any of it. Firstly, open-mindedness is crucial. Secondly, treat feedback sensitively. Thirdly, remember the prime directive. And fourth, there's no room for heroes on a strong team. So With the moral of this story over, you might be wondering what happened to the telegraph cable after the dramatic pamphlet incident. And you'll be pleased to learn that the Atlantic Telegraph Company did learn something about teamwork from their first iteration.

Speaker 2: They appointed William Thompson as the next chief electrician. This was great because in addition to having far more solid ideas about physics than his predecessor, he was also delightful to work with, and he knew how to treat his team with respect. And at this point, the Western world's demand for a transatlantic telegraph cable was so large, and the Atlantic Telegraph Company had by now had so much experience laying telegraph cable across the sea that they didn't have much trouble getting more money to try again, which was good. And this time they built the cables the exact opposite of White Houses. They were large in diameter and not quite so heavy, so that instead they could be flexible and move with sea currents and things like that, and they used a comparatively tiny amount of voltage to power them.

Speaker 2: So they built these, they put them on the ships, they went out to sea, they started to lay the cable out, and the first time they went out, the cable snapped again. But, because they've been through this so many times by now, Thompson and his crew knew enough that they were able to retrieve the cable from the ocean floor and repair it. And then they also went back for a second cable, and by the time that that second cable had reached both coasts, they'd repaired the one the up the first one and hooked it up as well. So then there were two cables across the Atlantic. And this time they worked. And after they had tested it to make sure that it really worked, the whole Western world had another wild go-life party. And eventually they laid undersea telephone

Speaker 2: lines, even more eventually there were undersea fiber optic lines and video chats and cat gifts, and we all lived happily ever after in an interconnected world. The end