Tim Brown

In December 2020, major US information technology firm SolarWinds discovered they were the victim of a highly sophisticated state-sponsored cyberattack. Russia-backed actor Nobelium planted malicious code into their popular Orion Platform software used by thousands of SolarWinds’ clients. Through this method, the hacker group could gain access to computer systems of targeted Fortune 500 companies and US government departments, making it one of the worst state-sponsored cyberattacks of the decade. The cyberattack was so severe that the Biden administration announced sanctions against the Russian government for their involvement in the SolarWinds hack. 

To get better insight into the human impact of this historic cyberattack, we spoke with the SolarWinds Chief Information Security Officer (CISO). Our interviewee, Tim Brown, shares with us what it means to find yourself caught in one of the most prolific cyberattacks to date. 

(Edited for brevity and clarity) 

 

DPN: Tell us a bit about yourself. 

TB: I’m Tim Brown, the CISO for SolarWinds. I’ve been at SolarWinds for about five years. I’ve been involved in security for 20-25 years and held many different positions in the field. 

 

Walk us through the day that the cyberattack was uncovered.  

Pretty standard Saturday morning. I had breakfast and thought it was going to be a good day at home. Then I got a call from the SolarWinds executive team, and they told me they got a call from Fire Eye Mandiant saying that we had been breached somehow and shipped tainted code. 

So, Saturday was spent marshaling the right people and pulling in the right teams. We all had to get as much information as possible remotely. This was at the height of COVID in the US, so all of our offices were shut down. Then, we found out that our email system was tainted. We had to get off our email and find alternative mechanisms to communicate in case the adversaries were still in our midst. We were probably at it till… one or two in the morning. 

 

Describe the moment you discovered SolarWinds was targeted by a cyberattack.  

I asked myself, “Is this real?” Normally in these incidents, we have to do a lot of investigation to determine if it is real. In this case, it was pretty easy to figure out that it was. Once we saw the source code, there was a little bit of shock and a little bit of awe. When the shock passed, it was simply, “Let’s get the job done.” 

 

How about the following day? 

On Sunday, we got the team to move forward. We had to bring folks into the office, so we had about 20 or 30 people in the building physically. We occupied two different rooms to work through everything that needed to happen. We needed to find out what exactly happened. Where did they come in? Had they affected our builds? Was it in the source code?  

The news didn’t break until essentially Sunday midday. Around two on Monday morning, right before the stock market opened, we issued a financial statement saying what we had known at that time, which was actually quite a bit, such as we knew that it was an advanced attacker and the number of customers that had downloaded it. 

 

How did your coworkers react to the attack? What were the feelings and atmosphere in the office like? 

Everybody handles stress and pressure a little bit differently. We knew it was important to keep a level head. Create a model of how to move forward. There’s your set of tactical things that are just extremely important to get done, so we had very little time to think about ourselves.  

But you can’t underestimate the kind of stress at that moment in time. We’re working, one, two, three in the morning, and then getting up and doing it again at seven. If you go to sleep before two, you wake up an hour later thinking about what you missed, what you could do better, and what you don’t know. 

 

When the severity of this attack became clear, what was your biggest concern? 

My biggest concern was how our customers were affected. We decided at the beginning of all this, and what essentially held us together, was to put the customers first. Work with them first. Get them ready first. Get them moving forward first. Everything else comes second.  

At that point, we initially said 18,000 customers, but that was simply the high number of those that downloaded the product. We wish we had known then that under 100 clients were truly targeted. But, at that point in time, we just said, “Okay, let’s play it safe. Here’s the high number.”  

 

Describe the weeks that followed the attack. What was it like in the office? How were your coworkers handling this whole ordeal? 

We did take Christmas off, but that was it. We were always in the office. Everyone’s lives were kind of upheaved. Remember, this is right around Christmas, so all kinds of family time and plans got scrapped at that point.  

When you’re dealing with these types of incidents, you have got many lawyers in the room, many things getting reviewed. Every word you say will be recorded and used for or against you in the future. You have to expect that level of scrutiny during these periods.  

So, the whole kind of stress in the system is pretty incredible. It’s not just us—our customers were a part of this too. When the news broke, customers were trying to figure out if they were affected. Although our Christmas and New Year’s Eve were ruined, so were many of our customers going through this. Interestingly enough, these major attacks always seem to occur right before Christmas. 

 

Even though this cyberattack targeted SolarWinds, and not you personally, could you explain how this cyberattack impacted your personal life?  Or the personal lives of others involved?  

Whether you’re directly affected or not doesn’t matter, you know a lot of people were mad that it happened on their watch. It’s like someone breaking into your house and changing things without you knowing. It takes a big toll on them. How did this happen? How did they get in? Then, doing a lot of work to figure out the answers.  

It affected everybody personally, and it went on for months. I lost 25-30 pounds in about two weeks. We were eating, but the stress just wiped me out. Yet, there’s little sleep. I tried to drive home that Sunday night, but I gave up. I stayed next to the office for the next couple of weeks. It was just relentless around the holidays.  

Even when you’re home, you’re simply so exhausted you just sleep, only to wake up and get back to it. You go from meeting to meeting to meeting, and when you get home, you don’t want to talk anymore. So, that affects things. Luckily, I have a very understanding family, so that helped. 

 

Speaking of which, how did this impact your family and friends?  

My wife heavily supported me. My son, often in another state, was also very supportive. Going through something like that, my family understood that this was different.  

The outreach from colleagues, friends, and others was incredibly important and incredibly helpful. Many great notes from people helped me a great deal.  

I think that’s something everybody should learn. When somebody is going through something hard, a kind word can go a long way. 

 

After this experience, what keeps you up at night as a cybersecurity expert? 

It’s something like this happening again. This is the scale which every CISO dreads, right? Even though this was a worst-case scenario, you still worry how it could have been worse. In this case, the code was very mission-centric, written to target specific entities. The code didn’t just do random harm.   

The next nightmare is if it did more – designed to do random harm. So, it could have done serious damage and affected the world in a much greater way, as opposed to affecting under 100 customers. 

 

What would you tell people who are unaware of the threat of cyberattacks and cyberwarfare, or who might not take these threats seriously? 

One thing we’ve been trying to stress to everybody is it’s very real. This isn’t a movie. Very sophisticated, very patient, mission-centric adversaries are behind these attacks. If you become a target of an adversary at this level, they will take a long time to do what they need. And why would they rush?   

Let’s take ransomware, for example. The very methods utilized by the nation-states are now being utilized by ransomware writers. These writers are realizing if they can spend a year and make $50 million or spend a month and make $100,000, they are going to spend that year and make $50 million. We’ll see more of it because the payoff is larger. We have to expect this is the wave of the future. 

 

What do you think digital citizens can proactively do to improve their cybersecurity habits, mitigating their exposure to cyber risks? 

Understand what your data is. Understand what you’re sharing. Understand how you’re sharing it. Understand simple things that can keep you safe, like implementing MFA on… everything. Make sure that you’re harder to cyberattack than you were last year. Take personal responsibility for your security. 

 

Anything else you would like to add? 

It’s an experience you’ll never forget, and one that can help you grow, but it’s not something I wish on anybody. I just hope this situation allows for greater transparency. That it allows people to speak up about cyberattacks and put a little less blame on the victim. If our situation has helped with that, I think that’s good for everybody. 

 

Thank you so much for your time. 

 

Have you been personally affected by a cyberattack? If so, we want to hear from you. Let us know by DMing our Twitter account @DigitalPeaceNow.