Dec 6, 2022

Measuring Security Debt With Garrett Smiley

TLDR;

Priority of security debt is always context driven. Businesses should reevaluate the risks depending on liability or exploitability of it and how it applies to the business.
Building a security centric culture always depends on two aspects. First is the messaging from the leadership of the execs. And the other one is how motivated the team is, how motivated the team is in general.
KPIs may not be the best metric to determine success of security team members. The more realistic metric could be Kri, where R stands for risk.

Transcript

Host: Hi, everyone. Thanks for tuning into our Scale to Zero show. I’m Purusottam, Co-founder and CTO of Cloudanix. Scale to Zero is a forum where we collect questions from curious security professionals and invite security experts to learn about their journey and also to get these questions answered. Our goal is to build a community where we learn about security together and leave no security questions unanswered. With that, let’s get started into today’s episode. For today, we have Garrett Smiley. Garrett is the CISO at Serco, where he oversees governance, risk management and compliance and overall cybersecurity efforts. He built the risk management program from ground up.

Prior to Serco, he was the CSO at General Dynamics Information Technology, and during his career, he has worked with many large security teams to design, implement, and scale security programs. Garrett, thank you so much for joining me in the show.

Garrett: Thanks for having me.

Host: Lovely.

So the way we do the episode is we have two sections. First section focused on security questions. And the second, the first one is the rapid fires section.

So let’s start with the security questions section. And I want to start with risk management in general. Most organizations follow a risk matrix where they categorize their risk by critical high, medium low into different categories, right? And because of business needs, sometimes the critical and highs are addressed, but medium minor lows are not addressed. And we received this question from a fintech growing, fast growing fintech startup. They want to understand, like, the items which get into the security debt.

How should they address the security debt?

So what are your thoughts on security debt?

How do you define them? And how do you measure security debt as well?

And once you have the security debt lined up, how do you prioritize them?

Garrett: Well, I guess first you would want to define what’s meant by security debt, right? I think most of us are familiar with the concept of technical debt which can be related. But in my mind, security debt are risks that you’ve taken on with some understanding of the risk that ideally you wouldn’t take on.

A good example would be like application whitelisting or two factor authentication. There’s a difference between what we deployed it versus what we fully implemented it or we fully implemented it with no exceptions. That would be kind of how I would define security debt. So if you’re like, well, we deployed it everywhere, but we haven’t implemented it over here because of tensions or constraints related to that particular business focus or whatever the reason or potential excuse may be. So that’s how I would define security debt. As far as tackling that and prioritizing that, you have to have context with how it impacts the business so you can say something is and generally the context when we say criticals and highs and don’t pay as much attention to the mediums and lows, that tends to be more of kind of a vulnerability scanning thing. But when you’re talking about risks for the organization, one you want to say how did you arrive at determining that that thing, whatever it is, is critical versus high? It doesn’t even matter.

I mean, is it relevant? It really comes back down to how does carrying this security debt present liability for the organization and how likely is it that liability will be realized?

How likely is this area to be exploited? So for example, if I have something that’s air gapped and sit in the middle of the desert, physical security is a lot more important there than it would be other places. Conversely, if somebody says, well, do you want to place a prioritization value on getting two factor put in place ahead of something else? I might say, well, maybe that edges out application whitelisting or maybe we’re talking about the value of getting EDR in place. If you don’t have MDR, for example, it all has to do with the type of organization you work for, the attack surfaces that are in place. So it’s very contextual and I’m probably not really giving a very specific answer, but that is the answer. You have to work with the leadership and the movers and shakers in your organization to help them better understand the risk that’s posed by security debt and then act on it and road map it and put the prioritizations in place to where you’re going to tackle this than that. Because as we all know, resources are not unbounded and that tends to be the biggest thing, right? It’s not just, well, hey, here’s a blank check never seen that doesn’t exist, right? And even if you do have money to do things, it still requires people.

Very often most of our people are already at 3% to 400%. And it’s just reality, right? Something’s got to give. It has to be done in partnership. I guess it’s the short answer, right?

Host: So I like two things that you mentioned. One is the context, right? It’s always about context, as you mentioned. If it is an air gap deployment, then you have a different set of priorities versus you have a web app facing end users, right? So that’s one. And then working in partnership with other teams. And that’s a good segue to my next question.

So often security teams work with other teams to get things implemented, let’s say, right? And often when it comes to business growth, security is seen as a roadblock.

So how can security teams work with other business units in the organization so that they help in increasing the revenue or improving the bottom line?

Garrett: Well, I would say it’s probably worse than that, if I’m honest. And what I mean is people say, oh well, security is a roadblock. Well, if nothing’s going through security, and security is not being used as a gate, we’re not a roadblock. We’re just ignored, we’re marginalized, we’re just pushed off to the side. So it is generally worse than that.

But to answer your question, security understands that we absolutely have to work with others because our role is risk advisory. If we’re completely blunt about it, at the end of the day, we are not administratively in charge of the systems that need to be configured in a more hardened way, or need to have agents put on them or whatever it may be that’s it operations or that’s cloud ops or whoever it is.

So of course we have to work with them because of, and appropriately so, separation of duties. We should not be in charge of the estate that we’re trying to provide oversight and governance for, which just doesn’t make any sense. So yeah, we’re not in that position and I’m not arguing that we should be. So without that partnership and if you don’t have a good working relationship with these groups, quite frankly, they’re not going to give you the time of day. They’re just not. There has to be a motivating factor on the part of those that you’re working with. And for me, my staff, I’ve been kind of focusing my examples on things that are a little bit more technically operationally focused, but I have to do the same thing with other departments and DU’s, when the topic is not a technical topic, particularly.

Maybe something that’s trying to get them to operationalize bake into their muscle memory and their daily tasks in procurement or in HR to do a thing more securely or with less risk, whatever that may be, whether it’s vetting a supplier or onboarding a new employee. But it still requires those relationships that have to be established and maintained through Rapport. And you have to care about what they care about. In other words, what their pain points are is what they care about and what their pain points aren’t. They don’t care. It’s just human nature. So if you’re like, I don’t want to take the time to learn that, oh, you don’t want to have rapport, you don’t want to get anything done.

That’s a great way to go about it. So that’s kind of what’s critical.

We have to be in the business of Rapport building and relationship establishment and maintenance. But I will say that that will only get you so far if your tone from the top doesn’t show any value or any deference to being secure and compliant. So both really have to be in place. And then you would say, well, it sounds like you need to do more report building efforts and more for maintaining efforts with senior executives and the board. And it’s an ongoing thing. Right?

Host: Right. Yeah. So there are two things, again, that you highlighted. One is the motivation of the teams to work with the security team and even the security team’s motivation to work with the other teams and messaging from the top how much priority or how much focus they pay to security. So this touches a little bit on the culture of the company. Right. So often we have seen that culture is what drives organizations and teams to work together. And every company has a unique culture. It can be engineering, sales or security driven

Garrett: Well, we do what we can. That is an ongoing challenge, I think, if I’m honest about it, where we have little excuse is definitely within our own team. So I like to start there because a lot of times people understandably, but I think incorrectly think of culture in an organization as a monolith. And that is just simply not true. It’s not true. There may be a pervasive culture that you find in a lot of places, but the culture within my Infosec team is extremely healthy. Why? Well, we’re very deliberate about who we hire, when we hire.

We are just as concerned about capability to learn things, problem solving, things of that nature as we are the fit in how this individual is going to be able to traverse dealing with difficult people. Let’s be honest with it. Sometimes we’re the difficult people, I understand that. But a lot of times others are the difficult people and they’re like, yeah, we’re just not doing that. It’s not a priority for us. We don’t care about it and we know that that’s wrongheaded. But you have to deal with that.

It’s like working in a hospital and you say, hey nurse, every patient you have is going to be awesome. And they’re not going to throw their lunch tray at you and you’re never going to have any problem. No, that’s ridiculous. Right? You say that way you’re like, no, you’re going to have some awful patients. Absolutely awful patients. You’re going to have to learn how to deal with that. It’s just part of it.

So yeah, it goes back to that rapport building and making sure that you’re continuously doing this and explaining your motivations helps. And I have found that with some of the most difficult customers I have had, explain to them, I’ll paraphrase, hey, my goal here is not to be a jerk. That may be a byproduct of this and they’ll laugh at me, but that’s not my goal. My goal is to help you do this thing you’re doing in a more secure and compliant way so that we can continue to be a business and all continue to get paychecks as well as not open ourselves up for liability and also in the business. That’s it. Period. It’s like, I’m not trying to get in your way of you doing what you need to do and you executing your mission.

I’m simply trying to get you to do your mission in a more secure and compliant way, not because I’m bored, because there’s a real consequence to not doing it that way.

Host: Yes, that makes a lot of sense. You touched upon one area around hiring. I want to double click on that a little bit. So there are several teams even in security, right? And say you’re a company of 50 people and you’re growing and you’re either in fintech or healthcare where you deal with a lot of PII phi data.

So in that case, how do you set up your security organization? What comes first? Who do you hire and how do you hire? Do you have any recommendations on that?

Garrett: Well, I’ll answer the question broadly. I think you’re correct when you say that the vertical that you’re in is probably going to have some influence there. And it would.

But I would also say that your organizational structure, how big your budget is, and how much you have to rely on external outsourced, third parties to kind of give that full staff augment picture, all that’s in place. But generically speaking, if you’re starting from scratch and you’re like, where do we start? You have to be able to react. So when stuff goes sideways, I need to have people that are kind of doing that sock analyst, incident response, those types of roles, because if I don’t have that in place when something’s going bad, oh yeah, great. Somebody’s exfiltrating all our data. Are we even in a position to respond to that? No, we’re not. Okay, well, we’re totally screwed.

Great. Awesome. Thanks. You’ve got to be able to react first. And so that was one of the first programs that I stood up and put shape to when I came here to Circle five years ago. Because it was the first and most important thing that needed to be done, and there were some really good elements that were already in place, but it really needed to be kind of better codified and pulled together, and everyone needed to have a better understanding of okay. When an incident occurs, we all understand our role and what swim lane we have in the predefined workflow. It’s like foreign policy.

You better know what you’re going to do before stuff goes down because if you don’t, it’s going to go really bad. And establishing that muscle memory around, no, no, no, that’s not your role. You need to do this. You need to inform this person and we need to put structure to it so that we just don’t even think about it. When an incident goes down, we tackle it. So that was the first thing and then coming after that it’s like, well, we need to get a handle on our attack surface of vulnerability management. Standing up a program there so I could go on and on, but I think you get the idea there that you’ve got to get your hands around the throat of it, so to speak.

I know it’s somewhat violent imagery, but I don’t mean it violently. I’m just saying you got to get a handle on it, right? So incident response is where you start and then you start taking a look at vulnerability management. And vulnerability management is going to point out the things like, hey, is our asset governance good or does it totally suck? By the way, most organizations totally sucks. Okay, well, we need to really shine a flashlight on the cockroaches there and say, we’ve got to do better because infosec cannot help you secure what you aren’t even aware of that you have. We aren’t aware of what you have. So it goes back to that continuous monitoring capability that you’re striving to get to if you don’t already have that in place. So anyway, I could go on, but I think you get the idea that’s kind of how I approach it just from, okay, we’re on fire, the buildings on fire. What do I need to do first? Well, first you need to get everybody on the building. Then you need to call fire department. Then you need to have them hose the building down or whatever they’re using titan right? You’re like, okay, well we got to start with safety and then we have to start triaging and getting things under control and then we can put together a plan of, well, what started the fire and so on and so forth.

Host: Okay, that makes sense. One question related to that is let’s say once you hire security engineers, generally every team does goal setting activity, right?

So how do you define success for security team members?

And what metrics are KPIs do you use for that? Let’s say you are planning for three months or six months or twelve months or 24 months plan?

Garrett: Well, that’s a good question. I would say that I’m stammering here a little bit. I’m not super wild about KPIs, just full disclosure. The reason I’m not super wild about them is because they are outside of something that’s contractually bound, which really kind of forces some honesty on this control costs x and requires y people. Doing KPIs to me is a bit of a waste of time. Because if you’re like, well, if I put in a request where I need ten people to COVID this area and I only have three, I think you understand what I’m saying. There things where you can honestly bound against performance for an individual.

I think that’s fine. Generally we tend to do those things along the amount of time it takes to address tickets and things of that nature. It can be challenging if we’re honest. Right. So do I have KPIs in place for where I’m leveraging third parties, where we’ve really bounded? It contractually, yeah. I don’t tend to do KPIs for internal people because if they’re shouldering a double workload, it doesn’t make sense. Right. The data I get back from that makes it look as if they’re not doing their job when literally they’re doing the job of two or three people. So yeah, I’m resistant to that.

One thing we do a lot more of is KRIs, which is not the same thing for performance and the are for risk. But we do KRIs because what you measure, you pay attention to, and what you pay attention to in theory has the tendency of improving. And there I’ve actually seen a lot of improvement because senior executives don’t like bad reports and it can be motivating. And I’m like, hey, why spend all your time trying to hide this? Why don’t we spend our energy on trying to fix it? Especially if it’s going to be about the same amount of effort, the value of KPIs? But for me, if you know that your staff augment is not where it needs to be, you get the data and you come back with it. So what does that tell me? It tells me that I wasted time doing this when I already know what the problem is. You need more people if you want to hit all tickets are closed in 5 seconds measurement. I’m like, yeah, we can do that, but I don’t know that you’re ready for the build. Right? Yeah.

Host: So I like the KRIsi like key risk indicator.

Right. In a way, you’re trying to find out how the team is performing when it comes to addressing risks. Maybe that is more accurate to measure security teams performance as I’m using.

Garrett: The interesting thing about KRIs is what it shows you is it shows you where risk is, shall we say, accumulating, for lack of a better word, across the entire organization. Now, in some instances, it may be the ITOps folks that are not keeping up with the Joneses.

In some instances it may be infosec, but you know, we have careis against phishing simulation so that’s the end users, right? So if they really suck wind on one campaign and they do better in one metric over here, but not so good on this metric, the information is varied. Really what it does is it shows you areas where you probably should focus and see if you can change the behaviors. Sometimes it’s a lot easier to change the behaviors because it’s just a matter of taking too long or you’re not following through on this process in the way that you should in things like campaigns. It’s a little bit more nuanced because they are the main reason why they’re clicking on links and entering credentials is because they’re going too fast and they’re not paying attention trying to fix that behavior is a lot harder as I think those security specials right

Host: Than following a specific indicator. Yeah, that makes a lot of sense. Yeah. So there are many things I learned, especially the Kri, that’s something I had never thought about, that can be an approach to assessing team members like overall success rate. Right. So, yeah, thank you so much for sharing these amazing insights. It was very helpful.

Rapid Fire:

Host: So now let’s move on to the Rapid fire section. So the first question is what advice would you give to your 25-year-old self starting in security and why?

Garrett: Well, I guess a couple of things there, right? Am I 25 now or am I 25 then? I guess we’ll just go with 25 then. If I’m 25 then and I’m telling myself that first I would say, hey, information security is going to be its own thing because 25 years ago that wasn’t the case. It was just it and security was not understood independent from that in any way, shape or form. It just wasn’t a thing. So yeah, that’s definitely what I would tell my 25 year old self, that hey, you’re going to pursue a career in this at some point. And at some point information security is going to become its own discipline and prepare for that because it’s interesting and it’s something that you’ll enjoy. So that’s what I would tell me at 25 or 25 years ago, whatever it is.

Host: I like that so the next question is, if you were a superhero of cyber security, which power would you choose to have in you?

Garrett: I will sell myself out here and say mind control. And the mind control powers I would use would be to make everybody understand what I actually do and have them be mentally encouraged to help me with that, because security is everybody’s job. I know that sounds like such a tired trope, but it’s so true, because you’re not sure if your people are not performing securely, you’re not secure if you don’t know what technology you have in your environment, it’s everything, right? So, yeah, I would be Professor X or whatever, and I just put on a helmet and I’m like, OK, guys, let’s straighten up our thinking here, because we’re doing things that are very risky, so that’s what I do. So bringing awareness around security so that people pay more attention to it. Yeah, correcting the thought processes there, because that’s kind of where it tends to derail, or at least where it starts to derail.

Host: Yeah, makes sense. The next one is, what’s the biggest lie you have heard in cyber security?

Garrett: Not so much in cybersecurity, but about cybersecurity. I would say when people come up to me, or people that work on my team and go, you guys make everything secure for us, I’m like, no, try again. I advise you on how you should be doing it. I do not have that locus of control generally. My joking response, although it’s not really a joke, is awesome. I just became the boss of everybody and all decisions go through me. Fantastic. And they’re like, that’s not what we meant. I was like, no, that’s what you meant, you just didn’t know it.

I’m here to identify and advise and consult on risk. That is why I’m here. And if you don’t understand that, I go back to my Professor X comment earlier and you’d be surprised how many people truly do not understand that. And it is a constant evangelization process that I undertake each and every day. But, yeah, I’m an advisor, I’m here to help. I go to my LinkedIn titles, where the first one is the fake titles that I like them Rich Consiglierity. This is like the Godfather lawyer.

A man don’t go to mattresses, don’t go to war. You have the decision. You will decide to go to war or not. I’m just advising you not to because the consequences are pretty bad.

Host: Yeah, I love that. Most of the time there is that disconnect between the team, so, yeah, thank you so much for adding that and that’s a great way to end the episode as well. So thank you so much, Garrett. It was very insightful discussion. Looking forward to learn more from you in the future.

Garrett: Thanks for having me, I appreciate it.

Host: And to our viewers, thanks for watching. Hope you have learned something new. If you have any questions around security. Share those at scaletozero.com. We’ll get those answered by an expert in the security space. See you in the next episode. Thank you.