When we think about improving efficiency in the SOC, we can often focus on tooling. Then why does it take so long to integrate new tools and get them up to speed?
Check out this post for the discussion that is the basis of our conversation on this week’s episode co-hosted by me, David Spark (@dspark), the producer of CISO Series, and Steve Zalewski. Joining me is our sponsored guest, Spencer Thompson, CEO, Prelude.
Got feedback? Join the conversation on LinkedIn.
Huge thanks to our sponsor, Prelude
Full Transcript
Intro
0:00.000
[David Spark] When we think about improving efficiency in the SOC, we often focus on tooling. Then why does it take so long to integrate new tools and get them up to speed? There’s got to be a better way. Welcome to Defense In Depth. My name is David Spark. I am the producer of the CISO series, and joining me for this very episode is Steve Zalewski.
Say hello to the audience, Steve.
[Steve Zalewski] Hello, audience.
[David Spark] I like how you and the other hosts just do what I say. It’s great. I love that. Our sponsor for today’s episode, responsible for bringing our awesome guests, is Prelude. They offer detection and response testing at machine speed, and we’re going to be talking about that speed in detection and response.
That’s going to be a big part of today’s conversation. It’s about working in the SOC. So let’s talk about that, Steve, because I want to set up this conversation. We all want our SOC to find and respond to threats faster. To do that, we’re often tempted to look at new tooling. Every vendor promises their solution will work great out of the box.
To really see efficiency gains for your SOC, they need time-intensive configuration. First, why does it take so long? And if a new threat is happening and we don’t have a detection for it, we’re essentially susceptible to the latest and greatest threats happening right now. I mean, is this the model that’s happening?
And if so, are we always in a state where we’re lacking readiness? What do you think, Steve?
[Steve Zalewski] So I say there are a couple of things going on here. One is the view of whether we are interested in finding what happened versus what’s happening. That’s the transition point we’re going to talk about. And the second thing is, if we’re interested in looking at what’s happening, then the needles we’re looking for in the speed of response and the integration points are also changing.
So I think we’ve got a couple of things going on here that we need to address.
[David Spark] Very good point. Well, the person to help us in that very discussion is our sponsor guest from Prelude. It’s none other than the CEO, Spencer Thompson. We asked for the guy who was working in the mailroom. They wouldn’t give him to us. They said, “You’re going to get the CEO and nobody else.” And that is Spencer Thompson.
Spencer, thank you so much for coming.
[Spencer Thompson] Thank you so much for having me.
Does anyone understand what’s going on?
2:33.802
[David Spark] Maxime Lamothe-Brassard of LimaCharlie said, “Get out of the habit of thinking only in acronyms. What you need isn’t SOAR. You need to automate the querying of additional context from some type of alerts in my EDR. Doing that allows you to scope your efforts not around a single product from a vendor, but around use cases.” And Erik Bloch of Atlassian said, “We want to buy a security outcome, not an acronym.” It’s interesting; both of them made this call on acronyms.
“Outcomes achieve goals you can measure. Acronyms get you a free dinner from a vendor when they come to town.” I like that line. So, there’s this desire in the industry to focus on categories. I need a SOAR product. I need an EDR product. I need whatever it is I need. But it’s really interesting. What you want is outcomes.
How do you have to think about your SOC differently if you’re operating like that, Steve?
[Steve Zalewski] So the first thing I’m going to pick up on is SOAR. SOAR was the holy grail, right? A couple of years ago, we had all the data we needed, and therefore we should be able to just automate outcomes and let the machine do its thing. And we’ve underdelivered on that. And I think part of what people are talking about here is we’ve underdelivered because we delivered a technical capability, but the people and process weren’t ready.
And so in revisiting that, what we’re starting to understand is what the outcome we want is and that we’re willing to let the machines do. And with that comfort, we have the ability to look at new technology and new integration points.
[Spencer Thompson] I think for me, the most important part of those quotes would be measuring the outcome. I think one of the great challenges, whether you’re talking about acronyms like SIEM and SOAR and everything else in between, is it’s really hard to define for SOCs what good looks like. And we have an industry where we basically outsource what good looks like to Gartner and Forrester and others.
That’s why we have magic quadrants because it’s actually very difficult to measure those things. So I think for a lot of organizations, sure, a SOAR is effectively a collection of if-then statements based on certain behaviors. But what are you actually trying to optimize for within your organization?
What actually matters most to you? There’s a lot of telemetry; it’s getting more complex. To Steve’s point, the theory for a long time was just throw more data at the problem. It’ll take care of itself because we can kind of route it into the right places. A lot of processes and tools are actually not ready for that.
So I very much agree.
[David Spark] So, okay, let’s go to the question you were asking earlier. What, exactly, are you measuring, or what does make an effective SOC? And if we don’t see it today, what is the idealized view of that, Spencer?
[Spencer Thompson] I think if you were to ask most organizations, the optimization might actually be different than we think it is. It might be just the reduction of false positives above all else. Like, if you think about most EDR products, XDR products, SIEM products, very often it’s the reduction of work for the manual kind of tier one, tier two, tier three SOC analysts.
And so, I think the big challenge is, assuming that is a solved problem, which it is not in today’s world, but assuming that is a solved problem, what is the actual goal? Is it full comprehensive detection across every single permutation of threat? That’s a lofty goal, but the world’s changing very quickly and AI-based attachments are making that kind of a must-have.
Is it routing to the right places at the right time? Is it responding by full prevention? I think there’s a whole list of questions you could actually ask within a SOC.
[Steve Zalewski] And I want to jump in here too, right, which is, we talked about efficiency, and then what we just did was we went to effectiveness. And I think we need to be very careful about understanding both of those are outcomes that we need to measure. But what is the metric? And to your point, being efficient is how we start looking at the problem.
How can I do more with less? But if we look at being effective, which is ultimately what we want to be measured on to stop the attack, the key is, how do we do less with less? Right. How are we pulling in just the right data and executing just in stopping the attacks, not with all the false positives and all the other things that are busy work that today we tend to want to measure as a way of demonstrating that there’s some ROI in our SOC.
What aspects haven’t been considered?
7:00.933
[David Spark] Mihir Mohanty said, “To make the SOC more effective, ah, you tipped us off. We’ve to start with the three core pillars of the SOC: people, process, and technology or tools. A major portion of our investment is in tools so far, and we have reached the point of diminishing returns. Any new investment in tools will move the needle marginally.
So it’s not time to focus on people and process.” Be intrigued on your takes on this, but let me also read Aqsa Taylor of Gutsy’s quote, who said, “You can have the best tools and people, but without a secure, consistent process that leaves no gaps or inefficiencies, it’s difficult to improve. But rarely do companies have visibility into the processes themselves.
There are tools to show you what’s wrong with your cloud environment, but you also need to see how your SOC team has worked to resolve it without leaving any open doors.” This is a really good point. I’m going to throw this to you, Spencer. Both Mihir and Aqsa make good points. I mean, how much can we change with people and process?
Because tools are critical to really understanding the environment. But as Aqsa said, if your process isn’t solid, you’re going to be failing all over the place. What say you?
[Spencer Thompson] Absolutely. One thing that I would start with is, and this is kind of a semi-obvious point, but I think it kind of dictates a lot of this, you know, do we collectively believe that there will be more or less data in the future? I think everybody would say more. And the rate of growth on that is not trivial.
And the point is, like, are we going to have this problem on steroids in the future? The answer is yes. And so which of those things actually scale? Can you scale a process? I think yes is the answer to that. Can you scale a bunch of other things? I think one of the great challenges in terms of process is that for the entirety of the SOC model, we have existed in a blacklist world, which is basically we’re looking for malicious behavior from the benign.
The way that I think about cybersecurity in general is how do you separate telemetry by malicious versus benign behavior? If we all lived in this perfect world, we could just block everything from happening on a computer, and we would have no cybersecurity problems except you couldn’t use anything. So, we have this kind of process of separating these things out.
One of the great changes that may occur—I’m not sure if we’re actually going to get there, this has been talked about in the past—is do we move towards a whitelist model? If there’s enough malicious behavior that starts compounding, do we actually have to block entire segments and then allow things to happen?
For example, do we basically block all spawn processes coming out of Microsoft Word or Adobe, right? But Microsoft has released attack surface reduction rules for this exact reason, and then you’re allowed to add exceptions or exclusions to that. That might be a model where you basically augment your tools with a process to actually minimize noise.
You’re basically setting a filter before it ever actually happens, versus this continuous needle in a haystack problem.
[David Spark] Steve
[Steve Zalewski] So, I want to kind of pick up on something Spencer said, which was about malicious versus benign. Okay. And we think that’s the core here. And another way of saying that is, am I looking at vulnerabilities, or can I focus on exploitabilities? Because a lot of vulnerabilities are benign; the exploitabilities are the execution of malicious intent.
So, if you’re willing to look at that and agree, white-listing and black-listing, another way of saying it is if I’m focusing on exploitability, then I can ignore the benign. Okay, dwell time doesn’t become interesting anymore. It’s about when the exploit is underway. How do I identify the breadcrumbs of an actual exploit?
And then I can do something about it because now I see what’s happening, not just try to monitor what happened. And now there is a need for some new technology because we’re looking at exploitability. We’re looking at a different way of monetizing the value of the SOC and that technology to get a contextualized view of what’s happening.
We need help there, but the people and the process now are adapting to where we have to go, and the technology is now a needed enabler. Spence, what do you think?
[Spencer Thompson] I think that’s perfectly put. The other way that I would potentially say the same thing would be, all the steps you mentioned around true exploitability maybe sums up into actual behavioral-based detection. Right. We’ve had this conversation for a long time. I think if you ask a lot of people, and some of our questions kind of hint at this, a lot of the detections that SOCs work on are very specific.
They’re very what we call brittle internally. They’re almost signature-based. And so the challenge is they’re looking for point-in-time things with certain flavors. Right? So if you’re doing step one, then step two, then step three, then step four, that might be treated as a very specific pattern. To your point, all of a sudden, that’s treated as a signature, and you go, “Oh, okay, we solved the problem you’re just talking about.” What if you do step one, then two, then three A, then four B, with the same outcome and the same goals, is that detection actually going to work?
And to your point, exploitability means all the permutations of steps three and four in that example are covered by something, and that starts to also narrow the scope from a process standpoint so that people can actually focus on the right things. Right now, I think we’re just kind of stuck in this very precision-based world.
[Steve Zalewski] What you just said, when you move to behavioral, what you’ve acknowledged is that historically we looked at our network edge and our data centers as our primary edge, and we mined them to identify attacks at the network level. We know the limits of that approach with logs, right? And if you’re into the cloud, what you have is a data edge.
Your service edge is all about data, and it’s hard to determine classification for data. So, what we realize is both of those perspectives look at identity. It’s identity and access management that becomes the behavioral component of understanding. Each thing has an identity. The contextualization of that allows us to determine deviation, whether accidental or malicious.
But then we can execute on it because I know what to do to stop an identity, whereas I don’t necessarily know what to do with a bad network packet or a misclassified piece of data. And so, I think that’s really what you’re hammering on, and I just wanted to bring that to a salient point.
[Spencer Thompson] I think that’s perfectly put. And it also hits on another point that we may get to later as well, which is cloud environments, in particular, usually have a reduction in surface area in terms of the size of the operating system versus a Windows kernel, where you have all the telemetry and all the pathways and all the different things you talked about.
There is a reason why we’re seeing cloud detection response tools start to come out. It’s a much more focused set of telemetry that might end up in the future of the SOC.
Sponsor – Prelude
13:52.997
[David Spark] If you’ve ever worked in security operations, security engineering, as a CISO, or a security liaison for a boardroom, you’re probably listening to the show. Let me start with that. But you’ve probably also been asked some form of the question, “Are we protected?” How can you know for sure unless you’re testing your defenses and at scale in production?
And how do you quickly fix your defenses when they’re not working as expected? If you’ve ever taken a stab at this process, you know it’s inefficient at best and near impossible at worst. Enter Prelude Detect. It is the world’s only production-scale detection and response testing platform. Automatically transform your threat intelligence into validated detections and preventions in less than five minutes.
You can integrate Detect with CrowdStrike, Microsoft Defender, SentinelOne, and more to enable machine speed detection and response engineering. That’s what we’re going for. You can learn more about just that over at Prelude Security.
What should we be measuring?
15:10.364
[David Spark] Yaron Levi, who’s the CISO over at Dolby, said, “We have to fundamentally change how we operate in the SOC. The SOC is based on a faulty concept of collecting more information and feeding the garbage factory, AKA SIEM.” Again, his comment, not mine. He goes on to say, “All the tools that were added over the years continue to fall into the same trap and feed the garbage factory.
We’re not lacking data or visibility. What we are lacking is proper context to use that data effectively. Instead, start with a scenario and use the data to either support that context or better refute it.” And Joshua Boyce of Cisco said, “We need much stronger correlation tooling. We focus too much on singular efforts, and we need to be faster and more accurate around the broader picture of chained, sequential events.” So, Spencer, this is kind of the point where I kind of want you to tell the Prelude story of what exactly you’re doing to speed and address this very issue.
Really, I think what Yaron is also getting to is that the garbage or the lack of knowing or context or the speed of responding, for that matter. What is Prelude doing here?
[Spencer Thompson] Sure. I mean, I think both of those comments are actually hitting at the same core point. And the way that I would describe this for us would be, if you take a step back and think about a modern detection response process within the SOC, and this is assuming that you’re not in some kind of instant response process, this is pre-instant response, your day-to-day kind of behavior, there tend to be three core buckets.
The first bucket is, how do you process threat intelligence? And so you maybe have a compliance requirement from CISA putting out advisories. Maybe you pay for premium threat intelligence from CrowdStrike or Mandiant or something else. But the first question is always, how do we do something with that information?
How do we read that, process that, turn that into action? The second is, how do we actually process that and turn that into a detection? That detection usually is the form, like we talked about in the previous segment, a brittle, precise detection, very focused on that specific threat that typically takes weeks, sometimes months, to produce.
There’s a tuning exercise there, there’s a creation exercise there. And the last step is, how do I validate that detection? I might have an offensive team, a pen-testing team, a red team, a purple team, but how do I validate that my defenses are actually working once that detection is installed? So if you take those three different steps, there are people associated with each of those steps.
There’s a bunch of manual work. And so what Prelude does is actually take a piece of threat intelligence, transform it into a detection that’s ready to be uploaded, uploaded into your defense instantly. It’s compatible with all sorts of different XDR products, and ultimately validates by testing that thing all within a couple of minutes.
And so the promise is that what typically takes you a month or a month and a half, and that’s kind of the speed that you’re moving at in today’s world, can happen in a few minutes, maybe an hour. And so the velocity increases in terms of the ability to actually go from asking a question of “Are we protected?” to an answer.
The amount of throughput increases because now I can produce a thousand detection sets instead of two a quarter, and ultimately the time it takes to do that actual process goes way down.
[David Spark] What Spencer is describing sounds like it should be the norm. When the threat is known, we should have a detection and response for it as quickly as possible. And it kind of is the philosophy of security professionals in general: new threat, know about it, be on top of it, right? Steve,
[Steve Zalewski] I love what Yaron Levi said, which is the “garbage factory,” and I’m going to hold him accountable for that next time I see him because I’m going to go, “I’m going to,” but what the garbage factory is, and I’m going to answer your question here, is an acknowledgment that the ROI on the SOC and the SIEM, after 15 years of investment, is the worst ROI of everything I have in security.
Why?
[David Spark] That’s actually a good point. And that’s why they keep saying more and more data because the answer’s in there somewhere. But the amount you have to ingest to find the quote “somewhere” answer is, yes, you’re spending a lot of money to get a little output.
[Steve Zalewski] The law put him, and I go, “Why?” That’s because we’re looking at vulnerabilities. That’s what we’re trying to do: prevent. And identify when a prevention failed. What we’re doing is moving from vulnerability now to what we just said, exploitability. So for 15 years, we’ve tried to solve the vulnerability management problem.
We’ve realized we can’t. We’re moving to exploitability, which is what’s happening now around behavior and identity. And so what you’ve seen in the last 18 months is the brutal truth of SOCs and SIEMs don’t work, and they’re too damn expensive, to understanding what the evolution wants to look like now, that resiliency and our ability to adopt a detect-contain methodology is one that we now realize is the way we have to move because we can’t keep doing what we’re doing.
It doesn’t work. And so this conversation today really is kind of a reimagining of the garbage factory to move from vulnerability to exploitability, to what happened, to what’s happening. So yes, David, it makes perfect sense, right? When you look at it that way, but like a child, we kind of had to go through right and learn how to walk and learn how to talk.
And now that we can do that, we can do it both more efficiently, but also more effectively communicate what we need to do.
[David Spark] I’m going to let Spencer answer this. ‘Cause this is in his wheelhouse. What say you, on Steve’s philosophy here?
[Spencer Thompson] I don’t think anything’s in my wheelhouse, but I’m happy to take a step. I agree so much, but we have an internal concept, which is, do we believe that patching will eventually be replaced by policies? Right, for a long time, going back to Steve’s core point, the answer to every vulnerability was, we’ll just patch it.
And don’t worry, auto-patching will happen. And so we don’t have a patching problem in the future, that will be solved, and it turns out life’s more complicated because you can’t take down systems and not everything can be patched, and there’s end of life, and that so we install controls. I think people sometimes lose sight of the reason why defenses exist in the first place.
And so you install controls to basically get at exactly what Steve’s mentioning, which is exploitability. You’re stopping behavior from happening, not just a single hash that’s actually just executing a piece of malware. And the challenge with that is we don’t always know if those things work. But the other challenge is we have a SIEM model and a SOC model designed for the former, to Steve’s point, and not for the latter.
And it’s actually very difficult to look at that behavior. And so it’s almost, it involves a retraining process, involves actually getting data out of the EDR or the control, which is why they’re all becoming XDRs. It’s why they’re all vertically integrating SIEMs into their products. If you look at why, you know, CrowdStrike is buying Humio, which is now called LogScale, it’s because their next-gen SIEM is actually just their own response and detection data sitting inside of the same platform.
So I actually think what’s happening in real time is what Steve’s saying. We’re just catching up to it.
What’s the next step?
22:24.639
[David Spark] David Ratner of HYAS said, “Many of the tools are aimed at answering the question, ‘What happened?’ Ah, you talked about happened and happening right at the beginning, Steve, it’s coming around again. David Ratner goes on to say, “While that is often important and useful, if you want to make your SOC more efficient and faster, that information needs to be able to bridge the gap and go from what happened to what is going to happen next.
The threat intelligence and other tools at their disposal need to allow the SOC to get proactive in adapting defenses and getting prepared against the nature of the threats and risks that they are actively facing versus always looking in the rearview mirror.” This seems like the philosophy of this entire episode.
Spencer, you’re nodding your head. Yes,
[Spencer Thompson] I am. I think it’s kind of the summary of a lot of what we talked about. I think one of the challenges is, you know, here it basically gets at, well, what are the preconditions that make that true? I think if you go all the way back to the beginning of this conversation, you say, okay, well, do we have a tool problem, a people problem, a process problem?
I think one of the challenges is like, unless you can actually commoditize and have equal access to threat intelligence and generating detections and protections, how are you supposed to shift to a proactive model? Because we’re still kind of struggling with the base layer right now, but that vision of the future, and that is a North star, I think is very central to how we think about the world.
[David Spark] Steve, wrap this sucker up here.
[Steve Zalewski] What we just said was the brutal truth is SOCs and SIEMs don’t work, and now we know why. But do we have it in us now to face that truth and do what David has said here, is move forward? The answer is we will. The question is, how long will it take us?
Closing
24:21.638
[David Spark] That is a good point. And that brings us to our next portion of the show, which is, which quote was your favorite and why? And let me start with you, Spencer, on this one. Taking a look at all these really good quotes, by the way, a good bevy of them. Which quote was your favorite and why?
[Spencer Thompson] I’m a little biased because it’s in the language that we use, but David’s quote around proactive security kind of holds very near and dear to our hearts at the company. And so that is, again, like a North star for the industry, very deeply resonates.
[David Spark] That’s David Ratner with HIAS. All right. Steve Zalewski, your favorite quote and why?
[Steve Zalewski] I like David’s. I also like Joshua Boyce, but I’m going with Yaron Levy, the CISO of Dolby. Which is, we have a garbage factory, so we’ve called the baby ugly without identifying the parents. And now that that’s true, we’ve talked about what we can do to fix the baby. So let’s go do it, and we now have a path forward to accomplish it.
[David Spark] Excellent. Well, that now brings us to the very end of the show. And I want to thank you, Spencer, of which I’ll let you have the very last word here. So hang tight. I want to first thank your company Prelude. Uh, remember their web address is PreludeSecurity.com, detection and response testing at machine speed.
So speed up the process. What you’re doing is essentially what you’re doing right now. But actually, it’s the speed you want to be, so you can actually be effective at what you’re trying to do. And eventually, the philosophy that Mr. David Ratner was also saying as well. Steve, any last thoughts before we take to our guest?
[Steve Zalewski] I want to thank the audience. There were some great quotes to pick from. And the fact that we had this quality of the quotes and the conversation, I think really shows us that we as an industry are moving in the right direction on this key topic.
[David Spark] Very good. Thank you, audience.
[Spencer Thompson] First thing is just to say thank you. The engagement via the quality of questions is kind of astounding, as Steve was mentioning. So we really appreciate it. We are hiring. We’re always hiring, but in particular, hiring security engineers, which everybody’s hiring security engineers.
But in particular, for us, we look at people that are former detection response engineers, so detection engineers, threat intelligence analysts, etc., and we really appreciate the opportunity to be here.
[David Spark] Awesome. Well, thank you very much. Thank you, audience. And thank you, Spencer, Prelude, Steve. We greatly appreciate your contributions and for listening to Defense in Depth.
[Voiceover] We’ve reached the end of Defense in Depth. Make sure to subscribe so you don’t miss yet another hot topic in cybersecurity. This show thrives on your contributions. Please write a review, leave a comment on LinkedIn or on our site CISOseries.com where you’ll also see plenty of ways to participate, including recording a question or a comment for the show.
If you’re interested in sponsoring the podcast, contact David Spark directly at [email protected]. Thank you for listening to Defense in Depth.
This post was originally published on 3rd party site mentioned in the title of this site
