I was Wrong about Endpoint Security

Years ago, when I was working in cloud security, I had a customer tell me something surprising. He said his mission that year was to kill off every last virtual machine in their cloud accounts. There were several reasons for this.

Death to Virtual Machines

First, the customer's company was cloud native and moved quickly. They had autoscaling groups with strong bootstrap and automated launch profiles, and they needed their compute capacity available more quickly than was possible with virtual machines. 

To top this off, their virtual machines were only in use for around 2.5 days maximum, which meant they had to continuously build and maintain these machines. This is a multi-step process, starting with selecting the right base OS, then installing required software, patching OS packages, running other hardening tasks, writing and maintaining all install scripts necessary, and finally, baking in an agent.

The company could no longer justify the overhead of this complex process, especially considering available alternatives such as containers, Kubernetes, and serverless function platforms. And this customer wasn’t alone in their thinking about virtual machines, either. 

My friend and fellow podcast host Ashish Rajan on LinkedIn brought up the issue again in a recent post:

The logical outcome of the transformation was a move towards microservices and data and application interactions over APIs. This let us know that APIs would be a high priority target for bad actors moving forward, and his thinking was what actually led us to starting FireTail, an end-to-end API security platform. Now, in 2024, we can see that we probably underestimated the scale of the problem.

EDR to XDR

Based on this trend in changing compute architectures, I expected that the Endpoint Detection and Response (EDR) companies were likely to shrink their overall install base, but instead, EDR has evolved into XDR or Extended Detection and Response (sidenote: it was hard to find an “EDR to XDR” reference not from a cyber tool vendor blog or marketing post). This evolution of EDR to XDR is another example of breaking down data silos in cybersecurity, or “platformization.” 

In addition to this, the virtual machine market grew more than I expected, likely fueled by a massive shift to the cloud from the pandemic.

Starting off, I also underestimated the increased complexity of cyber attacks, and how the correlation of data signals from multiple platforms would be increasingly useful going forward in cyber defense strategies. For more on this, see my last post on the consolidation of cloud security.

I discussed the trends with a former CISO friend of mine, and asked what use an EDR tool would be in these complex attacks. He pointed out that most cloud platforms still don’t see inside the operating system natively, at least not in real-time. 

For the most part, the operating system is still the best detection point for the active, in-progress exploitation of vulnerabilities, or for seeing the early-stage execution of malware. From the OS, one can observe what gets loaded into memory, software package execution, and other activity.

EDR platforms also have evolved to support container and serverless platforms with a much lighter footprint that focuses on the visible elements, while omitting the underlying operating system noise.

With the help of EDR data, attacks that start with a vulnerability exploit (such as CapitalOne and Equifax) can also be detected more quickly. This data can further be correlated with cloud infrastructure / CSPM to determine attack path and blast radius risks.

But what or who is an EDR vendor?

Many EDRs have evolved out of anti-virus and anti-malware technologies. 

Historically, this type of virus and malware analysis has required dedicated labs and teams back when malware detection was based on signatures, with a touch of heuristics. However, in the mid 2010s, we began to shift towards heuristics and ML being the core detection algorithm, and many abandoned signature-based detections altogether.

And now, open source tools like osquery and eBPF make it easier than ever to get the necessary OS-level data access and observability, enabling startups in the EDR space. If you work with data-based detections, it might be easier than ever to build an EDR/XDR technology for your platform.

Could Application Program Monitoring be EDR? Why or why not?

On the other hand, most EDR today is built almost exclusively on observability and telemetry data. The actual endpoint agent responsible for collection reports back to a central analytics engine, and potentially takes response actions to mitigate an attack.

So why couldn’t APM companies also be EDRs? In a recent conversation with a product manager, whom I consider an expert in the EDR space, I made the comment “Don’t (LEADING APM VENDOR) and (LEADING EDR VENDOR) collect 70% the same data?” And that person agreed. 

You can have a look at DataDog (source) and CrowdStrike (3rd party source - harder to find data and specifics on what is collected) charts below.

Limitations and opportunities of an EDR

Another common problem in endpoint security is vulnerability elimination. The term vulnerability management is the traditional name for this category of security activity, but I increasingly think that it’s a misnomer, but more on that in a later post. 

In order to correlate hosts or endpoints with vulnerabilities, you also need access to the files to see what vulnerabilities exist on a system, as well as a database of vulnerabilities and the memory to see what is being executed in order to understand what vulnerable software is being used. And ideally you want to see what gets loaded into memory to prioritize the vulnerabilities in software that actually gets used, as opposed to vulnerabilities that normally lie dormant on disk.

Takeaways

In an era of increasingly complex risk, the level of data visibility that can be gathered from the endpoint is invaluable. However, this data needs to be correlated - whether into an XDR system or a platform that overlays with infrastructure, depending on the architecture. Finding more tasks that can be managed using the EDR agent can increase the value of the EDR while maintaining the management or CPU overhead.

The challenge for companies moving quickly is finding the right “build, bundle and deploy” that gets them the security they need, without causing the management overhead that leads companies to abandon EDR. 

EDR is indeed a crucial component of an overall security strategy. I thought the macro trends indicating a large-scale shift away from virtual machines would make EDRs less valuable over time. I was wrong and I admit it.