Detection Engineering (DE) is an effective strategy to enhance an organization's security posture. DE involves using threat modeling, red teaming, sandboxing and pen-testing techniques in combination with threat modeling to identify vulnerabilities before they evolve into lethal threats.
DE is an innovative methodology for analyzing and hardening detections based on software engineering best practices. This approach reduces analyst fatigue while freeing teams to focus on more complex and meaningful alerts.
Threat modeling
Detection Engineering involves designing and developing detection capabilities to detect malicious activities in network traffic or host computers that bypass other security defences, such as firewalls or antivirus. This process is also known as Endpoint Detection and Response (EDR) on endpoints and Network Detection and Response (NDR) for networks; both involve various techniques designed to identify attacker patterns; this discipline forms part of any comprehensive security program.
Threat modeling is the foundation of detection engineering, consisting of creating visual representations of an application or infrastructure to identify potential threats and then identify and enumerate suitable mitigating controls.
Threat models can be built using various tools and techniques, including flow charts and attack trees. Flow charts help detect possible points of entry, while attack trees show all possible paths an attacker might use to breach your system. Furthermore, detection engineers can utilize risk analysis tools to rate an asset's vulnerability based on factors like damage, discoverability, exploitability and impact on affected users.
DE practitioners commonly rely on custom-tailored detection rules programmed using high-level languages like Python to integrate into security platforms. This approach allows developers to craft detections that fit each organization's environment and priorities while being easy to update, version control and programmatically manage.
Detection engineering is an evolving practice that constantly adapts to new threats. It includes researching methods of finding malicious activity in customer environments and translating threat bulletins into detection use cases. Furthermore, detection engineers often simulate attacks to test detection capabilities with Incident Response teams.
Threat engineers must utilize data from multiple sources - postmortems of real incidents and red/purple team exercises are two such sources - to build detections. After determining what behaviours can be detected and creating a detection model to incorporate those factors, threat engineers can use various techniques, including encoded PowerShell commands or Windows macro executions found in malware samples, to implement detections into their system.
Threat hunting
Threat hunting is a proactive security practice that looks for evidence of adversarial activity that traditional security systems might miss. By employing intelligence and expertise, threat hunters use intelligence-gathering to uncover attackers' tactics, techniques and means-of-compromise (TTPs and IoCs), triggering alerts or other threat prevention mechanisms. Threat hunters must examine networks carefully in search of any signs that indicate possible attacks - this process includes tools like sandboxing or pen testing and data from logs or security tools to identify anomalies that might indicate possible attacks.
Threat hunting seeks to minimize the dwell time between an initial compromise and its discovery by finding and analyzing suspicious activity within an organization's network, such as malware infections or unusual network traffic patterns. While it's essential to detect attackers quickly, threat hunting also plays an integral role in detection engineering lifecycle processes as it helps identify threats which might evade security controls - something which has made threat hunting an integral component.
Detection engineering refers to designing, creating and testing detection logic. As part of cybersecurity operations, detection engineering requires buy-in from all parties: content developers, analysts and risk management. Doing this improves detection quality while decreasing false positives, which leads to analyst fatigue.
A great detection engineer must be capable of creating effective detection rules based on real-world threat intelligence. To do so, they must understand various environments' complexity to craft rules that are more accurate and less likely to produce false positives. They should also frequently test these rules to identify false positives that warrant further investigation quickly.
Establishing effective detection rules can be challenging, requiring both knowledge and resources. Security teams often become overwhelmed with daily alerts and lack the bandwidth to investigate them; consequently, only about 1% of critical security alarms are investigated, leaving businesses vulnerable to attacks.
Detection as code
Detection as code is an approach that employs software engineering best practices for threat identification, allowing security teams to create scalable processes capable of detecting sophisticated threats across rapidly expanding environments. Detection as code facilitates rapid creation, testing, and deployment of detections into production. Although not a new concept in cybersecurity, detection as code has quickly gained prominence. It helps enhance SOC performance by automating manual processes, improving quality alerts, preventing false positives and eliminating false positives. Furthermore, detection as code works well alongside tools like threat modeling and hunting and SIEMs, EDRs, or XDRs for an enhanced detection experience.
An effective security team can use frameworks like YARA or Sigma to automate detections in a structured, automated fashion, then utilize a continuous integration/continuous deployment pipeline to test, linting, check and deploy them quickly - helping shift security left and speed response times.
Coding frameworks make it easy to manage changes to detections. Writing them in code is faster and more effective than manually editing a configuration file; additionally, this removes the need for security analysts to maintain an ongoing detection rules database.
Once detections are written in code, they can be versioned and deployed directly into production via a continuous integration and delivery (CI/CD) pipeline. This ensures they always run with the most up-to-date rules and reduces resolution time in production environments.
Use a coding framework compatible with the detections you are creating, and create a central repository accessible to all members of your security team for enhanced collaboration, test-driven development (TDD), and version control management.
A centralized repository can be hosted in either the cloud or an internal network and be accessed through either browser or command line access. This makes it easier for security engineers to collaborate on detections with team members while sharing them more quickly - speeding up both development and review processes.
Detection maintenance
Maintenance is an integral component of the detection engineering lifecycle. It entails minimizing false alerts, improving detection content and increasing threat actor visibility, identifying and closing gaps in detection capabilities and providing necessary resources for your team. For instance, a gas detection maintenance crew might be responsible for replacing worn sensors at industrial plants to help avoid potential danger from undiagnosed gas leaks or malfunctioning equipment.
Maintenance can be essential to security operations team operations, yet detection maintenance can be challenging. It is vital that your detection system works as intended and detects relevant IOCs to minimize the mean time to detection (and the impact of breaches) quickly and efficiently.
Different networks come with different configurations that may lead to different detection capabilities. Engineers must be mindful of this fact when creating detections across networks - otherwise, they risk creating many false positives, which could potentially waste resources and cause troublesome delays in production. A continuous integration and delivery (CI/CD) process for detections as code can help avoid this situation.
Deployment engineering's primary aim is to make it easy for network defenders to quickly identify any malicious activity on their networks so that they can act swiftly against it. To accomplish this effectively, a culture of support must exist for detection functions and an understanding of their complexities if producing accurate yet actionable detections is desired.
This can be a difficult challenge, but it can be accomplished by adhering to industry best practices. Detection engineering draws upon various methodologies - threat modelling, pen-testing, purple teaming and sandboxing, honeypot deployment, as well as automated testing and deployment tools, can speed development up considerably and create finely tuned detections that don't overload security teams' response times, resulting in alert fatigue and slow the response rate of security teams.
