How Fidelis Network Behavior Analysis Detects Advanced Threat Activity

Key Takeaways

Network behavior analysis detects advanced threats by modeling normal activity first.

Fidelis’ Deep Session Inspection® gives the telemetry needed for strong behavioral detection.

Protocol behavior is one of the strongest detection surfaces.

Correlation is what turns weak signals into high-confidence detections.

A mature network behavior analysis system reduces noise with context.

Advanced threat activity rarely looks dramatic at first.

It may start with a compromised workstation talking to a new internal system. A user account is accessing data at an unusual hour. A public-facing server is showing a quiet rise in invalid web requests. Or a device begins making DNS queries that deviate from its normal pattern.

On their own, these signals may look harmless, but together, they can point to command-and-control, lateral movement, reconnaissance, phishing, insider misuse, or data exfiltration.

That is where network behavior analysis becomes valuable.

Instead of waiting for a known malware signature or a single obvious indicator of compromise, Fidelis Network® Behavior Analysis looks at how users, devices, applications, protocols, and data flows normally behave. It then detects activity that falls outside those patterns and correlates related signals into a clearer threat picture.

Advanced attackers often hide inside legitimate protocols, encrypted sessions, valid credentials, and trusted network paths. A strong network behavior analysis system is built for that reality: finding activity that looks normal enough to pass through traditional controls, but abnormal enough to stand out when viewed in context.

Why Advanced Threats Are Hard to Detect on the Network

Modern enterprise networks generate constant noise. New applications appear, cloud services change, users travel, developers test tools, and administrators perform maintenance. Attackers take advantage of that noise by making their activity look like normal business traffic.

That creates four major detection challenges: legitimate access, encrypted traffic, trusted internal movement, and low-volume data theft.

Attackers Use Legitimate Protocols and Credentials

Many advanced attacks begin with access that appears valid.

A stolen credential, compromised session, abused service account, or misused internal tool can give an attacker a clean path into the environment. From there, the activity may look ordinary at the surface level. And traditional controls may treat those actions as acceptable because the account is valid and the protocol is allowed.

Network behavior analysis asks a sharper question: is this behavior normal for this user, device, application, or service?

Network user behavior analysis connects user activity with network activity, so compromised accounts are evaluated by behavior, not just by whether authentication succeeded.

Encrypted Traffic Creates Visibility Gaps

Encrypted traffic is necessary for business, but it also gives attackers cover.

Command-and-control traffic, malware callbacks, phishing infrastructure, and data movement can all ride over TLS. In many environments, decrypting every session is unrealistic because of privacy requirements, performance impact, certificate management complexity, and cloud traffic patterns.

Network behavior analysis helps by using the evidence still visible around the encrypted session. The payload may be encrypted, but the session still leaves behind valuable metadata: handshake characteristics, TLS fingerprints, certificate details, destination rarity, session timing, byte counts, directionality, and communication patterns.

Fidelis Network Behavior Analysis fingerprints TLS client and server behavior using handshake metadata such as JA3 and JA3S. When a new or rare TLS fingerprint appears on an enterprise asset, it can signal a newly introduced tool, unauthorized application, malware implant, or command-and-control channel. Fidelis increases confidence by correlating the fingerprint with supporting evidence such as suspicious certificate attributes, rare destinations, new geographies, and abnormal host activity.

Lateral Movement Often Happens Inside Trusted Zones

Once attackers gain a foothold, they usually move across the environment. That movement may happen inside trusted network zones where monitoring is weaker, and traffic is assumed to be safe.

This is where many attacks become difficult to detect. The traffic may never leave the organization. The attacker may use valid credentials. The protocols may be common administrative protocols. The activity may look like normal IT work unless it is compared against historical behavior.

Fidelis’ Network behavior monitoring helps expose this type of blind spot by looking for devices suddenly communicating with hosts they do not normally contact or using services they do not normally use.

For example, a workstation suddenly communicating with multiple servers it has never touched before is worth investigating.

Low-and-Slow Exfiltration Avoids Basic Thresholds

Attackers know that large, sudden transfers can trigger alarms. To avoid detection, they may exfiltrate data slowly, split files into smaller chunks, use common cloud services, or send data during periods that appear less suspicious.

Simple volume thresholds often miss this type of activity. Network behavior analysis looks at the pattern behind the movement: who is sending the data, where it is going, whether the destination is common or rare, whether the timing is normal, whether the volume fits the user or device, and whether the activity follows earlier suspicious behavior.

Network behavior monitoring guidance calls out low-and-slow exfiltration as a blind spot because attackers may move data in tiny pieces or embed it in normal-looking traffic, which makes baseline understanding of normal data flows critical.

What Network Behavior Analysis Looks at Under the Hood

Network behavior analysis starts with visibility, then adds context. First, the platform needs enough session-level detail to understand what is happening on the wire. Then it needs network baseline analysis to determine whether that activity is expected, unusual, or suspicious. Finally, it needs correlation across multiple contexts so weak signals can become high-confidence detections.

Here is how Fidelis uses network behavior analysis to detect advanced threat activity:

Deep Session Inspection Creates the Visibility Foundation

Behavioral analytics is only as strong as the telemetry behind it.

Fidelis Network® Behavior Analysis uses deep visibility into network traffic across ports and protocols. Deep Session Inspection® captures rich metadata from streaming traffic and turns raw activity into usable security evidence.

Basic flow data can show that the two systems communicated. Deep session-level visibility can show much more: which protocol was used, whether the protocol appeared on an unusual port, what kind of TLS handshake occurred, what DNS behavior appeared, what HTTP response patterns changed, whether a file or object was transferred, and how that behavior compares with historical activity.

Fidelis collects high-fidelity telemetry from streaming network traffic across all ports and protocols. It captures more than 300+ metadata attributes, giving analysts the session-level detail needed to understand what is happening on the wire. This telemetry becomes the foundation for baseline development, anomaly detection, alert enrichment, and response workflows.

Fidelis DSI – Advanced Data inspection and Threat Detection Capabilities

Baselines: Learning What Normal Looks Like

The word “anomaly” can be misleading. In a large enterprise, unusual activity happens constantly. New software gets deployed. Employees travel. Cloud services change IP ranges. Developers test tools. Administrators run maintenance. Business teams access new systems.

A useful behavior model has to separate harmless change from meaningful risk. Fidelis Network® Behavior Analysis does that by building statistical baselines for hosts, users, services, flows, applications, and protocols. Once normal activity is understood, the platform can flag meaningful deviations, such as new peer relationships, unusual port usage, abnormal data transfer patterns, unexpected external communication, rare protocol activity, or behavior that conflicts with an asset’s role.

Behavior analysis works because it considers the entity, the context, and the history together.

Multi-Context Analysis Connects the Signals

Advanced attacks are rarely visible through one signal.

A rare TLS fingerprint may be interesting. A rare TLS fingerprint connecting to a rare domain is more serious. Add suspicious certificate attributes, unusual internal access, abnormal data movement, or interaction with a deception asset, and the signal becomes much harder to dismiss as noise.

Fidelis Network® Behavior Analysis analyzes activity across five behavioral contexts to detect advanced threats with stronger confidence. It evaluates

External traffic for north-south communication and exfiltration risks,

Internal traffic for east-west movement and insider threat activity,

Application protocol behavior for protocol abuse,

Data movement for unauthorized transfer patterns,

Event context to correlate behavioral anomalies with rule-based and signature-based detections.

With this information, analysts are not left chasing isolated anomalies. The platform helps build a case by showing how events relate to each other.

How Protocol Behavior Reveals Advanced Threat Activity

Advanced attackers often hide inside protocols the business already allows, such as TLS, DNS, HTTP, and SMTP. Blocking these protocols is not realistic because the business depends on them. Modeling how they normally behave is more practical and more useful.

This is where behavior-based network traffic analysis becomes a strong detection layer. Protocols are inspected for how they behave, not just whether they match a known bad signature.

TLS Behavior: Detecting Suspicious Encrypted Communication

Encrypted traffic is a favorite hiding place for attackers, but the TLS handshake still exposes useful behavior. Fidelis Network® Behavior Analysis uses client and server handshake characteristics to identify fingerprints such as JA3 and JA3S. A new or rare fingerprint on an enterprise asset may indicate a new tool, unauthorized application, malware implant, or command-and-control channel.

Because legitimate software updates can also introduce new fingerprints, Fidelis correlates TLS fingerprint behavior with supporting signals such as suspicious certificate attributes, rare web domains, new destination geographies, and unusual host activity. This helps separate benign encrypted traffic from higher-risk communication that may indicate C2 activity.

DNS Behavior: Finding DGA and DNS Tunneling

DNS is widely allowed, noisy, and easy to overlook, which makes it attractive for attackers. Two DNS threat patterns matter most for behavior analysis: domain generation algorithms (DGA) and DNS tunneling.

For DGA detection, Fidelis Network® Behavior Analysis analyzes domain names observed through DNS and web activity to identify domains that appear algorithmically generated. That signal becomes stronger when paired with DNS failure behavior, especially elevated NXDOMAIN responses. A device repeatedly looking up algorithmically generated domains that do not resolve may be malware searching for active command-and-control infrastructure.

For DNS tunneling, Fidelis Network® Behavior Analysis models DNS query behavior against established baselines. Unusually long subdomain names, a high number of unique subdomains for a domain, rare domain usage, or abnormal DNS behavior for an asset can indicate that DNS is being used as a covert channel for command-and-control or data movement.

HTTP Behavior: Spotting Exploitation and Discovery Attempts

Web traffic creates another useful behavior surface because public-facing web servers are constantly scanned, probed, and tested. Internal web applications can also become targets during discovery and lateral movement. Fidelis Network® Behavior Analysis models HTTP behavior to identify activity that falls outside the normal pattern of a server or application.

For example, new invalid URL errors, repeated requests for non-existent paths, or an unusual rise in HTTP 400-range responses such as 404 errors can indicate probing, file and directory discovery, or attempted exploitation. These detections can also be mapped to relevant MITRE ATT&CK techniques, including exploitation of public-facing applications.

SMTP Behavior: Detecting Phishing and Internal Email Abuse

Email behavior carries strong threat signals because phishing often depends on trust. An external sender may spoof an identity, while a compromised internal account may abuse trusted access to reach a larger-than-normal group of recipients.

Fidelis’ Network Behavior Analysis models SMTP behavior to detect those patterns by identifying “From” and “Reply-To” mismatches, evaluating sender prevalence, and flagging unusually high internal recipient counts. These signals help surface phishing, compromised account misuse, and internal spear phishing without treating every unusual email as equally risky.

Other Protocols: Catching Rare or New Protocol Usage

Not every threat maps neatly to TLS, DNS, HTTP, or SMTP. Attackers may use administrative protocols, custom tools, non-standard ports, or unexpected protocol combinations to move through an environment.

Fidelis Network Behavior Analysis detects rare or new protocol usage by comparing an asset’s current communication behavior against its normal baseline. If a workstation suddenly uses a protocol it has never used before, a server begins communicating like a client, or a privileged asset uses a protocol outside its normal role, the platform can flag that activity as a suspicious deviation. This helps identify possible lateral movement, command-and-control activity, discovery, policy abuse, or unauthorized tool use.

ProtocolBehavior SignalPossible Threat

TLSRare fingerprint, suspicious certificateC2DNSDGA, NXDOMAIN spike, long subdomainsC2 / tunnelingHTTP400-range errors, invalid URLsDiscovery / exploitationSMTPFrom/Reply-To mismatch, high recipient countPhishing / account abuseUDP/ICMP/OtherNew or rare protocol usageLateral movement / C2

Protocol Behavior Signals Mapped to Threat Activity

How Fidelis Network Behavior Analysis Reduces False Positives

Not every anomaly is a threat. Fidelis Network Behavior Analysis reduces false positives by adding context before raising priority. It looks at how rare behavior is, how often it appears across the environment, which asset produced it, whether related signals are present, and how much risk the activity introduces. This helps security teams separate normal business change from behavior that may indicate command-and-control, lateral movement, data exfiltration, or account compromise.

Rarity and Prevalence Show What is Actually Unusual

Rarity helps determine whether behavior is common in the environment or unusual enough to investigate.

For example, a destination contacted by thousands of devices may be routine. The same type of destination contacted by one workstation for the first time may be more suspicious, especially if the asset has no history of similar activity.

Fidelis Network Behavior Analysis uses prevalence context to avoid treating every unusual event the same way. The platform evaluates how common the behavior is, which asset produced it, and whether the activity fits that asset’s normal baseline.

Correlation Connects Related Signals into an Investigation Path

A single signal may not prove an attack, but related signals can create a stronger case.

A compromised endpoint may first show a rare TLS fingerprint. Then it may query rare domains, communicate with an unfamiliar external destination, access internal systems it has never touched before, and move data in a pattern that does not match historical behavior.

Fidelis Network Behavior Analysis correlates these behavioral signals across protocols, assets, users, and activity stages. This gives analysts a connected investigation path instead of isolated alerts that have to be manually pieced together.

Risk Scoring Helps Analysts Focus on the Right Alerts First

Security teams cannot treat every alert with the same urgency.

Fidelis Network Behavior Analysis uses risk context to help prioritize what analysts should investigate first. A low-confidence anomaly may simply need monitoring. Multiple related anomalies involving a critical asset, rare destination, suspicious certificate, abnormal internal access, or threat intelligence match should move higher in the queue.

This makes network threat behavior analysis more actionable. The platform does not just show that something changed. It helps explain why the change matters, how serious it may be, and where analysts should focus first.

Threat Activity Fidelis’ Network Behavior Analysis Helps Detect

Network behavior analysis becomes most valuable when it connects technical signals to real attacker activity. When signals appear in the right sequence, they can reveal command-and-control, lateral movement, data exfiltration, or account compromise.

Fidelis Network Behavior Analysis helps security teams move from isolated anomalous behavior to a clearer threat picture by comparing activity against baselines, analyzing protocol behavior, and correlating related signals across users, devices, applications, and flows.

Command-and-Control Activity

Command-and-control traffic is often designed to stay quiet. It may be encrypted, low-volume, and routed through infrastructure that has not yet appeared on threat intelligence feeds.

Fidelis Network Behavior Analysis helps detect C2 by analyzing how a host communicates, where it communicates, and whether that behavior fits its normal activity. New or rare TLS fingerprints, suspicious certificate attributes, rare domains, new destination geographies, DGA patterns, DNS tunneling behavior, and unusual protocol usage can all contribute to a stronger C2 detection.

Lateral Movement

After attackers gain access, they usually need to move. That movement often happens inside trusted zones and may use valid credentials or common administrative protocols. This makes it difficult to detect with static rules alone.

Fidelis Network Behavior Analysis helps detect lateral movement by comparing internal communication against established baselines. New peer relationships, unusual east-west traffic, unexpected port or protocol usage, and activity that does not match an asset’s normal role can all indicate post-compromise movement.

Data Exfiltration

Data exfiltration is often built to avoid simple volume-based alerts. Instead of sending one large transfer, attackers may move data slowly, use approved channels, or send data to destinations that do not immediately look malicious.

Fidelis Network Behavior Analysis helps identify data movement that does not fit the normal behavior of the user, device, application, or flow. Repeated small transfers, unusual destinations, new geographies, after-hours movement, role-inconsistent data access, and covert channels such as DNS tunneling can all indicate possible exfiltration.

The key question is not only how much data was moved. It is who moved it, where it went, when it moved, how often it moved, and whether that activity fits the entity’s baseline.

Insider Threats and Account Compromise

Valid credentials can bypass many traditional controls, but they cannot always hide abnormal behavior. When an account, user, or device begins acting outside its normal pattern, network user behavior analysis can help surface possible misuse or compromise.

Fidelis Network Behavior Analysis helps identify suspicious behavior such as unusual access timing, unfamiliar systems, abnormal internal communication, unexpected file or data access, and high-volume internal email activity. These signals are especially important because insider misuse and account compromise can look similar at the network level. The account may be valid, but the behavior may be wrong.

Why Fidelis Network Behavior Analysis is Built for Advanced Threat Defense

Advanced threats depend on gaps between tools, teams, and telemetry. Fidelis Network Behavior Analysis helps close those gaps by combining deep session visibility, protocol behavior modeling, baselines, correlation, and contextual risk scoring into one detection approach.

The result is a stronger way to detect threats that hide inside normal-looking network activity.

Deep Visibility Across Ports and Protocols

Advanced attackers do not always use expected ports or obvious protocols. Deep visibility across ports and protocols helps reveal what is actually happening inside sessions, even when traffic patterns are designed to blend in.

This gives security teams better evidence than basic flow records alone.

Behavioral Modeling Across Users, Devices, Applications, and Flows

Network behavior analysis looks beyond individual events. It models how users, devices, applications, services, protocols, and flows normally behave.

That broader view helps detect activity that is technically allowed but behaviorally wrong.

Detection Across Encrypted, Internal, and Hybrid Traffic

Advanced threats frequently hide in encrypted traffic, internal east-west movement, and hybrid environments.

Behavior analysis helps expose those areas by using metadata, protocol behavior, baselines, and correlation. This makes it useful in environments where payload inspection is limited, internal movement is hard to monitor, or cloud traffic creates visibility gaps.

See What Metadata Sees — Your Network’s Secrets, Threats, and Patterns

Correlation That Turns Network Activity into Actionable Evidence

The real strength is correlation.

A rare fingerprint, unusual DNS behavior, new internal communication path, and abnormal data movement may each look small. Together, they form evidence. That evidence helps analysts understand what happened, how serious it is, and what action should come next.

Network behavior analysis gives security teams a way to move from “something odd happened” to “this activity matches a likely attack path.”

Frequently Asked Questions

What is network behavior analysis?

Network behavior analysis is a security approach that learns normal patterns across users, devices, applications, protocols, services, and data flows, then flags behavior that deviates from those patterns. It is especially useful for detecting advanced threats that use legitimate credentials, encrypted traffic, approved protocols, or slow-moving attack techniques.

How is network behavior analysis different from traditional signature-based detection?

Signature-based detection looks for known indicators such as malware hashes, known domains, or predefined attack patterns. Network behavior analysis looks for abnormal behavior, even when the specific malware, domain, certificate, or tool has not been seen before. That makes it useful for unknown threats, modified attacker tooling, and low-noise attacks that do not match existing signatures.

What types of threats can network behavior analysis detect?

Network behavior analysis can help detect command-and-control, lateral movement, DNS tunneling, DGA activity, phishing behavior, internal spear phishing, exploitation attempts, file and directory discovery, unusual data movement, insider misuse, and compromised account activity. The strongest detections usually come from correlating multiple behavioral signals instead of relying on one anomaly.

Why is network baseline analysis important?

Network baseline analysis defines what normal activity looks like for a specific environment. Without a baseline, unusual activity is difficult to judge. With a baseline, the platform can identify deviations such as new peer relationships, unexpected protocols, abnormal transfer patterns, rare destinations, unusual login behavior, or asset activity that does not match its normal role.

What are the most important network behavior analysis features?

Important network behavior analysis features include deep session visibility, baseline profiling, protocol behavior modeling, encrypted traffic metadata analysis, context-aware correlation, risk scoring, MITRE ATT&CK mapping, threat intelligence enrichment, integration with endpoint and identity telemetry, and response workflow support. These features help move analysts from raw anomalies to actionable evidence.

Unlock Powerful Network Security with Fidelis NDR

Comprehensive Threat Detection & AnalysisData Loss Prevention (DLP) & Email SecurityDeep Session Inspection & TLS ProfilingRead Datasheet See Fidelis NDR in Action

The post How Fidelis Network Behavior Analysis Detects Advanced Threat Activity appeared first on Fidelis Security.