What is Vulnerability Scanning?
Vulnerability scanning is the practice of scanning systems and networks to detect known flaws or weaknesses that could be exploited by adversaries. Through identifying security flaws, vulnerability scanning constitutes the first and the essential step of an overall vulnerability management process.
To conduct automated vulnerability scans, software tools that are called vulnerability scanners are used to detect and identify known vulnerabilities in a system. Just like an antivirus software, vulnerability scanners include a database of known security issues and check the systems being scanned against this database.
As a common practice, both system administrators (or security practitioners) and attackers conduct vulnerability scans, albeit for different purposes. System administrators use vulnerability scanners to detect and treat (mitigate or remediate) the identified vulnerabilities for improving the overall security posture of their network. Attackers on the other hand, make use of the vulnerability scanners to identify the weaknesses in a system, such as missing patches or weak passwords, before moving on to the exploitation step.
In addition to being a common and good security practice, vulnerability scanning is mandated by some industry standards or government regulations. For instance, Payment Card Industry Data Security Standard (PCI-DSS) require that quarterly scans be conducted, as well as upon configuration changes in the systems (such as network topology changes, new software/hardware installations, firewall rule changes etc.).
My message to companies that think they haven’t been attacked is: ‘You’re not looking hard enough.’
James Snook
Read more educational and inspirational cyber quotes at our page 100+ Best Cyber Security & Hacker Quotes.
Types of Vulnerability Scanners
According to the type of the targets scanned, there exists two types of vulnerability scanners, i.e., network-based or web application vulnerability scanners.
Network-based vulnerability scanners identify the assets that sit on a network and detect vulnerabilities that are exploitable by network-based attacks. In scanning the assets on the network, operating systems, software and firmware are checked for known flaws and misconfigurations. For this purpose, each detected software or firmware versions are compared against a vulnerability database to determine the specific vulnerabilities that exist on the scanned assets.
Web application vulnerability scanners on the other hand, check the code of the web applications and web servers to detect common types of flaws such as SQL/Command Injection, Directory Traversal, Broken Authentication, or Cross-Site Scripting (XSS) etc. Note that, web application scanners identify specific vulnerabilities on the application code rather than using a database of known vulnerabilities and misconfigurations.
Network-based vulnerability scanners can further be categorized into two groups according to the vulnerability scanning approaches they employ: Active Vulnerability Scanning (AVS) and Passive Vulnerability Detection (PVD).
Traditionally, vulnerability detection has been conducted by Active Vulnerability Scanners (AVS). This type of scanners are also known as intrusive scanners as they make use of vulnerability detection scripts that send transmissions to the hosts (such as workstations, servers, active network devices etc.) on the networks and examine the responses they receive to determine the existence of vulnerabilities.
For detecting vulnerabilities, AVS usually follow the steps described below:
- Identify the hosts that are up and running.
- Conduct port scans on the identified hosts to determine open ports and the transport layer protocols.
- Identify the operating system or the firmware of the host via fingerprinting.
- Identify the running services and installed software using a variety of methods such as banner grabbing, behavior analysis or reading system file/configuration data.
- Use signature matching to determine presence of known vulnerabilities.
Depending on the scanning configuration, AVS can further probe the hosts via exploiting the identified vulnerabilities to reduce their false positive results.
In general, active scanners are known to have significant side effects, such as disruptions on the hosts or the networks, due to the high volume of traffic incurred and the the intrusive nature of the vulnerability detection scripts. Active scanners are also criticized for having a high false positive rates.
Passive Vulnerability Scanning (PVD) on the other hand, is a technique that relies on the data captured via passively to determine running services and installed software on a system, rather than actively interacting or probing the hosts on the network. Upon discovery of the running services and installed software on the hosts, vulnerabilities are identified via comparing each existing product or service against a vulnerability database, such as National Vulnerability Database (NVD.)
There are 3 methods to identify open ports, runnings services and installed software in a system using the PVD approach. These are:
- Passive Network Monitoring
- Reading System Logs and Configuration Files
- Reading Data from Inventory Management Systems
In the passive network monitoring method, data is captured via a number of techniques, such as connecting to the span ports of the switches or via integrating with Intrusion Detection Systems. This method is limited to detect only software or services that generate traffic on the networks and requires considerably long monitoring durations for high coverage rates. For this reason, passive network monitoring method can miss existing vulnerabilities on the scanned systems if an installed application is not run or or do not generate traffic on the network during the monitoring period.
In the second PVD method, operating system and application logs and configuration files are analyzed to determine the running services and the installed software on the hosts.
In the last PVD method, inventory management systems are leveraged as a data repository for the installed applications on the host in a given network.
For a quick overview, the PVD method is regarded to be very fast, as all it functionally requires is a database lookup to determine the existence of vulnerabilities. In this respect, it allows for frequent vulnerability scans as opposed to the sporadic scans offered by the AVS. PVD also does not cause any side effects on the networks, as discussed previously. Lastly, PVD is claimed to return results with lower false positive and negative rates, compared to the AVS.
Types of Vulnerabilities That Can Be Detected
Vulnerability scanners are not capable of finding all types of vulnerabilities. Rather, only vulnerabilities arising from known flaws in operating systems, software and firmware or vulnerabilities due to misconfigurations can be detected.
However, vulnerabilities that are not known publicly (zero-day) and exploited in the wild can not be detected by vulnerability scanners since no records exist about them in the vulnerability databases.
Secondly, the flaws and misconfigurations that reside at the vulnerability databases are only technical in nature. Thus, vulnerabilities due to accidental or intentional human behavior can not be identified either via an automated vulnerability scanning. For instance, a default password used in a system can be detected technically, while not protecting a password securely (A vulnerability caused by humans inadvertently due to insufficient security training) can not be detected by vulnerability scanners.
Thirdly, vulnerabilities arising from missing, incomplete or unenforced security processes or procedures are also out of the scope of vulnerability scanners, though they are at least as critical as the technical vulnerabilities.
For this reason, vulnerability scanning should almost always be complemented with penetration tests, as part of a thorough vulnerability assessment process.
External vs. Internal Vulnerability Scanning
Network-based vulnerability scans can be conducted either externally or inside a network. Scanning outside the network perimeter helps determine exposure to attacks against network assets that are directly accessible from the Internet. In this respect, the goal of the external scans is to identify which assets are at risk of being exploited directly outside the network perimeter.
However, gaining a foothold on the network after a successful attack, an adversary usually conducts further exploits to move laterally to other assets on the network that are not facing the Internet directly. For this purpose, internal network scans are conducted to identify additional vulnerabilities on the local network. Internal scans are also useful to increase the security posture of a system against malicious insiders/attackers.
Authenticated vs. Unauthenticated Vulnerability Scanning
Network-based vulnerability scanners first try to identify the operating systems, services or applications uniquely in order to compare them against a list of vulnerability data on the vulnerability database. However, the accuracy of the identified applications or services depends on what type of vulnerability scanning is conducted, i.e., authenticated (credentialed) or unauthenticated (non-credentialed).
Authenticated scans use login credentials to directly access the network assets. They use remote administrative protocols such as Secure Shell (SSH), Remote Desktop Protocol (RDP) etc. to connect remotely to the scanned assets on the network. Authenticated scans allow the scanners to access low level data such as specific services and configuration details, installed software versions or missing/applied security patches that are not easily detectable without authenticating to the system under scan. Thus, authenticated scans result in more accurate vulnerability information with less false positives.
Unauthenticated scans on the other hand can collect less reliable information about the operating system or application versions, running services and applied patches or system configurations using techniques like fingerprinting. Thus, results of the unauthenticated scans are usually less complete and include more false positives. However, this method is still useful to determine the security posture of the assets externally, such as to find out about possible data leaks.
Common Vulnerability Scanning Pitfalls
When conducting vulnerability scanning, you should be cautious about the following issues:
- Up-to-Date Vulnerability Signatures: Before starting a vulnerability scan, first make sure that your your vulnerability scanner database is up-to-date with most current signatures. Otherwise, inevitably you will end up with detecting possibly less vulnerabilities than currently existing vulnerabilities (false negatives).
- Vulnerability Definition Feeds: It is also crucial to understand that having an updated vulnerability database in your product does not guarantee detection of all published or publicly known vulnerabilities. You should contact your vulnerability scanner vendor to learn more about how long it usually takes for them to include recently published vulnerabilities in their product’s vulnerability database. Note that, on average this period is longer than a month for most products.
- Vulnerability Scanning Period: One major issue with vulnerability scans is the fact that periodic vulnerability scans only provide a snapshot of your security posture for a specific time. In other words, the vulnerability scan results will be out-of-date as soon as vulnerability scanning is completed. In this respect, the longer the period between scans, the more you are susceptible to attacks that could exploit recently discovered vulnerabilities.
- Authenticated Scans: Check your vulnerability scanners logs to make sure that authenticated scans have been completed properly with successful authentications to the targeted assets on the network. Due to incorrectly typed credentials or technical issues like firewall blocks, remote authentications could fail for some of the assets and go with the unauthenticated scans instead without generating further alerts.
- Mitigated Vulnerabilities: Sometimes the preferred method of treating a detected vulnerability is mitigation rather than remediation. In other words, instead of updating or applying patches on a vulnerable application or operating system, additional control measures, such as blocking (or restricting) the related ports on a firewall, could be applied to remediate a vulnerability. In such a circumstance, future scans shall continue to detect and report these previously mitigated vulnerabilities as untreated. As a solution for this issue, you could choose a vulnerability scanner that allows you to label such vulnerabilities with a mitigation control when a remediation is not possible.
Final Remarks – Vulnerability Scanning Limitations and Shortcomings
Vulnerabilities scanners are invaluable tools in automating the vulnerability detection tasks. However, they can’t replace the expertise of trained security personnel. This is due to the fact that vulnerability scanners most of the time either alert on non-existing vulnerabilities (false positives) or overlook some of the existing ones (false negatives). Thus, data returned by the vulnerability scanners need to be analyzed carefully by the security professionals to eliminate the false positives and take action on the filtered true positive results. To identify the existing vulnerabilities missed by vulnerability scanners, security professionals should keep a good record of their inventory and check it against vulnerability databases (such as NVD) and make use of threat and vulnerability intelligence tools to detect imminent threats.
If you would like to learn more about vulnerabilities, you could also read our articles What Is a Security Vulnerability? or Why We Need to Change Our Current Vulnerability Scanning Practice?
