Passive Information Gathering for Pentesting

When you are conducting a penetration test, it is important to take a methodological approach to information gathering and divide the task up into two parts: passive information gathering and active information gathering. Passive information gathering should come first. It involves collecting public information from the internet about the company being assessed — without invoking any kind of communication with the target systems.

Passive information gathering involves using internet resources to find out publicly available information about the company that could help you exploit the company’s systems and bypass security controls while performing the pentest. There are different techniques to passive information gathering: you could surf public internet sites manually, query DNS, or use open-source intelligence (OSINT) gathering tools to automate the discovery of information. Most of these techniques are not technical in nature, but they do represent the mindset of a hacker, so you want to follow similar strategies when performing your pentest.

Open-source intelligence gathering

The term used for discovering information from public data sources available on the internet is open-source intelligence (OSINT) gathering. Through OSINT gathering, you can collect information about a company from the company’s website, social media sites, domain name system (DNS) information, blogs, and so on. The goal of OSINT gathering is to gather information such as contact names, email addresses, DNS records to aid in the penetration test.

Browsing internet resources

The first technique to use when information gathering is to surf the company website for information that could aid in an attack, such as software the company is using or email addresses and phone numbers of company employees that you could use in a social engineering attack.

Look for web pages, such as About Us, Job Postings or Careers pages, that could offer information like names, phone numbers, and email addresses of employees or upper management. This is great information to use in a social engineering attack. In addition, a Job Postings or Careers page may list active jobs that could help you understand the technologies the company is using. For example, if the company is looking for an Exchange Server 2016 Messaging Administrator, then you know the company is most likely running Exchange Server 2016.

For the PenTest+ certification exam, know that you can use tools such as the popular wget in Linux or the BlackWidow utility for Windows to copy the contents of a website to a local folder on your system so that you can leisurely review the contents offline.

Using Google hacking

Google hacking is the term used for an information gathering technique in which specific keywords are used to search Google or other search engines for specific information on the internet. Here are a few of the Google keywords you should be familiar with that I find quite useful:

• site: : The site keyword is used to search a specific website for a keyword. For example, if you are performing a security test for the Wiley publishing company, you could use site: www.wiley.com password to locate the login pages on the Wiley website. This could be useful if you wanted to test Wiley’s login pages against SQL injection attacks.
• intitle: : You can use the intitle keyword to search the title of a page for specific keywords. For example, if you want to find web pages that contain the word “intranet” in the title, you could use intitle: intranet.
• inurl: : The inurl operator will search the keyword given in the URLs found in the Google database. For example, if you want to locate sites that have the word “intranet” in the URL, you could use inurl: intranet.
• intext: : The intext operator searches a web page for specific text. For example, if you want to search my company site for pages that contain the word “video,” you could use site: dcatt.ca intext: video.
• filetype: : One of my personal favorites is the filetype operator, which you can use to find results containing a specific file type. For example, you could search the internet for sample penetration reports by filetype: pdf penetration test report.

Referencing online cybersecurity sources

In addition to browsing internet resources and using Google hacking to conduct your passive information gathering, research from many official sources is available for OSINT gathering, especially in the realm of cybersecurity information.

You should be familiar with the following sources of cybersecurity information for the PenTest+ certification exam:

• CERT: Short for Computer Emergency Response Team, there are many CERT groups available worldwide that share cybersecurity information. Example CERT groups are the US CERT group and the Canadian version.
• JPCERT: The PenTest+ certification exam makes special mention to JPCERT, which is the Japan CERT group used to share information on cybersecurity. You can visit the JPCERT
• NIST: The National Institute of Standards and Technology (NIST) is a standards organization that develops a number of documents related to cybersecurity known as special publication (SP) documents. For example, SP 800-115 is a guide to security testing and assessments, while SP 800-17 is a guide to risk management. There are a number of SP documents well worth reading.
• CAPEC: The Common Attack Pattern and Enumeration Classification (CAPEC) is an information resource provided by a company called MITRE that identifies and documents attack patterns. The MITRE site also provides information on mitigation techniques for the attacks.
• Full disclosure: You can subscribe to mailing lists that share information related to vulnerabilities and exploitation techniques known as full disclosure lists.
• CVE: The Common Vulnerabilities and Exposures (CVE) list is responsible for identifying known vulnerabilities by their name, number, and description.
• CWE: The Common Weakness Enumeration (CWE) list is a list of common weaknesses found in software and the mitigation techniques used to protect against those weaknesses.

Passive information-gathering tools

In addition to using Google or surfing the company website, you can use a number of passive OSINT tools to help collect such company information as contact names, email addresses, DNS information, and internet protocol (IP) addresses.

Whois

Whois is a widely used database search tool used to discover domain name information and IP address information about a company. The domain name information sometimes contains important contact information of senior IT professionals that you can use in a social engineering attack, while the IP information is the public IP addresses purchased by the company. Having this information handy will aid in the next phase of the pentest — discovering active hosts.

A number of Whois databases that you can search are available online. For example, you could go to www.godaddy.com/whois to perform a search, or you could go to www.networksolutions.com/whois, which is shown in the following figure. What is cool about the Network Solutions search page is you can search by domain name or IP address. Note that with the Whois lookup, you can collect information, like the organization’s name, the DNS servers hosting the DNS data, and sometimes contact information, such as email addresses and phone numbers of company employees.

Many people are now using private registration with their domain registration information, which helps protect the personal information by obfuscating the information that is displayed with Whois lookups.

You can also use Whois programs to discover domain name and IP address information. For example, Kali Linux comes with a Whois program you can execute from a terminal with the following command:

whois wiley.com

Another site with detailed Whois information is ARIN. When search results come back, choose the handle. You can then see the public IP addresses that are used by that organization.

theHarvester

theHarvester is a program in Kali Linux that you can use to perform passive information gathering to collect information such as employee names, email addresses, and subdomains, and discover hosts owned by the organization. You can use it to collect public information from Google, LinkedIn, Twitter, and Bing.

The following command searches LinkedIn users for Wiley:

theharvester -d wiley.com -b linkedin

To collect information from all sources such as Google, LinkedIn, and Twitter, use the following command:

theharvester -d wiley.com -b all -l 100

In this example, shown in the following figure, I limited the results to 100.

Shodan

Shodan is a search engine that collects information about systems connected to the internet, such as servers and internet of things (IoT) devices. To use Shodan, you need to register with a free account and then you can search the company or organization being assessed. When you perform a search in Shodan, you get a list of the target company’s publicly available servers and devices along with the IP address, the services running, and the ports that are open on that system. When you view the details for that system, you can get a list of its vulnerabilities. A map view shows the physical location of those servers as well.

Maltego

Maltego is OSINT software that shows a graphical representation of relationships between people, groups, webpages, and domains by analyzing online resources like Facebook, Twitter, DNS, and Whois information. For example, you could create a graphic and add a website address to the graphic, then use Maltego to search for additional information. This could be Whois information, phone numbers, location information, and email addresses associated with that website, and then you can have them added to the graph.

Recon-ng

Recon-ng is an OSINT tool built into Kali Linux that allows you to retrieve information like contact names, email addresses, DNS information, IP address information, and the like. Recon-ng is not as easy to use as theHarvester because it uses the module concept similar to the Metasploit framework, a modular penetration testing platform based on Ruby.

Let’s take a look at an example of Recon-ng you can use on Kali Linux. To start Recon-ng and add a workspace, use the following commands (a workspace represents a project you are working on):

recon-ng

workspaces add wiley

Now let’s add the domain names and company names to the Recon-ng database tables so that it uses them when performing all of the information gathering with future commands we use:

add domains wiley.com

add domains www.wiley.com

add domains dummies.com

add domains www.dummies.com

add companies Wiley~A publishing company

add companies Wiley Publishing~A publishing company

add companies ForDummies~A Wiley product line

To view the domains and company tables that have been populated, use these commands:

show companies

show domains

The Recon-ng tool has modules that you use to collect the different types of information from online resources.

Next, let’s collect the points of contact from Whois databases:

use recon/domains-contacts/whois_pocs

run

Now, let’s discover other domain names and hosts on the internet related to the company by using a Bing search and a Google search:

use recon/domains-hosts/bing_domain_web

run

use recon/domains-hosts/google_site_web

run

After running these commands, you can see the contact names and email addresses listed in the terminal, but it would be nice to output the information to a web page that you could use for a report. The following commands will load the reporting module and specify the creator of the report, the customer, and the report filename to generate:

use reporting/html

set CREATOR 'Glen E. Clarke'

set CUSTOMER 'Wiley Publishing'

set FILENAME /root/Desktop/Wiley_recon.html

run

If you open the HTML file on your desktop by double-clicking it, you will see a report similar to the report shown in the following figure. Keep in mind that if we would have used other modules to collect additional information (such as the IP ranges), that information would have been included in the report as well. Again, this is just a small example; know that there are a number of recon-ng modules that enable you to do things like view social media posts by an IP address.

Censys

Censys is another browser-based search engine that identifies hosts on the internet for a particular organization. In addition to identifying the hosts, Censys will also identify the services and ports that are open on those systems.

FOCA

Fingerprinting Organizations with Collected Archives (FOCA) is a tool used to scan documents to collect metadata that is typically hidden from the user. Some examples of document types that can be scanned by FOCA to extract the metadata are Microsoft Office files, Open Office files, and PDF files.

For the PenTest+ certification exam, remember that Whois, theHarvester, Maltego, Recon-ng, and Censys are all tools used for OSINT gathering.