CST8602 Lab #4 Recon & Footprinting

Objectives

 To use the steps of reconnaissance and footprinting to acquire information for penetration testing
Equipment:

 Printout of lab instructions  Lab manual for notes

 Lab VMs (see below)

V 2.0 November 20, 2018 Page 1 of 12

Lab Outcome:

 To complete the lab procedures & write a report of findings

Lab Deliverables

 Answers submitted to the Lab 4 dropbox on BlackBoard by Monday, Oct 15th, 2018 11:59pm.
o Yes, you get 2 weeks to work on it – Thanksgiving is Oct 10th 
 All submissions must be in PDF, one submission per student.
 Lab exercises may be performed outside the lab environment. However, be sure to indicate if
this is the case in your lab submission as some of the information may not conform to the
expectations.

References

 http://www.pentest-standard.org/index.php/Intelligence_Gathering
 http://www.pentest-standard.org/index.php/PTES_Technical_Guidelines#Intelligence_Gathering
 http://www.vulnerabilityassessment.co.uk/Penetration%20Test.html
 http://www.exploit-db.com/google-dorks/
 CST8602 Blackboard > Supplemental Information > Writing a Consulting Report
 Other supporting web sites

Procedure:

N.B.: Follow these procedures carefully. If at any time you are unsure or are having problems,
consult your lab instructor to ensure that you are not inadvertently damaging the
equipment. Don’t be afraid or embarrassed to have the lab instructor check your work
before going on to another step.

Remember: You learn more by asking questions than by protecting your ego!

 Good luck, Mr. Hunt…

(Imagine the theme music from Mission Impossible playing here…)
Section A- The Premise
 The director of Advosys Consulting Inc. wants to find out what hackers could learn about the
company and any of its Internet resources. He has asked you to do a full reconnaissance and
footprinting of their company and Internet-facing resources.

 As the city’s best penetration tester, perform a detailed reconnaissance of the organization to find
out as much as possible about it. Find all internet resources (web sites, remote access, DNS, third
party services) that the company uses and document them in detail.

► You are only to find information:

 Passive recon only, no active recon tools are allowed.
 You do not have permission to attack any resources found OR to contact any of the
company employees directly or indirectly.
► There is no requirement to identify vulnerabilities or security issues: only to perform a full
reconnaissance using open sources of information and then perform footprinting of the company’s
Internet services as we have covered in the previous lectures.
 All about what’s visible/available through passive recon only.

 As with every consulting engagement, the client expects a professional written report describing
what you did, how you did it, and what you found that is of importance.
Section B- Rules of engagement:
► This is a reconnaissance and footprinting engagement.
 You are NOT to perform vulnerability scans or penetration testing of any resources found.
 In other words, you are only authorized to do passive recon / OSINT research
 You may NOT use Nessus or any other active tool to find vulnerabilities, or use port scanners
such as nmap and other tools that help identify running services. Passive only, people!
 Use of passive sources of vulnerability information such as “google dorks” and other search
tools that do not touch the actual services is authorized. (see references)

► Except for the exclusions above, which tools you use is entirely up to you. Document which ones
were used for each information piece found (not the how, just the what). Tool documentation should
include at least the name of the tool, who makes it, what it does and where to find out more
information about it (e.g. the URL of the vendor or primary download site).

► You are trying to gain as much information as possible about the company, it’s employees and
external Internet services as possible using only open sources of information and tools that would
available to an actual attacker. Take notes as you go, not after!

► Documentation should be in the consulting report format based on samples resources on

Blackboard. Remember that the goal of every report is to communicate with the client, not bury
them in irrelevant technical details.

► Anyone found not following the rules listed above FOR ANY REASON will lose all of
their marks for the lab.

► N.B.: We are using Kali as the main O/S for demonstration purposes only.
 There are several other distributions available, with some (e.g. Buscador) designed specifically
for OSINT.
 There are also several other OSINT tools available we will not cover in this lab.
Section C- Initial Setup
 We will not be using any of the victim VMs in this lab.
 You’ll need Kali as the attack VM
► Make sure it`s setup to connect to the Internet (i.e. BLUE network)
 Nothing in this lab is considered “hacking” - simply information gathering and research.
► Set the network interface to use DHCP
► Ensure you have proper access to the Internet before continuing any further.

 Now it just so happens that Kali has a series of OSINT tools installed by default.
 Be aware that not all tools are passive recon tools. Some are active attack//exploit tools!

 But FIRST we want to add a couple of tools.

► First the Discover scripts
 root@kali:~# cd /opt
 root@kali:/opt# git clone https://github.com/leebaird/discover.git ./discover
 root@kali:/opt# cd discover
 root@kali:/opt/discover# ./setup.shpip insta
 We’re now ready to use the scripts

► Next, let’s add SpiderFoot

 root@kali:~# cd /opt
 root@kali:/opt# git clone https://github.com/smicallef/spiderfoot.git ./spiderfoot
 root@kali:/opt# cd spiderfoot
 root@kali:/opt/spiderfoot# pip install lxml netaddr M2Crypto cherrypy mako
Might get warnings …
 root@kali:/opt/spiderfoot# apt-get install swig
 root@kali:/opt/spiderfoot# python ./sf.py 127.0.0.1:5001
Spiderfoot will start its sever and connect a browser.
 You can now access and use spiderfoot

► We also want to ensure we have the latest version of a few installed tools
 root@kali:~# apt-get update && apt-get dist-upgrade [should take care of it all]
 root@kali:~# apt-get install maltegoce
 root@kali:~# apt-get install gufw
 root@kali:~# apt-get install recon-ng
Section D- Reconnaissance
 Firstly, you should be referring to the Hacker Playbook v3
► Chapter 2 cover the Passive Recon methodology and associated tools
► I’ll be waling you thru SOME of these tools…

 Remember that not every technique and/or site will give you results.
► The goal is to try as many techniques, using as varied a toolset and sites, to obtain as much
information as “humanly” possible.

 Kali has a long list of tools installed for OSINT already

► In Kali, Applications menu (Top Left) -> 01 – Information Gathering (click on it to see the menu)
 Whole suite of tools under each menu

 Now let’s find out as much public information as possible about “Advosys Consulting Inc.”
► Use open sources of information, other reconnaissance tools and public web sites you have
learned about to find the required information about the target.
 As a minimum you should be able to document the following:
 What the company does (products and services)
 Name(s) and details about the owner(s) and key people of the company
 Type of business (sole proprietorship, private corporation, public corporation, etc.)
 Physical street address, city and country
 Phone numbers and email addresses of key personnel for potential Social Engineering
 The date they opened for business
 All sub-domain names registered to the company
 When the domains were registered, when they expire.
 All key servers associated with the primary domain and the geographic location of each.
 All access points associated with the primary company
 (optional) Document more information about the company if you find it, but only information of
potential relevance to attackers.
 All Social Media information about the key personnel chosen above and/or the company
 Etc…

► Use your judgment as to what is relevant; but remember to think like an attacker!
► Also, even though it looks like I’m pawning off the “learning” aspects to online sites and tutorials,
this is also a part of the PenTester’s skillsets – being able to learn new tools on the fly!
► An expansive tool for OSINT we can use is Recon-NG,
Recon-NG preinstalled on Kali
 Start up recon-ng in Kali and follow the instructions
 root@kali:~# recon-ng
 Don’t worry if it complains about the API keys. Recon-NG c an use a suite of API
keys (currently 16+) to access online resources directly (e.g. Google), but they must
be individually registered - https://goo.gl/Kv4CNp
 Nice tutorial on how to use Recon-NG at https://hackertarget.com/recon-ng-tutorial/
 There’s help within Recon-NG, but it’s not entirely intuitive…
 Document EVERYTHING that you find using this tool
 Can be maintained in multiple files (i.e. one for each tool), but preferably in a single
directory related to the project.

► Next let’s try SpiderFoot

 Always best to confirm information thru multiple sources!
 Nice tutorial at https://goo.gl/KZTcy2
 Spiderfoot uses several PIA keys, but it is up to you to decide about using them or not.
 Document what info – even if duplicated - you find using this tool

► Another tool  Discover scripts (originally called Backtrack-scripts) which we installed earlier
 Go to https://github.com/leebaird/discover for more info on how to use it
 The tool’s menu included is somewhat self-explanatory, however…
 Once the processes finishes, you’ll have a new directory under /root with the domain name
which contains the results.
 i.e. /opt/discover/advosys.ca/
 To view the results:
 root@kali:~# firefox /opt/discover/advosys.ca/index.htm [directory might differ]
 OR browse to the appropriate directory with the browser
 You’ll see a web page with multiple drop-down tabs at the top.
 Navigate through the tabs and pages to see what the scripts managed to find out.
You might be surprized 

► Next, let’s try theharvester and see what it can do

 root@kali:~# theharvester
command options show up
 Nice tutorial at https://goo.gl/mo1wH4
 Document any new relevant information you find
 ► Next, let’s try MetagooFil and see what it can do
 root@kali:~# metagoofil -d advosys.ca -l 20 -f all -o advosys-mgf.html -t [chosen dir]
 If an error, you may need to add it using apt-get install MetagooFil
 Not a lot of tutorials for this tool - https://tools.kali.org/information-gathering/metagoofil
 Document any new relevant information you find

► One other tool we can use to add and/or verify what you’ve found so far is dmitry (Deepmagic
Information Gathering Tool)
 Click on dmitry from the Info Gathering menu
(terminal will open with command options displayed)
 root@kali:~# dmitry -wnse -o advosys.txt advosys.ca
We will not use the –p or –f as those are active recon methods.
 Once the tool is done, compare and update the information you found previously with any new
info that it found.

 You will have noticed that it looks like the web site was removed or under construction.
► Use the Wayback Machine (http://archive.org/web/web.php) to grab screenshots of the web site
from previous years.
 This site is a prime example of why OpSec professionals tell people that “once information has
been put up on the internet, it is possible to find a copy of virtually anything, no matter if it was
deleted or removed.”
 Document each time the web site underwent a major change of appearance (i.e. a screenshot
and a date).

 Remember, these are only a sampling of the many OSINT tools available.
► Go to Page 23, Chapter 2 in the Hacker Playbook v3 for more examples & tools
► http://securitytrails.com/blog/top-20-intel-tools
Section E- Footprinting
 Now let’s see if we can discover the technical resources and infrastructure used by the company:
► Use whois, open sources of information and other reconnaissance tools you have learned about to
find the following information about the target. As a minimum you should be able to document the
following:
 All IP addresses used by the company
 Who owns those blocks of IP addresses
 Other web sites hosted by the ISP (Hint: The Bing search engine is your friend)
 Parent Internet Service Provider (the internet provider one level up from the one hosting this
company's web site)
► Visit some of the more common OSINT sites for digging deeper into this information.
 http://toddington.com/resources/
 http://www.pentest-standard.org/index.php/Intelligence_Gathering#OSINT

► Use other passive footprinting tools to perform active footprinting of the company resources
(remember the rules):
 Identify what operating system the web site is using and a percentage of how certain you are.
 Yes, it is possible to do this without using NMap if you research if carefully.
 Identify what web server software it uses.
 Again, no NMap needed.
 Identify what products provide the web site (e.g. what content management system, if any).

 Next, we can dig a little deeper to find some of the Internet facing devices available
► Shodan is the search engine for Internet connected devices
 https://www.shodan.io/
► Censys is similar to Shodan for searchable servers, but may give different results
 https://censys.io/

► Try to find all “hidden” host names the company uses by scanning their DNS:
 In Kali, you can use the Fierce domain scanner in directory /pentest/enumeration/dns/fierce
to find as many hostnames as possible set up on the company’s domain.
 There are other varied DNS tools you can use to enumerate basic DNS entries and subdomain
entries.
 ► Use ping, traceroute, various DNS tools and other resources to document the following about
each of the host names found by the above methods:
 The IP address(es) in the company’s control, and which ones are being used for what
 Whether the record is an A record or a CNAME
 Which are actually alive (have services open on them)
 The name of the Internet service provider each ISP each active service is on.
 The operating system for each active service and a percentage of how certain you are.

 Later, you may need to run nmap or a VA tool against sme of those IPs.
► The problem is, as soon as you do, you’re exposing yourself to being seen/scanned/caught by
security mechanisms and observant security admins.
► Most of the active discovery tools end up being somewhat to entirely packet loud/noisy …

 There are tools, however, that can help with maintaining the information found with active recons
► HTTPScreenshot is designed to take screenshots of web site info it discovers
► EyeWitness creates an XML file from Nmap output and screenshots webpages, RDP servers and
VNC servers.
► Metasploit also has the ability to record any Nmap or Nesus/VA scanner reports into it’s database
Section F- Private/Personal Info
 So far, most of the tools we have used are oriented towards finding “technical” information about
the company and web site(s). But that’s not the whole story of OSINT…
► Technical info is great, as it can lead to Internet facing devices and/or sites that can be
accessed to find more about the company and its resources.
► But, to put it bluntly, not much was done to research information related to the actual people
involved in the company…,
 The thinking here is that “if it’s on the Internet and accessible without protection, it’s fair
game for OSINT!!”
► You could go through every social media site and research each person, but even that’s not a
complete picture of who that person is and may be missing key information
 It is much harder to social engineer someone if you don’t have a good “picture” of their
likes/dislikes, hobbies, family and locations, etc…

► Now, I realise that for some of you this might feel a bit intrusive and/or that you’re crossing a
line between privacy and getting info.
 Well, for a PenTester, it’s a line that sometimes MUST be crossed BUT any info found is to
be protected and not let loose!

► There are sites/search engines dedicated to trying to help with this side of the puzzle
 CheckUserNames will check across 160 social networks to see if the username you are
looking for exists on each one.
 https://checkusernames.com/
 HaveBeenPwnd is a web site that searches to see if an account that has been
compromised in a data breach
 https://haveibeenpwned.com/
 https://www.toddington.com/resources/
 E.g. pipl.com search engine
 And let’s not ignore the Google & Google Earth search engines..
 Imagine finding a picture of the actual address for the person !!

► Let your imagine roam and try and find as much as you can about the principle for the
company.
Section G- Report
Q1 - Write a consulting report. detailing what you did and what you found:
 Documentation should be in the consulting report format covered in class and as described the
resources on Blackboard.
 This report will not be as long as the Lab 1 report but must as a minimum cover “The Four
Things” (see “Writing a Consulting Report”)…
 Remember that the goal of every report is to communicate with the client, not bury them in
irrelevant technical details.

 In the Findings section of the report, be sure to point out everything you found that you feel is
particularly useful to an attacker (the “significant findings”). This could include such things as:
 Overly personal information that could be used for social engineering or identity theft.
 Version numbers of any software used
 Any potential vulnerabilities identified via your passive reconnaissance or that appeared in
the output of NMap or other footprinting tool
 Use your judgment as to what is a significant finding. If in doubt, please ask.

 Your report must have an “executive summary” of no more than two pages briefing describing
The Four Things and any significant findings.
 Any additional information is to be presented and referenced as addendums to the
executive summary.