Certified Penetration Testing Specialist Notes

Network Enumeration with Nmap

Nmap is one of the most used tools by network admins and IT security specialists, it can be used to Audit the security aspects of networks, simulate penetration tests, check firewall and IDS settings and configurations, test types of possible connections, network mapping, response analysis, identify open ports, and vulnerability assessments as well. Nmap offers many different types of scans, and can basically be divided into: Host Discovery, Port Scanning, Service Enumeration and Detection, OS Detection, and scriptable interactions with the target service through the nmap scripting engine. The basic syntax for nmap is

nmap <scan types> <options> <target>

The TCP-SYN (-sS) is one of the default settings and is one of the most popular scan methods. This scan method makes it possible to scan thousands of ports per second. The TCP-SYN scan sends one packet with the SYN flag, and never completes the 3-way handshake.

-If the target responds with SYN-ACK, nmap detects the port as open

-If the packet receives an RST flag, it is an indication that the port is closed.

-If Nmap does not receive a packet back, it will display the port as filtered. Depending on the firewall configuration, certain packets may be dropped or ignored by the firewall.

Host Discovery

A basic scan that can be used to search a subnet for alive hosts is

sudo nmap 10.129.2.0/24 -sn -oA tnet | grep for | cut -d" " -f5

where -sn disables port scanning, and -oA tnet will output the results in all formats starting with the name tnet. We can also scan hosts from a list using the -iL hosts.txt option, where hosts.txt is a list of hosts, 1 per line. If we disable the port scan (with -sn), Nmap automatically will ping scan with ICMP Echo Requests (-PE), where nmap will expect an ICMP Reply if the host is alive. We can confirm this with the --packet-trace option to ensure the ICMP echo requests are sent. Another way to determine why nmap has marked a host alive is with the --reason option.

Host and Port Scanning

There are a total of 6 different states for scanned ports that we can obtain.

State	Description
Open	This indicates that the connection to the scanned port has been established. These connections can be TCP connections, UDP datagrams as well as SCTP associations.
Closed	When the port is shown as closed, the TCP protocol indicates that the packet we received back contains an RST flag. This scanning method can also be used to determine if our target is alive or not.
Filtered	Nmap cannot correctly identify whether the scanned port is open or closed because either no response is returned from the target for the port or we get an error code from the target.
Unfiltered	This state of a port only occurs during the TCP-ACK scan and means that the port is accessible, but it cannot be determined whether it is open or closed.
Open|Filtered	If we do not get a response for a specific port, Nmap will set it to that state. This indicates that a firewall or packet filter may protect the port.
Closed|Filtered	This state only occurs in the IP ID idle scans and indicates that it was impossible to determine if the scanned port is closed or filtered by a firewall.

We can use the --top-ports=x flag to scan for x top ports, -p 21 to scan only port 21, or -p- to scan all ports, or -F for top 100 ports (fast scan). Also -n can be used to disable DNS resolution.

Connect Scan

The nmap TCP Connect Scan (-sT) uses the TCP 3-way handshake to determine if a specific port on a target is open or closed. The Connect Scan is useful because it is the most accurate way to determine the state of a port, and it is also the most stealthy, unlike other types of scans, such as the SYN scan, the connect scan does not leave any unfinished connections or unsent packets on the target host, which makes it less likely to be detected by IDS or IPS, and is useful when you want to scan a network but don’t want to disturb the services running behind it.

Filtered Ports

When a port is shown as filtered, it can be shown this way for several reasons, in most cases, firewalls have certain rules set to handle specific connections, The packets can either be dropped or rejected. When a packet gets dropped, nmap receives no response from our target, and marks the port as filtered. These will require further investigation later on, but for now there is no way to interact with the port.

Discovering Open UDP Ports

Since UDP is a stateless protocol, we do not receive any acknowledgment, consequently, the timeout is much longer, making UDP scans (-sU) Much slower than TCP scans. If the UDP port is open, we will only get a response if the application is configured to do so. If we get an ICMP response with error code 3 (port unreachable) we know that the port is indeed closed.

Saving Results

-oN for Normal output, -oG for Grepable output, and -oX for XML output. or -oA for all formats. With the XML output, we can easily create HTML reports that are useful for documentation as it presents the results in a detailed and clear way. To convert from XML to HTML we can use the tool xsltproc as such:

xsltproc target.xml -o target.html

Service Enumeration

-sV can be used to enumerate versions, after a successful 3-way handshake, the server often sends a banner for identification. This serves to let the client know which service it is working with. At the Network level, this happens with a PSH flag in the TCP header. However, some services may not provide this information immediately. We can use nc -nv ip port to grab the banner.

An example packet capture looks like this

18:28:07.128564 IP 10.10.14.2.59618 > 10.129.2.28.smtp: Flags [S], seq 1798872233, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 331260178 ecr 0,sackOK,eol], length 0

18:28:07.255151 IP 10.129.2.28.smtp > 10.10.14.2.59618: Flags [S.], seq 1130574379, ack 1798872234, win 65160, options [mss 1460,sackOK,TS val 1800383922 ecr 331260178,nop,wscale 7], length 0

18:28:07.255281 IP 10.10.14.2.59618 > 10.129.2.28.smtp: Flags [.], ack 1, win 2058, options [nop,nop,TS val 331260304 ecr 1800383922], length 0

18:28:07.319306 IP 10.129.2.28.smtp > 10.10.14.2.59618: Flags [P.], seq 1:36, ack 1, win 510, options [nop,nop,TS val 1800383985 ecr 331260304], length 35: SMTP: 220 inlane ESMTP Postfix (Ubuntu)

18:28:07.319426 IP 10.10.14.2.59618 > 10.129.2.28.smtp: Flags [.], ack 36, win 2058, options [nop,nop,TS val 331260368 ecr 1800383985], length 0

Where S is SYN, S. is SYN-ACK, . is ACK, P. is PSH-ACK, and . ACK confirms the receipt of the data at the end.

Nmap Scripting Engine

There are a total of 14 categories of scripts

Category	Description
Auth	Determination of authentication credentials.
Broadcast	Scripts, which are used for host discovery by broadcasting and the discovered hosts, can be automatically added to the remaining scans.
brute	Executes scripts that try to log in to the respective service by brute-forcing with credentials.
default	Default scripts executed by using the -sC option.
discovery	Evaluation of accessible services.
dos	These scripts are used to check services for denial of service vulnerabilities and are used less as it harms the services.
exploit	This category of scripts tries to exploit known vulnerabilities for the scanned port.
external	Scripts that use external services for further processing.
fuzzer	This uses scripts to identify vulnerabilities and unexpected packet handling by sending different fields, which can take much time.
intrusive	Intrusive scripts that could negatively affect the target system.
malware	Checks if some malware infects the target system.
safe	Defensive scripts that do not perform intrusive and destructive access.
version	Extension for service detection.
vuln	Identification of specific vulnerabilities.

There are several way to define the desired scripts in nmap

Default Scripts

sudo nmap <target> -sC

Specific Scripts Category

sudo nmap <target> --script <category>

Defined Scripts

sudo nmap <target> --script <script-name>,<script-name>,...

Nmap Aggressive scan

sudo nmap <ip> -p <port> -A

-A will perform service detection, OS detection, traceroute, and uses default scripts to scan the target

Vulnerability Assessment

sudo nmap <port> -p <ip> -sV --script vuln

These scripts interact with the service and find out more information about their versions and check various databases to see if there are known vulnerabilities.

Performance

There are many options for telling Nmap how fast (-T <1-5>), with which frequency (--min-parallelism <number>), which timeouts (--max-rtt-timeout <time>) the test packets should have, how many packets should be sent simultaneously (--min-rate <number>), and with the number of retries (--max-retries <number>) for the scanned ports that the targets should be scanned.

Timeouts

WHen nmap sends a packet, it takes some time (RTT=Round Trip Time) to receive a response from the scanned port. Generally nmap starts with a high timeout of 100ms. 1 way to speed up a scan is by:

sudo nmap 10.129.2.0/24 -F --initial-rtt-timeout 50ms --max-rtt-timeout 100ms

Were --initial-rtt-timeout 50ms sets the specified time value as initial RTT timeout, and --max-rtt-timeout 100ms sets the specified time value as maximum RTT timeout. However, too short of a time period may cause us to miss hosts in the output, so this exact command is not recommended.

Max Retries

Another way to increase the scans’ speed is to specify the retry rate. The default value is 10 but we can reduce this as such, which again can negatively impact the results:

sudo nmap 10.129.2.0/24 -F --max-retries 0

Rates

During an engagement, we might get whitelisted, and the network might have ample bandwidth to support such scanning, we can really speed things up with:

sudo nmap 10.129.2.0/24 -F -oN tnet.minrate300 --min-rate 300

Timing

nmap offers 6 different timing templates -T <0-5> for us to use. The higher the value, the more aggressive the scan. We can experience negative effects if we scan too fast, like missing hosts or getting picked up/blocked by security systems. The default timing nmap uses is -T 3.

• -T 0 / -T Paranoid

• -T 1 / -T Sneaky

• -T 2 / -T Polite

• -T 3 / -T Normal

• -T 4 / Aggressive

• -T 5 / Insane

Firewall and IDS/IDP Evasion

Firewalls are software components that monitor network traffic between the firewall and incoming data connections, and decides how to handle the connection based on pre-defined rules that have been set. Similarly, an IDS will scan the network for potential attacks, analyze them, and report any detected attacks. An IPS compliments an IDS by taking specific defensive measures, should an attack be detected. The analysis of such attacks is based on pattern matching and signatures.

Determine Firewalls and Their Rules

When a port is shown as filtered, it can have several reasons. In most cases, firewalls have certain rules set to handle specific connections. The packets can either be dropped or rejected. dropped packets are ignored, and no response is returned from the host. On the other hand, rejected packets are returned with an RST flag. These packets contain different types of ICMP error codes, or contain nothing at all. Such errors can be:

• Net Unreachable

• Net Prohibited

• Host Unreachable

• Host Prohibited

• Port Unreachable

• Port Prohibited

Nmap’s TCP ACK (-sA) method is much harder to filter for firewalls and IDS/IPS systems compared to regular SYN (-sS) or (-sT) because they send a TCP packet with only the ACK flag. Packets with the ACK flag are usually passed by the firewall because it can not determine whether the connection was first established from the external or the internal network.

Detect IDS/IPS

Detection of IDS/IPS is much more difficult because these are passive traffic monitoring systems. If you get picked up by an IDS/IPS, proceeding with the test will be much more difficult, as your ISP may be notified and your internet will be revoked, or you may just be blocked from the target environment. Ideally, you would have access to several VPS’s with different IPs to determine whether such systems are on the target network during a pentest, if one gets blocked, you would need to use the others in a more careful and stealthy manner.

Decoys

There are cases in which admins block specific subnets from different regions. This prevents any access to the target network. Another example is if an IPS exists and blocks us. For this reason, the Decoy Scanning Method (-D) is the right choice. With this method, nmap generates various random IP addresses inserted into the IP header to disguise the origin of the packet sent. It is critical that the decoys must be alive, otherwise the service on the target may be unreachable due to SYN-flooding security mechanisms.

sudo nmap 10.129.2.28 -p 80 -sS -D RND:5

The RND:5 generates five random IP addresses that indicates the source IP the connection comes from.

We can also use -S to specify a source IP, because it may be possible that specific subnets are blocked from accessing the resource, we can try our luck with another:

sudo nmap 10.129.2.28 -n -Pn -p 445 -O -S 10.129.2.200 -e tun0

DNS Proxying

DNS queries are traditionally made over UDP port 53, however due to IPv6 and DNSSEC expansions, many DNS requests are made over TCP port 53 now. Nmap gives us a way to specify DNS servers ourselves (--dns-server <ns>,<ns>). This method could be important to us if we are in a DMZ. The company’s DNS servers are usually more trusted than online ones, so for example we could use them to interact with the hosts of the internal network. We could make TCP port 53 our source port (--source-port) for our scans. If an admin uses the firewall to control this port and does not filter IDS/IPS properly, our TCP packets will be trusted and passed through.

Connect to the Filtered Port

If we discover that the firewall allows traffic from TCP port 53, we can use netcat to establish a connection as such

ncat -nv --source-port 53 <ip> <port>

Footprinting

FTP

Client and Server establish control channel through TCP port 21. The client sends commands to the server, and the server responds status codes, then both the client and server can establish a data channel over TCP port 20, which is used exclusively for data transmission.

TFTP

Trivial File Transfer Protocol is simpler than FTP and performs file transfers between client and server processes, however it does not provide user authentication and other features supported by FTP. While FTP uses TCP, TFTP uses UDP, making it unreliable for data transfers. Here are a few commands in TFTP:

Commands	Description
Connect	Sets the remote host, and optionally the port, for file transfers
Get	Transfers a file or set of files from the remote host to the local host
Put	Transfers a file or set of files from the local host onto the remote host
Quit	exits TFTP
Status	Shows the current status, like transfer mode (ascii or binary), connection status, time-out value, and so on
Verbose	Turns verbose mode on, which displays additional information

Unlike FTP, TFTP does not have a directory listing functionality

Default Configuration of vsFTPd

One of the most used FTP servers in linux-based distros is vsFTPd. The default config can be found in /etc/vsftpd.conf

Setting	Description
listen=NO	Run from inetd or as a standalone daemon?
listen_ipv6=YES	Listen on IPv6 ?
anonymous_enable=NO	Enable Anonymous access?
dirmessage_enable=YES	Display active directory messages when users go into certain directories?
use_localtime=YES	Use local time?
xferlog_enable=YES	Activate logging of uploads/downloads?
connect_from_port_20=YES	Connect from port 20?
secure_chroot_dir=/var/run/vsftpd/empty	Name of an empty directory
pam_service_name=vsftpd	This string is the name of the PAM service vsftpd will use.
rsa_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem	The last three options specify the location of the RSA certificate to use for SSL encrypted connections.
rsa_private_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
ssl_enable=NO

There is also a file /etc/ftpusers that is used to deny certain users access to FTP service. Users in this file are not allowed to log in to the FTP service.

Dangerous Settings

One of the most common things to test for when testing FTP is anonmyous access, these settings look like this:

Setting	Description
anonymous_enable=YES	Allowing anonymous login?
anon_upload_enable=YES	Allowing anonymous to upload files?
anon_mkdir_write_enable=YES	Allowing anonymous to create new directories?
no_anon_password=YES	Do not ask anonymous for password?
anon_root=/home/username/ftp	Directory for anonymous.
write_enable=YES	Allow the usage of FTP commands: STOR, DELE, RNFR, RNTO, MKD, RMD, APPE, and SITE?

Logging in Anonymously

ftp anonymous@<ip address>

Useful Commands and Settings

A useful setting for protecting FTP is hide_ids=YES, which means the UID and GUID representation of the service will be overwritten, making it difficult for us to see the rights that these files have. This prevents local usernames from being revealed.

Another useful setting is ls_recurse_enabled=YES, if this is enabled we can do ls -R which will give us a better overview of the FTP directory structure.

We can download a file with Get File\ Name.txt where the name of the file is File Name.txt, the \ is used to escape the space.

We can also use wget to download all files recursively as such:

wget -m --no-passive ftp://anonymous:anonymous@<ip address?

Or once logged in, we can prompt OFF, recurse ON and mget *

We can Upload files with the put command, like this put <filename>.ext

Footprinting FTP

We can start by updating the nse database as such

sudo nmap --script-updatedb

All the NSE scripts are generally located in /usr/share/nmap/scripts/, or we can find them with this simple command

find / -type f -name ftp* 2>/dev/null | grep scripts

Which will return all nmap scripts that start with ftp.

Service Interaction

nc -nv <ip> 21

telnet <ip> 21

It is a little different if the FTP server runs with TLS/SSL encryption, as we need a client that can handle that. We can use openssl as such.

openssl s_client -connect <ip>:21 -starttls ftp

SMB

Useful commands for enumerating SMB are:

#anonymously list shares
smbclient -L -N \\<IP/FQDN>\ 

and 
#connect to a share anonymously
smbclient -N \\<IP/FQDN>\<share>

We can also use rpcclient :

rpcclient -U "" <ip>

Then some usual commands are,

srvinfo-> Server information.

enumdomains-> Enumerate all domains that are deployed in the network.

querydominfo-> Provides domain, server, and user information of deployed domains.

netshareenumall-> Enumerates all available shares.

netsharegetinfo <share>-> Provides information about a specific share.

enumdomusers-> Enumerates all domain users.

queryuser <RID>-> Provides information about a specific user.

We can do enumdomusers, grab the user and group rids, then queryuser <rid> or querygroup rid

We can also use a tool from impacket samrdump.py to bruteforce RIDS as such:

samrdump.py <ip>

Another useful tool is enum4linux, which we can use as

./enum4linux-ng <ip> -A

NFS

Network File System has the same purpose as SMB, but it is only used between Linux and Unix systems, unlike SMB/Samba which are cross compatible with Windows/AD and Linux.

We can footprint the service with nmap

sudo nmap <ip> -p111,2049 -sC -sV

the rpcinfo NSE script retrieves a list of all currently running RPC services, their names and description, and the ports they use. Nmap also has some NSE scripts that we can run on NFS as such:

sudo nmap --script nfs* <ip> -sV -p111,2049

to show available NFS shares:

showmount -e 10.129.14.128

And then to mount the share

mkdir target-NFS
sudo mount -t nfs <ip>:/ ./target-NFS/ -o nolock
cd target-NFS
tree .

To unmount

cd ..
sudo umount ./target-NFS

DNS

There are several types of DNS servers:

DNS Root Server- The root servers of the DNS are responsible for the top-level domains (TLD). As the last instance, they are only requested if the name server does not respond. Thus, a root server is a central interface between users and content on the Internet, as it links domain and IP address. The Internet Corporation for Assigned Names and Numbers (ICANN) coordinates the work of the root name servers. There are 13 such root servers around the globe.

Authoritative Nameserver- Authoritative name servers hold authority for a particular zone. They only answer queries from their area of responsibility, and their information is binding. If an authoritative name server cannot answer a client’s query, the root name server takes over at that point.

Non-authoritative Nameserver- Non-authoritative name servers are not responsible for a particular DNS zone. Instead, they collect information on specific DNS zones themselves, which is done using recursive or iterative DNS querying.

Caching DNS Server- Caching DNS servers cache information from other name servers for a specified period. The authoritative name server determines the duration of this storage.

Forwarding Server- Forwarding servers perform only one function: they forward DNS queries to another DNS server.

Resolver- Resolvers are not authoritative DNS servers but perform name resolution locally in the computer or router.

There are also different DNS records used for DNS queries:

A- Returns an IPv4 address of the requested domain as a result.

AAAA-Returns an IPv6 address of the requested domain.

MX-Returns the responsible mail servers as a result.

NS-Returns the DNS servers (nameservers) of the domain.

TXT-This record can contain various information. The all-rounder can be used, e.g., to validate the Google Search Console or validate SSL certificates. In addition, SPF and DMARC entries are set to validate mail traffic and protect it from spam.

CNAME-This record serves as an alias. If the domain www.hackthebox.eu should point to the same IP, and we create an A record for one and a CNAME record for the other.

PTR- The PTR record works the other way around (reverse lookup). It converts IP addresses into valid domain names.

SOA- Provides information about the corresponding DNS zone and email address of the administrative contact.

Footprinting DNS

NS Query

dig ns <domain>.com @<dns IP>

Sometimes it is possible to query a DNS server’s version using a class CHAOS type query and type TXT. this entry must exist on the DNS server to work:

dig CH TXT version.bind <dns ip>

We can use the ANY to view all avaialbe records, which causes the service to disclose all available entries that it can share.

dig any <domain>.com @<dns ip>

Zone Transfer refers to the transfer of zones to another server in DNS, which usually happens over TCP port 53. This is abbreviated Asynchronous Full Transfer Zone (AXFR).

dig axfr <domain>.com @<dns ip>

or for internal

dig axfr internal.<domain>.com @<dns-ip>

We can also use a for loop in bash to query for individual A records.

for sub in $(cat /opt/useful/SecLists/Discovery/DNS/subdomains-top1million-110000.txt);do dig $sub.inlanefreight.htb @10.129.14.128 | grep -v ';\|SOA' | sed -r '/^\s*$/d' | grep $sub | tee -a subdomains.txt;done

Another tool that can do this is DNSenum

dnsenum --dnsserver <dns-ip> --enum -p 0 -s 0 -o subdomains.txt -f /opt/useful/SecLists/Discovery/DNS/subdomains-top1million-110000.txt <target domain>.com

SMPT

To interact with SMTP, we can use telnet to initialize a TCP connection.

telnet <ip>

Helo mail1.inlanefreight.htb

Depending how the server is configured, we can use the VRFY command to enumerate users

telnet <ip> 25

VRFY root

To send an email

telnet <ip> 25

EHLO inlanefreight.htb

MAIL FROM: <goober@goober.htb>

RCPT TO: <goober2@goober.htb> NOTIFY=success,failure

DATA

QUIT

Footprinting SMPT

sudo nmap <ip> -sC -sV -p25

We can also use NSE’s smtp-open-relay script to identify if the target server can be used as an open relay.

sudo nmap <ip> -p25 --script smtp-open-relay