Déjà vu with big time port scanning
Posted by Tate Hansen Wed, 23 Aug 2006 05:13:00 GMT
We have a potential pen. test project coming up with crazy numbers: 260,000+ range of publicly addressable IPs (spread over several non-contiguous blocks). Our previous experience with scanning large blocks aggressively via the internet has made us, well, super sensitive. It’s freakin’ hard.
260,000 x 65535 TCP ports x 2 (number of port query attempts) = 34,078,200,000 TCP SYN packets (nmap, default).
That is ~4x larger compared to our last "large" effort (~70,000 IPs).
No info yet on the bandwidth available per block, latency conditions, or other factors, but let’s run some numbers to see how this would work.
From recall, the Sun v40z’s were averaging ~3500pps (packets per second) outbound per server. I didn’t use packets per second to estimate previous effort, I created a quick excel chart similar to below to get a ballpark number (updated using 260,000 for the total number of IP addresses to scan):
| approx. number of IP addresses to scan | 260,000 |
|
| number of TCP ports to query per IP address | 65,535 |
|
total # of TCP ports to query |
17,039,100,000 |
|
| time-out setting per port query | 1.25 |
seconds (allow 1.25 seconds for query response) |
| number of query attempts per port | 2 |
which implies we need 34,078,200,000 SYN requests |
total seconds per port query |
2.50 |
seconds |
| number of parallel port queries | 2 |
(we set it to 100, but empirical evidence shows this to be ~2) |
| total # of hours to complete single IP port scan | 22.76 |
|
| total number of seconds to perform scan (if sequential) | 21,298,875,000 |
|
| total number of hours to perform scan (if sequential) | 5,916,354 |
|
| number of scans in parallel | 1,600 |
(2 servers, each running 8 unique nmap processes with min_hostgroup set to 100) |
| number of hours to complete scan | 3,698 |
|
| number of days to complete scan | 154 |
Now I realize pps is an important factor. Remember these are still pretty fast servers: quad dual core opterons with 16 GB RAM. Doing a sanity check with our previous results of obtaining ~3500 pps per server goes as follows:
| total # of SYN requests | 34,078,200,000 |
|
| average outbound packets per second / per server | 3500 |
|
| number of dedicated servers | 2 |
|
| total number of seconds to complete scan |
|
|
| number of hours to complete scan |
|
|
| number of days to complete scan | 56 |
So the numbers don't match. The first chart says it'll take ~154 days, the second says ~56 days. In the first chart I set "number of scans in parallel" to 1600. I got that by having 8 unique nmap processes each running with min_hostgroup of 100 (800 hosts being scanned simultaneously per server; we have two dedicated scanners, so the total is 1600 hosts). The number that is probably way off is the "number of parallel port queries" in the first chart. Although we had set the value to 100 (meaning scan 100 ports simultaneously on each host) it often seemed like we were only getting 2 or so in parallel (observed from watching tcpdump output). That was probably after running into the memory issues I reported with nmap (which Fyodor subsequently reported to fix).
I just ran a few quick checks on a Sun Fire x2100 we have (dual core opteron 175 / 4 GB of RAM) using a newer nmap version (4.11). Firing off a bunch of nmap scans on one server with parameters similar to below resulted in a sustained 9000+ pps (~4.5 Mbits/s).
/usr/local/bin/nmap -vv -sS -P0 -p 1-65535 -n --min_hostgroup 100 --max_rtt_timeout 1250 --min_parallelism 100 <a_/24_block>
I didn't let the test run for very long to see if issues arise like before. If we could sustain 9000 pps per server and be allowed to push ~9Mbits/s, then the overall time is greatly reduced. It drops to ~22 days. I really doubt we'll be able to or allowed to push 9Mbits/s, but at least we now have ballpark figures to play with.
Who else has had this kind of crazy port scanning fun?
Related entry: maximizing nmap scans for accuracy
