ClearNet Security: Tag port scanning

Follow-up on using unicornscan for a big scan (400,000+ public IPs)

tate@ClearNetSec.com (Tate Hansen) — Thu, 27 Dec 2007 12:36:00 -0700

I’m happy to report our growing experience using unicornscan for large discovery sweeps is a positive one. Our confidence in using this tool has increased and it is now our preferred weapon of choice for scanning large IP swaths.

To recap: We performed a sweep of 400,000+ public IPs across multiple continents by configuring the scans to do a full TCP port scan of each IP, sustained ~55 Mbits/s using between 3 and 5 systems, and completed it in a matter of days.

This is pretty good considering by sending two SYN probes per port it meant sending ~52.5 billion packets and producing some 3 Terabytes of data.

Nmap is often our preferred tool, and we used it to spot check our results with unicornscan, but from now on it will come down to the details of the gig to make the choice.

Tech note: We avoided problems with table overflows and other like issues by placing the systems directly on the internet and with iptables turned off.

Trying out unicornscan

tate@ClearNetSec.com (Tate Hansen) — Sun, 14 Oct 2007 22:10:00 -0600

We’ve hit a new high. We’ve soaked ourselves in a bandwidth bath on behalf of a client whom would like us to discover active services across a range of six public /16 blocks plus some scattered /17s, /24s, etc. The range is close to a total of 400,000 IPs.

We started out with five dual Xeon systems running 20 to 40 instances of nmap, each tuned, and each instance targeting 64 IPs. This client wants the job completed in weeks, so we decided it was a good time to get more experience with unicornscan.

By luck, we tapped into a Danish provider that is allowing us to push 55Mbits/s. I have no idea how much that amount of bandwidth would normally cost, especially if sustaining it 24x7 for a few weeks, but I’m guessing it is way over $10,000. Our client would allow us to go up to 100Mbits/s, alas, our luck doesn’t go that far.

Anyway, we now have faster dual-core systems each pushing ~25 Mbits/s via unicornscan like so:

sudo nohup /usr/local/bin/unicornscan -mT -p –r25000 -vv xxx.zz.0.0/16:a -w unicorn.output.for.xxx.zz..0.0.fullTCP > unicorn.output.fullTCP &

We have lots of results from nmap; so far unicornscan is matching the nmap results. Having the ability to specify packets per second with unicornscan is super nice.

We’ll create a follow up post on how all our scanning worked out on this gig when we’re finished (sometime in late November).

nmap junkie?

tate@ClearNetSec.com (Tate Hansen) — Thu, 23 Nov 2006 23:02:00 -0700

We’re playing w/RoR and building a few web apps. I wrote a quick and easy one that helps you build a command line for nmap, nothing particularly special.

http://nmaptweaker.clearnetsec.com/

I did add some extra content: tuning tips, examples, and a time estimator. Basically you can use it to build a command line, then copy and paste it to wherever you want to run it.

It does not do syntax checking for nmap options which require input yet (e.g. targets, output file name, etc.), but the app does do some error checking for conflicting options.

It’s a little rough, but hey, it might save you a few minutes of time if you’re not an nmap option sage.

Déjà vu with big time port scanning

tate@ClearNetSec.com (Tate Hansen) — Tue, 22 Aug 2006 23:13:00 -0600

We have a potential pen. test project coming up with crazy numbers: 260,000+ range of publicly addressable IPs (spread over several non-contiguous blocks). Our previous experience with scanning large blocks aggressively via the internet has made us, well, super sensitive. It’s freakin’ hard.

260,000 x 65535 TCP ports x 2 (number of port query attempts) = 34,078,200,000 TCP SYN packets (nmap, default).

That is ~4x larger compared to our last "large" effort (~70,000 IPs).

No info yet on the bandwidth available per block, latency conditions, or other factors, but let’s run some numbers to see how this would work.

From recall, the Sun v40z’s were averaging ~3500pps (packets per second) outbound per server. I didn’t use packets per second to estimate previous effort, I created a quick excel chart similar to below to get a ballpark number (updated using 260,000 for the total number of IP addresses to scan):

approx. number of IP addresses to scan	260,000
number of TCP ports to query per IP address	65,535
total # of TCP ports to query	17,039,100,000

time-out setting per port query	1.25	seconds (allow 1.25 seconds for query response)
number of query attempts per port	2	which implies we need 34,078,200,000 SYN requests
total seconds per port query	2.50	seconds

number of parallel port queries	2	(we set it to 100, but empirical evidence shows this to be ~2)
total # of hours to complete single IP port scan	22.76

total number of seconds to perform scan (if sequential)	21,298,875,000
total number of hours to perform scan (if sequential)	5,916,354

number of scans in parallel	1,600	(2 servers, each running 8 unique nmap processes with min_hostgroup set to 100)
number of hours to complete scan	3,698
number of days to complete scan	154

Now I realize pps is an important factor. Remember these are still pretty fast servers: quad dual core opterons with 16 GB RAM. Doing a sanity check with our previous results of obtaining ~3500 pps per server goes as follows:

total # of SYN requests

34,078,200,000

average outbound packets per second / per server

3500

number of dedicated servers

total number of seconds to complete scan

4,868,314.29

number of hours to complete scan

1,352.31

number of days to complete scan

So the numbers don't match. The first chart says it'll take ~154 days, the second says ~56 days. In the first chart I set "number of scans in parallel" to 1600. I got that by having 8 unique nmap processes each running with min_hostgroup of 100 (800 hosts being scanned simultaneously per server; we have two dedicated scanners, so the total is 1600 hosts). The number that is probably way off is the "number of parallel port queries" in the first chart. Although we had set the value to 100 (meaning scan 100 ports simultaneously on each host) it often seemed like we were only getting 2 or so in parallel (observed from watching tcpdump output). That was probably after running into the memory issues I reported with nmap (which Fyodor subsequently reported to fix).

I just ran a few quick checks on a Sun Fire x2100 we have (dual core opteron 175 / 4 GB of RAM) using a newer nmap version (4.11). Firing off a bunch of nmap scans on one server with parameters similar to below resulted in a sustained 9000+ pps (~4.5 Mbits/s).

/usr/local/bin/nmap -vv -sS -P0 -p 1-65535 -n --min_hostgroup 100 --max_rtt_timeout 1250 --min_parallelism 100 <a_/24_block>

I didn't let the test run for very long to see if issues arise like before. If we could sustain 9000 pps per server and be allowed to push ~9Mbits/s, then the overall time is greatly reduced. It drops to ~22 days. I really doubt we'll be able to or allowed to push 9Mbits/s, but at least we now have ballpark figures to play with.

Who else has had this kind of crazy port scanning fun?

Related entry: maximizing nmap scans for accuracy

Scan fast and evade triggers

tate@ClearNetSec.com (Tate Hansen) — Tue, 14 Mar 2006 03:48:00 -0700

I've wanted to build this for a long time, alas the pain and costs of obtaining disparate public IPv4 blocks is high. I want to perform 65k port scans fast, accurately, and avoid 95% of the IDSes, IPSes, or whatever other ‘smart’ devices are in my way. It can be done.

Buy or lease some servers
Find a few data centers that connect to different Tier 1 providers
Justify and purchase IP blocks from ARIN (or another regional registry)
Setup scan server(s)
Setup NAT server(s)
Write some code to distribute port scans
Feel cool when you can scan like crazy
Feel really cool when no ‘smart’ devices alert, block, or rate limit you because you haven’t triggered any threshold ‘rules’
Act surprised when the client mentions his team didn’t see or report any anomalous behavior

Here is a high-level diagram of what I want:

Of course, there are some realities which make this hard to build. Registries prefer to hand out contiguous net blocks, but it would be far more desirable to have a bunch of smaller non-contiguous net blocks. Some ‘smart’ devices do detect scans based on the source net block, not just via a single source IP. Bandwidth and latency conditions are always in play. I still want it. A scan setup like this can increase accuracy, be fast, is distributed, and raises the difficulty for detection.

FYI: Initial costs from ARIN for different net block sizes

Category	Initial Registration Fee (US Dollars)	Assignment Size
X-small/ Micro-allocation	$1,250	/24 - < /20
Small	$2,250	/20 - /19
Medium	$4,500	> /19 - /16
Large	$9,000	> /16 - /14
X-large	$18,000	> /14