maximizing nmap scans for accuracy
There have been some recent newsgroup postings about consistency problems with nmap results (especially when scanning over the internet). In my experience, if you want the best chance of obtaining accurate nmap scan results, you need to:
- check the latency to your targets (ping some device on the target net block for awhile, what is the average rtt value?)
- use the average to configure the --max_rtt_timeout nmap option (e.g. if your ping time is 300ms, you obviously want to set this option to more than 300ms to prevent nmap for missing an open port if/when a probe takes longer than 300ms. Maybe set it to 500ms or even higher. Speed suffers, but we're chasing accuracy)
- if you can, check the available bandwidth limit for you and the target net (e.g. T3:45mbps, T1:1.544mbps, DSL, etc.)
- use this to configure the --min-hostgroup and --max-hostgroup nmap options and the --min-parallelism and --max-parallelism options (i.e. the more bandwidth each side has available, the more hosts and ports you can scan in parallel). This can significantly reduce overall scan time while keeping accuracy as the highest priority. I typically use IPTraf to watch pps (packets per second) to guage my bandwidth usage. If my options are too aggressive (i.e. consuming too much bandwidth), I'll kill the current scan and start over with different tuning values. The objective here is to set --min-hostgroup and min-parallelism to something that both sides can handle without sacrificing accuracy. If bandwidth permits, I like to set min_parallelism to 100 (number of parallel SYN requests or whatever scan type you're doing). I've gone up to 300 for --min-hostgroup (had lots of bandwidth and plenty of server horsepower). You don't want to cause your scan server, the target, or any device the probes traverse to drop packets, so the more information you have about all the components involved the better you'll be able to tune these values.
- if you can, verify there are no 'smart' devices impeding scans (e.g. IPS, rate limiting device, etc.)
- run multiple nmap scans of the same target, preferably at different times of the day and different days of the week (e.g. nightly backups can saturate certain hosts/links which causes nmap to fail to discover open ports). I like to run multiple services scans (~1600 ports) and do one or two 65k scans depending on circumstances. If time permits, the confidence boost you get knowing you've maximized accuracy by running repeated scans is satisfying. It also shows your assiduity; you can tell your customer you fired off 5 scans at different times to obtain your final list of exposed services. If you missed something after repeated scans (assuming you used safe tuning values), something is likely wrong.
- set --host_timeout to something you are comfortable with.
- if you determine a scan may take 20 hours to perform a 65k port scan of the target device, then set this option accordingly. Note: you can use 's', 'm', or 'h' to specify seconds, minutes, or hours.
- set --max-scan-delay to something low if nothing is impeding scans
- I typically set this option to 0 because I'm confident in my other tuning values (no delay between probes, just fire them off)
While tuning nmap options can do lots of good things it is still important to verify your scan server can keep up. We've knocked down 8-way Opteron systems w/16GB of RAM, so optimal nmap options may not be realistic for your hardware.
OpenPGP smartcards
I've been playing with a batch of OpenPGP cards that I got a while back. Very cool. You can use it for secure storage of your normal PGP keys but you can also integrate it with PAM and SSH to use it for securing logins to your computer. Passwords aren't enough anymore.
I haven't got it working with OS X yet but it looks like a pretty straight forward operation to get it setup.
Here is some more info.
FireBug
Sloshing around in rails and playing with AJAX is fun and all, but we hit a roadblock the other day when trying to view the html source of AJAX rendered components. You can't just hit cntrl-u in Firefox to see the source. After some googling, FireBug is the tool that rocks.
Not only is this helpful for debugging web applications, but it'll help when performing web assessments. I particularly liked the feature which highlights the elements on the html page when you select the corresponding code in the FireBug console.FireBug lets you explore the far corners of the DOM by keyboard or mouse. All of the tools you need to poke, prod, and monitor your JavaScript, CSS, HTML and Ajax are brought together into one seamless experience, including an error console, command line, and a variety of fun inspectors.
tough to find where to begin
I shouldn't be shocked, but I am. A piece of the conversation today we had with a client went something like this:
client: yeah, we also just found out we have an ex-employee logging in from the internet to our servers and helping other nurses with some computer tasks
us: um, you have an ex-employee logging into your servers remotely?
client: yes
Talk about scary. I wish I could say more. Let's just say this is relatively minor compared to other illegitimate activities this particular client is suffering from (e.g. knowledgeable attackers with clear targets). It is quickly turning into one of those scenarios whereby you can’t trust the integrity of anything electronic.
On top of that, it’s another flare on why it is so important to just know what is and should be happening on your network. Forget about all the fancy security solutions; what is important first is to understand why and how devices talk. Do these systems over here need to talk to these systems here? No. Why are they talking then?
This client has security point solutions in place, but they haven’t a clue what is happening or why. If you spend the time to define the relationships, catching potentially illegitimate activity is a LOT easier.
idealism vs realism debate, great points
Here is a great post about security products and the idealist vs the realist. Below are two snippets, but go read it, it's good.
"Idealist : all security products designed to stop attacks/attackers are useless and snake oil, because a skilled enough attacker can always evade the HIPS/evade the NIPS/defeat the heap protection/own you."
"Realist : security products are useful and worth purchasing because they can stop unskilled attackers armed with off the shelf (freely downloadable) exploit frameworks like metasploit (although hd's recent talk at cansec stated that the new nips evasion techniques evade almost every product) and they stop actual malware as seen on the internets."
bug tracking, web-based tracking tools
http://geekswithblogs.net/flanakin/articles/CompareWebTrackers.aspx
"wacky govt project"
Dave Aitel referenced an interesting government project I hadn't heard about until reading a posting by him on dailydave. The project: http://cryptome.org/traceback.htm. To spark your interest, I grabbed some snippets:
- " We seek to develop tools and techniques for the traceback of attacks carried out over information networks to their originating source."
- " We are soliciting research that will significantly improve the science and practice of network traceback, and are seeking tools and techniques to increase our knowledge of the true source of an attack. We are seeking solutions for both IPv4 and IPv6 networks. We are particularly interested in tracing attacks involving confidentiality and integrity of information on IC networks. Therefore techniques designed for tracing anonymous packet flooding attacks causing denial-of-service (DDOS and DOS) in IC networks back to their source are not of interest."
- "We are focused on tools and techniques for tracing attacks that involve single packets, encrypted payloads, "stepping stones" (compromised hosts), and similar attack attributes. We are seeking traceback solutions that perform in one or more of the following network environments: cooperative, non-cooperative, and hostile. Solutions for the non-cooperative and hostile network scenarios are of particular interest. We seek to develop a suite of IP traceback techniques that require:
- Low to no Internet Service Provider (ISP) involvement
- A minimum number of packets for traceback to include a single packet for certain traceback scenarios
- Low memory requirements
- Little or no overhead to the router."
- White papers should identify what concealment methods their proposed tool can use to mask its operation, and what concealment methods used by an adversary it can overcome. An adversary's obfuscation techniques can include:
- Introducing random delays before a packet departs from a stepping stone
- Inserting chaff (padding) into stepping stone connections
- Encrypted payloads
- Single packet triggers for prepositioned malicious software
- Spoofed IP addresses.
- We are specifically interested in traceback techniques that can operate under one or more of the following conditions:
Can perform without violating current protocol semantics
- Can perform without changes in the core routing structure
- Are difficult to detect and evade by the attacker
- Are useful for asymmetric communications (i.e., half duplex, in which only one direction of the communication is available)
- Can operate in a passive mode, without requiring interventions
- Are likely to be preserved across a long connection of stepping stones
- Work through multiple internet hops, across jurisdictions and with non-cooperative or hostile Internet Service Providers (ISPs)
- Can be performed without requiring interactive operational support from ISPs
- Can be performed "post-mortem" after an attack has completed
- Can be efficiently implemented and incrementally deployable.
- We are seeking techniques that can trace the origin of a single IP packet delivered by a TCP/IP network in the recent past. The techniques to track individual packets in a network must be accomplished in an efficient, scalable fashion.
Internet Routing Tables, BGP, and lots of numbers.
Since working for Quova back in 2000, I've watched the default internet routing table grow from ~84,000 routes to 186,545 routes. A great mailing list to subscribe to for keeping abreast of statistics based on BGP summaries is [bgp-stats]. This APNIC page lets you subscribe to [bgp-stats] and you can learn about other APNIC mailing lists on this page.
One of the values I watched was the 'Number of addresses announced to Internet'. Currently it is 1,505,834,848 IP addresses. It was important at the time because Quova attempts to map every public IP address to a physical location. You can see % of available address space allocated, % of address space announced, and % of available address space announced. Anyway, chock-full of sometimes interesting numbers.
Below is a snippet (only the top section) of a single full analysis report:
Analysis Summary
----------------BGP routing table entries examined: 186545
Prefixes after maximum aggregation: 103149
Unique aggregates announced to Internet: 91293
Total ASes present in the Internet Routing Table: 21958
Origin-only ASes present in the Internet Routing Table: 19079
Origin ASes announcing only one prefix: 9097
Transit ASes present in the Internet Routing Table: 2879
Transit-only ASes present in the Internet Routing Table: 69
Average AS path length visible in the Internet Routing Table: 4.5
Max AS path length visible: 24
Prefixes from unregistered ASNs in the Routing Table: 9
Special use prefixes present in the Routing Table: 0
Prefixes being announced from unallocated address space: 10
Number of addresses announced to Internet: 1505834848
Equivalent to 89 /8s, 193 /16s and 55 /24s
Percentage of available address space announced: 40.6
Percentage of allocated address space announced: 59.9
Percentage of available address space allocated: 67.8
Total number of prefixes smaller than registry allocations: 92023
To see the full analysis with all kinds of interesting information, here is the report for Sunday April 16th.
Forensic tools
I did some recent forensic work on a Terminal Server and I found NetAnalysis from Digital Detective a great tool to quickly analyze users' internet browsing activities. Not to mention it is relatively cheap (~$200) compared to the more popular commercial forensics tools. One of my ex-colleagues performs forensic work full-time and provided me his hit list of preferred tools (what he uses 90% of the time):
EnCase: what is cool is you can mount via EnCase and boot an image in VMWare
AccessData's FTK: good for email and quick searching, and has a protected storage viewer so you can reveal passwords stored by IE, Outlook, list Autocomplete strings and passwords, etc.
Snapview, UltraEdit, IrfanView, SMART
Friday Fun
Friday Fun Crossword Puzzle
I have been reading the Head First series of books and they have been very entertaining. I would say they are the most exciting technical books I have read. So in the spirit of learning through fun I have created a crossword puzzle for you guys to solve. It is compiled from a hodge podge of facts and should be fun to solve.
Good luck, and here are the answers.
Apple Mania
Sun Fire T2000 Server Review: Encryption Routines
I'm no fan boy; in fact I seem to carry a good dose of skepticism around with me any more. I like science, I like numbers, I like facts, it's really really easy to talk and it's usually even easier to pull out some numbers and then you don't need to talk. So few people do it though, this whole industry is filled with sales people and talk a lot's. I'm a Sun shareholder but I'm also a former IBMer and I know the difference. I'm not a shill but I'm also not a hater, I won't just advertise for Sun or Dell like Tate, either. So I'm helping to kick around a Niagara, uncut, my opinions and observations. Maybe some Sun people will hear me and it'll put a fire under them because frankly, my first impressions aren't so great.
Test 1, openssl speed. I know this is not fair, the Niagara is "optimized for threads." It doesn't have a great deal of cache (relatively speaking) and it doesn't have a lot of the out of order stuff many modern processors have, but it's late, I haven't slept a lot the last few days and it's easy. All things being equal, I'd personally still expect decent results from the machine.
I had grandiose plans of testing on many of the machines I have around here but my old imac in my kitchen that primarily is a browsing and email machine did a good enough job so I stopped there for now and never got to any of the fast machines.
Machine 1: Sun Fire T2000, 8 cores and costs around $15,000.
Machine 2: iMac, G5 1.6Ghz, 1GB of RAM. "The Breakfast nook machine", costs about $1,000 give or take.
FYI, "/usr/sfw/bin/openssl speed" segfaults out of the box. I'm not sure what the rules on this machine or what all has been done to it; I assume that's how it came though. Maybe someone else can try it on their T2000. If you give it an argument to run a specific test it will work.
As you can see, my dinky old iMac blew this beast clean out of the water, across the board. Again, this isn't a terribly fair test but I was a little shocked and I didn't even get out the big mac or opteron machines. FWIW, Aqua is running and the T2000 is damn near idle, I might have a Norton AV scan going on too, I have 2 browsers up, OpenOffice, 3 terminals and iTunes all running also.
So some observations, first, gcc didn't work real well yet so I haven't rebuilt openssl on the T2000 to provide optimized numbers, I'd assume that Sun did an okay job of building it though while Apple clearly didn't (but I chalk up to Apple supporting many more processors at the time, G3s, G4s, and G5s all run that code out of the box)
Second, DES didn't optimize so well, it might be a GCC 4.0 regression because everything else got a nice and noticeable boost with the proper optimizations. The RSA stuff in particular was impressive.
So initially on non-threaded integer stuff, which isn't exactly what Sun claims the T2000 is good at (although they kind of act like it's a world beater at integer math) it looks like an old v9 ultrasparc multiplied by 8 which was kind of interesting back in the 20th century...
I'm sure that the T2000 will show some form as we beat it up with other things.
technorati tags: Sun, T1, Performance, marketing, sales, advertising
A quick summary on how DHCP quarantining works
When a standard DHCP client connects to a network, it sends a network broadcast message requesting an IP address. In response, a listening DHCP server may dynamically allocate and assign an IP address to the requesting client. The idea of quarantining devices is straight-forward when restricting the scope to only DHCP clients. The concept is simply to temporarily assign the client an IP address which is outside the range of your valid internal network blocks. For example, if your internal network is all within the 10.0.0.0/8 block, you could configure a special DHCP server to only allocate IP addresses within the 192.168.1.0/24 block. Devices receiving a 192.168.1.xxx address would, in theory, be unable to communicate with any services on the 10.0.0.0/8 block – hence quarantining each. Now, without additional functionality, this solution is incomplete. Additional features are needed to create a solution that works. To develop a basic system there needs to be a method to test or obtain information about connecting clients and a method to assign each a new IP address after the device is deemed acceptable. The criteria for testing may be as basic as checking the end-point device for open ports. If ports are open which are prohibited, the device remains quarantined; otherwise the device passes the policy checks and may receive a new non-quarantined IP address. Here is a simplified diagram illustrating the flow:

DHCP quarantining works by dispensing IP addresses based on the state of the client. If you create a list of properties you want all devices to exhibit before being permitted to access network resources, then you can develop a policy used to verify client state. The DHCP server then becomes a component of a system to enforce the policies you create.
In an upcoming blog I'll toss out several ways to circumvent DHCP quarantining "security".
Bought a duo core laptop
I broke down and bought a new core duo laptop (Dell 6400). I hate to spend money on laptops; I usually elect to spend my computer bugdet on keeping my workstation tricked out. I can't tell if Ian is serious or not, but he's saying this $1000 core duo laptop will outrun my dual Opteron 252 workstation. I'll just be super happy to leave my freakin' slow Inspiron 8100 behind when traveling now.

library paths, ldconfig & crle
Fixing up the shared library path is a frequent necessity. What lots forget is Solaris has an equivalent command to Linux's ldconfig for modifying the default system wide search path, named crle. The shared library path basically tells the operating system where to look for shared libraries required by programs to run. It is especially important when you've installed multiple versions of libraries and need a way to guarantee the right libraries are used by the right programs. For example, we installed a pre-compiled version of the Apache web server on a Sparc/Solaris8 system which was linked to a specific Openssl library version. Since there were already earlier versions of the Openssl libraries on the system we had to modify the shared library path based on which user was executing what to ensure proper linking.
Set the LD_LIBRARY_PATH variable:
% export LD_LIBRARY_PATH=/opt/usr/local/lib:/opt/usr/local/libexec:/opt/usr/local/pgsql/lib:/opt/usr/local/pgsql/libexec:/opt/usr/local/ssl/lib (removed several dirs to keep this short)
To verify an executable can find and link to the right shared libraries, use ldd:
% user@server /opt/usr/local/sbin> ldd lighttpd (ldd -s is more powerful)
libpcre.so.0 => /opt/usr/local/lib/libpcre.so.0
libdl.so.1 => /usr/lib/libdl.so.1
libsendfile.so.1 => /usr/lib/libsendfile.so.1
libresolv.so.2 => /usr/lib/libresolv.so.2
libnsl.so.1 => /usr/lib/libnsl.so.1
libsocket.so.1 => /usr/lib/libsocket.so.1
libc.so.1 => /usr/lib/libc.so.1
libgcc_s.so.1 => /opt/usr/local/lib/libgcc_s.so.1
libmp.so.2 => /usr/lib/libmp.so.2
/usr/platform/SUNW,Ultra-80/lib/libc_psr.so.1
You can use Linux's ldconfig command and Solaris' crle command to create default system wide search paths and use LD_LIBRARY_PATH to override the path when necessary.
Below is a chart describing the correct shared path variable names for different OSes:
OS |
32bit |
64bit (if different from 32) |
Delimiter |
AIX |
LIBPATH |
: (colon) |
|
HP-UX |
SHLIB_PATH |
LD_LIBRARY_PATH |
: (colon) |
Solaris |
LD_LIBRARY_PATH |
LD_LIBRARY_PATH_64 |
: (colon) |
Linux |
LD_LIBRARY_PATH |
: (colon) |
|
Tru64 |
LD_LIBRARY_PATH |
: (colon) |
|
SCO |
LD_LIBRARY_PATH |
: (colon) |
|
Unixware |
LD_LIBRARY_PATH |
: (colon) |
|
Windows |
PATH |
; (semicolon |
Lost Onfolio
![]()
I am a heavy user of Onfolio. Along with Firefox it has served as my primary RSS reader and Information Organizer. I went to their website to check out if anything new was coming and I was quite surprised to see Microsoft all over their page. Although I have paid $100+ for this tool it now appears it is a free plugin for the upcoming "Windows Live Toolbar" (which is in beta). The thing that really sucks is they, of course, discontinued development and support for every brower except IE. On top of that, they removed features which will not be available in the 'Live' plugin version. I'm not super religious with technology, but IE is far from first choice and I'm guessing I'll be searching for a replacement.
Scan fast and evade triggers
I've wanted to build this for a long time, alas the pain and costs of obtaining disparate public IPv4 blocks is high. I want to perform 65k port scans fast, accurately, and avoid 95% of the IDSes, IPSes, or whatever other ‘smart’ devices are in my way. It can be done.
- Buy or lease some servers
- Find a few data centers that connect to different Tier 1 providers
- Justify and purchase IP blocks from ARIN (or another regional registry)
- Setup scan server(s)
- Setup NAT server(s)
- Write some code to distribute port scans
- Feel cool when you can scan like crazy
- Feel really cool when no ‘smart’ devices alert, block, or rate limit you because you haven’t triggered any threshold ‘rules’
- Act surprised when the client mentions his team didn’t see or report any anomalous behavior
Here is a high-level diagram of what I want:

Of course, there are some realities which make this hard to build. Registries prefer to hand out contiguous net blocks, but it would be far more desirable to have a bunch of smaller non-contiguous net blocks. Some ‘smart’ devices do detect scans based on the source net block, not just via a single source IP. Bandwidth and latency conditions are always in play. I still want it. A scan setup like this can increase accuracy, be fast, is distributed, and raises the difficulty for detection.
FYI: Initial costs from ARIN for different net block sizes
| Category | Initial Registration Fee (US Dollars) | Assignment Size |
|---|---|---|
| X-small/ Micro-allocation |
$1,250 | /24 - < /20 |
| Small | $2,250 | /20 - /19 |
| Medium | $4,500 | > /19 - /16 |
| Large | $9,000 | > /16 - /14 |
| X-large | $18,000 | > /14 |
Tools for fingerprinting apps, services, and OSes
I was wondering how many different network-based fingerprinting tools are out there which use unique detection techniques. I know several commercial network scanners use Nmap, so if you decide to run Nmap by yourself and commercial tool X to see how they compare, you may (or even likely) be running the same thing. Obviously it can be a lot more helpful to have a handful of tools in which each has their own way to guess what the remote OS version is, or application version, or service. I've started to compile my own list and I haven't delved into the details of how each performs fingerprinting, but here is the list so far.
| Tool | Date of last version | version | OS | Service | Protocol |
| nmap | Feb, 2006 | 4.01 | yes | yes | yes |
| xprobe2 | Feb, 2005 | 0.2.2 | yes | no | no |
| p0f | Sep, 2004 | 2.0.6 | yes | no | no |
| amap | Sep, 2005 | 5.2 | no | yes | yes |
| nessus | Mar, 2006 | 3.02 | yes | yes | yes |
| winfingerprint | Mar, 2006 | 0.6.x | yes | yes | yes |
| httprint | Dec, 2005 | 301 | no | no | yes |
| queso | Aug, 1998 | 980922 | yes | no | no |
| NTP-fingerprint | Feb, 2005 | 0.1a | yes | no | no |
| ike-scan | Dec, 2005 | 1.8 | no | yes | yes |
| thcrut | May, 2003 | 1.2.5 | yes | no | no |
| smtpmap | Dec, 2001 | 0.6 | no | yes | no |
| smtpscan | May, 2003 | 0.5 | no | yes | no |
| snacktime | Jun, 2003 | 0.5 | yes | no | no |
| synscan | Apr, 2004 | 0.1 | yes | no | no |
| telnetfp | Jan, 2001 | 0.1.2 | yes | no | no |
| ldistfp | May, 2001 | 0.1.4 | yes | no | no |
| telnet | N/A | N/A | yes | yes | yes |
| siphon | May, 2000 | 666 | yes | no | no |
| ring | 0.0.1 | ||||
| scanssh | Mar, 2005 | 2.1 | no | yes | yes |
| hackbot | Dec, 2003 | 2.21 | no | yes | yes |
| hping3 | Nov, 2005 | 3.0.0 | yes | no | no |
| induce-arp.pl | May, 2000 | 0.27 | yes | no | no |
| vmap | Aug, 2003 | 0.6 | no | yes | yes |
| disco | Jul, 2003 | 1.2 | yes | no | no |
| k9 | yes | no | no | ||
| ettercap | May, 2005 | NG-0.7.3 | yes | ||
| Net::SinFP | Mar, 2006 | 1.00 | yes | no | no |
| Archaeopteryx | Jul, 2001 | 1.0 | yes | no | no |
| iQ | Apr, 2002 | 0.2 | yes | no | no |
| sprint | Mar, 2003 | 0.4.1 | yes | no | no |
Rubies, Rubies, Ruby
So what is the furor over the Ruby programming language lately? I have known about Ruby for a few years but never got into it much until recently. I had always heard of it in context of Python vs. Ruby on the Python programming list usually with the Python guys bashing Ruby over this our that. So what is it that is making Ruby so popular now?
Now Ruby on Rails is a framework that helps you create a web application that can render dynamic content quickly and easily. What the hell does that mean? It basically means that Ruby on Rails has a lot of code and functionality ALREADY built for you to use. In a matter of minutes (after installing all the stuff of course!) you can have a web page that queries your database and displays the data.
So what does this have to do with Ruby exactly? Well Ruby enables Rails to be so simple and easy to use. So by now you are thinking what is it that makes Ruby the language so good… The official list most people will say is:
- It is object-oriented down to its toe nails
- Simple syntax, not too many non-alphanumeric in use
- It is interpreted, making prototyping fast
- It is cool as of March 6th, 2006
So what? Python (or insert other language) is Object oriented, simple, and interpreted. Well I will list some of MY items that make me want to continue to master Ruby.
First off the ability of an object to know all its methods is great. In many languages you have to pass a value to a function (I know sounds technical!?) to get that value to do something. A method is basically all the functions an object can execute and a function is a stand alone operation that is not associated with an object. The way this manifests is in the way you call each. A method is called like object.method and a function is function(value). An example would be making a string into a number:
The Python function:
- x = "11" <-- This is a string because of the double quotes.
- int(x) <-- We call a function called int() to make "11" in 11.
The Ruby method:
- x = "11" <-- Again this is a string saved to x.
- x.to_i <-- The string object x has a method to convert a string to integer this case 11.
Python of course has many objects and methods itself and the langauage is actually really cool too, it is just that I like Ruby that much more.
Second is the use of block code and iterators in Ruby instead of using the stereotypically looping constructs. This is great as you can essential build smarter “loops” as the objects themselves know how to iterate over themselves instead of you knowing (or learning) how to iterate over them. For example how you would iterate over a string is different than an array or hash right? How would you iterate over a custom object you create? In ruby it is simple to iterate over an object like so:
Ruby iteration:
- x = [1,2,3] <-- this is now an array (one object) of three things, 1, 2, and 3.
- x.each {|i| puts i} <-- This is a loop basically a for loop!
Now what happened there and what is all that stuff? Well first off x.each is a method call for the object x which happens to be an array. The each method will return each item in an array one by one. Then each will pass the item to the block which is everything between the {}. The block will put the item into the variable i then execute the statement "puts i" which prints the value to the screen. This will be done for each item passed to it by the objects “each” method. Sounds hard but it is easier than this:
A Java for loop:
- for ( i=1; i<6; i++ ) { <-- This will assign 1 to i and only run the loop if i is less than 6. Also i is added to each iteration. How do I know this from looking at it? I don't, a book told me.
System.out.println(i); } <--Prints what is stored in i.
Third, Ruby has CPAN like functionality. I don't mean that Ruby is watching the White House press room for late breaking info. What I mean is that Ruby has similar functionality to the Perl Comprehensive Archive Network. Well CPAN is where you go if you are a Perl programmer that needs ready made code for something like SSHing, parsing XML etc.
Odds are that someone has done that task and placed it on the web. CPAN allows perl programmers to easily retrieve and install these modules of code and use them. So what is the Ruby equivalent? It is called Ruby Gems. Gems is no where as Comprehensive as Perl yet, cause Perl is as old as the Rocky Mountains but it has lots of functionality already. The gem program is how you get rails installed onto your system.
So you want Rails to go with your ruby ring? Step into my gem room and we will see what we can do:
Red ~ # gem install rails
Attempting local installation of 'rails'
Local gem file not found: rails*.gem
Attempting remote installation of 'rails'
Updating Gem source index for: http://gems.rubyforge.org
Install required dependency rake? [Yn] y
Install required dependency activesupport? [Yn] y
Install required dependency activerecord? [Yn] y
Install required dependency actionpack? [Yn] y
Install required dependency actionmailer? [Yn] y
Install required dependency actionwebservice? [Yn] y
Successfully installed rails-1.0.0
Successfully installed rake-0.7.0
Successfully installed activesupport-1.2.5
Successfully installed activerecord-1.13.2
Successfully installed actionpack-1.11.2
Successfully installed actionmailer-1.1.5
Successfully installed actionwebservice-1.0.0
Installing RDoc documentation for rake-0.7.0...
Installing RDoc documentation for activesupport-1.2.5...
Installing RDoc documentation for activerecord-1.13.2...
Installing RDoc documentation for actionpack-1.11.2...
Installing RDoc documentation for actionmailer-1.1.5...
Installing RDoc documentation for actionwebservice-1.0.0...
Red ~ #
Hmm well I will let you guys off for now with this thought. Most times I find that I struggle with the language more than I struggle with the problem I am trying to solve. Ruby has helped me with this one issue. Doh!!! Now I have no excuse for not solving my issues...
Firewalls part 3
Another issue with firewalls which isn't always on radar is that dropping (or the correct thing which is rejecting) a packet is a pretty substantial thing to do. It's kind of like the death penalty. The justice system has a wide range of punishments and while the consensus in the US is that there are crimes which justify the death penalty most do not. When we build firewalls we tend to be heavy handed in that regard and if there is a well define policy then it's the safest thing to do.
I watch my logs too much and every now and then I see an IP doing "too much," so I whois him, it's some random ISP somewhere serving up DHCP, depending on my temperment at the time, maybe I'll nmap him to see what I can learn and based on that I might make some new firewall rules. This is my "hacker computer geek security response" someone is poking me so what do I do? I poke right back, it's not like I'm showing someone a gun on an Los Angeles freeway.. Mind you, I'm also a software engineer and not a network guy but I've got some hammers and my network has some nails. As I get older, wiser and have less time to just screw around I find myself responding this way less frequently but regardless I still don't particularly like it when someone creates enough traffic on my firewall for me to notice and I'm not about to simply start dropping packets because I don't like them, a more mature response is needed. It's even more ironic, they might have been responding in kind and I'm pretty sure that I've scanned someone and caused them to scan me right back before, sort of a high tech submarine collision.
I also happen to be the victim of working on an IDS/IPS project for a couple years. You'll have to give me a couple of beers to get my complete feelings on the subject, for here I'll just say that they can be nice tools to provide more visibility in to a network, making sense of that visibility is a different matter and acting on it is something else that is still all together different. One of the overriding themes from that episode was how for many customers an IPS is a much larger hammer than they usually need and we routinely feed them a bunch of BS about "tuning," like we were helping them put a really large "racing muffler" on their Honda Civic rather than breaking in the Cadillac that we had told them we were selling them, and that they need to figure out what is on their network before they start just blocking stuff regardless of the policy that they didn't have to begin with... I'll shut up before I compromise my professionalism too much more than I did by working at that place in the first place. A different response is needed, much more so with IDS/IPS type products.
So what do we do? Rate limiting. At a glance this may not seem like a resonse. If you're actively being attacked and possibly exploited, slowing the data flow down doesn't really "fix the problem." However if you have questionable data about a situation such as an alert from an IDS, your network is critical for business (or maybe it's just not critical because it's at home) or you simply don't know what the traffic is and you don't get a good feeling about it then rate limiting is much better than the death penalty.
Linux provides a fairly robust and full featured set of tools for this, even better, they integrate very nicely with the firewall. Once you've created some traffic shaping policies you can easily make firewall rules to shape packets based on IP, protocol, flags and port and anything you'd put in your firewall.
Foregive the slop, I just cranked this out and it's not optimal..
# Define some match bits for different types of rate limiting.
MATCH1="111"
MATCH2="222"
IPTABLES=/sbin/iptables
$IPTABLES -n RATELIMITRULES
$IPTABLES -n RATELIMIT1
$IPTABLES -n RATELIMIT2
# For packets on both interfaces limit them to "rate 1" by setting match 1
$IPTABLES -I RATELIMIT1 -t mangle -i eth0 -j MARK --set-mark $MATCH1
$IPTABLES -I RATELIMIT1 -t mangle -i eth1 -j MARK --set-mark $MATCH1
# For packets on both interfaces limit them to "rate 2" by setting match 2
$IPTABLES -I RATELIMIT2 -t mangle -i eth0 -j MARK --set-mark $MATCH2
$IPTABLES -I RATELIMIT2 -t mangle -i eth1 -j MARK --set-mark $MATCH2
# Tell packets that are forward to go through the RATELIMITRULES table.
$IPTABLES -t mangle -I FORWARD -j RATELIMITRULES
The in the RATELIMITRULES table you can place rules for IPs you wish to rate limit, I created 2 different rates as punishments, RATELIMIT1 and RATELIMIT2, one is really slow and one is just kind of slow. A rule might look like this:
$IPTABLES -I RATELIMITRULES -s www.badguy.com -j RATELIMIT2
Now how do we set up the actual rate limiting? Your kernel needs queuing disciplines. I'll explain the many options there in a future article.
TC=/sbin/tc
$TC qdisc del dev eth0 root
$TC qdisc del dev eth1 root
## This part is lame, I made is completely symmetrical which probably isn't what's wanted.
$TC qdisc add dev eth0 root handle 1: htb default 30
$TC qdisc add dev eth1 root handle 1: htb default 30
# Define your network pipe
$TC class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 100mbit
$TC class add dev eth1 parent 1: classid 1:1 htb rate 100mbit burst 100mbit
# Define a 15kbit pipe.
$TC class add dev eth0 parent 1:1 classid 1:10 htb rate 15kbit burst 15kbit
$TC class add dev eth1 parent 1:1 classid 1:10 htb rate 15kbit burst 15kbit
# This pipe is only 4kbit
$TC class add dev eth1 parent 1:1 classid 1:20 htb rate 4kbit burst 4kbit
$TC class add dev eth0 parent 1:1 classid 1:20 htb rate 4kbit burst 4kbit
## Here is the magic, it hooks the queues up to the firewall match.
$TC filter add dev eth1 protocol ip parent 1:0 prio 1 handle ${MATCH1} fw flowid 1:20
$TC filter add dev eth0 protocol ip parent 1:0 prio 1 handle ${MATCH1} fw flowid 1:20
$TC filter add dev eth1 protocol ip parent 1:0 prio 1 handle ${MATCH2} fw flowid 1:10
$TC filter add dev eth0 protocol ip parent 1:0 prio 1 handle ${MATCH2} fw flowid 1:10

