Tech note on Syslog, TCP, and Cisco ASA/PIX

Posted by Tate Hansen Fri, 24 Aug 2007 01:05:00 GMT

Absent of Cisco wizard skills caused me a little pain yesterday. I remotely configured my Cisco ASA to forward syslog via TCP to a central log host. When I subsequently rebooted the central log host, I lost the ability to establish new connections to anything behind the ASA.

Luckily, I had an established session to a system with a serial connection which enabled me to recover.

I hadn’t run into this before, but I confirmed my experience:

1. If it is unable to log via a defined TCP syslog session, a PIX will not create any new connections (although connections opened before the failure of the session will continue to work). The PIX will log a message to the console stating that it is disallowing new connections.
2. In order to re-establish connection activity, the privileged set logging command, with the correct parameters, will have to be entered or the PIX reloaded.

Tags , , , , , , ,  | 1 comment

Compromised? Where are the logs?

Posted by Tate Hansen Fri, 10 Nov 2006 05:55:00 GMT

Full compromises are crazy – it happens, but nearly every one I touch still sends a little shiver down my spine mixed with a “holy shit” moment.

Case in point: we got a call last week to help out with an incident and to do an investigation of some Linux based security devices. Financial industry, lots of customers.

Their engineers were working feverishly to scope the incident and to learn all the details then slam – they hit a wall. Their firewall logs everything. Their only-way-out-is-via-a-proxy-server logs everything.

And their security point solution in the DMZ? Oops, it only logs locally (it also happens to be a party to the fully compromised club inside this trusted net). And what about their desktops which appear to have offered free remote admin VTYs? Well, this all has a kicker for an answer.

It happened a year ago. Ouch. Talk about sensing a cold sweat and suddenly not trusting anything. By the way, the desktops exhibiting the worst behavior traveled the evolutionary path and have been wiped clean and rebuilt, upgraded, or replaced since the break-in. No substantive logging or backups of mischievous desktops; no way to reconstruct the perpetrator’s methods. Security point solution in DMZ had standard local logging (i.e. full rotation in weeks), therefore it is unlikely things we're looking for are there (still investigation). No network device logging. There are lots of unanswered questions.

The details of this should strengthen your neuron paths connecting logging to your “holy shit” moments. That is to say…

  • If you got important data to protect, log everything you reasonably can. All the “security” in this scenario failed and failed to help reconstruct events.
  • Do you have one of those semi hands-off security appliances that you presume is working fine because you can connect to the web admin portal? Make it forward logs to somewhere.
  • Do you have workstations which touch sensitive data anytime? Yes. Then boost the priority to configure central logging, stop procrastinating, then take comfort you’ll be in better shape than the poor souls at this company fighting to salvage their pride, and maybe their jobs.

Tags , , , , ,  | 2 comments

Time for an IDS?

Posted by Cory Stoker Fri, 27 Oct 2006 21:37:00 GMT

Often we run into a scenario where a client wants to improve security by implementing an IDS. Now this is OK but often we find out that they are not exactly "ready". What I mean by this is to effectively deploy an IDS on a network you should first cover your bases with what you already got. Before going IDS wild, are you using your existing infrastructure to get the knowledge you need first?

Of course just throwing out an IDS is a quick hit; it can cover your ass if the regulatory audit kids drop by, but for lots it becomes a security lame duck. Do you know how to really make it a valuable component of your security?

"Yeah we have [insert IDS vendor here] in place off the 6509."


"Sweet, what does your ruleset look like?"


"Well I think we have all rules enabled right now."


"Ok, what networks are you watching?"


"We want to see both ways so we are not restricting the nets."


"Do you know what kind of traffic and how much are you seeing?"


"Well, not really, but we have all the usual suspects: SMTP, HTTP, HTTPS, FTP and even IM."


"Do you know how much activity you’re seeing for each of those?"


"Man, we just threw it out there, its working!"


Yeah, that $20k went far. The point is to get off this copy-cat buy and forget.


First, I am a believer in doing the macro before the micro. You can’t pick whether a rule should be enabled or not until you know what’s going on in your network. Enabling all will protect you is bullshit. In fact, enabling all can make things worse (easier to exhaust resources, get blinded, or rev up the false positive counters). And forget about dealing with IDS evasion techniques.

Take logging, a macro thing. Logging is an enabler. Deducing traffic patterns is a macro enabler. You need to do these things first before considering an IDS. Engineers like to complain how hard it is to watch logs and I can empathize, but it is really not that hard to get some valuable numbers. Which firewall rules are taking hits, which are not? What kind of traffic are you seeing and how much? A couple scripts can get you those numbers.

So before you do the IDS thing, do these things at a minimum:

  • Grab your firewall rulesets and the counters for each allow/deny rule. With this you info you can teach your IDS where to look (vitally important both direction so ingress and egress).
  • Learn traffic patterns. What protocols are in use, which networks, and in what amounts. High, average, low - all good numbers. Use this to tune the rulesets. Now you can identify something like your top 100 talkers. If some server catapults to #1, check it out.\
  • What apps are running. Learn all the things your environment is hosting. When do your backups run, which servers like to talk to which? Again, information vital to getting value from an your infrastructure that will enable you to get even more value from your IDS.

Tags , , , , , ,  | 4 comments