Immutable Security

Here are the five or more links that I found interesting for this week:

• Dave Hoelzer from SANS provides some very useful “AuditCasts.” These are short, instructional videos on various topics. This week, Dave talked about the benefits of split DNS. One implementation of split DNS that Dave did not mention specifically is to not have the internal DNS servers forward requests at all; rather, you may rely on your proxy to do the queries for you. This can be helpful if a malware infection is trying to phone home. If they are relying on DNS for the call-back, it might fail.
• What should you do after compromising a system (on which you had permission, of course)? These Linux and Windows community docs provide some good tips. I wouldn’t recommend running these commands or tools without knowing exactly what the outcome will be. They could be dangerous.
• The Morto RDP worm takes advantage of poorly chosen passwords. Microsoft has a pretty good write-up of the behavior of the worm. OSSEC should detect the invalid logon attempts. I have some ideas on how a few rules could detect this general class of infection.
• Barracuda, the company that has a pretty decent anti-spam gateway, offers their RBL for free. I have been using the Spamhaus Zen RBL for quite some time and decided to give this a try. Putting it before the Spamhaus list, the result has been pretty good. Spamhaus now rejects only 3-5 messages a day since the BRBL is blocking lots of spam that Spamhaus would have caught. So far, I know of no false-positives. Using both seems to work pretty well.
• Lenny Zeltser reminds us that security is best designed with failure in mind. Security controls will fail, but that doesn’t necessarily have to lead to an information breach. A good security design plans on these controls failing with the information remaining safe.

That’s it for today. Have a great weekend!

Here are the five links that I found interesting for this week:

• The Shadowserver foundation is comprised of a group of volunteer security professionals who gather information about Internet-based crime. One of the more interesting projects is a compilation of how various antivirus vendors fare against 0-day threats. How does your vendor hold up?
• Logs are not much good if you can’t trust them. Maintaining log integrity is vital to a robust incident response process. Here is a great article on how to protect your logs from tampering. It’s not fool-proof, but it can go a long way.
• Information security is a profession that necessitates a solid ethical foundation. Security professionals are often trusted with the most sensitive of data. This presentation, from the Honeynet Project, tackles some of the more thorny situations about performing ethical research.
• Looking for a really awesome way to store and compare your Cisco configs? Rancid, or the Really Awesome New Cisco confIg Differ, may be just the tool for you. It stores Cisco configs in CVS and can let you know something changed. By the way, OSSEC is also capable of something very similar.
• Are you looking to use virtualization in your PCI program? It can be done, but like most technologies, has to be approached carefully. This guide will show you some of the things that need to be considered.

That’s it for today. Have a great weekend!

Here are the five links that I found interesting for this week:

• Mitigating the Apache Range Header Attack. This is a pretty good overview of several ways you can protect yourself for little to no cost. Also, see my post, Detecting the Apache Range Header DoS Attack with OSSEC.
• Automatically encrypt all inbound email part I and part II. Even if you have full-disk encryption, it does not protect you if someone can access your account. This method allows you to keep the private key off the server and does not rely on convincing other people to encrypt email to you. Very impressive.
• Process Monitor is a tool that helps you to see what it really happening under the Windows hood. It’s truly indispensable for Windows troubleshooting and incident response. These filters are specifically designed for malware analysis. I imagine they will be very useful on my next incident.
• Have you ever wanted to open a command prompt as SYSTEM? Most people think that having administrator rights is the same thing, but there can be subtle differences. This short little script allows you to become SYSTEM for those rare situations where you may need to be.
• Would you know if your web site was compromised? Here are eight tips for detecting a web site compromise.

That’s it for today. Have a great weekend!

If you run Apache, you may have heard about the DoS vulnerability last week. Apache suffers from a condition where an attacker can remotely cause the web server to consume huge amounts of memory. This causes the system to be unstable and eventually, maybe even crash.

The question was raised: “Can OSSEC detect this attack?” I got to thinking about this and the answer is “probably.” Since OSSEC is primarily a log-based HIDs, we first have to look at the logs to see if there is anything juicy in there that we can use. We also need an exploit and a vulnerable system so we can reproduce the conditions of the attack. Since my server wasn’t vulnerable, FrankS in the #OSSEC channel on IRC offered to lend a hand in the rule research efforts. The first thing we noticed is several logs like this:

172.16.0.1 – - [27/Aug/2011:21:42:53 +0000] “HEAD / HTTP/1.1″ 206 354 “-” “-”

There are two things about this log that don’t quite look right: one, there are multiple HEAD requests to the root of the domain (/) and two, there are several 206 HTTP status codes. Generally, you would see 206 status codes  in the context of requesting compressed content, and the request would probably be a GET. The other thing we noticed in the logs was a page allocation failure coming from Apache, like so:

Aug 27 21:59:43 hostname kernel: [ 1181.719148] apache2: page allocation failure. order:0, mode:0×20

This was promising. If we could simply look for multiple HEAD requests to / with 206 status codes in a very short amount of time, followed by a ‘page allocation failure,’ it’s probably an attack. OSSEC can do that.

There’s a certain amount of art and experience which goes into writing IDS rules. The goal is to make the rule as accurate as possible: if it does not detect the attack (false-negative), you will lose faith in the IDS; on the other hand, if it detects things which aren’t really an attack (false-positive), then you will also lose faith in the IDS and miss potential attacks. Finally, it is best to avoid making the rule exploit-specific. This can result in a situation where a small change in the exploit can avoid the rule being triggered.

Knowing what we do about the logs, how can we make a rule or set of rules that will trip when the host is attacked and be somewhat accurate? Multiple HEAD requests to / are certainly suspicious, but does the attack rely on HEAD to be successful? Section 10.2.7 of the the RFC for HTTP specifically refers to GET requests, and the attack does not seem to be specific to the HTTP method, so we can’t necessarily rely on the HEAD as an indicator. Next, we see the series of 206 codes. That is also not necessarily an indicator of the attack, but it likely is something that the webmaster may want to know about. If there are several of them in a small amount of time, we can alert the analyst to the condition for further inspection. Still, we aren’t really sure if it is the Range Header DoS. In this case, we’ll need two rules. The first rule detects the 206 status code but does not send an alert, while the subordinate rule looks for 10 of them in a 5 second interval coming from the same location (agent) and from the same source IP:

<rule id=”100002″ level=”5″>
<if_sid>31108</if_sid>
<id>^206$</id>
<description>Web Server 206 Error Code</description>
</rule>

<rule id=”100003″ level=”10″ frequency=”8″ timeframe=”5″>
<if_matched_sid>100002</if_matched_sid>
<same_location />
<same_source_ip />
<description>Multiple Web Server 206 Error Codes </description>
<description>from Same Source IP</description>
<group>web_scan,recon,</group>
</rule>

At this point, the analyst will get alerted to the DoS condition, but we are not necessarily confident that it is the Range Header attack. There’s one more rule we can create that looks for the ‘page allocation failure’ occurring within five minutes of rule 100003 firing. If we see all of this, we are reasonably confident in what is going on:

<rule id=”100004″ level=”12″ timeframe=”300″>
<if_matched_sid>100003</if_matched_sid>
<if_sid>1002</if_sid>
<program_name>kernel</program_name>
<match>page allocation failure</match>
<description>Apache Range Header DoS Attack</description>
<group>attack,</group>
<info type=”cve”>2011-3192</info>
</rule>

So, what can go wrong? Lots. The system may be so unstable that it cannot send logs to the manager. The attack might be successful with fewer than ten 206 requests within five seconds. In some cases, the ‘page allocation failure’ does not appear in the logs, although the attack still might trigger rule 10003. At least this would give the analyst a chance to visually inspect the sample of logs OSSEC sends in the alert.

Are there better ways to detect this? Certainly. Mod_Security can not only detect but also prevent the attack. Tools such as Snort are better positioned than OSSEC in situations like this. Of course, OSSEC can monitor the logs of those tools, and still alert you. The point here was to demonstrate that there is an OSSEC-only way to detect attacks like this.

Do you see a problem with the rules? Can they be subverted easily? Let me know in the comments.

One of the reasons I started this blog was to share things I had encountered in the security and privacy world. I have done quite a bit of editorializing, but not too many of the quick and useful posts. I thought it might be helpful to post about five of my favorite reads and links on Fridays–unless I get too busy. So let’s start off with a few interesting links:

• PacketFence is a free and open source NAC system. I haven’t used it so I can’t vouch for it either way, but it’s nice to see a NAC in the free software world. NACs are good at preventing things like man-in-the-middle attacks, help you with asset control and help to keep the worm-of-the-day off your network when a contractor plugs in his laptop. Free software can also be a good way to meet a requirement even with limited or no budgets.
• Need a forensics tool? Maltego may fit the bill. It’s also free to use, but not free software in the sense that it doesn’t seem to have an OSI compatible license. Like PacketFence, there are also commercial support and versions available.
• Jamie Riden wrote a very nice piece on his/her response to an SSH attack. There are some nice recovery and lessons-learned aspects to the article. Another possible countermeasure would be the use of OSSEC along with its active response capabilities. This might have been able to prevent the compromise entirely.
• Would you like to have a log of all commands entered on a Cisco router? This is something that can be very useful for audit and compliance, as well as change management needs. This is a great one for PCI environments.
• The ‘nix mtr tool can be useful for troubleshooting network problems. The WinMTR does pretty much the same thing from a Windows host. It’s also free software.

That’s it for today. Have a wonderful weekend!

The OSSEC team is pleased to announce the general availability of v2.6. This version includes support for IPV6, a new tool for key management of ‘nix agents, an option to increase the block timeout for repeat offenders, and many other goodies.

Major kudos for this release go to Dan Parriott (ddpbsd). Dan is the most active person helping OSSEC users today on the mailing list and IRC. He also seems to find time to write documentation, which–let’s face it–no one really likes to do, and writes new rules and decoders. Thanks, Dan.

If you would like to see what I am up to in the OSSEC world, check out my repository on Bitbucket. My commits are generally tested and ready for integration into the next release, so try them out and let me know how they work for you. The tickets section is basically my task list of things I am already working on or plan to implement.

As always, thanks to everyone who contributes and supports our work. If you have some free time, stop by #ossec on freenode to say hi.

One of the things I have been trying to do lately with OSSEC is to bring it into the “real world.” I have been exploring how to add support for useful and interesting devices such as phones, TVs and facility alarm systems. I have been tinkering with one device, a VoIP ATA adapter made by Grandstream and I thought I would share what I had come up with. You’ll probably see this in a future release of OSSEC, so let me know what you think.

There are a couple of interesting (to me, anyway) rules. One of them is when someone dials an emergency number like 911. This might be useful if you have family members at home and you are out and about. In fact, I am somewhat sorry to say that this one proved to be effective very recently in an actual emergency (everything is OK now).

Another useful rule might be the one which tracks calls to premium-rate numbers (think psychics and sex lines). If someone is doing that, you may (or may not) want to know about it right away to avoid a huge bill.

Businesses could write rules to be alerted when a competitor’s number is dialed. It might be nothing, but what if it happens only during hours when the boss isn’t around?

These rules do not detect traditional IP-based attacks, but we can certainly expand them to do so. The point is simply that you may be able to use OSSEC in ways that you didn’t originally conceive.

If you have a Grandstream 286, 502 or 503 ATA adapter, download these files and check it out. The grandstream_decoder.txt stuff can go in your local_decoder.xml and the grandstream_rules.txt can be placed in your local_rules.xml (watch for rule ID collisions).

Let me know if you run into any problems. What interesting things have you done recently with OSSEC?

Open source log analysis types sometimes need to be crafty. We often don’t have relationships with the companies who’s products we support, but that doesn’t stop the users from wanting to use software like OSSEC to analyze logs from big, enterprise-type closed source stuff.

We scour the Internet for log samples and documentation–anything, really–that might help us out.  So today when I came across some documentation for IBM’s Compliance Insight Manager, I thought I would pass along what I found.

Under the Configuring auditing for specific platforms section, you’ll find instructions on how to configure auditing for dozens of enterprise systems. It even goes far beyond what to click or which file to edit. There is info on audit capabilities, monitoring recommendations, suggested audit settings and tuning.

This is a worthwhile resource to bookmark. Until next time..

One of the things us log analysis types love is good documentation. It’s rare to find well-formatted, well-documented logs, so when we do find good log info, it’s like being a kid in a candy store. So without further ado, I present all of the links you will need to describe every Windows Server Security Event log that exists:

Windows NT (please tell me you don’t really need this):

KB174074 – Security Event Descriptions

Windows 2000

KB299475 – Windows 2000 Security Event Descriptions (Part 1 of 2)

KB301677 – Windows 2000 Security Event Descriptions (Part 2 of 2)

Windows 2003

Windows 2003 Security Guide, Chapter 4, Audit Policy

Windows Vista to Present

KB947226 – Description of security events in Windows Vista and in Windows Server 2008

Security audit events for Microsoft Windows Server 2008 and Microsoft Windows Vista (This one is fantastic–every log in spreadsheet form. Export to csv for an easily-parsable and machine-readable format!)

Of course, just having the documentation doesn’t necessarily tell you the whole picture. Sometimes it’s nice to have a discussion about the log event. Two of my favorite sites for this are Ultimate Windows Security and EventID.net. In particular, the Security Log Encyclopedia is a great reference when you want to quickly know what a particular event ID means, but also get some contextual and practical information about the event.

Remember that documentation isn’t always correct. There may be subtle differences in what events are logged or how they are presented depending on the service pack level. What other resources have you found?

I have been studying Oracle logging for the last couple of weeks. Oracle can log to the SYS.AUD$ table within the database, a flat file, XML file, or it can use the OS logging facility (in Windows this is the event log; in ‘nix, it is syslog).

Preferring ‘nix-based solutions, I downloaded Oracle XE 11g for Linux and configured it for logging to syslog. It wasn’t long before I had logs like this:

Dec 28 22:32:41 localhost Oracle Audit[4958]: ACTION : ‘CONNECT’ DATABASE USER: ‘/’ PRIVILEGE : SYSDBA CLIENT USER: username CLIENT TERMINAL: pts/2 STATUS: 0

At first glance, it looked like a pretty standard syslog message, but after having some issues getting OSSEC to pre-decode it properly, I decided to check the RFC to see if it was technically compliant. Here is the relevant part of RFC 3164.

The MSG part has two fields known as the TAG field and the CONTENT field. The value in the TAG field will be the name of the program or process that generated the message. The CONTENT contains the details of the message. This has traditionally been a freeform message that gives some detailed information of the event. The TAG is a string of ABNF alphanumeric characters that MUST NOT exceed 32 characters. Any non-alphanumeric character will terminate the TAG field and will be assumed to be the starting character of the CONTENT field. Most commonly, the first character of the CONTENT field that signifies the conclusion of the TAG field has been seen to be the left square bracket character (“[“), a colon character (“:”), or a space character. This is explained in more detail in Section 5.3.

Did you spot it? The problem is that Oracle likely intended the string ‘Oracle Audit’ to compromise the TAG field (program or process name); however, the TAG field in this case really is just ‘Oracle’ since it is terminated by a non-alphanumeric character (the space).

So is it compliant? I would have to say yes, but I don’t think it is compliant in the way they intended. Simply removing the string ‘Audit’ in this case would have made a clearly compliant and understandable message.