Immutable Security

Here are the five or more links that I found interesting for this week:

• PDF-XRAY is a site where you can submit suspect PDFs for analysis. Now you can download the code behind the site and have a go at the file locally. This might be a better alternative than sending a potentially sensitive file to an unvalidated web site.
• Are you wondering whether that weird looking exe in a startup location is malicious? Virus Total is one of my favorite sites for getting a second opinion. It will scan a file you submit with multiple AV engines and let you know what they think. A file that comes back clean is not necessarily clean, but if multiple AV engines tell you that it is infected then it probably is. Another feature, which seems to be new since my last visit, is the ability to scan a web site. It will check the index file and let you know if any obvious malware is being delivered from the site.
• Threat Expert is a similar site to Virus Total, with the added advantage that it returns a pretty detailed report. The report tells you what the file actually attempted to do. I like to use both sites on a suspect file.
• Feeling pretty confident in your firewall egress policy? What if I told you that systems can be controlled with ping packets, or ICMP. Ok, I’ll wait while you pick your jaw up from the ground.
• As we become more and more of a gadget society, we’ll be running into things like this quite often. Jerome Radcliffe discovered some vulnerabilities with an insulin pump. Since the pump controls the dosage of insulin to the patient, a security vulnerability in a device like this is not just inconvenient, it can be fatal. Medtronic, the company behind the device, seems to be doing a very poor job at handling what seems to be a very honest and ethical disclosure. Information security concerns may be new to these types of devices, but we can’t let the companies PR their way around these issues. The stakes are simply too great.

That’s it for today. Have a great weekend!

Here are the five links that I found interesting for this week:

• The Shadowserver foundation is comprised of a group of volunteer security professionals who gather information about Internet-based crime. One of the more interesting projects is a compilation of how various antivirus vendors fare against 0-day threats. How does your vendor hold up?
• Logs are not much good if you can’t trust them. Maintaining log integrity is vital to a robust incident response process. Here is a great article on how to protect your logs from tampering. It’s not fool-proof, but it can go a long way.
• Information security is a profession that necessitates a solid ethical foundation. Security professionals are often trusted with the most sensitive of data. This presentation, from the Honeynet Project, tackles some of the more thorny situations about performing ethical research.
• Looking for a really awesome way to store and compare your Cisco configs? Rancid, or the Really Awesome New Cisco confIg Differ, may be just the tool for you. It stores Cisco configs in CVS and can let you know something changed. By the way, OSSEC is also capable of something very similar.
• Are you looking to use virtualization in your PCI program? It can be done, but like most technologies, has to be approached carefully. This guide will show you some of the things that need to be considered.

That’s it for today. Have a great weekend!

If you run Apache, you may have heard about the DoS vulnerability last week. Apache suffers from a condition where an attacker can remotely cause the web server to consume huge amounts of memory. This causes the system to be unstable and eventually, maybe even crash.

The question was raised: “Can OSSEC detect this attack?” I got to thinking about this and the answer is “probably.” Since OSSEC is primarily a log-based HIDs, we first have to look at the logs to see if there is anything juicy in there that we can use. We also need an exploit and a vulnerable system so we can reproduce the conditions of the attack. Since my server wasn’t vulnerable, FrankS in the #OSSEC channel on IRC offered to lend a hand in the rule research efforts. The first thing we noticed is several logs like this:

172.16.0.1 – - [27/Aug/2011:21:42:53 +0000] “HEAD / HTTP/1.1″ 206 354 “-” “-”

There are two things about this log that don’t quite look right: one, there are multiple HEAD requests to the root of the domain (/) and two, there are several 206 HTTP status codes. Generally, you would see 206 status codes  in the context of requesting compressed content, and the request would probably be a GET. The other thing we noticed in the logs was a page allocation failure coming from Apache, like so:

Aug 27 21:59:43 hostname kernel: [ 1181.719148] apache2: page allocation failure. order:0, mode:0×20

This was promising. If we could simply look for multiple HEAD requests to / with 206 status codes in a very short amount of time, followed by a ‘page allocation failure,’ it’s probably an attack. OSSEC can do that.

There’s a certain amount of art and experience which goes into writing IDS rules. The goal is to make the rule as accurate as possible: if it does not detect the attack (false-negative), you will lose faith in the IDS; on the other hand, if it detects things which aren’t really an attack (false-positive), then you will also lose faith in the IDS and miss potential attacks. Finally, it is best to avoid making the rule exploit-specific. This can result in a situation where a small change in the exploit can avoid the rule being triggered.

Knowing what we do about the logs, how can we make a rule or set of rules that will trip when the host is attacked and be somewhat accurate? Multiple HEAD requests to / are certainly suspicious, but does the attack rely on HEAD to be successful? Section 10.2.7 of the the RFC for HTTP specifically refers to GET requests, and the attack does not seem to be specific to the HTTP method, so we can’t necessarily rely on the HEAD as an indicator. Next, we see the series of 206 codes. That is also not necessarily an indicator of the attack, but it likely is something that the webmaster may want to know about. If there are several of them in a small amount of time, we can alert the analyst to the condition for further inspection. Still, we aren’t really sure if it is the Range Header DoS. In this case, we’ll need two rules. The first rule detects the 206 status code but does not send an alert, while the subordinate rule looks for 10 of them in a 5 second interval coming from the same location (agent) and from the same source IP:

<rule id=”100002″ level=”5″>
<if_sid>31108</if_sid>
<id>^206$</id>
<description>Web Server 206 Error Code</description>
</rule>

<rule id=”100003″ level=”10″ frequency=”8″ timeframe=”5″>
<if_matched_sid>100002</if_matched_sid>
<same_location />
<same_source_ip />
<description>Multiple Web Server 206 Error Codes </description>
<description>from Same Source IP</description>
<group>web_scan,recon,</group>
</rule>

At this point, the analyst will get alerted to the DoS condition, but we are not necessarily confident that it is the Range Header attack. There’s one more rule we can create that looks for the ‘page allocation failure’ occurring within five minutes of rule 100003 firing. If we see all of this, we are reasonably confident in what is going on:

<rule id=”100004″ level=”12″ timeframe=”300″>
<if_matched_sid>100003</if_matched_sid>
<if_sid>1002</if_sid>
<program_name>kernel</program_name>
<match>page allocation failure</match>
<description>Apache Range Header DoS Attack</description>
<group>attack,</group>
<info type=”cve”>2011-3192</info>
</rule>

So, what can go wrong? Lots. The system may be so unstable that it cannot send logs to the manager. The attack might be successful with fewer than ten 206 requests within five seconds. In some cases, the ‘page allocation failure’ does not appear in the logs, although the attack still might trigger rule 10003. At least this would give the analyst a chance to visually inspect the sample of logs OSSEC sends in the alert.

Are there better ways to detect this? Certainly. Mod_Security can not only detect but also prevent the attack. Tools such as Snort are better positioned than OSSEC in situations like this. Of course, OSSEC can monitor the logs of those tools, and still alert you. The point here was to demonstrate that there is an OSSEC-only way to detect attacks like this.

Do you see a problem with the rules? Can they be subverted easily? Let me know in the comments.

Awhile back, I posted about how certain versions of Windows always have the capability to lose logs. I encourage you to read the full post to understand the issues involved, then come back here and continue reading.

The basic problem is that Windows loads the full log into a shared memory space, and if it’s too big, then logs will be silently dropped. That’s why it is very important to have a centralized logging solution with logs sent in real-time or pretty close to it.

So how does centralized logging affect the local retention settings? To know that, you have to make certain assumptions about log size, average number of events generated per day, and so-on. Let’s start with this quote from Microsoft’s Threats and Countermeasures Guide:

Although there is no simple equation to determine the best log size for a particular server, you can calculate a reasonable size. The average event takes up about 500 bytes within each log, and the log file sizes must be a multiple of 64 KB. If you can estimate the average number of events that are generated each day for each type of log in your organization, you can determine a good size for each type of log file.

For example, if your file server generates 5,000 events per day in its Security log and you want to ensure that you have at least 4 weeks of data available at all times, then you would want to configure the size of that log to about 70 MB. (500 bytes * 5000 events/day * 28 days = 70,000,000 bytes.)

70MB doesn’t sound so bad. It’s probably well within the architectural limits of this memory-mapped I/O thing, but even that may be too big.

A better way of calculating what the local log size should be is to consider the recovery time objective and recovery point objective of the centralized log server. That is, consider how long it might take you to get your log server back online in the event of a disaster, then base your local Windows logging settings on that.

For example, if your recovery point objective is three days, you want to make sure that the local Windows logs will collect for at least three days, but not much more than that. Ideally, you also want to make sure that the solution you’re using can forward the logs collected during the downtime, or have a high-availability solution as part of the plan.

Remember, the goal is to get the local log sizes as small as possible, not as large as possible. You want to make sure you’re not losing any logs, and the two best ways to achieve that are to use centralized logging and try to avoid maxing out the services.exe process. This is counter-intuitive, but when you think it through, you’ll realize that it makes sense.

I recently blogged about security practices in a hospital environment that I was witness to. It was interesting to see how security worked (or perhaps didn’t work), rather than post about another standard, tool or best-practice.

Today, I bring you the story of Grandpa. Grandpa had a little information security incident and, well, it didn’t go so well. In fact, it went something like this:

Scene opens with me exiting out of the bedroom after a good night’s sleep…

“Grandpa thinks he has a virus,” my wife dutifully said.

“Coffee,” was the best reply I could muster.

You see, I knew that viruses were really no big deal. I’ve battled many a virus and have always been victorious.  I certainly knew that it could wait until after a cup o’ joe, so I relegated to call him a bit later.

I forgot.

While I was forgetting, Grandpa was in a mini-panic. Malware was telling him he was infected with all kinds of other malware, using a fake scan window of course, and that the only resource was to pay $49.95 for the program.

He bought it. Of course, that didn’t fix his problem.

Your first reaction might be something along the lines of, “what a dummy.” Maybe you’re thinking, “doesn’t he know not to click on links?” Perhaps you think he got what he deserved. While all of those responses may be justified, and maybe even correct given a certain perspective, let’s take another look at this.

Grandpa can’t hear. He is almost completely deaf. Since he grew up hearing, he doesn’t know sign language. He finds family functions to be extremely saddening and frustrating. When the little ones try to tell him something, he doesn’t know what to say because he didn’t hear a word they said.

Being a retiree, he also doesn’t have much income. In fact, there’s really nothing extra. Spending that $50 meant that he had to take $50 from another, probably essential, area. It wasn’t planned; it was more along the lines of, “I need to do this now!”

So, why did he pay this money when I would have helped him for free? Why did he fall victim to the scam? Why did he make an impulsive decision and not think it through?

The answer has nothing to do with how he has or hasn’t been educated about not clicking on untrusted links or installing untrustworthy software. He became a victim because, in that moment, he felt as if he had no other choice. His fight-or-flight response kicked in. He emotionally made a decision because he felt as if he was losing control over something very important. If the computer were no longer his, how would he communicate? What would he do with this time? How would he take his mind off of the multiple health problems that fills his thoughts?

I was able to fix his computer and even helped to get the charge reversed, but the point of this story is that security problems usually aren’t technical in nature. Obviously, trying to teach people to become security experts just to be able to have some fun or do their job effectively isn’t working. The approaches have to change.

We need to understand people better. The age of the security geek is passing. If we are to truly make progress on issues like this, the solutions will be less technical and more people-focused. If Grandpa had felt secure in knowing that I would help him no matter what, perhaps he would have made a different choice. If he had known that his computer would not be relinquished to this scammer unless he paid money, he might have taken awhile to watch a movie until I had called him back.

Security is about people. Let’s make that a mantra for 2010.

I have spent the last several days responding to a 0-day worm outbreak. We didn’t have signatures when the you-know-what hit the fan. Fortunately, some tooling we already had in place allowed us to contain the initial spread while we waited for emergency signatures from our AV vendor.

The last stage of any incident response is the lessons learned phase. This is where you examine the response and analyze all the things that went well and all of the things you could have done better.  There are always ways to improve the response.

I got to thinking that simulating a worm outbreak would be an excellent way to test a response, along with the detection tools you currently have in place. Although it wouldn’t cover all scenarios, and maybe not even most of them, creating an EICAR worm seems like an excellent way to set this up.

For the uninitiated, an EICAR signature is a string identified by AV vendors as a “test virus.” It is meant to show that AV is working.

So why not take this one step further and create a custom worm from the EICAR string? It would be simple: simply create a batch file with the EICAR string and attempt to loop through a list of network shares using a privileged account.

There are risks, of course. First and foremost, there would have to be absolutely no way for this to “infect” a system not under your management. That would simply be inexcusable. Second, there may need to be a way to throttle and control the spread. Some kind of “dead man’s switch.” Finally, there needs to be a way to pull-the-plug should things go really awry.

While this may sound like a crazy idea at first, which is better: to have finely tuned responses based on well-practiced and controlled scenarios, or to struggle through the next worm incident?

From the “What can we do to stop terrorism, without actually addressing terrorism” department, comes the news that scientists are researching how to sniff out scared people at checkpoints.

In the research, scientists discovered that they could literally detect the pheromones produced when someone is afraid. That’s not so surprising, but what is mind-boggling is that one of the proposed implications of the research is to be able to identify “scared” terrorists.

I’m not even sure where to begin with this one, but let’s give it a try. Here are just some of the potential vulnerabilities in this stupid idea:

• Many terrorists seem to be so brainwashed into believing that they are about to get 72 virgins that they’re probably more likely to be a bit “happy,” rather than scared if you know what I mean.
• Sociopaths won’t be scared.
• When we recently took my three year old daughter through an airport checkpoint she probably would have been tagged. It would have been because they took her Cabbage Patch doll to scan for hidden bombs.
• We better hope there are no nearby spiders and arachnaphobes.

Do I really need to continue?

Fighting terrorism with stupid ideas like this only serves to take the focus off those areas where we need to pay attention. With limited resources, we can’t afford to divert our attention from those techniques law enforcement has been using for years and which are proven to detect and stop criminals.

This idea smells stupid because it is.

A few of my posts have involved debating the ethics of public web app testing by security professionals. When the good guys poke and prod public web apps it raises a bunch of ethical questions, besides being legally questionable. Rather than recap my thoughts again, I invite you to read the article which I wrote for this month’s “ISSA Journal.” If you like the article and like some of the articles in the Journal archive, please consider supporting ISSA by joining. I have found it to be pretty valuable.

As always, I welcome your feedback. Feel free to challenge my assertions. All constructive comments will be let through moderation.

Don Norman is my hero for today.

Rarely do I read something and come away thinking, “this hits the nail right on the head. I can’t possibly think of a more eloquent way to say what is being said.” But Don manages to do just this in his essay about usability in security.

Don understands that security measures have diminishing points of return. He intuitively knows that asking more and more of users doesn’t necessarily make them more secure. He sees the underlying reasons why security is failing. He gets it.

I could go on and on, but I’m just going to ask you to take a few minutes out of your day and read his essay. Read it slowly and carefully, then print out copies and hand them to your colleagues.

Finally, think about some of the ways you may be unwittingly making security problems worse. Consider if your approach could use some fine tuning. Simplify, re-examine and take chances.

One of the big problems in information security is how to effectively teach small businesses safe data handling. They’re too small to have dedicated security budgets and they can’t be expected to publish volumes of security policies; yet, they have needs to handle information safely above and beyond what a normal consumer has to deal with.

NIST attempts to fill this gap with the Small Business Information Security: The Fundamentals guide. In the guide they detail what a small business should minimally be concerned with, along with some extra measures they may want to take.

While it has a little ways to go (it is a draft, after all), it’s a great start to filling this much needed void. Check it out and see if they manage to walk the fine line by making security simple, yet effective enough for small business.