Immutable Security

Most people that use the OSSEC Windows agent download a pre-compiled copy from the OSSEC site. While that is a good option for many individual users, it may not suit those with more specific needs and/or those in enterprise environments. Users who fall into those categories could benefit from customizing the agent and maintaining internal builds in order to suit their individual needs.

There are already instructions on how to compile the Windows agent on Linux, but ironically the process doesn’t work so well on Windows. I had a need to make this work on Windows, so I thought I would share the process with you.

First, there are some prerequisites.  You’ll need:

• The Nullsoft Scriptable Install System (NSIS)
• The Minimalist GNU for Windows compiler (MinGW)
• My gen_win batch file.  Simply rename from gen_win.txt to gen_win.cmd.
• 7-Zip for Windows
• The public domain Unix2DOS utility
• The latest OSSEC for Unix/Linux (this contains the Windows source code)

Here are the steps:

1. Download and install the required programs. Be sure to pay special attention to the steps for properly installing and configuring MinGW, particularly the part about modifying the PATH environment variable.
2. Next, we’re going to extract OSSEC using 7-Zip. To do so, simply right-click on the file and select 7-Zip, extract to “folder name.tar,” where folder name is the name of the package. This decompresses the archive. Navigate within that folder and repeat this step to untar the archive. At this point, you should see all of the files in the package.
3. Place gen_win.txt in the src\win32 folder and rename the extension to .cmd.
4. Download Unix2DOS and place it in the src\win32 folder
5. Open a command prompt. Navigate to src\win32, make any desired customizations, and execute gen_win.cmd. This should gather all of the required files and place them in src\win-pkg.
6. Next, we compile the Windows agent by navigating to src\win-pkg and executing make.bat (I assume you have the chops to know how to change directories :) ).
7. Now we have all of the files we need but no way to effectively install it. To generate the installer, simply execute the NSIS compiler like so: “c:\Program Files\NSIS\makensis.exe” ossec-installer.nsi

If you see no errors and a binary named ossec-win32-agent.exe, everything was successful. Congratulations, you now have a custom-made version of OSSEC!

There are four or five responses to risk, depending on who you ask. They are: mitigate, accept, transfer, reduce, and sometimes, ignore. Ignoring a risk is just a lame way of burying your head in the sand and pretending it doesn’t exist. Let’s just say there are four legitimate responses to risk.

What we usually hope for is mitigation and reduction. These responses deal with risk directly, but most importantly, they show the greatest amount of responsibility in dealing with the problem.

Transferring risk can be a legitimate response or it can be a way to essentially ignore it. It can be legitimate when there is a thoughtful decision to transfer to another party, and that party accepts. This can be an insurance policy, for example. It is illegitimate when transferring it to another party is a way of ignoring it, essentially just another way of saying, “it’s not my problem.”

That leaves acceptance. Acceptance means, “I know that this might happen, and if it does, I am prepared to accept the consequences of my decision.” The problem with acceptance is that it is rarely tied to consequences.

I see this problem in the real world all the time. Security professionals present risks to senior management and they simply do not respond. They don’t respond unless there is an incentive to do so, and agreeing to put themselves on the line with little or no positive return doesn’t win many fans. If the senior executive fails to respond and the risk is realized, they can just say, “I never saw the risk analysis,” or worse, “they failed to effectively communicate the risk to me.”

We need two things to make risk acceptance work. One, the risk acceptor can not be in a position of authority over the risk advisor. This creates a situation where the advisor might water down the risk out of fear for his own job. Two, there have to be consequences that tie directly to the risk being realized and for choosing to not mitigate, transfer or reduce it. Essentially, this means that ignoring the risk becomes default acceptance with a particular consequence. In both cases, the consequences of acceptance (either by default or explicitly) need to be commensurate with the risk.

Consequences tend to motivate people. Risk acceptance without consequences is not risk acceptance at all.

In a recent point/counterpoint over at techtarget.com, Marcus Ranum responded to Bruce Schneir on the issue of the risks of hiring hackers. Marcus said one thing in particular that struck me as very insightful. He writes:

We hear a lot about “thinking like a hacker,” but I think that’s largely nonsense — it’s really just a matter of “thinking like an engineer” and performing an in-line failure analysis along with your design analysis.

I think he’s mostly right.

Information security defenses are generally built based on compliance, the current new attack and logical analysis of the threats, in that order. But even when we perform a thoughtful risk analysis, we tend to over-emphasize the rare but sensationalized threats, and under-emphasize the mundane threats. Because information security is seen as a solution to the problem of other people actively attacking systems, we tend to design our countermeasures around those threats.

While it’s true that attacks are happening all the time, the vast majority of those are non-targeted attacks that can be resolved by simple countermeasures such as patching. What we tend to underestimate is loss from areas where information security practitioners have traditionally not had a foothold. We don’t design good countermeasures around forgetting to renew the domain name, a service provider going out of business, or someone hitting the delete key by mistake. But the end result is the same–a loss of confidentiality, integrity or availability.

So where does thinking like a hacker come in? Do we need to think like a hacker to protect against a hacker? I suppose that depends on your definition of what a hacker is. If that definition involves creative thinking, considering how things break, and coming up with solid solutions, then yes, thinking like a hacker is good. But then that is really just good security engineering.

Take this site for example. Before it went live, I played the “what if” game. I started to ponder the many ways this site could break. Along with considering the usual stuff like SQL injection and XSS, I wondered what would happen if my DNS provider suddenly went offline. I assumed that someone would steal the server from the data center, or that a government subpoena might inadvertently include this site along with the site they were targeting. I assumed I would turn off some of the security measures temporarily to see if something started to work again. I then assumed I would forget to turn them back on.

I designed countermeasures for all of these scenarios to a certain extent. At some point it wasn’t worth going any further–it is a blog after all. The point is that I didn’t so much design to be attacked as I designed it to be resilient to failure. Does that mean it is fail-proof? Absolutely not. The site could still be defaced. The server could crash and I might have a problem with restoring the backup. I might get hit by a bus. Despite all this, I am comfortable with the level of security engineering I put into the site.

To sum this up, thinking like a hacker is not that different than thinking like a security engineer. It’s two sides of the same coin. To consider how systems can be resilient to failure (the security engineer), we need to consider how they will break (the hacker).

And sometimes, just sometimes, they break in spectacular ways.

I had the good fortune recently to take a few days off. We decided to travel to a city a few hours away by train, a method of travel that is generally comfortable and relaxing.

Being a security guy, I couldn’t help but notice the lack of security. My ID was checked only once when heading to our destination, but not on the leg back. The guy obviously didn’t even look at it very closely and he didn’t use one of those little ultraviolet lights. My luggage wasn’t inspected at all, nor was my person. There were no metal detectors. All I noticed was a few cameras and a sign telling me to watch out for suspicious stuff.

It would have been trivial to load myself up with automatic weapons or even pack my suitcase with explosives. At a minimum, I could have destroyed the portion of the train I was on and killed everyone on it.

I got to thinking–in the post 9/11 world we live in, how could this be? Didn’t they think to secure the rail system? Wouldn’t an attack on the railway instill fear in America and be an easy target? Of course they must have thought about this. There must be other explanations.

Perhaps the intelligence indicates that the railway really isn’t a target. Perhaps Amtrak doesn’t have enough money to implement something like the airlines have, and since they haven’t been given a mandate or been taken over by the government, they haven’t done anything. Maybe it’s “in the works.” Maybe there aren’t enough resources to go around and this is at the bottom of the list.

Whatever the case, I came to realize that I really, really enjoyed the lack of security. I didn’t feel any less safe by not having my luggage swabbed. I realized that I most likely had a much higher risk of dying in the car on the way to the Amtrak station than I did by a terrorist attack on the actual train.

Would I feel differently if there had been a recent terrorist attack on a train, or if I had survived a terrorist attack anywhere? Possibly. But not having gone through that experience, I realize that fearing an attack on a train is a mostly irrational fear and a risk that may not be worth doing anything about.

Now the lack of on-board wi-fi is another matter entirely…

Logrotate is a fantastic little utility for, well, rotating logs. It has several options available. It can rotate by date, file size, set specific owners on rotated files, and it even gives you the capability to run pre and post-rotate scripts.

Getting it to work on a centralized log server, however, can be a bit tricky. The issue lies in the unique way these log servers are often configured. Rather than a static set of directories, log servers often dynamically create directories based on the hosts that are logging to them. It’s not uncommon to see a directory structure something along the lines of /mnt/logs/2010/May/18/host/messages. While logrotate can use wildcards in the directory structure definition, it gets a bit hairy when trying to define the rotated log directory.

There are two options for where to put the rotated logs. You can put them in another directory, or in the same directory as the rotated files. Another directory is usually the better option, as rotated logs in the same directory could be rotated again and again, unless you are careful with the way logs are created.

So what happens if the directory you specify for the rotated log files doesn’t exist (remember, the log server is creating them dynamically based on the hosts that report in)? In that case, logrotate will complain and refuse to rotate your logs.

By now you might be thinking, “Why, I’ll just use that handy pre-rotate option. Then I can have it run a script prior to the rotation.” Sorry, no dice. Logrotate will simply check the config prior to even starting and refuse to do anything useful.

The solution is to use a script, completely outside the scope of logrotate, to create these rotated logs directories just before logrotate runs. Let’s say all remote hosts log to /mnt/logs/yyyy/mmm/dd/<hostname>/<log-name>. Your logrotate.conf file might look like this:

/mnt/logs/*/*/*/* {
weekly
dateext
compress
maxage 365
olddir rotated

}

This would rotate all logs within that directory structure to the relative directory called “rotated.” But of course we need the rotated directory to exist. So let’s create a small script to do that:

#!/bin/bash

# Create “rotated” directory so logrotate has somewhere to put the rotated files.
# Logrotate will not do this by itself, even when putting the script in the “prerotate” section

for i in /mnt/logs/*/*/*; do
if ! [ -d $i/rotated ]; then
/bin/mkdir $i/rotated
/bin/chown root.log_analyst $i/rotated
/bin/chmod 550 $i/rotated
fi
done

In the above script, we’re checking to see if the directory exists, and if it doesn’t, we create it. Then we set proper permissions, owner and group in order to ensure integrity of the rotated log directory. I am assuming a group of log_analyst, so change as appropriate.

The final step it to ensure the script runs just before the logrotate happens. In RHEL, logrotate is called from within the cron.daily directory. So we’ll just name the script make_rotated.sh and link it in this directory in such a way that it runs before logrotate:

[root@hostname log]# ll /etc/cron.daily/ | grep logrotate
total 64
lrwxrwxrwx 1 root root 31 Apr 8 15:19 01_pre-logrotate -> /usr/local/sbin/make_rotated.sh
-rwx—— 1 root root 180 Feb 26 2009 logrotate

There are certainly other ways to solve this problem, but this seems to work well. Until next time…

The new version of Ubuntu includes integration with the 7digital.com music store. No longer do open source users have to rely on proprietary applications like iTunes. They can now purchase good-quality music using only free software.

Like most other online music stores today, 7digital sells non-proprietary, but patent-encumbered MP3 files. This is leaps and bounds better than the DRM-laden files which used to be sold by stores like Amazon and iTunes.

And while there is a bug which discusses some of the issues surrounding relying on a patent-encumbered format, rather than something free like Vorbis, there is one critical issue that free software users are not debating.

Allow me to demonstrate by quoting from the 7digital.com Terms and Conditions:

(i) You are authorized to use the Content only for personal, non-commercial use, and not for redistribution, transfer, assignment or sublicense, to the extent permitted by law.

So, unlike a traditional CD you buy at the store, you are contractually agreeing not to lend it to your friend, donate it to your library, or when you’re tired of it, throw it up on eBay for sale. In short, it makes it a breach of contract to share your music or even to sell it when you’re you have outgrown it or lost your job.

The Terms and Conditions continue:

(ii) You are authorized to use the Content on up to five authorized devices at any time. 7 reserves the right to limit the number of authorized devices further and the number of authorized downloads to comply with the wishes of its licensors.

And there we have the DRM. The difference here is that it is implemented in a contract, not in technology.

And just when you thought it might be over, they throw in this little gem:

(iii) You may not use Content as a musical “ringer” in connection with mobile phone calls.

Take all the fun away, why don’t you.

Ubuntu certainly isn’t the only one guilty of implementing the DRM through a contract rather than technology, but honestly, I hoped for better from a distribution that espouses to be freedom-focused.

If the free software community can accept this, what hope do we have for retaining the rights under U.S. copyright law that we have enjoyed for so long? If we contractually give up those rights on music, video and most importantly, books, what does that say for the future of the independent after-markets, fair use and archival?

For those that did not see me at the Rochester Security Summit last year, and who would like to see me present my OSSEC in the Enterprise presentation, I will be giving it again at the ISSA Ft. Worth chapter meeting, next Wednesday. Details are as follows:

Meeting Location: Devry University, 301 W. Commerce Street, Fort Worth, TX 76102

Phone: 817-810-9114

Parking is up to you, there is a parking garage, lots & meters.

FWT Consulting will be sponsoring the meeting & providing lunch, so please RSVP to steve.streiffert <a t> gmail.com by Monday, May 17, 2010.

Encrypted is good. Clear-text is bad. AV is good. Not having AV is bad. Those are the messages we have been receiving and teaching for some time now. And while they do contain sound security advice, it is important to also consider the context of the situation at hand.

A colleague of mine recently requested a firewall exception for FTP traffic. The security policy did not normally allow clear-text traffic, which is generally a good policy to have, but in this case it was justified.

He was trying to get DAT updates for AV on a Linux box (we’ll get to that in a minute), and anonymous FTP was the best way to do it.

Consider this:

1. There was already an FTP service running, so adding an additional service would mean one more to patch, secure, etc.
2. The access allowed was only anonymous, making the issue of clear-text password transition moot.
3. Only read access was allowed, thereby mitigating file integrity issues.
4. Since these were ultimately publicly available DAT files, file confidentiality was not an issue.

After careful examination, it was clear that clear-text FTP was the best option.

Antivirus on Linux is another area where the cure may be worse than the disease. Although you’re not likely to win over your auditor friends, sometimes it can be demonstrated that running AV on Linux is a bigger threat than not running it.

Antivirus software is software, just like every other kind of software. It is prone to security failures and attacks. Is an AV agent with a history of vulnerabilities more of a liability on a system which is not likely to be a target of viruses? Perhaps, perhaps not. Again, it depends a lot on context.

Context helps us to make good security decisions–decisions that are based on risk, reason and facts, not necessarily best-practices or requirements. Keep an open mind.

Awhile back, I posted about how certain versions of Windows always have the capability to lose logs. I encourage you to read the full post to understand the issues involved, then come back here and continue reading.

The basic problem is that Windows loads the full log into a shared memory space, and if it’s too big, then logs will be silently dropped. That’s why it is very important to have a centralized logging solution with logs sent in real-time or pretty close to it.

So how does centralized logging affect the local retention settings? To know that, you have to make certain assumptions about log size, average number of events generated per day, and so-on. Let’s start with this quote from Microsoft’s Threats and Countermeasures Guide:

Although there is no simple equation to determine the best log size for a particular server, you can calculate a reasonable size. The average event takes up about 500 bytes within each log, and the log file sizes must be a multiple of 64 KB. If you can estimate the average number of events that are generated each day for each type of log in your organization, you can determine a good size for each type of log file.

For example, if your file server generates 5,000 events per day in its Security log and you want to ensure that you have at least 4 weeks of data available at all times, then you would want to configure the size of that log to about 70 MB. (500 bytes * 5000 events/day * 28 days = 70,000,000 bytes.)

70MB doesn’t sound so bad. It’s probably well within the architectural limits of this memory-mapped I/O thing, but even that may be too big.

A better way of calculating what the local log size should be is to consider the recovery time objective and recovery point objective of the centralized log server. That is, consider how long it might take you to get your log server back online in the event of a disaster, then base your local Windows logging settings on that.

For example, if your recovery point objective is three days, you want to make sure that the local Windows logs will collect for at least three days, but not much more than that. Ideally, you also want to make sure that the solution you’re using can forward the logs collected during the downtime, or have a high-availability solution as part of the plan.

Remember, the goal is to get the local log sizes as small as possible, not as large as possible. You want to make sure you’re not losing any logs, and the two best ways to achieve that are to use centralized logging and try to avoid maxing out the services.exe process. This is counter-intuitive, but when you think it through, you’ll realize that it makes sense.

Over at the Apache blog, you’ll find a nice and detailed incident report on the recent, successful attack on Apache.org. I thought it might be worth a few minutes to share my thoughts on their write-up.

First, I would like to say that the level of transparency in this response is truly commendable. Rather than sweep this under the rug, they have chosen to share the details of what happened, why it happened (more on that in a moment) and what their plans are to, hopefully, prevent future breaches.

I encourage you to read the entire post, because it is a good account of how actual, real-world attacks happen. Targeted attacks take advantage of trust (both in people and machines), shared and weak passwords, too much privilege and an assortment of other security 101 vulnerabilities.

The attacks consisted of a XSS vulnerability, brute-force logins, a shortened URL, password sniffing, password re-use and social engineering. Pretty typical stuff, really.

What I find most interesting about this report is the emphasis on technical countermeasures in the What are we Changing? section, when the attack succeeded primarily due to vulnerabilities in human beings.

• The Infrastructure Team members clicked on a cloaked and untrusted link, which launched a XSS attack.
• A brute force login succeeded against a poorly chosen password. But prior to it being successful, no one seemed to be getting alerts on so many failed login attempts.
• They once again exploited the Infrastructure Team by getting them to log in with a password that the team members, themselves, did not choose.
• They took advantage of cached passwords on the server.
• Slicehost didn’t respond to the attack when notified, which enabled one host to continue its attack against someone else.

These are all people issues. It’s the same stuff that we security types have been trying to hammer into the brains of people for years now. There are certainly technical countermeasures which could have helped, but this was an attack on people.

It’s easy for us to play armchair quarterback and be critical of the response, however that is not my intention, Rather, it is my intent to simply cast another light on the response so we can all learn and secure our assets more effectively.