Immutable Security

Open source log analysis types sometimes need to be crafty. We often don’t have relationships with the companies who’s products we support, but that doesn’t stop the users from wanting to use software like OSSEC to analyze logs from big, enterprise-type closed source stuff.

We scour the Internet for log samples and documentation–anything, really–that might help us out.  So today when I came across some documentation for IBM’s Compliance Insight Manager, I thought I would pass along what I found.

Under the Configuring auditing for specific platforms section, you’ll find instructions on how to configure auditing for dozens of enterprise systems. It even goes far beyond what to click or which file to edit. There is info on audit capabilities, monitoring recommendations, suggested audit settings and tuning.

This is a worthwhile resource to bookmark. Until next time..

Sometime when I wasn’t paying attention, a bunch of marketing folds must have gotten together to come up with a new, catchy acronym. I imagine the meeting must have gone something like this:

Joe: We’re not selling enough of our <insert product here>. We need a way to really connect with people.

Linda: The problem is branding. Cross-site Request Forgery doesn’t really roll off the tongue too well.

Bill: Hmm… Advanced.. Problem.. Fixer..

Tom: That’s a good start. How about Advanced Persistent.. Software..?

Joe: Wait for it… Advanced Persistent Threat!

All in unison: Awesome! Let’s go for drinks.

OK, so maybe it didn’t go down exactly that way, but it’s fun to imagine.

So, what exactly is an Advanced Persistent Threat, or APT? According to Wikipedia, APT attacks utilize traditional attack vectors such as malware and social engineering, but also extend to advanced attacks such as satellite imaging. It’s a low-and-slow attack, designed to go undetected.  Finally, there is a specific objective behind it, rather than the incoherent activity of some fifteen-year-old hacking away in a basement for brownie points with his buddies.

It’s not just vendors getting in on the game–companies are increasingly blaming their security failures on APT, as if it was far too sophisticated for them to possibly defend against.

There’s no doubt that such attacks exist. Corporate espionage and nation-state attacks are very real and, in some cases, extremely sophisticated. But these attacks are very rare. The truth is that the vast majority of attacks are not very advanced because they don’t need to be. It is extremely difficult to defend against all known attack vectors. The defenders have to get everything right, all of the time. The attackers only have to find one or a few small holes to work their way in. That’s just the current state of information security.

I think we should generally avoid using the term Advanced Persistent Threat. There are two main reasons I feel this way.

1. It’s highly likely that it’s not an APT at all. Even if you have a great security program with the smartest security people in the world, a company of any appreciable size is going to have hundreds of ways in. You can have everything patched and the front-desk lady will give up her password. You can require two-factor authentication for all remote access until the smart phones come along. You can have great perimeter security until your business partner gets compromised, and you realize that your perimeter really doesn’t begin and end at the firewall. Face it, your company is insecure on its best day.
2. Using terms such as APT, while sexy, encourage us to gloss over the actual facts. Just as intellectual property is a way of lumping together things like copyright and trademark, APT discussions keep us from focusing on which attacks were actually used and how our defenses failed. There is no APT attack. There are SQL injection attacks. There are social engineering attacks. There are buffer overflows in software. There are default passwords left on systems. There are insecure trust relationships. APT is a dangerous umbrella term.

Even those few who do face what people call APT attacks need to break them down into their core elements in order to understand and defend against them. For the rest of us, let’s go back to discussing how our security design failures could lead to a compromise. And if one should occur, let’s speak to the specifics of the attacks so we can learn our lesson, even if a little humility is in order.

One of the things us log analysis types love is good documentation. It’s rare to find well-formatted, well-documented logs, so when we do find good log info, it’s like being a kid in a candy store. So without further ado, I present all of the links you will need to describe every Windows Server Security Event log that exists:

Windows NT (please tell me you don’t really need this):

KB174074 – Security Event Descriptions

Windows 2000

KB299475 – Windows 2000 Security Event Descriptions (Part 1 of 2)

KB301677 – Windows 2000 Security Event Descriptions (Part 2 of 2)

Windows 2003

Windows 2003 Security Guide, Chapter 4, Audit Policy

Windows Vista to Present

KB947226 – Description of security events in Windows Vista and in Windows Server 2008

Security audit events for Microsoft Windows Server 2008 and Microsoft Windows Vista (This one is fantastic–every log in spreadsheet form. Export to csv for an easily-parsable and machine-readable format!)

Of course, just having the documentation doesn’t necessarily tell you the whole picture. Sometimes it’s nice to have a discussion about the log event. Two of my favorite sites for this are Ultimate Windows Security and EventID.net. In particular, the Security Log Encyclopedia is a great reference when you want to quickly know what a particular event ID means, but also get some contextual and practical information about the event.

Remember that documentation isn’t always correct. There may be subtle differences in what events are logged or how they are presented depending on the service pack level. What other resources have you found?

For the past couple of weeks, I have been reading with great interest the coverage of Sony deciding to bring suit against George Hotz. George, or GeoHot, as he is known, and others like him, hacked the PS3 after Sony removed the “Other OS” feature. It was this “Other OS” feature that appealed to people like George in the first place, since it allowed the more technical among us to use the PS3 in interesting ways–such as to run a custom version of Linux.

There are many elements to this story: Is What GeoHot did illegal, as Sony claims? Will it lead to more piracy? If it did lead to more piracy, would it even matter? Was what Sony did by removing a feature of the device illegal or unethical? But I think the two main questions above all are: Who owns the device, and How far does free speech go?

We live in an era where corporations are asserting more and more control over the devices we purchase. Unlike a physical book or a lamp, the technology of today allows for interactivity between the manufacturer of the product and the product, itself. Companies like Apple and Sony have attempted to use this to their economic advantage by restricting what valid purchasers of the product can do with their own device. They restrict what apps you run, if you can resell it, and can even take the product back on a whim. Make no mistake: the rights we have enjoyed over our own property are under full frontal assault. Companies like Sony would like nothing better than to convince you that you don’t actually own the product you purchased–that it is really just a long-term rental–that they get to decide the rules.

The other main question besides device ownership is: How far does free speech go? We already know that free speech is not unlimited. You can’t yell “fire” in a move theater and not expect consequences. But at the same time, we are actually very tolerant of free speech. The fourth ammendment guarantees the rights of white supremacists just as it does those whose speech we find copasetic.

“But, wait!” observant readers will say. Sony is not the government and this is a civil issue. If Sony thinks it has been harmed by the free speech that is the release of the private key, they have a right to address that in the courts. Surely, if the secret formula to Coca-cola were to be released, you would expect an army of lawyers to descend on that person.

While it is true that speech can harm a company, a large corporation like Sony can’t necessarily be expected to win a case just because it doesn’t like someone’s speech. They actually have to prove harm in some way, be it libel, slander, trademark infringement or what-have-you. If that weren’t true, and if the law didn’t provide some protection even in cases where the government wasn’t involved, companies like Sony would successfully sue every security researcher every time a new flaw is found.

This case is not about piracy. If George is to be believed, he has never used the Sony online service, never assented to the EULA and never pirated a game. This is about Sony attempting to send a message: The PS3 is ours, not yours. Play by our rules or we will ruin you financially. To anyone else: freely discuss the hack, or, for that matter, look at it, and we will come after you, too.

I have considered everything from staying completely silent on this issue–certainly the safe choice career-wise–to getting a tattoo of the leaked key. But I can stay silent no more. Sometimes you have to speak up for what you believe is fair and right. And I believe it was fair and right for George Hotz to use his device in any way he chose to. I believe it was wrong for Sony to remove a feature that people paid for and had a reasonable expectation to be able to use. And I believe it is right for everyone to freely and openly discuss the hack, including the key, so that they may use their own device in any way that does not involve piracy, and to further the discussion about what device security means.

If the facts are truly what they appear to be, then I support George Hotz and I wish him well in his case.

I have a dirty little secret. It doesn’t have anything to do with the NSA, a leaked memo or pink leotards. But it’s a secret just as earth-shattering, just as awe-inspiring and just as potentially devastating as any other well-hidden secret that has been revealed. Intrigued? Ok, here it is: I have sometimes knowingly agreed to, or even recommended less than secure solutions. Let me give you a minute to collect yourselves.

Yeah, that’s right. Call the ISSA Ethics Committee. Call ISC(2). Call the President. I have been a bad, bad infosec professional. Do what you must, but I have a right to a defense. Allow me to explain my side of the story…

When I was just a young lad starting out in the security field, I had a pretty firm idea of what I thought it meant to be secure. I attended the right conferences, read the hardening guides and dutifully regurgitated what I had learned. Why would anyone choose to be insecure? I thought. That just didn’t make sense.  Security was about correctness. To be less-than-secure was like a mistake on an exam. It needed to be corrected.

As I went along, I learned that the take-it-or-leave-it approach had a few drawbacks. People didn’t really think the same way I did. They had their own jobs to worry about. When security got in the way, it was easier to route around it. When everyone was able to do their jobs every day, the “hackers are coming” argument became less and less effective.

Eventually, I noticed something interesting. The people that I had developed good relationships with were a bit more amenable to my suggestions. When I listened to the the challenges they faced and explained the benefits of what I was trying to do, they saw things in a different light. I made a bit more progress.

Today, I still try to build these important relationships. And when dealing with people who are not security minded, I am much more likely to make compromises. When asked to exclude a location from antivirus scanning that I know doesn’t need to be excluded, I might offer to make exclusions on read only, so that anything newly written will be scanned. When hardening a system, I might purposely leave out a setting that I think will likely cause the administrators to experience a lot of trouble and subsequently develop a negative association with security. I might even recommend that a risk not be addressed at this time, so as to focus limited resources on more important issues.

As infosec practitioners, we are the diplomats of our profession. We want others to come to us with security concerns, not try to route around us. We want to be partners with those we serve. And just like any other form of diplomacy, that involves compromise.

Everyone is doing it. Whenever the new year rolls around, security bloggers feel the urge to predict the year ahead. We invent new acronyms like APT (Advanced Persistent Threat), talk about mobile malware shutting down communications networks and warn about cyber attackers as if they were just waiting to pounce on some undefendable threat. I am here to tell you that none of that matters.

It’s true that attacks do get more sophisticated. As technology gives us new options to communicate, we have to consider the security implications of using that new technology. We might need to update our tactical strategy to deal with these threats, but our fundamental strategy likely doesn’t need to be changed.

Security engineering principles transcend technology. If the principles are sound, they will survive new technology with little need for change. Core principles such as least privilege, compartmentalization, kneed-to-know and fail-safe apply even to threats that we don’t know about yet.

So, don’t sweat the new stuff. When considering new threats, remember that they are likely just old threats, repackaged. Keep your eye on the ball. Look at your existing countermeasures and, if you have done a good enough job applying the core principles, you may find that you are already protected.

Model likely threat scenarios. What would it mean to incident response if a worm did take out the cell network? Had you already considered that a natural disaster might occur and then implemented something like walkie-talkies? If so, then congratulations–you just protected yourself against the worm threat.

For myself, I’ll just keep doing what I have been doing. Oh, and while I have your attention: “Get off my lawn!”

I have been studying Oracle logging for the last couple of weeks. Oracle can log to the SYS.AUD$ table within the database, a flat file, XML file, or it can use the OS logging facility (in Windows this is the event log; in ‘nix, it is syslog).

Preferring ‘nix-based solutions, I downloaded Oracle XE 11g for Linux and configured it for logging to syslog. It wasn’t long before I had logs like this:

Dec 28 22:32:41 localhost Oracle Audit[4958]: ACTION : ‘CONNECT’ DATABASE USER: ‘/’ PRIVILEGE : SYSDBA CLIENT USER: username CLIENT TERMINAL: pts/2 STATUS: 0

At first glance, it looked like a pretty standard syslog message, but after having some issues getting OSSEC to pre-decode it properly, I decided to check the RFC to see if it was technically compliant. Here is the relevant part of RFC 3164.

The MSG part has two fields known as the TAG field and the CONTENT field. The value in the TAG field will be the name of the program or process that generated the message. The CONTENT contains the details of the message. This has traditionally been a freeform message that gives some detailed information of the event. The TAG is a string of ABNF alphanumeric characters that MUST NOT exceed 32 characters. Any non-alphanumeric character will terminate the TAG field and will be assumed to be the starting character of the CONTENT field. Most commonly, the first character of the CONTENT field that signifies the conclusion of the TAG field has been seen to be the left square bracket character (“[“), a colon character (“:”), or a space character. This is explained in more detail in Section 5.3.

Did you spot it? The problem is that Oracle likely intended the string ‘Oracle Audit’ to compromise the TAG field (program or process name); however, the TAG field in this case really is just ‘Oracle’ since it is terminated by a non-alphanumeric character (the space).

So is it compliant? I would have to say yes, but I don’t think it is compliant in the way they intended. Simply removing the string ‘Audit’ in this case would have made a clearly compliant and understandable message.

As a young infosec practitioner, I quickly learned that morals and ethics had to be intertwined with everything I did. Someone with the knowledge of how to defend systems usually has a pretty good grasp on how to attack them. Had I not considered morals and ethics as a foundation to all I did professionally, I faced a limited career and even the possibility of jail time.

Throughout the years, I have learned that this perspective is somewhat unique. Infosec practitioners are usually not the owners of the data; rather, we provide guidance and feedback on how to keep data safe and secure. It is up to the data owners to either take our advice or not, and when it comes to the ethical and moral considerations within security, we often don’t see eye-to-eye.

From my perspective, a data breach is more than just a technical or process failure. When it involves custodianship of other people’s information, it is a breach of trust–a broken promise to those who entrusted their information to you. With that broken promise comes a moral imperative: an honest effort to apologize and make things whole again. From the perspective of those not within the security field, a breach ranges from something that simply needs to be fixed, to something that no one can ever know about, even at the expense of laws.

During my career, I have been asked to do things by data owners that seem to be on-the-edge, ethically speaking. In some cases, I have politely declined–such as when someone asked me to change a ticket type so an auditor for an upcoming audit would not discover the problem. In other cases, the situation hasn’t been so black-and-white–such as when I was recently asked to recommend an alternative antivirus for Windows NT. I could have simply recommend an antivirus solution as asked, with the caveat that I know it would not adequately protect something as insecure as Windows NT. Instead, I gave choices in order of the preferred solutions: move off the platform, invest in an IPS, or accept AV knowing that it will not protect the system. Still, I wonder if I should have even presented AV as an option.

We have an implicit moral imperative to act responsibly. Just as a responsible electrician would not shove frayed wires back into a wall cavity, so do we have a responsibility to promote only the highest standards and, in some cases, actually refuse to act in a way that a data owner may ask us to.

There will never be completely clear answers on how we should act in the face of situations that are somewhat muddy, but it is clear that a line has to be drawn somewhere. Where that line exists depends on the individual, and on their own set of unique circumstances. If the infosec industry is to be considered trusted and professional, each person has to have a line that they will not cross, and there should be a line that we as a group will not cross.

Where is your line?

When I bought a laptop about three years ago, I booted it up, read the Windows Vista EULA and decided it wasn’t for me. A quick reboot and install of Ubuntu took care of my concerns and has served me well since then. So when that laptop bit the dust, I already knew that Windows wouldn’t be on the laptop long enough to boot to the EULA.

Even though I am using predominantly free software, there are trade-offs and decisions to be made. Do I want to use the free ATI driver at the expense of 3D acceleration and performance or the more fully-featured non-free version? Do I want to use the Adobe Flash player (from a company who had Dimitry Sklyarov arrested for legal activities in his own country) or the free-but-somewhat-buggy Gnash player? Would I be willing to give up contributing to OSSEC Windows Agent development, even though the agent, itself, is free?

Perspectives on software freedom range from purists such as Richard Stallman, who believe all software should be free, to people like my wife, who really don’t care and would rather just have it work. For myself, I am most interested in maintaining a healthy marketplace where free and non-free software can offer users viable alternatives–a marketplace that ensures information can be exchanged freely and easily. Using and maintaining proficiency in free software allows me to easily make that choice if the developer of a non-free software application presents unacceptable terms.

We’re entering a new era of computing. It’s an era where phones and tablets are finally making their mark, while desktop computing takes a back seat. It’s also an era where user choice is being annihilated by companies like Apple, who make it abundantly clear that they consider the device they sold to the consumer to still be theirs, and who act as the gatekeeper deciding exactly how you can use your device. It’s an era where the bundling of the browser to the OS is the least of our worries; now the companies control the entire platform.

Freedom is all about choice. It’s also about evaluating the trade-offs. When there is a clear free and non-free solution to my problem, I try to default to the free option. By doing so, I help to keep the ecosystem alive and thriving, which, in some small way, ensures the free flow of our information now and in the future.

Update: A perfect example of Apple trying to control the free flow of information can be found in this article, in which Apple is described as removing the Wikileaks app. It is not for Apple to decide whether or not its customers should be the consumer of such information on devices they own. Enough said.

Awhile back, I blogged about how not to handle notification of a possible breach. In that case, I began to receive spam to a very unique address only used at one place. When I attempted to report the potential breach, I was at first stonewalled, and then “cautioned” against publicly discussing the issue.

Unfortunately, the stakes have risen from spam to outright malicious e-mails and this time the suspected company is Payscale.com. When I received a malicious PDF to a very unique address only used for that site, I wanted to let them know about a possible breach. So I sent an e-mail to every one of the contacts I could find on their web site. I wanted to make sure someone knew about this.

After over a week with no response from anyone, I received another e-mail, this time a malicious link posing as an Adobe Flash Player update. Still no response.

I think the right thing to do is to publicly discuss the issue. I think that when a company doesn’t respond to concerns such as this, and the public entrusts their data to them, it is ethical and right to publicly discuss the issue so people can make an informed choice about doing business with that company.

I can’t say that Payscale.com has been breached because I don’t know. What I can say is that the source of these malicious e-mails seems to have a strong connection to this company, and that they did not respond to a possible breach notification. Consumer beware.

Update: I am happy to report that, as a result of this post, I was contacted by someone at Payscale.com. My contact does seem genuinely concerned with looking into the issue. Again, this may or may not be anything, but the point of the post seems to have succeeded–getting someone to acknowledge that a security issue may exist.