Immutable Security

I have a dirty little secret. It doesn’t have anything to do with the NSA, a leaked memo or pink leotards. But it’s a secret just as earth-shattering, just as awe-inspiring and just as potentially devastating as any other well-hidden secret that has been revealed. Intrigued? Ok, here it is: I have sometimes knowingly agreed to, or even recommended less than secure solutions. Let me give you a minute to collect yourselves.

Yeah, that’s right. Call the ISSA Ethics Committee. Call ISC(2). Call the President. I have been a bad, bad infosec professional. Do what you must, but I have a right to a defense. Allow me to explain my side of the story…

When I was just a young lad starting out in the security field, I had a pretty firm idea of what I thought it meant to be secure. I attended the right conferences, read the hardening guides and dutifully regurgitated what I had learned. Why would anyone choose to be insecure? I thought. That just didn’t make sense.  Security was about correctness. To be less-than-secure was like a mistake on an exam. It needed to be corrected.

As I went along, I learned that the take-it-or-leave-it approach had a few drawbacks. People didn’t really think the same way I did. They had their own jobs to worry about. When security got in the way, it was easier to route around it. When everyone was able to do their jobs every day, the “hackers are coming” argument became less and less effective.

Eventually, I noticed something interesting. The people that I had developed good relationships with were a bit more amenable to my suggestions. When I listened to the the challenges they faced and explained the benefits of what I was trying to do, they saw things in a different light. I made a bit more progress.

Today, I still try to build these important relationships. And when dealing with people who are not security minded, I am much more likely to make compromises. When asked to exclude a location from antivirus scanning that I know doesn’t need to be excluded, I might offer to make exclusions on read only, so that anything newly written will be scanned. When hardening a system, I might purposely leave out a setting that I think will likely cause the administrators to experience a lot of trouble and subsequently develop a negative association with security. I might even recommend that a risk not be addressed at this time, so as to focus limited resources on more important issues.

As infosec practitioners, we are the diplomats of our profession. We want others to come to us with security concerns, not try to route around us. We want to be partners with those we serve. And just like any other form of diplomacy, that involves compromise.

As a young infosec practitioner, I quickly learned that morals and ethics had to be intertwined with everything I did. Someone with the knowledge of how to defend systems usually has a pretty good grasp on how to attack them. Had I not considered morals and ethics as a foundation to all I did professionally, I faced a limited career and even the possibility of jail time.

Throughout the years, I have learned that this perspective is somewhat unique. Infosec practitioners are usually not the owners of the data; rather, we provide guidance and feedback on how to keep data safe and secure. It is up to the data owners to either take our advice or not, and when it comes to the ethical and moral considerations within security, we often don’t see eye-to-eye.

From my perspective, a data breach is more than just a technical or process failure. When it involves custodianship of other people’s information, it is a breach of trust–a broken promise to those who entrusted their information to you. With that broken promise comes a moral imperative: an honest effort to apologize and make things whole again. From the perspective of those not within the security field, a breach ranges from something that simply needs to be fixed, to something that no one can ever know about, even at the expense of laws.

During my career, I have been asked to do things by data owners that seem to be on-the-edge, ethically speaking. In some cases, I have politely declined–such as when someone asked me to change a ticket type so an auditor for an upcoming audit would not discover the problem. In other cases, the situation hasn’t been so black-and-white–such as when I was recently asked to recommend an alternative antivirus for Windows NT. I could have simply recommend an antivirus solution as asked, with the caveat that I know it would not adequately protect something as insecure as Windows NT. Instead, I gave choices in order of the preferred solutions: move off the platform, invest in an IPS, or accept AV knowing that it will not protect the system. Still, I wonder if I should have even presented AV as an option.

We have an implicit moral imperative to act responsibly. Just as a responsible electrician would not shove frayed wires back into a wall cavity, so do we have a responsibility to promote only the highest standards and, in some cases, actually refuse to act in a way that a data owner may ask us to.

There will never be completely clear answers on how we should act in the face of situations that are somewhat muddy, but it is clear that a line has to be drawn somewhere. Where that line exists depends on the individual, and on their own set of unique circumstances. If the infosec industry is to be considered trusted and professional, each person has to have a line that they will not cross, and there should be a line that we as a group will not cross.

Where is your line?

In the security research community, it is commonly held that the ethical thing to do when discovering a vulnerability is to contact the software developer. Only after a lack of response, after the vulnerability has been fixed, or after the vulnerability has not been fixed within a reasonable time, should the information be disclosed.

But what is the right thing to do when there is an indication that a company may have been compromised? Is it ethical to disclose at all? Is it ethical to not disclose, since others would presumably entrust their data to the company without having this important piece of information? This is the question I have been pondering for some time.

While I think there is still room for healthy debate, I have come to the tentative conclusion that the company should be notified first and given an opportunity to respond, and only if they either do not respond or respond in such a way as to indicate that they are not going to consider the issue, should your concerns be disclosed publicly. I think that companies who may have had a breach have the right to deal with it internally. However, if it affects others and the company does not give an indication that they have looked into the issue, the right thing to do is to publicly discuss the issue so as to warn others about the potential consequences of entrusting their data with that company.

Publicly discussing a potential breach could be just the catalyst a company needs to begin an investigation that helps them to limit their losses. And while this shouldn’t be necessary, the reality of the world we live in is that information protection is rarely practiced well, let alone incident response.

On the other hand, this may only work well (or not) in the consumer area. If there is anything that the Wikileaks debate has taught us, it is that–regardless of whether you think the release of this kind of information is a good thing–there are consequences. One has to carefully consider the potential ethical and legal consequences when choosing to make a public disclosure. What if it initially looked like a breach, but was simply a misunderstanding? What if your words lean towards libel and not opinion? What if disclosing the potential breach results in people losing their jobs?

While there are no right or wrong answers, it is an area that deserves attention and discussion in the security community. As protectors of information, we need to be diligent about the way we handle it, and that involves careful reflection about the ethics and consequences of our actions.

What do you think?

When I went searching for a better interest rate for my emergency fund, I ran across a bank that offered over 5%, with relatively few restrictions. I thought this might be a good bank to work with.

So I set up an account, and immediately noticed some things that had me concerned from an information security perspective. The password length was limited to six characters, e-mails to their support contact resulted in a bounce message from two people who apparently no longer worked there, and they were leaking other information that gave me details about their internal network. This was all noticed without actively probing them in any way at all. Finally, when I brought the issues to their attention, the best response I got was that they were planning a system upgrade sometime around September.

In that case, the actual price to me of taking an action to protect my information security, assuming I had invested $25,000 with them, would have been almost $100 a month in lost interest.

Cost does not necessarily equate to price, though. Price can be expressly measured in very finite terms, where cost is often a collection of values. In the example above, we might say that the price is around $100 a month, but the cost could include stress, time and aggravation from a resulting security breach.

I am reminded of this lately with the recent news surrounding the backscatter x-ray machines currently in operation at US airports, combined with the “enhanced” pat-down procedures for those that refuse to go through the machine. I don’t think its too strong to say that we are left with the choice of having a revealing picture taken of us, or being fondled by a TSA agent.  This is the trade-off we are supposed to accept for enhanced security.

This might be an acceptable trade-off if the technology and procedures significantly reduced the risk of dying in a terrorist attack. But according to reason.com, the risk of dying in a plausible attack is actually far lower than the risk of dying by crossing the street, and this was before the machines. And when asked about the technology, Rafi Sela, a security expert at Ben Gurion airport in Tel Aviv, Israel, had this to say:

“I don’t know why everybody is running to buy these expensive and useless machines. I can overcome the body scanners with enough explosives to bring down a Boeing 747″

On the surface, it would seem that the machines simply aren’t worth it.

Now ask yourself how you would feel if the system stopped a major attack, and it could be shown that no other countermeasures would have stopped it? How would you feel if someone close to you was killed by a terrorist?

Or maybe the question is, how would you feel about your three-year old daughter being photographed by one of these machines?

These provocative questions go beyond the statistics to the essence of what we are–human beings. They at the same time expose our fears and our fallacies; our fear of attack and the fallacy of not being able to accurately intuit risk.

We’re being asked to pay a higher and higher price for our safety and security, but at what cost? Are these trade-offs worth it? Have we taken a rational look at the alternatives? Are we being led by fear, allowing our liberties to be usurped by largely ineffective security measures? Will our children accept this as normal and routine? How far will it go? Before you answer, could you have envisioned this even five years ago?

At what point will the answer simply be, “No!” Security always has trade-offs. What are you willing to give up in the name of security?

Today, we thank Daniel Cid for creating OSSEC.

Daniel has been working on OSSEC for a long time now. He started on it long before being snatched up by Third Brigade, having already put thousands of hours into the project. He chose to make it free and open so everyone could benefit.

Some interesting features of the plaque:

• The date of the alert is the day it was presented to him. The time is the current repo’s first check in at 1127488204 (Fri, 23 Sep 2005 15:10:04 GMT)
• The log name of communitylog and hostname of allhosts represent the great community that surrounds OSSEC
• Of course, the rule number is meaningful in that it is the first rule in the user space
• Nothing other than a level 15 alert…
• Notice the new daemon, ossec-awardd :)
• The PID is the alpha representation of Daniel’s initials (d=4,b=2,c=3)
• The log is fairly well-formatted for parsing and is an RFC-compliant syslog

Please join me in thanking Daniel for providing this free software as a service to the community.

Since version 1.2, OSSEC has generated daily, chained MD5/SHA1 checksums of the alerts log, and if logall is enabled, the archives.log. As Daniel notes in his blog post, this can provide you with a mechanism to prove that a particular log was not tampered with. If you compare a hash that was e-mailed to you with one on the manager, you can reasonably say the log has not been tampered with.

Or can you? What if the e-mail server was on the same network as the manager? What if they were running the same OS version and had the same vulnerability exposure? Can you still say the hash has not been tampered with?

One way to have a high degree of confidence in the integrity of the hash is to get it out to as many people as possible. The more remote, unrelated computers that have a copy of the hash, the higher degree of confidence and non-repudiation in that hash. Since the hash is a one-way function and does not contain the actual log data, the confidentiality of your logs remains intact.

Here are just a few ways this can be done.

• Twitter. Tweet your hashes, maybe even to an account set up especially for this. Make sure they are public. Here’s an example of what I mean.
• Facebook. Set up a facebook page for your hashes and make it public.
• Your web site. Set up a cron job to update a page daily that has all of your hashes. Make sure it gets indexed.

Again, the more unrelated sites that have these hashes, the higher degree of assurance you will have. Although I am not a lawyer, I can imagine a situation where one would make a very compelling argument that the logs were intact, because in order for the other side to repudiate that argument, they would have to prove that all of the external sources were also compromised and the hash modified.

What other ideas do you have for log integrity and non-repudiation? Speak up in the comments!

Last year I spoke at a conference on OSSEC and, in celebration, decided to create an entire week of blog posts about OSSEC. It was received pretty well. A few people were even inspired to contribute their own posts.

Well, there are a lot of us out there, and OSSEC is one of those rare communities in the open source world where everyone is respectful and tries to help. So, I thought to myself, “Self, why not make this a community event?”

I am proposing that we have a community “Week of OSSEC” to occur next month, October 17-23. October is aptly named National “Cyber Security Awareness Month,” so I think it’s a good fit.

How can I help? What can I contribute, you say?

-Spread the word
-Blog about OSSEC – how do you find it useful? What tips and tricks do you have?
-Find, report, and/or fix bugs
-Brainstorm ideas on how we can make OSSEC better. What kinds of rules should we have? What additional functionality is needed? What research can we incorporate to better detect attacks?
-Help with documentation (this one is huge!)
-Submit some code, a patch, or a new feature
-Tell us a story about what makes it difficult. What rules constantly fp?
-Proudly wear your OSSEC shirt if you have one. Send in a pic. Bonus points for standing next to a commercial SIEM-type booth. :)
-More..

The idea is simple: if everyone contributes just a little–even if you don’t think the contribution matters much, then we will have helped one another immeasurably. And isn’t that what free software is all about?

Yeah, I haven’t updated the blog in over two months..

When I started this up, I decided I didn’t want to make this a second job. I decided that if I didn’t update it in awhile, I would be OK with that. So I took some time over the summer to just work and do some fun stuff.

Lucky for you, one of the things I consider fun is OSSEC! So I spent some time working with the rest of the OSSEC team on the next release, which should be in beta soon. And I suppose it’s about time to resume musings about the general state of security, liberty and so-on.

Until next time…

There are four or five responses to risk, depending on who you ask. They are: mitigate, accept, transfer, reduce, and sometimes, ignore. Ignoring a risk is just a lame way of burying your head in the sand and pretending it doesn’t exist. Let’s just say there are four legitimate responses to risk.

What we usually hope for is mitigation and reduction. These responses deal with risk directly, but most importantly, they show the greatest amount of responsibility in dealing with the problem.

Transferring risk can be a legitimate response or it can be a way to essentially ignore it. It can be legitimate when there is a thoughtful decision to transfer to another party, and that party accepts. This can be an insurance policy, for example. It is illegitimate when transferring it to another party is a way of ignoring it, essentially just another way of saying, “it’s not my problem.”

That leaves acceptance. Acceptance means, “I know that this might happen, and if it does, I am prepared to accept the consequences of my decision.” The problem with acceptance is that it is rarely tied to consequences.

I see this problem in the real world all the time. Security professionals present risks to senior management and they simply do not respond. They don’t respond unless there is an incentive to do so, and agreeing to put themselves on the line with little or no positive return doesn’t win many fans. If the senior executive fails to respond and the risk is realized, they can just say, “I never saw the risk analysis,” or worse, “they failed to effectively communicate the risk to me.”

We need two things to make risk acceptance work. One, the risk acceptor can not be in a position of authority over the risk advisor. This creates a situation where the advisor might water down the risk out of fear for his own job. Two, there have to be consequences that tie directly to the risk being realized and for choosing to not mitigate, transfer or reduce it. Essentially, this means that ignoring the risk becomes default acceptance with a particular consequence. In both cases, the consequences of acceptance (either by default or explicitly) need to be commensurate with the risk.

Consequences tend to motivate people. Risk acceptance without consequences is not risk acceptance at all.

In a recent point/counterpoint over at techtarget.com, Marcus Ranum responded to Bruce Schneir on the issue of the risks of hiring hackers. Marcus said one thing in particular that struck me as very insightful. He writes:

We hear a lot about “thinking like a hacker,” but I think that’s largely nonsense — it’s really just a matter of “thinking like an engineer” and performing an in-line failure analysis along with your design analysis.

I think he’s mostly right.

Information security defenses are generally built based on compliance, the current new attack and logical analysis of the threats, in that order. But even when we perform a thoughtful risk analysis, we tend to over-emphasize the rare but sensationalized threats, and under-emphasize the mundane threats. Because information security is seen as a solution to the problem of other people actively attacking systems, we tend to design our countermeasures around those threats.

While it’s true that attacks are happening all the time, the vast majority of those are non-targeted attacks that can be resolved by simple countermeasures such as patching. What we tend to underestimate is loss from areas where information security practitioners have traditionally not had a foothold. We don’t design good countermeasures around forgetting to renew the domain name, a service provider going out of business, or someone hitting the delete key by mistake. But the end result is the same–a loss of confidentiality, integrity or availability.

So where does thinking like a hacker come in? Do we need to think like a hacker to protect against a hacker? I suppose that depends on your definition of what a hacker is. If that definition involves creative thinking, considering how things break, and coming up with solid solutions, then yes, thinking like a hacker is good. But then that is really just good security engineering.

Take this site for example. Before it went live, I played the “what if” game. I started to ponder the many ways this site could break. Along with considering the usual stuff like SQL injection and XSS, I wondered what would happen if my DNS provider suddenly went offline. I assumed that someone would steal the server from the data center, or that a government subpoena might inadvertently include this site along with the site they were targeting. I assumed I would turn off some of the security measures temporarily to see if something started to work again. I then assumed I would forget to turn them back on.

I designed countermeasures for all of these scenarios to a certain extent. At some point it wasn’t worth going any further–it is a blog after all. The point is that I didn’t so much design to be attacked as I designed it to be resilient to failure. Does that mean it is fail-proof? Absolutely not. The site could still be defaced. The server could crash and I might have a problem with restoring the backup. I might get hit by a bus. Despite all this, I am comfortable with the level of security engineering I put into the site.

To sum this up, thinking like a hacker is not that different than thinking like a security engineer. It’s two sides of the same coin. To consider how systems can be resilient to failure (the security engineer), we need to consider how they will break (the hacker).

And sometimes, just sometimes, they break in spectacular ways.