Het Bijstere Spoor

Interesting news passed by today, apparently a fraudelent SSL was issued by Diginotar, effectively allowing wrong-doers to perform MITM attacks for all google services. Normally fake certificates will clearly error up in the browser, but because Diginotar is a trusted CA (certificate authority) it won't.

This says something about how much we can trust SSL. All it takes is one corrupt employee at a trusted CA and it falls down. CNET has pretty good coverage of the story.

The problem with Apache's approach to dealing with multiple clients, is that there's only ever a limited amount of Client processes available. This is usually is around a few hundred on common webservers.

Because of this, it becomes necessary to handle HTTP requests as quickly as possible. As soon as a request is handled, it can go on serving the next. If a client happens to have a slow connection, this can have a direct effect on the scalability of your frontend server.

A common way to fight this, is to put a caching server in front of your webserver, such as Varnish or Squid. These webservers are better suited to deal with many clients. This will allow your Apache server to send back HTTP responses quickly to the reverse proxy, and let the proxy deal with sending back the response to the client.

However, this doesn't deal with slow requests. Generally, these proxy servers will open connections directly to the backend webserver to avoid having to buffer larger request bodies.

Because PHP installations generally use apache 'prefork mpm', the number of possible connections is considerably low. This is also often the case with Fast-CGI based webservers, such as nginx and lighttpd. So if you were to just able to open up a few hundred connections, and drip in the bytes for the request body it would be very easy to take these servers down.

To test this theory, I wrote a simple python script that does exactly this, you can grab it from github. To use it, try something like this:

python slowdeath.py --threads 200 http://localhost/

In my case my webserver was limited to 150 connections. It took about a second for it to stop serving requests.

Big warning: This tool is for research purposes only. Use at your own risk, and only on servers you own.

To take out a server, simply specify a number of threads higher than the MaxClients or whatever setting your webserver happens to use. Note that I only tested this on a few servers, so results may vary. Side effects include diarrhea, rashes, blackouts and death. Do not use while driving.

Samy, famous for his worm, released evercookie this week. Evercookie stores cookies is various storage mechanisms such as Flash Local Shared Objects (also known as flookies), HTML5 storage mechanisms and even in the history and cache. When any of these are wiped by the user the script will repopulate it, making it very hard to get rid of your cookies.

This is technique is common to circumvent a users' privacy wishes, which Clearspring recently got sued for, but it's put in overdrive.

One good use for it is banning users. In the past I've used ips + cookies to ensure a user stays banned, but it doesn't take much to change your ip address and clear your cookies. All these techniques together make it a lot harder to get through. Because Flash stores it's flookies in a central place in the operating system, the cookies often even live in multiple browsers and private browsing sessions.

Most of all, I think the tool is made to make a point. It's very hard for the average user to clear all the tracking information. It should be doable with a press of a button, without losing all your settings and history for every other site.

I blogged about Content Security Policy about 2 year ago when it was still called 'Site Security Policy'. It started as a specification and an add-on, and turned into a patch a bit later. Finally it made it into Firefox 4 beta 1. I think CSP is the next web security revolution, so make yourself aware of how it works and the implications.

So what is it? The short version is that it's a very effective measure against cross-site scripting. By specifying a policy through the 'X-Content-Security-Policy', you can specify exactly from which locations you accept javascript and other content. This allows you to block scripts from any domains unknown to you, and inline scripts altogether.

A simple example

X-Content-Security-Policy: allow 'self'

A simple PHP example to see this in action:

<?php
 
header("X-Content-Security-Policy: allow 'self'"); 
 
?>
<html>
  <head>
    <title>CSP test</title>
  </head>
  <body>
 
<script type="text/javascript">
 
alert('XSS!');
 
</script>
 
  </body>
</html>

If the above code is opened in Firefox 4.0 beta1, the script will not execute, and a warning is added to the "Error Console" (in the Tools menu).

Not only does this header block inline scripts, it also blocks the following:

• eval(). This important for people using eval() to parse json responses.
• setTimeout and setInterval if the function is provided as a string.
• javascript: urls
• HTML event attributes (onclick, onload, etc.).
• All images, plugin objects (flash, quicktime etc.), audio, video, html frames and fonts not served from the same domain as the html page.
• XMLHttpRequest to domains other than the source domain.

Fortunately there are fine grained controls about what you want to allow from which domains. Here are some examples from the specification.

X-Content-Security-Policy: allow 'self'; img-src *; \
                           object-src media1.com media2.com *.cdn.com; \
                           script-src trustedscripts.example.com

This example starts with "allow 'self'", allowing only content from the same domain. The "img-src *" rule allows images from any domain. "object-src: media1.com media2.com" allows <object> tags to use files from media1.com, media1.com and the same domain as the html was served from. To learn more about these, I would recommend just taking a good look at the directives list in the specification.

Options and reporting

Using the 'options' directive it's possible to turn on specific measures. Valid values for options are 'eval-script' and 'inline-script'.

X-Content-Security-Policy: allow 'self'; options inline-script, eval-script

The preceding example allows inline scripts (using html event attributes, or the script tag) as well as the 'eval()' function. In general I would try to avoid this though.

When a security rule is violated, it's possible to get the browser to send a report back to the server. For example, if an image is referenced from a blocked domain, the browser can send a simple report to a url you specify.

X-Content-Security-Policy: allow 'self'; report-uri http://example.org/cspreport.php

This allows you to detect any problems with your policy, or successful attempts by your evil users to inject code. An example of such a report is the following:

{
  "csp-report":
    {
      "request": "GET http://index.html HTTP/1.1",
      "request-headers": "Host: example.com                                                        
                          User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.3a5pre) Gecko/20100601 Minefield/3.7a5pre
                          Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8  
                          Accept-Language: en-us,en;q=0.5                                          
                          Accept-Encoding: gzip,deflate                                            
                          Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7                           
                          Keep-Alive: 115                                                          
                          Connection: keep-alive",
      "blocked-uri": "http://evil.com/some_image.png",
      "violated-directive": "img-src 'self'",
      "original-policy": "allow 'none'; img-src *, allow 'self'; img-src 'self'"
    }
}

Final notes

Using CSP does not mean you can go easy on other security measures. At the moment a very limited amount of users will have support for CSP, so everybody else still needs to be protected. However, it's still a great idea to implement. Your Firefox users will automatically be protected better, and because of the reporting functionality, they automatically help you detect holes which benefits everybody.

My guess is that CSP is going to be very important, and is here to stay. There are two things you can do to prepare for the future:

1. Figure out your policy. It's a good idea for your web application to know anyway where resources are coming from. Especially advertisers tend to be bad at using many different domains and scripts using other scripts.
2. Try to avoid any inline scripting, html event handlers and eval(). They are all avoidable, and in my opinion it is a good idea to keep your javascript out of html anyway. This is a big one, because both inline scripts and html events are still very popular. With the popularity of libraries such as jQuery, I do think it will be easier to just grab most of the inline scripts and move them to an external script.

Our session system is due for an upgrade. Currently all PHP sessions are stored in the database, and some things are getting a bit slow. There have been a couple of approaches I've been considering, one of which is simply storing all the information in a browser cookie.

First I want to make clear I don't necessarily condone this. The reason I'm writing this post, is because I'm hoping for some more community feedback. Is this a really bad idea? I would love to know.

The benefits

If all the session data is stored in the browser, it means that I don't need to store it on the server. I actually don't care all that much for having the data on the server (unless it's the only secure way), it's mostly a gigantic map with session tokens and user id's (along with some other info).

I also feel it's more natural for HTTP, as it makes it a bit more stateless.

Sample code

<?php
 
class BrowserSession {
 
    public $secret = 'this will need to be a cryptographic random number';
    public $currentUser = null;
 
    // Sessions time out after 10 minutes
    public $timeout = 600;
 
    function init() {
 
        if (!isset($_COOKIE['MYSESSION'])) {
            echo "No session cookie found\n";
            return;
        }
 
        list($userId, $time, $signature) = explode(':',$_COOKIE['MYSESSION']);
 
        // The cookie is old
        if ($time> time() + $this->timeout) {
            echo "The session cookie timed out\n";
        }
 
        if ($signature !== $this->generateSignature($userId,$time)) {
            echo "The secret was incorrect\n";
        }
 
        $this->currentUser = $userId;
 
        echo "Logged in as user: $userId\n";
 
    }
 
    function login($userId) {
 
        $this->userId = $userId;
 
        $time = time();
 
        $cookie = $this->userId . ':' . time() . ':' . $this->generateSignature($userId,$time);
 
        setcookie('MYSESSION',$cookie,$time+$this->timeout,null,null,null,true);
 
        echo "Set cookie: $cookie\n";
 
    }
 
    function generateSignature($userId,$time) {
 
        $stringToSign =
           $userId . "\n" .
           $time . "\n" .
           $_SERVER['HTTP_USER_AGENT'] . "\n" .
           $_SERVER['REMOTE_ADDR'];
 
        return hash_hmac('SHA1',$stringToSign,$this->secret);
 
    }
 
}
 
ob_start();
$session = new BrowserSession();
$session->init();
 
if (isset($_GET['login'])) $session->login($_GET['login']);
else {
 
    echo '<br /><a href="?login=1234">Log in as user 1234</a>';
 
}
?>

A few notes:

• The preceeding code was just intended as a proof of concept, it's missing some validation.
• Currently the secret would be the same for every user. I was thinking of appending some per-user information to the secret. If somebody does guess or bruteforce the secret, they would only have access to a single users' information.
• If a user changes their password, existing sessions should expire. To do this the signature should also include a sequence number that changes when the password changes.
• Currently this only stores a user id. It could be extended to contain more data, but this is all I need.

So, is there anything fundamentally wrong with this approach? In general the client should never be trusted, but for setups where the security requirements aren't as high (highly subjective, I know) I feel this might be strong enough. OAuth, OpenID and Amazon AWS all seem to trust HMAC+SHA1, but those applications do work differently.

Credit where it's due

I first asked this question on stack overflow. The users there already gave some great suggestions and pointed out some of the flaws. Thank you!

Rant warning

We enter our usernames and password on pretty much all the sites we commonly visit. Authentication is probably one of the first things you're being taught when starting to work with PHP. For some reason, in 99% of the cases this is done through an HTML form, with the username and password submitted as a urlencoded string.

You probably know that HTTP also has native authentication, in the form of Basic and Digest authentication (read my older article if you want to know how). Every browser and pretty much any HTTP client does too. There's some big benefits to that, because it provides a very standardized mechanism to authenticate a client, whether you're a machine or human.

What baffles me is that HTTP authentication hasn't been developed further. HTTP Digest is pretty secure by itself, and has some nice features (hashed password, protection against man in the middle and replay attacks, message digests) which is way more advanced than an HTML POST form with a session cookie can provide.

What's missing?

1. There's no way for a user to see if they are authenticated to a site. Perhaps a username in the addressbar?
2. Pretty much everybody always wonders how they can code a logout mechanism. Because there are no session cookies that can be destroyed, there are some hacks that trick the browser to ask for credentials again. There should be no need for the server to provide this functionality. The browser knows it's logged in, and HTTP applications are stateless. We need an in-browser log-out button.
3. Less important, some javascript hooks that allow developers to still use html forms to setup HTTP authentication.

Mozilla is doing some interesting things with their Account Manager Add-on for firefox, but even that add-on does not support HTTP authentication. With Account Manager they are jumping through some hoops with javascript hooks so it works with regular authentication systems, but you'd think that if HTTP Authentication was used, things could be a lot more straightforward. The browser knows exactly who is logged in.

So, does anyone know how this happened? Is there a major flaw in HTTP authentication I'm just missing?

Two examples of escaping data are the following:

• Before you insert a value into a SQL query, using for example mysqli::real_escape_string() or PDO::quote().
• Before you insert data into your output HTML, using htmlspecialchars().

The question I'd like to ask today is, when to do this? There are two possible moments:

1. Right when the data comes in. For SQL this used to be done with 'magic quotes' quite a bit in PHP-land. In general I don't see this happening a lot anymore for SQL. I do however see data encoded using htmlentities/htmlspecialchars before entering the database.
2. The other way to go about it, is to only escape when you know how you're going to use it. For example, only call htmlspecialchars right before you echo() your data into your document.

I would personally argue that #2 is the best way to go about things. The first reason is that you don't know exactly how your data might be used in the future. If you pre-encoded everything using htmlentities, but at some point in the future you need the data to be used in an XML feed, you're going to be in trouble. The reason for this, is that the only valid entities in XML are &amp;, &lt;, &gt;, and &quote;. If you are going to need to need to output to CSV, very different rules apply. Other examples are: escaping for urls, escaping for command-line arguments, escaping for javascript and escaping for mime-headers.

In the illustrated example, this is no big disaster. A workaround would be to call htmlspecialchars_decode() or html_entity_decode() first, and then escape for your desired output. A worse case is filtering. If you have been stripping out all, or some html tags before saving it do the database, and later on your decide you wanted to show some of them anyway, that data is now lost.

Conclusion

So my argument is to store raw data. Only encode right before you know where you going to need it. If you're worried about the overhead of escaping right before output in an html page, cache the output.

Whichever route you go, make sure this is clearly documented. There's 2 ways this can go wrong:

1. Escaping is done on input and output. Now you see literal &amp;'s in your html, or quotes prepended by slashes. (\'hello\').
2. Escaping is forgotten at both ends. Now you might be vulnerable to SQL injection attacks, XSS attacks or data corruption.

What do you think? I'm especially interested in the other side of the argument.

Clickjacking allows an attacker to trick your users into clicking parts of your interface without their consent. A simple way to describe describe this is, an attacker will embed your application in their site as an iframe. On top of the iframe they can show a completely different interface. You're thinking you're clicking buttons on your own interface, while in fact you are hitting the 'Delete my account' button in for example GMail.

Because this technique completely operates with frames, it can be circumvented by using a 'Frame busting' technique. As a bonus, this will also disallow for example Digg to steal and monetize your content.

Frame busting can be achieved with a simple javascript technique:

<script type="text/javascript">
if (top !== self) top.location.replace(self.location.href);
</script>

Security through javascript?

If you think this sounds like a bad idea, you are probably right. Users might simply have javascript disabled, and I also don't like relying on UI developers too much to implement preventive security measures (although I realize in most cases you do have to).

In Internet Explorer the situation is worse, IE allows you to specify the non-standard attribute security="restricted":

<iframe src="http://www.rooftopsolutions.nl/ security="restricted"></iframe>

This attribute tells IE to not allow executing of javascript in the iframe, which actually is not a bad security measure for other types of attacks. In this case however, it allows the attacker to disable the framebusting script.

X-Frame-Options

Thankfully, Internet Explorer 8 introduces a new feature that allows the site owner to disallow frames altogether, which is in my opinion an even better protection mechanism, because it doesn't rely on javascript to be executed.

The name of the http header is specified as such:

X-FRAME-OPTIONS: SAMEORIGIN
X-FRAME-OPTIONS: DENY

You only have to specify one of these two, 'sameorigin' means the page can only be framed from an html page hosted on the same domain, deny will kill framing altogether.

PHP example:

<?php
header('X-FRAME-OPTIONS: DENY');
?>

Firefox also appears to have started implementing this feature, and there's a feature request for webkit open as well.

Protecting yourself

Unfortunately you can safely assume most sites don't implement either of these security measures. For firefox users I would therefore strongly recommend using the NoScript plugin. Not only does it implement the X-FRAME-OPTIONS for firefox, it also actively detects clickjacking attempts.

Reference: hackademix.net

Escaping user-input in your HTML is essential for preventing worlds #1 vulnerability.

When you're embedding user input into javascript, a simple htmlspecialchars won't cut it, you'll need to make sure you're escaping other things, like \n (line endings), and \ (slashes). Google doctype has a good list of characters in need of proper escaping to prevent users breaking your javascript.

However, when I dropped the question if a simple string replacement would be good enough, the members of the Web security mailing list gave me a different answer.

When escaping or filtering output using a blacklist (such as the one published on google doctype) browser/unicode escaping bugs are not taking into consideration. Some new vulnerability might appear in the future, which would immediately open a hole in your app. For this reason its wiser to go with a much more defensive white-list approach, essentially only letting things through you know is safe.

Introducing Reform

Reform is a tool that does exactly this. Reform allows you to escape your data for a javascript, xml, html or vbscript (yes it still exists) context. It provides libraries for Java, .NET, PHP, Perl, Python, Javascript and ASP. Pretty cool!

One dislike I have is that it only considers I really small set of unicode codepoints safe, especially when dealing with non-latin languages this is going to add a great deal to the bandwidth usage and the legibility of your sourcecode. One would think there has to be more ranges considered 'safe'.

PHP example:

<?php
  // Assuming the Reform class is included..
 
  echo '<script type="text/javascript"> var myString = ', Reform::JsString($userInput), '; </script>';
 
?>

I made a couple of changes in the PHP version, specifically:

• Prepended the 'static' keyword to every method to make it work in PHP5's strict mode.
• Removed the UTF-8 checks, I'm in a controlled environment, mbstring is installed, and the internal encoding is utf-8.
• Added a parameter to Reform::JsString to not automatically put the string between quotes (').

Today on the IE blog a big announcement was made regarding the upcoming security features in Internet Explorer 8.

Definitely check it out! Among things it includes an XSS protection filter, HTML sanitizing built straight into the scripting engine and a way to disable the infamous 'content sniffing'. I'd still hope to see the content-sniffing 'feature' to be opt-in, instead of the proposed opt-out solution.. but hey, at least it allows us to plug the hole.

To serve files as text/plain, serve the document with the Content-Type header as:

Content-Type: text/plain; authoritative=true;

I have to say, I'm quite impressed how IE is catching up with things like standards and security.