Het Bijstere Spoor

RFC 6585 has been published quite recently. This document describes 4 new HTTP status codes.

So in case you were wondering, yes.. HTTP is still evolving :), and these new statuses may be quite useful for developing your REST, or otherwise HTTP-based service. This post describes why they are important, and when you should use them.

428 Precondition Required

A precondition is something a client can send along with a HTTP request. This condition needs to be met in order for the request to complete.

A good example is the If-None-Match header, which is often used along with GET requests. If the If-None-Match is specified, the client can request to only receive the response if the ETag changed.

Another example of a precondition, is the similar 'If-Match' header. An If-Match header is usually sent along with PUT requests to indicate to only update the resource if it hasn't changed. This is useful if multiple clients are using a HTTP-based service, and they want to make sure they are not overwriting each others contents.

Using the 428 Precondition Required status, the server can now indicate that the client *must* send along one of those headers to perform the request. This is effectively a way for the server to force clients to prevent this 'lost update' problem.

429 Too Many Requests

This status code is useful in cases where you want to limit the amount of requests a client may want to do on your API (also known as rate limiting).

In the past different status codes have been used, such as '509 Bandwidth Limit Exceeded'. Twitter uses 420 for some stuff (which is an unused status code). Thus it was important enough to give it it's own code.

So if you limit the number of requests clients may do on your server, 429 Too Many Requests is the way to go. Include a 'Retry-After' response header to indicate to a client when they are allowed to make requests again.

431 Request Header Fields Too Large

I was surprised to see that this was a common enough usecase to warrant it's own status code, but here it is!

In a case where a client is sending a HTTP request header that's too big, the server can respond with 431 Request Header Fields Too Large to indicate exactly that.

I have no idea why they skipped over 430 though. I tried to search around, but couldn't quite find the reasoning. My best guess is that a lot of people may have mistyped '403 Forbidden' as '430 Forbidden', and they wanted to avoid complications. If you know, let me know!

511 Network Authentication Required

This status code is very interesting to me. You will not have to deal with this if you're writing a server, but it can be important if you're writing a (desktop) HTTP client.

If you move around with your laptop or smartphone a lot, you may have noticed that a lot of public wifi services now require you to accept their license agreement, or just log in before the web works.

This is generally done by intercepting the HTTP traffic and presenting a redirect and login when the user tries to access the web. Quite nasty, but that's the way it is.

Using these 'intercepting' clients can have some nasty side effects. There are two great examples mentioned in the RFC to illustrate this.

• If you hit a website before logging in, the network device intercepts the first request. These devices also tend to have a 'favicon.ico' stored. After logging in, you'll notice that the favicon is now cached for the website you tried to visit, and it may follow you around for quite some time.
• If a client uses HTTP requests to find documents, the 'network' may respond with a login page, instead of the json or other document you expected. Your client may (in error) assume it's a 'normal' response and use that instead. This can put clients in a broken or irrecoverable state. I've noticed this in real life a few times working on a CalDAV system as well.

So to fix this 511 Network Authentication Required is introduced.

So if you write an application that runs on an desktop or phone and use HTTP, you should ideally check for this HTTP response code. In a way, it simply means that a network is not yet available and you should pretty much ignore anything coming back until it is. You could even provide the user with the returned login page, like iOS and OS X 10.7 do.

I've been at this for a bit, but now it's official: I'm now based in Münster, and working at fruux as a co-founder / tech-lead.

The feeling of having this amount of control over a product is great. We've also recently got funded from High-Tech Gründerfonds and netSTART Venture and moved into a new office so things are indeed pretty serious. We have some flexibility to focus on making a great product, in what we feel is 'the right way' (whatever that may mean). It will be an exciting time :)

So if you were looking at SabreDAV, and needed something way easier to use, fruux may be a good option. Regardless, for SabreDAV the future is bright too, as there will be plenty of cross-pollination.

Also, if you're looking for a job, or know someone who's looking: drop me a line! We're in the market for good PHP and front-end developers.

A quick update about CSP. Browsers are well on their way to all adopt the specification.

An early draft was already adopted by Firefox 4, and I just found out that it's also working in Chrome, Safari and IE 10.

IE10 and FF are using the following header:

X-Content-Security-Policy: default-src 'self'

While Safari and Chrome use:

X-Webkit-CSP: default-src 'self'

When the specification is finalized, the X- will be dropped, and it will simply be 'Content-Security-Policy'.

A call for support

Hi Developers! Start implementing this feature! It's important for the future and security of the web. The web's biggest vulnerability, from what I understand, is still XSS, but if people start to properly implement CSP, XSS can effectively be a thing from the past.

So even if you don't want to risk using CSP on a production environment, at least consider adding the headers in your development environment. It will force you to write better code, by not embedding javascript directly into the HTML source. By considering this right now, you will also make it much easier if you do decide to adopt CSP at some point in the future.

I'm implementing CSP full-on in a new project, and one of the things I've noticed already is that some of the javascript we embed from 3rd parties use eval() and inline html events (onclick & friends). For the sake of security we will most likely decide to only use 3rd party code if they are indeed CSP-compatible.

In my work I have to look a lot at the IETF rfc documents, and I always have to jump through a bunch of hoops to find the document I want. Google often links me to the text version, and often I'm looking at obsoleted versions of documents when a new rfc has been published.

Well, so I created a simple OpenSearch provider to solve this. It does exactly what I need, and I only tested in Firefox. If it's useful to anyone else, you can add the provider from rfcsearch.phpfogapp.com.

It works with a simple hardcoded list of known keywords, and it will fall back to "Google's I'm feeling lucky" if it doesn't know the keyword. Feel free to fork it or laugh at my source code.

My PHP Advent article just got published. It's a list of best practices around dealing with dates and times in PHP. Have a read and tell me what you think. Also, be sure to follow @phpadvent or subscribe.

While profiling SabreDAV, I noticed a few times more than half of the request time was spent in the autoloader.

So instead of autoloading, now I prefer to unconditionally include every file for each package (there are 5 packages). For a while I manually maintained these files manually, but a while back I automated this process.

This is how you run it:

phpincludes . includes.php

This will generate a file such as:

<?php
 
// Begin includes
include __DIR__ . '/Interface1.php';
include __DIR__ . '/Class1.php';
include __DIR__ . '/Class2.php';
include __DIR__ . '/Class3.php';
// End includes

You can edit everything before "// Begin includes" and after "// End includes". Subsequent edits will only replace the lines in between those comments.

The script will automatically expand classes and interface dependencies and load them in the correct order. It also has support for a PHP 5.2-compatible syntax (dirname(__FILE__) instead of __DIR__).

If you like it, head over to github, or install it with:

pear channel-discover pear.sabredav.org
pear install sabredav/PHPIncludes

I've finally taken the plunge and moved the SabreDAV source from Google code to GitHub.

The main reason is that I'm hoping that people are more likely to fork and contribute. This was possible as well on googlecode, but I think people had a bit more trouble getting used to the feature.

Mercurial is still the best source control system, but I feel switching to Git was worth it, solely for GitHub.

You can find the source at https://github.com/evert/SabreDAV.

Interesting news today. The Olson database, which is used in countless operating systems and other software (including PHP) has been shut down due to a Copyright claim.

Details, which I won't repeat here can be found on Stephen Colbourne's blog.

While working on an application I ran across a huge bottleneck which I isolated down all the way to the use of the iconv_substr function. If you ever wonder which is better to use, this should help your decision:

Benchmark script

<?php
 
$str = str_repeat("foo",100000);
$time = microtime(true);
 
iconv_substr($str,1000,90000,'UTF-8');
 
echo "iconv_substr: " . (microtime(true)-$time) . "\n";
 
$time = microtime(true);
 
mb_substr($str,1000,90000,'UTF-8');
 
echo "mb_substr: " . (microtime(true)-$time) . "\n";
 
$time = microtime(true);
 
substr($str,1000,90000);
 
echo "substr: " . (microtime(true)-$time) . "\n";
?>

The results widely varied between machines, operating systems and PHP versions; but here are two results I recorded.

First, PHP 5.3.4 on OS/X:

iconv_substr: 0.014400005340576
mb_substr: 0.00049901008605957
substr: 3.7193298339844E-5  # Note the E-notation, this was actually something like 0.00003 seconds.

As you can see iconv took 0.01 seconds, while mbstring took only 0.0004 seconds. Already a significant difference (2800% slower), but the difference became more apparent when running this on a Debian box with PHP 5.2.13.

iconv_substr: 8.3735179901123
mb_substr: 0.00039505958557129
substr: 4.8160552978516E-5

Yup, it took 8.3 seconds. That's an increase of over 2100000%. So next time you're wondering which of the two may be smarter to use, this may help you decide.

Important to note that OS/X uses libiconv as the 'iconv implementation' and my Debian test machine 'glibc', so it looks like libiconv is much, much faster than glibc. mbstring still leaves both in the dust though.

I'm interested to hear what your results are, especially if they differ.

Interesting news passed by today, apparently a fraudelent SSL was issued by Diginotar, effectively allowing wrong-doers to perform MITM attacks for all google services. Normally fake certificates will clearly error up in the browser, but because Diginotar is a trusted CA (certificate authority) it won't.

This says something about how much we can trust SSL. All it takes is one corrupt employee at a trusted CA and it falls down. CNET has pretty good coverage of the story.