Het Bijstere Spoor

My PHP Advent article just got published. It's a list of best practices around dealing with dates and times in PHP. Have a read and tell me what you think. Also, be sure to follow @phpadvent or subscribe.

While profiling SabreDAV, I noticed a few times more than half of the request time was spent in the autoloader.

So instead of autoloading, now I prefer to unconditionally include every file for each package (there are 5 packages). For a while I manually maintained these files manually, but a while back I automated this process.

This is how you run it:

phpincludes . includes.php

This will generate a file such as:

<?php
 
// Begin includes
include __DIR__ . '/Interface1.php';
include __DIR__ . '/Class1.php';
include __DIR__ . '/Class2.php';
include __DIR__ . '/Class3.php';
// End includes

You can edit everything before "// Begin includes" and after "// End includes". Subsequent edits will only replace the lines in between those comments.

The script will automatically expand classes and interface dependencies and load them in the correct order. It also has support for a PHP 5.2-compatible syntax (dirname(__FILE__) instead of __DIR__).

If you like it, head over to github, or install it with:

pear channel-discover pear.sabredav.org
pear install sabredav/PHPIncludes

While working on an application I ran across a huge bottleneck which I isolated down all the way to the use of the iconv_substr function. If you ever wonder which is better to use, this should help your decision:

Benchmark script

<?php
 
$str = str_repeat("foo",100000);
$time = microtime(true);
 
iconv_substr($str,1000,90000,'UTF-8');
 
echo "iconv_substr: " . (microtime(true)-$time) . "\n";
 
$time = microtime(true);
 
mb_substr($str,1000,90000,'UTF-8');
 
echo "mb_substr: " . (microtime(true)-$time) . "\n";
 
$time = microtime(true);
 
substr($str,1000,90000);
 
echo "substr: " . (microtime(true)-$time) . "\n";
?>

The results widely varied between machines, operating systems and PHP versions; but here are two results I recorded.

First, PHP 5.3.4 on OS/X:

iconv_substr: 0.014400005340576
mb_substr: 0.00049901008605957
substr: 3.7193298339844E-5  # Note the E-notation, this was actually something like 0.00003 seconds.

As you can see iconv took 0.01 seconds, while mbstring took only 0.0004 seconds. Already a significant difference (2800% slower), but the difference became more apparent when running this on a Debian box with PHP 5.2.13.

iconv_substr: 8.3735179901123
mb_substr: 0.00039505958557129
substr: 4.8160552978516E-5

Yup, it took 8.3 seconds. That's an increase of over 2100000%. So next time you're wondering which of the two may be smarter to use, this may help you decide.

Important to note that OS/X uses libiconv as the 'iconv implementation' and my Debian test machine 'glibc', so it looks like libiconv is much, much faster than glibc. mbstring still leaves both in the dust though.

I'm interested to hear what your results are, especially if they differ.

Over the last month I've been working hard at the Atmail office in sunny Australia to get CardDAV support built into SabreDAV; and I've finally completed all the steps to do this release.

So there it is, CardDAV. Unfortunately there are not yet a lot of clients who actually use it, and it mainly comes down to iOS and OS/X, but I've been asked about CardDAV a lot and suspect more people will become interested in this protocol (especially if more vendors start supporting it).

So that's pretty much it; head over to download page to fetch a copy. I've had to break a couple of minor api's, you can read about those in the migration document.

• Full changelog
• Migration document for 1.4.x users
• CardDAV documentation

I tried my best to write good documentation for the new stuff, but it's always very time consuming, and not as good as I'd like If you have time and the will to write more, let me know!

Lastly, a big thank you to Nick Boutelier for creating the new SabreDAV logo!

Although PHP's loose comparison type juggling tends to invoke some negative responses, I don't think it has really ever worked against me, and works quite comfortably in my opinion. As long as you make sure you always use strict checking (=== and !==) where you can, and fall back to the loose checks when you must.

As a PHP developer, I think it's very important to understand and memorize exactly how these work, whether you're writing new code, or maintaining existing code.

A few days ago on PHP-internals I saw a behavior that was completely new to me, and very much counter-intuitive.

if( '20110204024217300000' == '20110204024217300264' )
  echo 'equal';
else
  echo 'not equal';

Guess what the output is.

PHP will for loose comparisons always try to convert numeric strings, even when both sides of the comparisons are strings. Because the numbers are too big to fit in an integer, they are converted to floats. For both numbers this conversion ends up in the number: "2.0110204024217E+19" (give or take, based on the standard precision settings).

In my mind it makes sense to do this type juggling when a comparison is done with <, >, <= or >=, but it definitely feels like a bug when doing an equals check.

The moral is: always do strict checks when you are able to.

Thanks to Matt Palmear for pointing this out.

Last Saturday I put up version SabreDAV 1.4.

It's taken a while to get this one out. Much longer than I thought. The result was that there's been very little released over the past few months. In an effort to change this, I decided to release 1.4.0 as soon as possible, rather than when all the features are ready. I believe this is better for the end-user and for me as well (release early, release often, etc).

So there it is. These are the new major features:

• WebDAV ACL support. This part is not 100% done. It can be integrated into existing API's, but there's no central ACL store or ability to modify ACL's through the WebDAV protocol yet. These additional features will be added in subsequent versions.
• CalDAV proxy support. This is a proprietary apple extension, allowing users to delegate calendar access to other users.
• Integrated the 'VObject' library, which provides an easy way to read and write iCalendar objects with an api similar to SimpleXML.
• Added the ICSExportPlugin, allowing you to export iCalendar-formatted calendars.

full changelog

To allow for a proper ACL implementation, much of the 'principal' functionality has been moved from Sabre_DAV_Auth to Sabre_DAVACL. There's a Migration guide available with all the details.

As usual, if you're not ready to migrate to 1.4 because of the API breaks or because it's still considered beta, I'll be maintaining 1.3 for at least another year. However, I'll be doing this on a strictly on-demand basis. So if you need a bugfix backported or a release, feel free to ask on the mailing list.

Lastly, thanks to all the users. The number of deployments and feedback is steadily growing and that's very rewarding.

Download here.

During Rob Allen's ZF2 talk at PHPBenelux an audience member shouted this really useful tip, which I thought was worth sharing.

If you're running PHP 5.3 and you have to use pesky old code that uses long class prefixes (yea, so, pretty much all PHP code out there), you can still make use of namespace features to shorten them.

<?php
use Sabre_DAV_Auth_Backend_PDO as AuthBackend;
use Zend_Controller_Action_Helper_AutoComplete_Abstract as AutoComplete;
 
$backend = new AuthBackend();
?>

Might have been super obvious to most of you, but it just hadn't occurred to me.

I've just finished an iCalendar vCard parser for PHP. It's done almost completely with a 'natural' simplexml-like interface, so it should (hopefully) be just as easy to parse, and also modify iCalendar / vCard objects (ics/vcf files).

To install using pear, run the following:

pear channel-discover pear.sabredav.org
pear install sabredav/Sabre_VObject-alpha

Or download from pear.sabredav.org.

For testing, I used this iCalendar file: icalendartest.ics.

To load in an object, you use the Reader class:

// Link to the correct path if you manually dowloaded the package
include 'Sabre/VObject/includes.php';
 
// Reading an object
$calendar = Sabre_VObject_Reader::read(file_get_contents('icalendartest.ics'));

iCalendar objects consist of components (VEVENT, VTODO, VTIMEZONE, etc), properties (SUMMARY, DESCRIPTION, DTSTART, etc) and parameters, which are to properties what attributes are to elements in XML. To show a listing of all events in a calendar, this snippet would work:

echo "There are ", count($calendar->vevent), " events in this calendar\n";
 
// Looping through events
foreach($calendar->vevent as $event) {
 
    echo (string)$event->dtstart, ": ", $event->summary, "\n";
 
}

You can easily modify properties:

$calendar->vevent[0]->description = "It's a birthday party";

Creating new objects uses the following syntax:

$todo = new Sabre_VObject_Component('vtodo');
$todo->summary = 'Take out the dog';
$calendar->add($todo);

And to turn your newly modified calendar back into an ics file:

file_put_contents('output.ics', $calendar->serialize());

Lastly, parameters are accessible through array-syntax:

echo (string)$calendar->vevent[0]->dtstart['tzid'], "\n";

I had fun building this, I hope it's useful to you as well. It's 100% unittested, but bugs might still appear due to the complex nature of API. Use at your own risk :). This library will be part of the SabreDAV project, which is also where you can go for the source, report bugs or make suggestions.

Since may 11 TLD's (top-level domainnames) have been added. In order for this to work successfully, a lot of applications will have to be fixed.

Many email-validation scripts might use an approach like this:

$ok = preg_match('/^[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,6}$/i', $email);

This one is pretty simple, it matches the most common address formats, as long as the tld (.com, nl, .uk, etc) is under 6 characters. For a bit more sophistication you might want to ensure that the tld is a bit more valid:

$ok = preg_match('/^[A-Z0-9._%+-]+@[A-Z0-9.-]+\.(?:[A-Z]{2}|com|org|net|edu|gov|mil|biz|info|mobi|name|aero|asia|jobs|museum)$/i',$email);

Note: both these regexes were taken from regular-expression.info. The top google hit, and decent examples.

The new TLD's use non-ascii characters, and they might become aliases for existing top-level domains, or new tld's altogether. Here are the currently working examples:

• http://مثال.إختبار - Arabic.
• http://例子.测试 - Chinese (simplified)
• http://例子.測試 - Chinese (traditional)
• http://παράδειγμα.δοκιμή - greek
• http://उदाहरण.परीक्षा Hindi
• http://例え.テスト - Japanese
• http://실례.테스트 - Korean
• http://مثال.آزمایشی - Persian
• http://пример.испытание - Russian

At first sight these look like regular utf-8, characters, but if you look at the sourcecode of this page, you'll notice that it's actually encoded differently.

The korean url http://실례.테스트, is actually encoded as http://xn--9n2bp8q.xn--9t4b11yi5a/. This is called Punycode.

If you want support for these new urls (and thus domainnames in emails), you should have support for punycode. You will likely receive UTF-8 encoded domainnames for email address (example@실례.테스트), but internally you must make sure that you only deal with the punycode representation.

This translating is also what modern browsers do. If you were to paste "http://xn--9n2bp8q.xn--9t4b11yi5a/" directly in the firefox address bar, it will show you the UTF-8 characters instead. Firefox will re-encode to punycode though and use that format for HTTP requests.

The best way really to check for valid email addresses is to use a very liberal regex, but verify with a simple MX record lookup if a mailserver exists for the given domain. This example is an expansion on the first regex.

$email = 'example@xn--9n2bp8q.xn--9t4b11yi5a';
 
if(preg_match('/^[A-Z0-9._%+-]+@([A-Z0-9.-]+\.[A-Z0-9-]{2,})$/i', $email,$matches)) {
    $hostname = $matches[1];
        if (!getmxrr($hostname, $hosts)) {
            echo "Host has an MX record\n";
        } else {
            echo "Host does not exist or does not have an MX record\n";
        }
} else {
    echo "Email address did not match regular expression\n";
}

The preceeding code does not convert UTF-8 to punycode though. There's not yet an easy native way in PHP to do this, but Pear's Net_IDNA2 provides a way. The implementation seems very complex though, and leaves me wondering if there's an easier way to go about it.

Our session system is due for an upgrade. Currently all PHP sessions are stored in the database, and some things are getting a bit slow. There have been a couple of approaches I've been considering, one of which is simply storing all the information in a browser cookie.

First I want to make clear I don't necessarily condone this. The reason I'm writing this post, is because I'm hoping for some more community feedback. Is this a really bad idea? I would love to know.

The benefits

If all the session data is stored in the browser, it means that I don't need to store it on the server. I actually don't care all that much for having the data on the server (unless it's the only secure way), it's mostly a gigantic map with session tokens and user id's (along with some other info).

I also feel it's more natural for HTTP, as it makes it a bit more stateless.

Sample code

<?php
 
class BrowserSession {
 
    public $secret = 'this will need to be a cryptographic random number';
    public $currentUser = null;
 
    // Sessions time out after 10 minutes
    public $timeout = 600;
 
    function init() {
 
        if (!isset($_COOKIE['MYSESSION'])) {
            echo "No session cookie found\n";
            return;
        }
 
        list($userId, $time, $signature) = explode(':',$_COOKIE['MYSESSION']);
 
        // The cookie is old
        if ($time> time() + $this->timeout) {
            echo "The session cookie timed out\n";
        }
 
        if ($signature !== $this->generateSignature($userId,$time)) {
            echo "The secret was incorrect\n";
        }
 
        $this->currentUser = $userId;
 
        echo "Logged in as user: $userId\n";
 
    }
 
    function login($userId) {
 
        $this->userId = $userId;
 
        $time = time();
 
        $cookie = $this->userId . ':' . time() . ':' . $this->generateSignature($userId,$time);
 
        setcookie('MYSESSION',$cookie,$time+$this->timeout,null,null,null,true);
 
        echo "Set cookie: $cookie\n";
 
    }
 
    function generateSignature($userId,$time) {
 
        $stringToSign =
           $userId . "\n" .
           $time . "\n" .
           $_SERVER['HTTP_USER_AGENT'] . "\n" .
           $_SERVER['REMOTE_ADDR'];
 
        return hash_hmac('SHA1',$stringToSign,$this->secret);
 
    }
 
}
 
ob_start();
$session = new BrowserSession();
$session->init();
 
if (isset($_GET['login'])) $session->login($_GET['login']);
else {
 
    echo '<br /><a href="?login=1234">Log in as user 1234</a>';
 
}
?>

A few notes:

• The preceeding code was just intended as a proof of concept, it's missing some validation.
• Currently the secret would be the same for every user. I was thinking of appending some per-user information to the secret. If somebody does guess or bruteforce the secret, they would only have access to a single users' information.
• If a user changes their password, existing sessions should expire. To do this the signature should also include a sequence number that changes when the password changes.
• Currently this only stores a user id. It could be extended to contain more data, but this is all I need.

So, is there anything fundamentally wrong with this approach? In general the client should never be trusted, but for setups where the security requirements aren't as high (highly subjective, I know) I feel this might be strong enough. OAuth, OpenID and Amazon AWS all seem to trust HMAC+SHA1, but those applications do work differently.

Credit where it's due

I first asked this question on stack overflow. The users there already gave some great suggestions and pointed out some of the flaws. Thank you!