Rogers takes over my browser

Feb 27, 2009 11 Comments , , , and

Rogers is one of Canada's biggest ISP's. I've pretty much been subscribed to them ever since I moved here. The last few months I've been seeing some shady behaviour, which lead to find out:

  • They have some sort of transparent proxy in place, not only intercepting tcp packages; but sometimes even change them.
  • Every DNS lookup to an unknown host will be responded to with a rogers ip.

Both of these are a little scary. The first time I noticed they were rewriting packages, was when I was just browsing around. All of a sudden rogers injected some html telling me my bills were due. This one is the most scary, who knows what else they log or modify. Wouldn't this cause some privacy concerns in most western countries?

I didn't make a screenshot at the time, but I'm not the only one.

DNS intercepting

This one has primarily been a major annoyance. I've been used to just type a word in the addressbar, and expect a search engine to bring me to the top page. Firefox does this by default when a non-existant domain is requested.

With rogers, I get this:

Rogers is watching you

Notice the first 3 items are spam.

I wonder if they put thought in the potential side-effects to applications. Some can definitely rely on negative replies from DNS servers. Firefox is a simple example, but similarly a ping to a misspelled domain will always succeed.

  1. $ ping -c 4 thisiscreepy.rogers
  2. PING thisiscreepy.rogers (8.15.7.107): 56 data bytes
  3. 64 bytes from 8.15.7.107: icmp_seq=0 ttl=54 time=63.920 ms
  4. 64 bytes from 8.15.7.107: icmp_seq=1 ttl=54 time=48.662 ms
  5. 64 bytes from 8.15.7.107: icmp_seq=2 ttl=54 time=50.744 ms
  6. 64 bytes from 8.15.7.107: icmp_seq=3 ttl=54 time=84.603 ms
  7.  
  8. --- thisiscreepy.rogers ping statistics ---
  9. 4 packets transmitted, 4 packets received, 0% packet loss
  10. round-trip min/avg/max/stddev = 48.662/61.982/84.603/14.311 ms

I wouldn't be surprised if there's security related implications as well. All a bit scary to me. If you're currently a rogers customer, I would definitely recommend switching DNS providers to OpenDNS, which promises to be safe, and as a bonus; i've definitely noticed much faster DNS lookups as well.

A few tests:

RogersOpenDNS
me.evertpot.com427 msec236 msec
www.rooftopsolutions.nl381 msec33 msec
www.weddav.org4370 msec53 msec

And who could forget: net neutrality. I feel it's time to switch isp's.

SabreDAV 0.6

Feb 17, 2009 4 Comments , and

I just finished up a new release of SabreDAV. The mailing list is getting some traction, and it's shows! There were quite a bit of suggestions and patches recently, so the changelog is relatively big. The best thing that can happen is of course people liking and using it, and it makes me also more excited to work on it :).

Download it here.

Most notable changes:

  • Added: We're now using streams for passing contents of files around everywhere. Not only is this a big speed improvement, but it also reduces memory consumption quite a bit for larger files. It should be noted that this breaks API compatibility as the ->put and ->createFile methods will now no longer receive strings.
  • Added: HTTP Digest authentication helper.
  • Added: Support for HTTP Range header, ETags and overriding default Content-Type.
  • Added: 49 unittests.
  • Updated: Using Clark notation for serializing custom properties. If you use custom properties this affects you, because properties are no longer serialized as http://namespace#tagName, but {http://namespace}tagName.
  • Removed: SabreDAV is no longer distributed as a pear package. The main reason for this is that I had too much trouble figuring out how it works. I use phing as my build-tool, and although there's some good integration points with pear packaging, I couldn't get it to do what I wanted. So now it's a regular gzip tarball.
  • Bugfixes!

Full Changelog.

Special thanks to Andreas Gohr, Luidnel Maignan and Stephan Wenz for patches and suggestions.

Search engines to support 'canonical urls', or: how to reinvent the wheel

Feb 14, 2009 3 Comments , , , and

Google, Yahoo and Microsoft all announced support for a so-called canonical url's, allowing us developers to avoid duplication in search engines.

Great feature, but it seems that we already had a standard for it.

HTTP Basic and Digest authentication with PHP

Feb 12, 2009 11 Comments , , , and

HTTP authentication is quite popular for web applications. It is pretty
easy to implement and works for a range of http applications; not to mention
your browser.

Basic Auth

The two main authentication schemes are 'basic' and 'digest'. Basic is pretty
easy to implement and appears to be the most common:

  1. <?php
  2.  
  3. $username = null;
  4. $password = null;
  5.  
  6. // mod_php
  7. if (isset($_SERVER['PHP_AUTH_USER'])) {
  8.     $username = $_SERVER['PHP_AUTH_USER'];
  9.     $password = $_SERVER['PHP_AUTH_PW'];
  10.    
  11. // most other servers
  12. } elseif (isset($_SERVER['HTTP_AUTHENTICATION'])) {
  13.    
  14.         if (strpos(strtolower($_SERVER['HTTP_AUTHENTICATION']),'basic')===0)
  15.           list($username,$password) = explode(':',base64_decode(substr($_SERVER['HTTP_AUTHORIZATION'], 6)));
  16.          
  17. }
  18.  
  19. if (is_null($username)) {
  20.    
  21.     header('WWW-Authenticate: Basic realm="My Realm"');
  22.     header('HTTP/1.0 401 Unauthorized');
  23.     echo 'Text to send if user hits Cancel button';
  24.    
  25.     die();
  26.    
  27. } else {
  28.     echo "<p>Hello {$username}.</p>";
  29.     echo "<p>You entered {$password} as your password.</p>";
  30. }
  31.  
  32. ?>

Well it's a bit difficult I suppose, but you might have noticed the username
and password are sent over the wire using base64 encoding. Not really secure, unless
you have SSL in place.

Digest

Digest is designed to be more secure. The password is never sent over the wire
in plain text, but rather as a hash. The implications of the usage of a hash is that
it can never be decrypted. We can only validate the hash by applying the same hash function
to the password we have. If the hashes match, the password was correct.

Lets first see how Digest auth should work:

Client requests url

  1. GET / HTTP/1.1

Server requires authentication

  1. HTTP/1.1 401 Unauthorized
  2. WWW-Authenticate: Digest realm="The batcave",
  3.   qop="auth",
  4.   nonce="4993927ba6279",
  5.   opaque="d8ea7aa61a1693024c4cc3a516f49b3c"

Client authenticates

  1. GET / HTTP/1.1
  2. Authorization: Digest username="admin",
  3.   realm="The batcave",
  4.   nonce=49938e61ccaa4,
  5.   uri="/",
  6.   response="98ccab4542f284c00a79b5957baaff23",
  7.   opaque="d8ea7aa61a1693024c4cc3a516f49b3c",
  8.   qop=auth, nc=00000001,
  9.   cnonce="8d1b34edb475994b"

Information coming from the server:

realmA string which will be used within the UI and as part of the hash.
qopCan be auth and auth-int and has influence on how the hash is created. We use auth.
nonceA unique code, which will be used within the hash and needs to be sent back by the client.
opaqueThis can be treated as a session id. If this changes the browser will deauthenticate the user.

Information from the client:

usernameThe supplied username
realmSame as server response.
nonceSame as server response.
uriThe authentication uri
responseThe validation hash.
opaqueSame as server response.
qopSame as server response.
ncNonce-count. This a hexadecimal serial number for the request. The client should increase this number by one for every request.
cnonceA unique id generated by the client

So how do we know if the password was correct? We van validate using the following formula (pseudo code).

  1. A1 = md5(username:realm:password)
  2. A2 = md5(request-method:uri) // request method = GET, POST, etc.
  3. Hash = md5(A1:nonce:nc:cnonce:qop:A2)
  4.  
  5. if (Hash == response)
  6.   //success!
  7. else
  8.   //failure!

Or, using PHP:

  1. <?php
  2.  
  3. $realm = 'The batcave';
  4.  
  5. // Just a random id
  6. $nonce = uniqid();
  7.  
  8. // Get the digest from the http header
  9. $digest = getDigest();
  10.  
  11. // If there was no digest, show login
  12. if (is_null($digest)) requireLogin($realm,$nonce);
  13.  
  14. $digestParts = digestParse($digest);
  15.  
  16. $validUser = 'admin';
  17. $validPass = '1234';
  18.  
  19. // Based on all the info we gathered we can figure out what the response should be
  20. $A1 = md5("{$validUser}:{$realm}:{$validPass}");
  21. $A2 = md5("{$_SERVER['REQUEST_METHOD']}:{$digestParts['uri']}");
  22.  
  23. $validResponse = md5("{$A1}:{$digestParts['nonce']}:{$digestParts['nc']}:{$digestParts['cnonce']}:{$digestParts['qop']}:{$A2}");
  24.  
  25. if ($digestParts['response']!=$validResponse) requireLogin($realm,$nonce);
  26.  
  27. // We're in!
  28. echo 'Well done sir, you made it all the way through the login!';
  29.  
  30. // This function returns the digest string
  31. function getDigest() {
  32.  
  33.     // mod_php
  34.     if (isset($_SERVER['PHP_AUTH_DIGEST'])) {
  35.         $digest = $_SERVER['PHP_AUTH_DIGEST'];
  36.     // most other servers
  37.     } elseif (isset($_SERVER['HTTP_AUTHENTICATION'])) {
  38.      
  39.             if (strpos(strtolower($_SERVER['HTTP_AUTHENTICATION']),'digest')===0)
  40.               $digest = substr($_SERVER['HTTP_AUTHORIZATION'], 7);
  41.     }
  42.  
  43.     return $digest;
  44.  
  45. }
  46.  
  47. // This function forces a login prompt
  48. function requireLogin($realm,$nonce) {
  49.     header('WWW-Authenticate: Digest realm="' . $realm . '",qop="auth",nonce="' . $nonce . '",opaque="' . md5($realm) . '"');
  50.     header('HTTP/1.0 401 Unauthorized');
  51.     echo 'Text to send if user hits Cancel button';
  52.     die();
  53. }
  54.  
  55. // This function extracts the separate values from the digest string
  56. function digestParse($digest) {
  57.     // protect against missing data
  58.     $needed_parts = array('nonce'=>1, 'nc'=>1, 'cnonce'=>1, 'qop'=>1, 'username'=>1, 'uri'=>1, 'response'=>1);
  59.     $data = array();
  60.  
  61.     preg_match_all('@(\w+)=(?:(?:")([^"]+)"|([^\s,$]+))@', $digest, $matches, PREG_SET_ORDER);
  62.  
  63.     foreach ($matches as $m) {
  64.         $data[$m[1]] = $m[2] ? $m[2] : $m[3];
  65.         unset($needed_parts[$m[1]]);
  66.     }
  67.  
  68.     return $needed_parts ? false : $data;
  69. }
  70.  
  71. ?>

As you can see we need to have a plain-text version of the password in order to
validate the user. It's not a good idea to store the plain-text password, therefore
it's strongly recommended to store the result of $A1 instead.

Security improvements

  • It's smart to validate the contents of opaque, nonce and realm. If you have
    the data stored on the server, why not check it.
  • The nc should be an ever increasing number. You could store the number and
    track to make sure it doesn't make any big jumps. It's not wanted to be extremely
    strict about the sequence, because you might miss a number, and requests could come in
    be out of order.
  • 'qop' is quality of protection. This serves as an integrity code for the request.
    A hacker could steal all your HTTP Digest headers and simply change the body to make it do
    something else. If 'qop' is set to 'auth', only the requested uri will be taken
    into consideration. If 'qop' is 'auth-int' the body of the request will also be used in the hash. (A2 = md5(request-method:uri:md5(request-body))).

References:

 1

About

My name is Evert, and I've been writing semi-regularly on this blog since 2006.

I'm currently available for contract work.

more info.

Subscribe

Dropbox

Dropbox is a simple cross-platform online backup and sync application. The first 2GB of space is free, and both you and me get an extra 250MB extra space if you sign up through this link.