Comments
a+ is wrong, that is append only, meaning that you can't ftruncate and rewind to replace the content.
You need to handle the case where the file didn't exist with some logic in your code with appropriate locking. You might find the PHP specific x+ mode useful for that purpose.
flock() is moot if you fopen($filename, 'w') since that will truncate the file.
You should use r+ which allows you to write; you should then flock() and ftruncate() the file.
Thanks for the correction Wez. I totally didn't think of that situation.
I updated the code in the article
What about memory_limit value in php.ini ?
// fetching all the data and putting it in an array
while($row = mysql_fetch_assoc($result)) { $data[] = $row; }
when $data gets bigger than memory_limit, depending on the server profile, the cache never gets stored, the script dies ....
This cache system needs to improvement on memory management, or some sort of pagination functionality to handle big resultsets.
Tobozo,
It's a good idea to have the pagination in the SQL query and also store the 'page info' in the key of the cache.
function getUsers($start,$count) {
global $cache;
// A somewhat unique key
$key = 'getUsers:selectAll:' . $start . ':' . $count;
.....
klhjjklh
It's worth noting that the largest payload that memcached can store is 1MB. Admittedly, this is a lot of data but I've seen serialized objects that are this large.
Gavin,
I'm going to see if this is really the case. I have not seen this limit myself, but I'll try it out..
Did you ran into this problem yourself, or is there an online resource which confirms this?
Evert
UPDATE: fixed the FileSystem cache class to use the a+ mode instead of the r+. Before, it wouldn't write to the file if it didn't exist already.
And by the way, APC now works for my code! yay!
Err, I'm confusing myself with open modes ;-)
Disregard my last comments about x+, but a+ is still not what you want to use.
Thanks for your comments wez.
I have to say that this worked for me on linux.. Maybe I am assuming incorrect, but from the work I've seen from you, you seem to be a windows guy for a big part (I could be way off with this one).
The simple solution is to do a file_exists().. but I'm going to have to do some more serious cross-platform testing with this one and post results tomorrow.
Again, thanks very much.
That is a possibility..
perhaps it might be even easier to make the first line the TTL and the rest just the unserialized data.. that would speed up things..
That is probably still faster than having multiple files..
I dont like having to unserialize the file each time I need to check if the file is valid.
What about making two files: one for ttl values and one for serialized data.
This will save disk and CPU time..
while (feof($h)) $data.=fread($h,4096);
slight (but show stopping) error here, how about:
while (!feof($h)) $data.=fread($h,4096);
Thanks David,
I actually found out file_get_contents is a faster alternative. I probably misread somewhere that feof is better, but in this case file_get_contents will be faster..
I'll update this now
Usefull article, gonna try that and write what i really think about it later :)
Great article!
About the use of +a in fopen(): even though the behaviour of this open mode is different between Windows and Linux, the desired effect of automatic file creation is identical; you just need to make sure to fseek() before ftruncate() ;-)
Instead of using file_get_contents() after flock(), consider using stream_get_contents()
I completely missed the part about needing zlib-devel, and spent a good hour trying to figure out what I was getting a configure error when trying to install memcache. Doing a yum install memcache fixed that.
I just wrapped memcache so that it's also backed by MySQL. It's a simple table with good indexing and I haven't yet experienced any slow behavior. Anything to big or to long lived to fit in memcache will just ignore memcache and anything that dropped from memcache will just reload as needed from MySQL. Works pretty well.
Thank you for this article, was very helpfull
Great article, helps me a lot. because this i will give back this snippet:
class S_FileCache_Output extends S_FileCache {
private $_ttl;
private $_key;
function start($key, $ttl=0) {
$this->_ttl = $ttl;
$this->_key = $key;
$data = $this->fetch($key);
if ($data !== false) {
echo($data);
return true;
}
ob_start();
ob_implicit_flush(false);
return false;
}
function end() {
$data = ob_get_contents();
ob_end_clean();
$this->store($this->_key, $data, $this->_ttl);
echo($data);
}
}
Now you can yous it like:
$cache = new S_FileCache_Output();
$key = 'OutPut:Buffer';
if (!($cache->start($key))) {
// Some data
echo $data;
$cache->end();
}
I was wondering if you could use filemtime rather than having to open the file and then read part of it to check. Not sure if this would be faster, but logic suggests it would be.
if ((time() - filemtime($filename)) >= $ttl) {
// cache has expired
// do something
} else {
// read file and show
}
Not sure if this is a good idea or not, would love to hear some feed back.
Disregard last post, I just been doing some testing, and it performs better using the ttl in the file over using filemtime.
I opened a two column table with 10 fields using various expiry times, doing this 1000 times for each test. Using the ttl in the file averaged better in every run.
Thats good news,
the biggest reason I put the ttl in file instead of using filemtime, is because i wanted to put the expiry upon storing the data, not while loading..
This allowed me to keep a consistent api across all cache stores..
I'm a little shocked that would be faster though
Hi Evert, I was looking for something like this class, glad I found it here. I have a cople of questions though: What happens if unlink is executed on a file that is currently locked ? will the call to unlink fail ? (if it fails then the delete method wont be able to flush the cache)
Thanks in advance !
Hey Steve,
The unlink will succeed, and if you are on linux and some other process currently has that file open, there will still be no issues.
Hope that helps,
Evert
APC rocks.
Its very good also on multiple servers.
When setting up APC on windows machine, be sure that you use right dll for your PHP version, otherwise you get a small amount of Apache/PHP fatals.
APC rocks.
Its very good also on multiple servers.
When setting up APC on windows machine, be sure that you use right dll for your PHP version, otherwise you get a small amount of Apache/PHP fatals.
This is very good resource for php caching using PECL and APC.....
It is amazing in web development....
I think it is extremely useful in speed up web application and improve performance..
Sandeep Verma
http://sandeepverma.wordpress.com
@cnjax:
I don't see any situation when you really have to cache data larger than 1MB. Though, if you really need to, then just split the data into chunks, like $chunkname = $key.'_'.$chunkId;
But to be honest I don't really think it's the most optimized thing to send >1MB items, even trough your local network. if you have 100/100 on your local network, then you'll only be able to send about 12 items simultaneously per second. I don't know the size of usage on your app, but I do think one should remember that memcached is a caching daemon for fast-access-data, and not a storage-mechanism for harddrive-offloaded-files...
Has someone used Adodb for PHP with memcached? It just doesnt work on my server.
Hi
Thank you for great sharing...
Could you make an example code on how to use it?
As I am noob in OOP php :)
I would serialize the data before openning/locking a file, just to keep the time down that the file is actually locked.
You can also avoid loading the file cache to check its ttl validity by testing its file date by touching the cache file with time() + $ttl like this
function store($key,$data,$ttl) {
// Opening the file in read/write mode
$h = fopen($this->getFileName($key),'a+');
if (!$h) throw new Exception('Could not write to cache');
// Serializing along with the TTL
$data = serialize(array(time()+$ttl,$data));
flock($h,LOCK_EX); // exclusive lock, will get released when the file is closed
fseek($h,0); // go to the beginning of the file
// truncate the file
ftruncate($h,0);
if (fwrite($h,$data)===false) {
throw new Exception('Could not write to cache');
}
fclose($h);
if (!touch($this->getFileName($key), time() + $ttl)) throw new Exception('Could not touch cache file');
}
function fetch($key) {
$filename = $this->getFileName($key);
if (!file_exists($filename)) return false;
// if the 'touched' file time on the cache is older then the current time, then
// delete the file and return false;
if (time() > @filemtime($filename)) {
unlink($filename);
return false;
}
$h = fopen($filename,'r');
if (!$h) return false;
// Getting a shared lock
flock($h,LOCK_SH);
$data = file_get_contents($filename);
fclose($h);
$data = @unserialize($data);
if (!$data) {
// If unserializing somehow didn't work out, we'll delete the file
unlink($filename);
return false;
}
return $data[1];
}
Thanks for this, I was about to build a class to handle many caching methods and you just did half of it for me =)
If TTL is 10 seconds and I access the file at 9th second it gets deleted after the next second. Does that make sense? TTL should always be 10secs from the last accessed time. I think we can achieve this if we store TTL out of the data in some way...any thoughts?
Is there a way I can refresh FILE CREATION TIME/FILE ACCESS TIME without actually writing or modifying the data - so that i can use filemtime to identify TTL
touch and filemtime can be used to implement refreshing TTL on every cache access.
Should try how better I can write a script which will remove all files whose creation/accesstime is less than current time - ttl.
If I have the cachedir with more than 10 million files, listing and then deleting all expired files wont be a good idea as this process itself is resource consuming. Any comments please...







