Re: how to redirect and let the server do some lengthy job in the background?

On Feb 28, 2011, at 13:55, Tan Cheng wrote:

> This is a great idea! So whichever script starts encoding, it takes
> place in the $LOCKS folder, then everytime before a new script starts
> running it will always check if the "$LOCK room" is empty, and runs
> only if it's empty. Brilliant! See how far I can go with all the
> ideas...

I hadn't meant for encoding to occur in the locks directory; I had only meant for the locks directory to contain locks -- empty files with carefully-chosen names that signify when an instance of the encoder daemon is running. The encoding itself (i.e. the creation by ffmpeg of temporary files related to encoding a particular video) should occur in a different directory; $APP/tmp/encoder maybe.

Also, just checking if the directory is empty before starting isn't enough; that allows a race condition. Consider: two instances of the encoder daemon are started simultaneously for some reason and run more or less in parallel, because the OS is multithreaded. Both instances check the directory and find it to be empty. Both scripts then decide they should continue, both scripts create their locks, and both scripts begin encoding the same videos. My suggestion of first creating the lock, then checking if there are any other locks, should guarantee that you will never have two encoder daemons running at once, which is the situation that must be prevented. There's still a race condition leading to the possibility that both instances will decide to quit without doing any work, but that's ok; it'll run again in a short interval anyway, via cron.

Just be sure your script doesn't have a bug in it that causes it to exit prematurely and not clean up the lock file.

Another idea, instead of lock files, would be to simply create a specific directory when beginning to encode. PHP's mkdir returns false (and triggers a warning) if the directory already exists -- if so, it's because another encoder daemon is running so you just exit.

<?php

$encoder_tmp_dir = "$APP/tmp/encoder";
if (@mkdir($encoder_tmp_dir)) {
$videos = array();
do {
$videos = getUnencodedVideos();
foreach ($videos as $video) {
encodeVideo($video);
}
} while (!empty($videos));
rmdir($encoder_tmp_dir);
}

?>

(This assumes PHP's mkdir() function doesn't itself have a race condition.)

Race conditions are challenging to handle correctly, and I'm sure somebody else can suggest a better way to do this, but that's what I came up with for now.

By the way, even if you decide to go for an always-running single-instance daemon as originally discussed, this lock stuff is good to implement anyway. If your daemon relies on the fact that there's only one copy of itself running, it's a good idea to write the code so that if more than one instance is inadvertently started, it won't break things.

--
Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org
Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions.

To unsubscribe from this group, send email to
cake-php+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php