Monday, February 28, 2011

Re: how to redirect and let the server do some lengthy job in the background?

On Feb 28, 2011, at 01:48, Zaky Katalan-Ezra wrote:

> You have the ff log to check if every thing is ok and the progress.
> You can create PID file to check if the process ended.

These ideas are fine if everything is running on a single server, but falls apart if there are multiple frontend web servers. What if frontend web server 2 wants to know about an encode job that was started on frontend web server 1? What if you decide video encoding will be delegated to a dedicated video encoding server? My suggestion supposes that you have a single daemon running on a single server, which handles all video encoding, just by checking the database. If the video encoding turns out to be so CPU-intensive that you need more than one server doing video encoding, then the daemon could be rewritten in some way to allow multiple instances of itself to not interfere with one another.


> Form my experience when dealing with files its better to inspect the filesystem in order to know what you have.
> The information from the database may conflict with the filesystem because of a bug.
> Way save the information twice?, the filesystem should be treated as a data source also.

You're already doing a database query, for example to get a list of all videos. It's nice to be able to limit this query to those videos that are encoded. (If you're trying to display a list of videos someone can watch, why bother fetching information from the database about videos that can't be watched?) IMHO the database should be the authoritative source of information, and the app should be written so that changes to files on disk are accompanied by a corresponding db update.

Also, I don't know whether ffmpeg does this, but when you ask it to convert x.mpg to x.flv, it's possible it starts writing to x.flv right away. How should your app know whether the x.flv that's there has just begun being encoded or has already finished being encoded? My suggestion eliminates this ambiguity by recording definitively in the database when encoding is done. (Another option would be to encode to a temporary location and only move the encoded video to its final location once encoding is done. It's also possible ffmpeg already does something like this internally.)

Recording in the database that encoding is done could be expanded to also save other information about the video into the database -- original and/or encoded size on disk, width, height, audio/video codecs, etc. Yes, this information could also be fetched from the original and encoded files whenever they're needed, but again, that makes it hard to write queries that search based on these criteria.


--
Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org
Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions.


To unsubscribe from this group, send email to
cake-php+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php

No comments: