Sunday, July 26, 2009

Re: keep hitting mysql memory limit, even in Cake Shell when exporting large xml file

great suggestion and it worked to disconnect from the database...but i
still ran into the same error.

I have also tried:

$this->Model->cacheQueries = false;
with the idea being to turn of cakephp model caching...no luck

also tried this which seemed promising:
mysqli_next_result( $mysqli->connection);
http://www.nabble.com/SQL-Error:-2014-td17315466.html
the idea being to prepare mysql for the next result set
no luck though still same error....
i might be implementing this incorrectly so i will need to do some
more testing.


On Jul 26, 5:41 am, majna <majna...@gmail.com> wrote:
> Try with :
> $mysqli = ConnectionManager::getDataSource('default');
> $mysqli->disconnect();
> $mysqli->connect();
>
> debug($mysqli->connected);
> debug($mysqli->connection);
>
> On Jul 26, 7:55 am, JamesF <usaexportexpe...@gmail.com> wrote:
>
> > MY PROBLEM:
> > i am running into a wall with this one. basically i have a large xml
> > file that i am rendering using XmlHelper. We are talking about a
> > 25-50mb file. This is basically a product data feed. I have tried
> > quite a few methods to overcome the memory limit but no luck.
>
> > MY ERROR:
> > Fatal error: Allowed memory size of 83886080 bytes exhausted (tried to
> > allocate 53 bytes) in /home/username/usr/cakedev/cake/libs/model/
> > datasources/dbo/dbo_mysqli.php on line 402
>
> > WHAT I HAVE DONE TO TRY AND FIX IT:
>
> > 1) Initially i set this up to render through the web in one
> > shot....this cause execution limit timeouts and memory errors.
>
> > 2)I broke up the data requests in seperate chunks of 500 records using
> > internal method calls, like $listings = $this->getListings($start_id,
> > $limit); This caused the same problem
>
> > 3) I set the whole thing up via the cake shell interface. It works
> > great except for the fact that i keep running up against the same
> > limit, even with staggered chunks of records in different method
> > calls.
>
> > MY GUESS:
> > is that Cake is keeping an open connection with the database for the
> > life of the script.
>
> > MY SPECIFIC QUESTIONS:
> > can i disconnect and reconnect from the database inside of the script?
> > specifically can i do this in a cake shell script? will this solve my
> > memory problem? am i going about this the wrong way entirely?
>
> > thanks again anyone who has some advice!
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "CakePHP" group.
To post to this group, send email to cake-php@googlegroups.com
To unsubscribe from this group, send email to cake-php+unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.com/group/cake-php?hl=en
-~----------~----~----~----~------~----~------~--~---

No comments: