Re: keep hitting mysql memory limit, even in Cake Shell when exporting large xml file

i have paginated the results and no matter how i chunk it it still
fills up the mysqli result set and hits the memory limit. i need to be
able to flush the result set between each iteration.

On Jul 26, 6:33 am, Jaydeep Dave <jaydipd...@gmail.com> wrote:
> You can try something like this in your shell....
>
> <?php
>
> .......
> ......
>
> $page = 1;
>
> while(1) {
>
>          $records = $this->findAll("conditions' => array(bla bla bla),
> 'fields' => 'Model.*' ,'limit' => 50, 'page' => $page);
>
>          if($records) {
>               // Write Your XML in a File.
>          }else{
>                break;
>          }
>
>         $page = $page + 50;
>
> }
>
> This will help you a lot.
>
> Regards,
>
> Jaydeep Dave
>
>
>
> On Sun, Jul 26, 2009 at 1:41 AM, majna <majna...@gmail.com> wrote:
>
> > Try with :
> > $mysqli = ConnectionManager::getDataSource('default');
> > $mysqli->disconnect();
> > $mysqli->connect();
>
> > debug($mysqli->connected);
> > debug($mysqli->connection);
>
> > On Jul 26, 7:55 am, JamesF <usaexportexpe...@gmail.com> wrote:
> > > MY PROBLEM:
> > > i am running into a wall with this one. basically i have a large xml
> > > file that i am rendering using XmlHelper. We are talking about a
> > > 25-50mb file. This is basically a product data feed. I have tried
> > > quite a few methods to overcome the memory limit but no luck.
>
> > > MY ERROR:
> > > Fatal error: Allowed memory size of 83886080 bytes exhausted (tried to
> > > allocate 53 bytes) in /home/username/usr/cakedev/cake/libs/model/
> > > datasources/dbo/dbo_mysqli.php on line 402
>
> > > WHAT I HAVE DONE TO TRY AND FIX IT:
>
> > > 1) Initially i set this up to render through the web in one
> > > shot....this cause execution limit timeouts and memory errors.
>
> > > 2)I broke up the data requests in seperate chunks of 500 records using
> > > internal method calls, like $listings = $this->getListings($start_id,
> > > $limit); This caused the same problem
>
> > > 3) I set the whole thing up via the cake shell interface. It works
> > > great except for the fact that i keep running up against the same
> > > limit, even with staggered chunks of records in different method
> > > calls.
>
> > > MY GUESS:
> > > is that Cake is keeping an open connection with the database for the
> > > life of the script.
>
> > > MY SPECIFIC QUESTIONS:
> > > can i disconnect and reconnect from the database inside of the script?
> > > specifically can i do this in a cake shell script? will this solve my
> > > memory problem? am i going about this the wrong way entirely?
>
> > > thanks again anyone who has some advice!
>
> --
> Regards,
>
> Jaydeep Dave
> Mobile: +919898456445
> Email: jaydipd...@yahoo.com
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "CakePHP" group.
To post to this group, send email to cake-php@googlegroups.com
To unsubscribe from this group, send email to cake-php+unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.com/group/cake-php?hl=en
-~----------~----~----~----~------~----~------~--~---