PHP memory problems

Status
Not open for further replies.
S

spyderman4g63

New member
Mar 8, 2007
313
2
0
Ohio
www.teamtutorials.com
I thought I had a huge memory leak in an app I am writing, but anything I run keeps eating memmory. For example:

PHP: 
<?php 
function loop(){
global $i,$startmem;

    print "$i-mem: ".(memory_get_usage() - $startmem)." bytes<br/>";
    $i=$i+1;
    unset($startmem);
    
    if ($i==20){
    print('end loop');
    exit();
    }else{
    loop();
    }




}
$startmem = memory_get_usage();
$i = 1;
loop();

?>
Why does that code eat memory? Here is the output:

1-mem: 232 bytes
2-mem: 516 bytes
3-mem: 652 bytes
4-mem: 788 bytes
5-mem: 924 bytes
6-mem: 1060 bytes
7-mem: 1196 bytes
8-mem: 1332 bytes
9-mem: 1468 bytes
10-mem: 1604 bytes
11-mem: 1740 bytes
12-mem: 1876 bytes
13-mem: 2012 bytes
14-mem: 2148 bytes
15-mem: 2284 bytes
16-mem: 2420 bytes
17-mem: 2556 bytes
18-mem: 2692 bytes
19-mem: 2828 bytes
end loop



I don't get it.
 


Edit

I rewrote the script above to see if it would make a difference:

PHP: 
<?php 
$i = 1;
$startmem = memory_get_usage();
for($i = 0; $i < 100; $i++) {
   print "$i-mem: ".(memory_get_usage() - $startmem)." bytes<br/>";
}
?>


Now the output is:
0-mem: 164 bytes
1-mem: 312 bytes
2-mem: 312 bytes
3-mem: 312 bytes
4-mem: 312 bytes
5-mem: 312 bytes
6-mem: 312 bytes
7-mem: 312 bytes
8-mem: 312 bytes
9-mem: 312 bytes
10-mem: 312 bytes
11-mem: 332 bytes
12-mem: 332 bytes
13-mem: 332 bytes
14-mem: 332 bytes
15-mem: 332 bytes
16-mem: 332 bytes
17-mem: 332 bytes
18-mem: 332 bytes
19-mem: 332 bytes
20-mem: 332 bytes
etc...

So I take it you cannot loop functions with PHP?
 
You can recursively call (or loop as you put it), but it seems to create more memory overhead for some reason. I think because you're basically instantiating a new function call while the current call hasn't finished, all the variables are stored again in memory and php isn't very good at cleanup.

*edit*
Code: 
$startmem = memory_get_usage();
echo "startmem = $startmem<br/><br />";
$i = 1;

loop2($i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);
loop2(++$i, $startmem);

function loop2($i, $startmem){    
    print "$i-mem: ".(memory_get_usage() - $startmem)." bytes<br/>";
}
returns:

startmem = 65520

1-mem: 384 bytes
2-mem: 384 bytes
3-mem: 384 bytes
4-mem: 384 bytes
5-mem: 384 bytes
6-mem: 384 bytes
7-mem: 384 bytes
8-mem: 384 bytes
9-mem: 384 bytes
10-mem: 408 bytes
11-mem: 408 bytes
12-mem: 408 bytes
13-mem: 408 bytes
14-mem: 408 bytes
15-mem: 408 bytes
16-mem: 408 bytes
17-mem: 408 bytes
18-mem: 408 bytes
19-mem: 408 bytes


So it's definitely the recursion doing something

-the mole
 
  • Like
Reactions: spyderman4g63
themole said:
So it's definitely the recursion doing something

-the mole
Click to expand...
I noticed the OP defined startmem as a global in the beginning of the function... yet he unset it before it calls loop again, hence unsetting the global startmem before the 2nd recursion can even start.... so what is the memory_usage minus NULL? think about it...

Say startmem is 16000
1st Loop - $startmem = 16000
2nd Loop - $startmem = NULL
3rd Loop - $startmem = NULL
4rd Loop - $startmem = NULL
and so on... (PHP acts funky if yous start dividing by null)


Btw its been a while since I read the PHP white papers, but isn't there a limit to how many recursions a function can perform? You would think after so many you would get a stack overflow error on the server.
 
  • Like
Reactions: spyderman4g63
If you think your code confused you, check this out... what I consider a proper recursive function.

PHP: 
<? 
function loop($i)
{
    global $start;
    
    if ($i > 0)
        echo $i."-mem: ".(loop($i-1) - $start)." bytes<br/>\n";

    return memory_get_usage();
}

$start = memory_get_usage();
loop(20);

echo "end loop";
unset($start);
?>
Output:

1-mem: 11080 bytes
2-mem: 11112 bytes
3-mem: 11112 bytes
4-mem: 11112 bytes
5-mem: 11112 bytes
6-mem: 11112 bytes
7-mem: 11112 bytes
8-mem: 11112 bytes
9-mem: 11112 bytes
10-mem: 11112 bytes
11-mem: 11112 bytes
12-mem: 11112 bytes
13-mem: 11112 bytes
14-mem: 11112 bytes
15-mem: 11112 bytes
16-mem: 11112 bytes
17-mem: 11112 bytes
18-mem: 11112 bytes
19-mem: 11112 bytes
20-mem: 11112 bytes
end loop
Click to expand...

So question for ya... from 1 to 20... is it showing memory usage forward in time, or backwards in time? :P
 
kblessinggr, I don't think that makes any noticeable difference. Get rid of $startmem altogether and the memory leak is still there.
themole is right in that PHP isn't very good at memory management. My guess is the function creates a new local instance of $i on every iteration, and doesn't probably remove them once the function finishes running.
 
Stanley said:
kblessinggr, I don't think that makes any noticeable difference. Get rid of $startmem altogether and the memory leak is still there.
themole is right in that PHP isn't very good at memory management. My guess is the function creates a new local instance of $i on every iteration, and doesn't probably remove them once the function finishes running.
Click to expand...

Thats kinda why I thought it was funny, we're using this recursion to look at memory usage... but its pointless in determine the problem in the script itself that's causing the problem.

Basically my last code (which has a new instance of $i in each call) shows that themole is incorrect in saying.
"You can recursively call (or loop as you put it), but it seems to create more memory overhead for some reason."

Its not the recursion that causes overhead, but whatever it is you're doing inside of it.

So... the OP's exercise in looping the memory is pointless if he's not looking at the actual script itself that has the problem.
 
Question... to the OP
is it PHP4 or PHP5?

My code was ran on PHP5 which could make a difference in memory management.

Cuz if I run the EXACT same code as in the original post. this is my result.

1-mem: 0 bytes
2-mem: 0 bytes
3-mem: 0 bytes
4-mem: 0 bytes
5-mem: 0 bytes
6-mem: 0 bytes
7-mem: 0 bytes
8-mem: 0 bytes
9-mem: 0 bytes
10-mem: 0 bytes
11-mem: 0 bytes
12-mem: 0 bytes
13-mem: 0 bytes
14-mem: 0 bytes
15-mem: 0 bytes
16-mem: 0 bytes
17-mem: 0 bytes
18-mem: 0 bytes
19-mem: 0 bytes
end loop

(btw $startmem was before the loop call : 99392)

... so that gets me curious as to which version of PHP you're using.

Edit Course as Stanly said, the script could be perfect and there is still a link.

So here is my thought... your script (the one you claim the problem is) is still running in the background, and the loop isn't causing the memory leak, but is actually just reporting the leak happening by the other script still running. If that is the case, your other script has an infinite loop going on somewhere.
 
More food for thought: (or beating a dead horse ;) )

Code: 
<?php 
function loop($i,$startmem)
{    
    print "$i-mem: ".( memory_get_usage()- $startmem)." bytes (".memory_get_peak_usage().")<br/>";    
    ++$i;
    if ($i==20)
    {
        print('end loop<br /><br />');        
    }
    else
    {
        loop(&$i,$startmem);
    }
}
$startmem = memory_get_usage();
$i = 1;
loop(&$i,$startmem);
print "$i-mem: ".(memory_get_usage() - $startmem)." bytes<br  />";
loop(19,$startmem);
loop(19,$startmem);
loop(19,$startmem);
?>
which returns

1-mem: 528 bytes (70392)
2-mem: 1016 bytes (70392)
3-mem: 1544 bytes (70392)
4-mem: 2072 bytes (70392)
5-mem: 2600 bytes (70392)
6-mem: 3128 bytes (70392)
7-mem: 3616 bytes (70392)
8-mem: 4184 bytes (70392)
9-mem: 4712 bytes (70392)
10-mem: 5264 bytes (70392)
11-mem: 5792 bytes (70392)
12-mem: 6320 bytes (70392)
13-mem: 6848 bytes (70392)
14-mem: 7376 bytes (70392)
15-mem: 7904 bytes (70392)
16-mem: 8960 bytes (70392)
17-mem: 9488 bytes (70392)
18-mem: 10016 bytes (70392)
19-mem: 10504 bytes (70392)
end loop

20-mem: 4512 bytes
19-mem: 4856 bytes (70392)
end loop

19-mem: 4896 bytes (70392)
end loop

19-mem: 4896 bytes (70392)
end loop
Click to expand...
Now in this code notice the lack of the memory_get_peak_usage()

Code: 
<?php 
function loop($i,$startmem)
{    
    print "$i-mem: ".( memory_get_usage()- $startmem)." bytes ()<br/>";    
    ++$i;
    if ($i==20)
    {
        print('end loop<br /><br />');        
    }
    else
    {
        loop(&$i,$startmem);
    }
}
$startmem = memory_get_usage();
$i = 1;
loop(&$i,$startmem);
print "$i-mem: ".(memory_get_usage() - $startmem)." bytes<br  />";
loop(19,$startmem);
loop(19,$startmem);
loop(19,$startmem);
?>
and that returns

1-mem: 456 bytes ()
2-mem: 872 bytes ()
3-mem: 1288 bytes ()
4-mem: 1744 bytes ()
5-mem: 2200 bytes ()
6-mem: 2656 bytes ()
7-mem: 3112 bytes ()
8-mem: 3568 bytes ()
9-mem: 4024 bytes ()
10-mem: 4504 bytes ()
11-mem: 4960 bytes ()
12-mem: 5416 bytes ()
13-mem: 5872 bytes ()
14-mem: 6328 bytes ()
15-mem: 6784 bytes ()
16-mem: 7768 bytes ()
17-mem: 8224 bytes ()
18-mem: 8680 bytes ()
19-mem: 9136 bytes ()
end loop

20-mem: 9176 bytes
19-mem: 9200 bytes ()
end loop

19-mem: 9200 bytes ()
end loop

19-mem: 9200 bytes ()
end loop
Click to expand...

It seems like memory_get_peak_usage() does some sort of memory cleanup once the function recursions are all done...but at the same time it also doesn't seem to as I tried calling it after each of the "loop(19,$startmem);" and it made no difference with either code. Also passing $i by reference used slightly less memory than passing by value, but the results were still the same otherwise.

Anyways recursively calling the function and passing in $i and $startmem doesn't make a difference. Memory usage still goes up because it's still having to make new copies of the variables in memory to work with.

On unsetting a global variable php.net says: "If a globalized variable is unset() inside of a function, only the local variable is destroyed. The variable in the calling environment will retain the same value as before unset() was called." That reinforces it's making a new copy of the variables used regardless imo.

Fun stuff...
 
I also noticed that if I use multiple file handlers in a loop i will get a leak.

Basically I am scraping -> caching the file (closing this handle) -> reopening(new handle or reuse the same causes the same outcome). It is random also. say memory will stay stable for 20 times, then increase, then work for 5 or so and increase. It is different every time I run it.

I also tried to keep the file open, write to it, read from it and it immediately fails with the "tried to use X amount of memory).

I am using PHP 5.2.6

I tried to write to one file, and read a different file and had no memory problems. The reason I reopen the file is to read it line by line to an array. I use either fread or fopen. I also notice the php.net says fopen needs to have the handle closed, however php will not use the variable as a valid resource.

$lines = fopen('file.dat','r');
fclose($lines);
--not valid



edit: Somebody said that it is possible that I have an infinite loop running in the background causing the memory usage (which is very possible since I have been screwing around so much). Is there a way to kill this on my server before running new code?
 
Thanks for the replies guys. My first thought was that this had to to with me not completely understanding the variable scope in php. I have actually been messing with this over he last 3 nights, commenting out sections of the actual script to try to hunt down the memory problems. I am still making no progress.
 
I found a cron job running on my test site that started everynight and appears to have an infinite loop. I think that was the main problem causing the leak. I worked on this for the past 3 nights until somebody mentioned that in this thread. Now it appears to be stable. The memory usage goes up and down though, I think this is due to the size of the pafe I am scraping not being static because of rotating ads? So thanks for the help everyone.
 
spyderman4g63 said:
I found a cron job running on my test site that started everynight and appears to have an infinite loop. I think that was the main problem causing the leak. I worked on this for the past 3 nights until somebody mentioned that in this thread. Now it appears to be stable. The memory usage goes up and down though, I think this is due to the size of the pafe I am scraping not being static because of rotating ads? So thanks for the help everyone.
Click to expand...
Up and down may not necessarily be a bad thing. Constantly going up on the other hand would be.

Good to hear that it's more stable though.
 
Status
Not open for further replies.
Top Bottom