matplotlib-users Mailing List for matplotlib

Hi Stan,

Seems numpy was the culprit. I upgraded to numpy-1.5.1 and now it works.
Python and runtime updates were not sufficient.

Thanks

Daniel

On Thu, Nov 18, 2010 at 4:50 PM, Benjamin Root <ben...@ou...> wrote:
>
> Caleb,
>
> Interesting analysis.  One possible source of a leak would be some sort of dangling reference that still hangs around even though the plot objects have been cleared.  By the time of the matplotlib 1.0.0 release, we did seem to clear out pretty much all of these, but it is possible there are still some lurking about.  We should probably run your script against the latest svn to see how the results compare.
>
> Another possibility might be related to numpy.  However this is the draw statement, so I don't know how much numpy is used in there. The latest refactor work in numpy has revealed some memory leaks that have existed, so who knows?
>
> Might be interesting to try making equivalent versions of this script using different backends, and different package versions to possibly isolate the source of the memory leak.
>
> Thanks for your observations,
> Ben Root
>

Sorry for the double post; it seems the first is not displaying
correctly on SourceForge.

I conducted a couple more experiments taking into consideration suggestions
made in responses to my original post (thanks for the response).

First, I ran my original test (as close to it as possible anyway) using the
Agg back end for 3 hours, plotting 16591 times (about 1.5Hz). Memory usage
increased by 86MB. That's about 5.3K per redraw. Very similar to my original
experiment. As suggested, I called gc.collect() after each iteration. It
returned 67 for every iteration (no increase), although len(gc.garbage)
reported 0 each iteration.

Second, I ran a test targeting TkAgg for 3 hours, plotting 21374 times. Memory
usage fluctuated over time, but essentially did not increase: starting at
32.54MB and ending at 32.79MB. gc.collect() reported 0 after each iteration
as did len(gc.garbage).

Attached are images of plots showing change in memory usage over time for each
experiment.

Any comments would be appreciated.

Following is the code for each experiment.

Agg
-----

from random import random
from datetime import datetime
import os
import gc
import time
import win32api
import win32con
import win32process

import numpy

import matplotlib
matplotlib.use("Agg")
from matplotlib.figure import Figure
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas

def get_process_memory_info(process_id):
    memory = {}
    process = None
    try:
        process = win32api.OpenProcess(
            win32con.PROCESS_QUERY_INFORMATION|win32con.PROCESS_VM_READ,
            False, process_id);
        if process is not None:
            return win32process.GetProcessMemoryInfo(process)
    finally:
        if process:
            win32api.CloseHandle(process)
    return memory

meg = 1024.0 * 1024.0

figure = Figure(dpi=None)
canvas = FigureCanvas(figure)
axes = figure.add_subplot(1,1,1)

def draw(channel, seconds):
    axes.clear()
    axes.plot(channel, seconds)
    canvas.print_figure('test.png')

channel = numpy.sin(numpy.arange(1000) * random())
seconds = numpy.arange(len(channel))
testDuration = 60 * 60 * 3
startTime = time.time()

print "starting memory: ", \
    get_process_memory_info(os.getpid())["WorkingSetSize"]/meg

while (time.time() - startTime) < testDuration:
    draw(channel, seconds)

    t = datetime.now()
    memory = get_process_memory_info(os.getpid())
    print "time: {0}, working: {1:f}, collect: {2}, garbage: {3}".format(
        t,
        memory["WorkingSetSize"]/meg,
        gc.collect(),
        len(gc.garbage) )

    time.sleep(0.5)


TkAgg
---------
from random import random
from datetime import datetime
import sys
import os
import gc
import time
import win32api
import win32con
import win32process

import numpy

import matplotlib
matplotlib.use("TkAgg")
from matplotlib.figure import Figure
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg \
    as FigureCanvas

import Tkinter as tk

def get_process_memory_info(process_id):
    memory = {}
    process = None
    try:
        process = win32api.OpenProcess(
            win32con.PROCESS_QUERY_INFORMATION|win32con.PROCESS_VM_READ,
            False, process_id);
        if process is not None:
            return win32process.GetProcessMemoryInfo(process)
    finally:
        if process:
            win32api.CloseHandle(process)
    return memory

meg = 1024.0 * 1024.0

rootTk = tk.Tk()
rootTk.wm_title("TKAgg Memory Leak")

figure = Figure()
canvas = FigureCanvas(figure, master=rootTk)
axes = figure.add_subplot(1,1,1)

def draw(channel, seconds):
    axes.clear()
    axes.plot(channel, seconds)

channel = numpy.sin(numpy.arange(1000) * random())
seconds = numpy.arange(len(channel))

testDuration = 60 * 60 * 3
startTime = time.time()

print "starting memory: ", \
    get_process_memory_info(os.getpid())["WorkingSetSize"]/meg

draw(channel, seconds)
canvas.show()
canvas.get_tk_widget().pack(side=tk.TOP, fill=tk.BOTH, expand=1)

rate = 500

def on_tick():
    canvas.get_tk_widget().after(rate, on_tick)

    if (time.time() - startTime) >= testDuration:
        return

    draw(channel, seconds)

    t = datetime.now()
    memory = get_process_memory_info(os.getpid())
    print "time: {0}, working: {1:f}, collect: {2}, garbage: {3}".format(
        t,
        memory["WorkingSetSize"]/meg,
        gc.collect(),
        len(gc.garbage) )

canvas.get_tk_widget().after(rate, on_tick)
tk.mainloop()

> From: D [mailto:da...@gm...] 
> Sent: Friday, November 12, 2010 16:52
> 
> I am running Python 2.5.2 on win xp sp3 on an Intel Core2 Duo. After
> installation of matplotlib-1.0.0.win32-py2.5.exe I get a Windows crash
> on importing pylab.
> import pylab -> An unhandled win32 exception occured in 
> python.exe [5048]
> import matplotlib works
> 
> I did not add anything to the matplotlibrc. I uninstalled matplotlib
> and re-installed with the same problem.
> matplotlib-0.99.3.win32-py2.5.exe gives me the same problem.
> The older version I had installed before (forgot which one) 
> worked fine.

Can you 'import numpy' and 'import matplotlib.pyplot'?  What is your
numpy.__version__?

So I tried updating and rebuilding Matplotlib - no luck - something (at
least on my system) is goofy with the GTK(Agg),WX(Agg), and
Fltk(Agg) backends.  But the TkAgg backend works great.  So I will
stick with that, but if anyone else sees this problem and it becomes an
issue, I'd be happy to help debug it in any way I can.

-C

 On Wed, 17 Nov 2010 19:46:34 -0600
Benjamin Root <ben...@ou...> wrote:

> On Wed, Nov 17, 2010 at 7:38 PM, Collin Day <dcd...@gm...>
> wrote:
> 
> > On Wed, 17 Nov 2010 19:00:54 -0600
> > Benjamin Root <ben...@ou...> wrote:
> >
> > Another data point -
> >
> > I tried Qt4Agg - it also works interactively - ie it goes back to
> > the ipython cmd line.  I also noticed when I start ipython --pylab,
> > the following error messages occur:
> >
> > ** Message: pygobject_register_sinkfunc is deprecated (GtkWindow)
> > ** Message: pygobject_register_sinkfunc is deprecated (GtkInvisible)
> > ** Message: pygobject_register_sinkfunc is deprecated (GtkObject)
> >
> >
> Interesting.  It is possible that the gtk backend might have been
> compiled against gtk development libraries that do not match your
> current gtk library.  What have you updated recently, and how did you
> do it?
> 
> Ben Root

Hello,

I see that for a legend you can do the following:

ax = plt.scatter(x,y,label='test data')
p_leg = mpl.font_manager.FontProperties(size='8')
ax.legend(prop=p_leg)

But, how do you do set font properties for the colorbar tick labels?

Thanks!