matplotlib-devel Mailing List for matplotlib

On Sun, Oct 10, 2010 at 12:49 PM, Friedrich Romstedt
<fri...@gm...> wrote:
>
> Second, I would be very interested in more information about the
> ipython -pylab threading.  What do you mean with "running all user
> code in a single separate thread"?  When it's not the thread importing
> Tkinter, the problem persists.
>

Two things to note:

- ipython, even in the 0.10 series, uses threads for the Qt, Gtk and
Wx backends, but *not* for Tk, because python automatically pumps the
Tk event loop when the console blocks on reading (via the C API
PyOSInputHook call).  So there is exactly *zero* threading added by
ipython in the specific case of a Tk backend.

- in the 0.11 ipython series, we abandoned threading altogether (it's
just too brittle) and moved to a model similar to the Tk one for *all*
backends, we now use PyOSInputHook with all mpl backends.

hth,

f

On 10/10/2010 09:49 AM, Friedrich Romstedt wrote:
> 2010/10/2 Eric Firing<ef...@ha...>:
>> On 10/01/2010 09:40 AM, Friedrich Romstedt wrote:
>>> There were several question on the user's list in the recent past
>>> reporting hangs and similar when using TkAgg backend&    interactive
>>> mode.
>>>
>>> It is known that Tkinter doesn't play well with threading, as long as
>>> one isn't done very carefully.
>>>
>>> It could be that matplotlib has implemented it in a way not safe for
>>> Tkinter.  This applies if there are Tkinter methods called from
>>> threads other than those which imported Tkinter.
>>>
>>> It results in unpredicatable behaviour with exactly the features
>>> reported: Hang-ups, blankening, partial hangup of part of the threads.
>>>
>>> Is a rewrite of the Tk interactive code necessary?
>>
>> I don't see any use of the threading module in backend_tkagg.py or
>> backend_bases.py.  Is that what you are referring to?
>>
>> Ipython -pylab does use threads (version 0.10), but is intended to avoid
>> problems by running all user code in a single separate thread.
>
> I just dived a bit into the matplotlib source code, and come up with
> partial answers:
>
> When interactive mode is on, the mainloop isn't running at all (see
> backend_bases.py:ShowBase).  This explains hang-ups when using
> interactive mode in a non-shell Python call.

No, I don't think it does.  Before going farther, I suggest that you 
provide specific test cases that fail with mpl from svn.  As far as I 
know, everything works--in or out of interactive mode, in or out of a 
script, in or out of ipython.  There may be problems with some other 
environments (idle?).

>
> Second, I would be very interested in more information about the
> ipython -pylab threading.  What do you mean with "running all user
> code in a single separate thread"?  When it's not the thread importing
> Tkinter, the problem persists.

It is the thread importing Tkinter (via the user code), and there is 
generally no problem.

>
> Tkinter is always cumbersome and needs lots of care ... (at least as
> soon as the program wants to do something useful :-)
>
> Also, I cannot see (with reasonable effort) how Threads play a role in
> matplotlib.  I see there is the Scheduler class in cbook.py, and in
> backends/backend_tkagg.py there is "a dict from func->  cbook.Scheduler
> threads", but I have no idea how this works.  A small hint in the
> correct direction would be appreciated!  I think it's just a lot
> faster although I'm borrowing your time for writing the answer ... I
> cannot find any reference to ``.sourced`` in any matplotlib code.
> Maybe it's simply dead?

That's the point--threading does not play any role in mpl by default. 
It looks like John thought about using cbook.Scheduler in backend_tkagg, 
but ended up not doing so.  Hence Scheduler (and its two subclasses, 
Timeout and Idle) is just a miscellaneous tool in cbook, available to 
users, but not used in any backend.  Yes, that "sourced" attribute is 
dead; I have now deleted the associated dead code from svn trunk.

Eric

>
> Friedrich

2010/10/2 Eric Firing <ef...@ha...>:
> On 10/01/2010 09:40 AM, Friedrich Romstedt wrote:
>> There were several question on the user's list in the recent past
>> reporting hangs and similar when using TkAgg backend&  interactive
>> mode.
>>
>> It is known that Tkinter doesn't play well with threading, as long as
>> one isn't done very carefully.
>>
>> It could be that matplotlib has implemented it in a way not safe for
>> Tkinter.  This applies if there are Tkinter methods called from
>> threads other than those which imported Tkinter.
>>
>> It results in unpredicatable behaviour with exactly the features
>> reported: Hang-ups, blankening, partial hangup of part of the threads.
>>
>> Is a rewrite of the Tk interactive code necessary?
>
> I don't see any use of the threading module in backend_tkagg.py or
> backend_bases.py.  Is that what you are referring to?
>
> Ipython -pylab does use threads (version 0.10), but is intended to avoid
> problems by running all user code in a single separate thread.

I just dived a bit into the matplotlib source code, and come up with
partial answers:

When interactive mode is on, the mainloop isn't running at all (see
backend_bases.py:ShowBase).  This explains hang-ups when using
interactive mode in a non-shell Python call.

Second, I would be very interested in more information about the
ipython -pylab threading.  What do you mean with "running all user
code in a single separate thread"?  When it's not the thread importing
Tkinter, the problem persists.

Tkinter is always cumbersome and needs lots of care ... (at least as
soon as the program wants to do something useful :-)

Also, I cannot see (with reasonable effort) how Threads play a role in
matplotlib.  I see there is the Scheduler class in cbook.py, and in
backends/backend_tkagg.py there is "a dict from func-> cbook.Scheduler
threads", but I have no idea how this works.  A small hint in the
correct direction would be appreciated!  I think it's just a lot
faster although I'm borrowing your time for writing the answer ... I
cannot find any reference to ``.sourced`` in any matplotlib code.
Maybe it's simply dead?

Friedrich

--- On Sun, 10/10/10, Eric Firing <ef...@ha...> wrote:
> Do you know anything about how Enthought packages their
> python and mpl for the Mac?  Would it be as a framework?

Sorry, no idea. I haven't used the Enthought packages.

> If you think there is something that really should be fixed
> for the Mac  to prevent future problems, and if it can be
> fixed without creating more problems, please do so.  Your
> suggestion above could be implemented in the svn trunk, at
> your discretion.  I'm not going to do it because I 
> don't understand it and can't test it.

I think it's possible, but I would like to hear from people who ran into this problem if Python not being a framework install is really the cause. It may also be that enthought packages an older version of matplotlib, which doesn't check for framework installs. If that is the case, then from a matplotlib perspective the problem has been fixed already.

--Michiel.


      

On 10/09/2010 06:43 PM, Michiel de Hoon wrote:
> The problem described in that bug report sounds similar to what happens if Python was not installed as a framework. The Mac OS X backend checks for that automatically, but the other backends do not as far as I know. If this is indeed the problem, we should consider checking this when importing matplotlib. I believe
>
>>>> import sysconfig
>>>> sysconfig.get_config_var("WITH_NEXT_FRAMEWORK")
>
> should do the job.
>
> --Michiel.

Michiel,

Thanks to you, Tony, and Matthew for your quick responses.  I am going 
to close the ticket.

Do you know anything about how Enthought packages their python and mpl 
for the Mac?  Would it be as a framework?

If you think there is something that really should be fixed for the Mac 
to prevent future problems, and if it can be fixed without creating more 
problems, please do so.  Your suggestion above could be implemented in 
the svn trunk, at your discretion.  I'm not going to do it because I 
don't understand it and can't test it.

Eric

>
>
>
> --- On Sat, 10/9/10, Eric Firing<ef...@ha...>  wrote:
>
>> From: Eric Firing<ef...@ha...>
>> Subject: [matplotlib-devel] SourceForge.net: matplotlib: Modify: 2973874 - Text box, Save dialog for mac
>> To: "matplotlib development list"<mat...@li...>
>> Date: Saturday, October 9, 2010, 10:30 PM
>> https://sourceforge.net/tracker/?func=detail&aid=2973874&group_id=80706&atid=560720
>>
>> Would someone with a Mac please look at this bug and say
>> whether it is
>> occurring with our release of mpl?  If not, I will
>> close the ticket.
>>
>> Thank you.
>>
>> Eric
>>
>> ------------------------------------------------------------------------------
>> Beautiful is writing same markup. Internet Explorer 9
>> supports
>> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and
>> DOM L2&  L3.
>> Spend less time writing and  rewriting code and more
>> time creating great
>> experiences on the web. Be a part of the beta today.
>> http://p.sf.net/sfu/beautyoftheweb
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>
>
>

The problem described in that bug report sounds similar to what happens if Python was not installed as a framework. The Mac OS X backend checks for that automatically, but the other backends do not as far as I know. If this is indeed the problem, we should consider checking this when importing matplotlib. I believe 

>>> import sysconfig
>>> sysconfig.get_config_var("WITH_NEXT_FRAMEWORK")

should do the job.

--Michiel.



--- On Sat, 10/9/10, Eric Firing <ef...@ha...> wrote:

> From: Eric Firing <ef...@ha...>
> Subject: [matplotlib-devel] SourceForge.net: matplotlib: Modify: 2973874 - Text box, Save dialog for mac
> To: "matplotlib development list" <mat...@li...>
> Date: Saturday, October 9, 2010, 10:30 PM
> https://sourceforge.net/tracker/?func=detail&aid=2973874&group_id=80706&atid=560720
> 
> Would someone with a Mac please look at this bug and say
> whether it is 
> occurring with our release of mpl?  If not, I will
> close the ticket.
> 
> Thank you.
> 
> Eric
> 
> ------------------------------------------------------------------------------
> Beautiful is writing same markup. Internet Explorer 9
> supports
> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and
> DOM L2 & L3.
> Spend less time writing and  rewriting code and more
> time creating great
> experiences on the web. Be a part of the beta today.
> http://p.sf.net/sfu/beautyoftheweb
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 


      

On Oct 9, 2010, at 10:46 PM, Matthew Brett wrote:

> Hi,
> 
> On Sat, Oct 9, 2010 at 7:30 PM, Eric Firing <ef...@ha...> wrote:
>> https://sourceforge.net/tracker/?func=detail&aid=2973874&group_id=80706&atid=560720
>> 
>> Would someone with a Mac please look at this bug and say whether it is
>> occurring with our release of mpl?  If not, I will close the ticket.
> 
> Not for me (python.org 2.6 32 bit, mpl 1.0.0) with the macosx backend
> or the tkagg backend.  I don't have a 64 bit python / matplotlib to
> test with - maybe that's the trick?  Or maybe it was for a different
> backend?
> 
> Best,
> 
> Matthew

Works fine on my system: python 2.6 64 bit (system install), mpl svn r8726 with macosx, tkagg, and qt4agg.

Best,
-Tony

Hi,

On Sat, Oct 9, 2010 at 7:30 PM, Eric Firing <ef...@ha...> wrote:
> https://sourceforge.net/tracker/?func=detail&aid=2973874&group_id=80706&atid=560720
>
> Would someone with a Mac please look at this bug and say whether it is
> occurring with our release of mpl?  If not, I will close the ticket.

Not for me (python.org 2.6 32 bit, mpl 1.0.0) with the macosx backend
or the tkagg backend.  I don't have a 64 bit python / matplotlib to
test with - maybe that's the trick?  Or maybe it was for a different
backend?

Best,

Matthew

https://sourceforge.net/tracker/?func=detail&aid=2973874&group_id=80706&atid=560720

Would someone with a Mac please look at this bug and say whether it is 
occurring with our release of mpl?  If not, I will close the ticket.

Thank you.

Eric