matplotlib-devel Mailing List for matplotlib

Depending on the exact use case you can get pretty good mileage out of
blitting (See http://matplotlib.org/api/animation_api.html#funcanimation for
an explanation or how it is used in the widgets module).

The best way to make things faster is to just do less work :)

Tom

On Thu, Sep 21, 2017 at 5:14 PM Chris Barker <chr...@no...> wrote:

> On Wed, Sep 13, 2017 at 12:31 AM, Francesco Faccenda <
> f.f...@gm...> wrote:
>
>> I have to admit I already stumbled on VisPy while doing my research on
>> the web. Still, I've got a lot of code already working with *matplotlib*.
>> Indeed, not only I plot data with it, but i manage a lot of *mpl events*
>> to provide the users usefool tools, like lines picking, tooltips, lines
>> copy/paste, square selectors for multiple selections, context menu and so
>> on. Moreover, I got matplotlib *embedded *on *wxpython *as well. While
>> at the beginning few lines were managed and noone complained, now that big
>> amout of data has to be displayed, the non-GPU core of the library is
>> starting to show its limits.
>>
>> Since matplotlib is a reference library for this kind of  applications, I
>> thought it deserved an update in this direction.
>>
>
> Well, As I understand it, VisPY made some effort to be compatible with the
> MPL API -- but that is going to depend on how much you use the lower-level
> parts f the API -- things like the transform, etc. to take advantage of GPU
> rendering, all the transforms, etc needs to be pushed to the GPU, so the
> architecture (and API) needs to be quite different.
>
>
>> If anyone is willing to do so, I'm available to discuss possible
>> solutions and also provide any help I can give.
>>
>
> As Ben pointed out, and I was trying to make clear -- it really isn't a
> matter of "just" writing an OpenGL backend -- there really needs to be a
> major restructuring.
>
> And VisPy is pretty much that project. Whether it is feature complete,
> robust and maintained enough for your use-cases, I have no idea, but even
> if not -- you'll probably be better off contributing to that effort than
> starting all over with trying to make an GPU_based OPenGL back-end.
>
> However -- maybe there is another option:
>
> Taking full advantage of GPUs does require a full restructuring, but maybe
> there are other ways to get better performance -- for instance, optimizing
> the transform code, etc:
>
> Using the GPU with PyCuda or [what the heck is the name of the more
> general GPU-using lib??]
>
> using numba
>
> Maybe there is still room for Cython, etc....
>
> In short, profiling MPL carefully, to see where the performance
> bottlenecks are:
>
> With modern hardware, actually rendering stuff is no longer the slow part
> of visualization. Rather, it's pushing data to the renderer, transforming
> data etc.
>
> This is why to take advantage of the GPU, you need to do the
> transformations, etc on the GPU -- which the MPL architecture doesn't make
> easy by dropping in a new back-end.
>
> Which is why VisPy built a new architecture from the bottom up.
>
> -CHB
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chr...@no...
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

On Wed, Sep 13, 2017 at 12:31 AM, Francesco Faccenda <f.f...@gm...
> wrote:

> I have to admit I already stumbled on VisPy while doing my research on the
> web. Still, I've got a lot of code already working with *matplotlib*.
> Indeed, not only I plot data with it, but i manage a lot of *mpl events*
> to provide the users usefool tools, like lines picking, tooltips, lines
> copy/paste, square selectors for multiple selections, context menu and so
> on. Moreover, I got matplotlib *embedded *on *wxpython *as well. While at
> the beginning few lines were managed and noone complained, now that big
> amout of data has to be displayed, the non-GPU core of the library is
> starting to show its limits.
>
> Since matplotlib is a reference library for this kind of  applications, I
> thought it deserved an update in this direction.
>

Well, As I understand it, VisPY made some effort to be compatible with the
MPL API -- but that is going to depend on how much you use the lower-level
parts f the API -- things like the transform, etc. to take advantage of GPU
rendering, all the transforms, etc needs to be pushed to the GPU, so the
architecture (and API) needs to be quite different.


> If anyone is willing to do so, I'm available to discuss possible solutions
> and also provide any help I can give.
>

As Ben pointed out, and I was trying to make clear -- it really isn't a
matter of "just" writing an OpenGL backend -- there really needs to be a
major restructuring.

And VisPy is pretty much that project. Whether it is feature complete,
robust and maintained enough for your use-cases, I have no idea, but even
if not -- you'll probably be better off contributing to that effort than
starting all over with trying to make an GPU_based OPenGL back-end.

However -- maybe there is another option:

Taking full advantage of GPUs does require a full restructuring, but maybe
there are other ways to get better performance -- for instance, optimizing
the transform code, etc:

Using the GPU with PyCuda or [what the heck is the name of the more general
GPU-using lib??]

using numba

Maybe there is still room for Cython, etc....

In short, profiling MPL carefully, to see where the performance bottlenecks
are:

With modern hardware, actually rendering stuff is no longer the slow part
of visualization. Rather, it's pushing data to the renderer, transforming
data etc.

This is why to take advantage of the GPU, you need to do the
transformations, etc on the GPU -- which the MPL architecture doesn't make
easy by dropping in a new back-end.

Which is why VisPy built a new architecture from the bottom up.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Folks,

Matthias as just helped us to install the meeseeks bot than ipython /
jupyter is using to help with their backporting onto matplotlib.

For details, see https://github.com/MeeseeksBox/MeeseeksDev .

The most important command is

@MeeseeksDev backport [to] {branch}

to automatically open a PR with the backport against the correct branch.

Currently the bot will only respond to:

        'tacaswell',
        'QuLogic',
        'anntzer',
        'NelleV',
        'dstansby',
        'efiring',
        'choldgraf'

We will open this up more as we sort out if we like this and how to use it.


Tom

Thank you, Ben.

unfortunately I don't have experience developing opengl neither, but I'd
like to make an attempt riesuming this subject.

I am starting from this: https://github.com/ChrisBeaumont/mplgl

Can anyone confirm this is the last update we have for this project?

Francesco Faccenda

2017-09-13 16:21 GMT+02:00 Benjamin Root <ben...@gm...>:

> A bit of background regarding efforts in this area...
>
> A long time ago, some experiments were done to see if an opengl backend
> could be made for matplotlib. The results weren't that great, particularly
> for text rendering, so the effort was dropped. The core developers all
> agree that an opengl backend would be neat to have, but we all have
> limited, if any, experience developing opengl. Furthermore, such a backend
> would likely require a lot of rapid development and trial-n-error. So, we
> encouraged others to go develop a package or two separately, with the eye
> for bringing it back into matplotlib when it was ready. Glumpy and a few
> other projects were born from that prodding.
>
> VisPy was an attempt to consolidate the development efforts for those
> projects. The matplotlib devs had very fruitful discussions with some VisPy
> devs back at SciPy 2015, but the project became non-responsive shortly
> afterwards.
>
> Ben Root
>
>
> On Wed, Sep 13, 2017 at 3:31 AM, Francesco Faccenda <
> f.f...@gm...> wrote:
>
>> Thank you Chris for your reply.
>>
>> I have to admit I already stumbled on VisPy while doing my research on
>> the web. Still, I've got a lot of code already working with *matplotlib*.
>> Indeed, not only I plot data with it, but i manage a lot of *mpl events*
>> to provide the users usefool tools, like lines picking, tooltips, lines
>> copy/paste, square selectors for multiple selections, context menu and so
>> on. Moreover, I got matplotlib *embedded *on *wxpython *as well. While
>> at the beginning few lines were managed and noone complained, now that big
>> amout of data has to be displayed, the non-GPU core of the library is
>> starting to show its limits.
>>
>> Since matplotlib is a reference library for this kind of  applications, I
>> thought it deserved an update in this direction. If anyone is willing to do
>> so, I'm available to discuss possible solutions and also provide any help I
>> can give.
>>
>> Best regards,
>> Francesco Faccenda
>>
>> 2017-09-13 0:46 GMT+02:00 Chris Barker <chr...@no...>:
>>
>>> On Tue, Sep 12, 2017 at 8:47 AM, Francesco Faccenda <
>>> f.f...@gm...> wrote:
>>>
>>>> But there’s a good news, I have a nice GPU available (an NVIDIA Tesla
>>>> K40c), so I’d like to know if there is a way to make matplotlib run on it,
>>>> or maybe wrap it on some GPU/CUDA wrapper and make it run smoothly.
>>>>
>>>
>>> I tihnk you want VisPy:
>>>
>>> https://vispy.readthedocs.io/en/latest/
>>>
>>> It's a plotting package with a kinda like  matplotlib API, built on
>>> OpenGL.
>>>
>>> Unfortunately, it doesn't look like it's been updated in a while -- from
>>> teh docs. But the gitHub project is active:
>>>
>>> https://github.com/vispy/vispy
>>>
>>> So maybe it's only the docs that haven't been updated!
>>>
>>> But probably  a much better option than trying to shoehorn GPU rendering
>>> into MPL.
>>>
>>> The problem is that while MPL was designed to be "backend" independent
>>> -- so it is "easy" to plug in an alternative renderer, the rendering model
>>> is not really well suited to GPU rendering -- it would take a lot of
>>> refactoring to really be able to take advantage of the graphics card.
>>>
>>> -CHB
>>>
>>>
>>> --
>>>
>>> Christopher Barker, Ph.D.
>>> Oceanographer
>>>
>>> Emergency Response Division
>>> NOAA/NOS/OR&R            (206) 526-6959   voice
>>> 7600 Sand Point Way NE
>>> <https://maps.google.com/?q=7600+Sand+Point+Way+NE&entry=gmail&source=g>
>>>   (206) 526-6329   fax
>>> Seattle, WA  98115       (206) 526-6317   main reception
>>>
>>> Chr...@no...
>>>
>>
>>
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@py...
>> https://mail.python.org/mailman/listinfo/matplotlib-devel
>>
>>
>

Thank you Chris for your reply.

I have to admit I already stumbled on VisPy while doing my research on the
web. Still, I've got a lot of code already working with *matplotlib*.
Indeed, not only I plot data with it, but i manage a lot of *mpl events* to
provide the users usefool tools, like lines picking, tooltips, lines
copy/paste, square selectors for multiple selections, context menu and so
on. Moreover, I got matplotlib *embedded *on *wxpython *as well. While at
the beginning few lines were managed and noone complained, now that big
amout of data has to be displayed, the non-GPU core of the library is
starting to show its limits.

Since matplotlib is a reference library for this kind of  applications, I
thought it deserved an update in this direction. If anyone is willing to do
so, I'm available to discuss possible solutions and also provide any help I
can give.

Best regards,
Francesco Faccenda

2017-09-13 0:46 GMT+02:00 Chris Barker <chr...@no...>:

> On Tue, Sep 12, 2017 at 8:47 AM, Francesco Faccenda <
> f.f...@gm...> wrote:
>
>> But there’s a good news, I have a nice GPU available (an NVIDIA Tesla
>> K40c), so I’d like to know if there is a way to make matplotlib run on it,
>> or maybe wrap it on some GPU/CUDA wrapper and make it run smoothly.
>>
>
> I tihnk you want VisPy:
>
> https://vispy.readthedocs.io/en/latest/
>
> It's a plotting package with a kinda like  matplotlib API, built on OpenGL.
>
> Unfortunately, it doesn't look like it's been updated in a while -- from
> teh docs. But the gitHub project is active:
>
> https://github.com/vispy/vispy
>
> So maybe it's only the docs that haven't been updated!
>
> But probably  a much better option than trying to shoehorn GPU rendering
> into MPL.
>
> The problem is that while MPL was designed to be "backend" independent --
> so it is "easy" to plug in an alternative renderer, the rendering model is
> not really well suited to GPU rendering -- it would take a lot of
> refactoring to really be able to take advantage of the graphics card.
>
> -CHB
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE
> <https://maps.google.com/?q=7600+Sand+Point+Way+NE&entry=gmail&source=g>
>   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chr...@no...
>

On Tue, Sep 12, 2017 at 8:47 AM, Francesco Faccenda <f.f...@gm...>
wrote:

> But there’s a good news, I have a nice GPU available (an NVIDIA Tesla
> K40c), so I’d like to know if there is a way to make matplotlib run on it,
> or maybe wrap it on some GPU/CUDA wrapper and make it run smoothly.
>

I tihnk you want VisPy:

https://vispy.readthedocs.io/en/latest/

It's a plotting package with a kinda like  matplotlib API, built on OpenGL.

Unfortunately, it doesn't look like it's been updated in a while -- from
teh docs. But the gitHub project is active:

https://github.com/vispy/vispy

So maybe it's only the docs that haven't been updated!

But probably  a much better option than trying to shoehorn GPU rendering
into MPL.

The problem is that while MPL was designed to be "backend" independent --
so it is "easy" to plug in an alternative renderer, the rendering model is
not really well suited to GPU rendering -- it would take a lot of
refactoring to really be able to take advantage of the graphics card.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Greetings to everyone,



I'm Francesco Faccenda and I'm developing a sfotware in *python* where I
need to show some charts, so I chose matplotlib as a plot library. Anyway,
since lots of data are shown on matplotlip stages (thousands of curves with
hundreds of points each), the charts become very heavy, interactive events
and functions start lagging and users experience becomes unacceptable.



But there’s a good news, I have a nice GPU available (an NVIDIA Tesla
K40c), so I’d like to know if there is a way to make matplotlib run on it,
or maybe wrap it on some GPU/CUDA wrapper and make it run smoothly.



Do you have any suggestion how to do it? I never went through matplotlib
source code before and I don’t know if this feature has already been
scheduled for future releases or it’s something completely new.

I would really appreciate some suggestions on this subject.



Thank you in advance.



Best regards,

Francesco