matplotlib-devel Mailing List for matplotlib

On 2012/12/29 6:25 AM, Shriramana Sharma wrote:
> On Sat, Dec 29, 2012 at 9:24 PM, Thomas Kluyver <th...@kl...> wrote:
>>
>> If it helps, I have a PPA with builds of the development version of
>> matplotlib, including Python 3 builds on precise. I use those packages
>> myself, and it's been pretty stable.
>>
>> https://code.launchpad.net/~takluyver/+archive/matplotlib-daily
>
> Thank you very much -- I should have searched the PPAs myself. Now
> have installed.
>
> Just one more question: why is inkscape a build-dep?

Only because it is used in the tests, to test the svg backend.

Eric

>

On Sat, Dec 29, 2012 at 9:24 PM, Thomas Kluyver <th...@kl...> wrote:
>
> If it helps, I have a PPA with builds of the development version of
> matplotlib, including Python 3 builds on precise. I use those packages
> myself, and it's been pretty stable.
>
> https://code.launchpad.net/~takluyver/+archive/matplotlib-daily

Thank you very much -- I should have searched the PPAs myself. Now
have installed.

Just one more question: why is inkscape a build-dep?

-- 
Shriramana Sharma

On 29 December 2012 13:53, Shriramana Sharma <sa...@gm...> wrote:

> Hello. I don't use Matplotlib directly (yet) but I wish to use
> https://github.com/scios/IPyXt which needs it, and I'm using IPython
> and SymPy both with Python 3. (Current version on my Kubuntu Precise
> system is 3.2.3.) So I'm trying to backport v1.2.0 from Kubuntu 13.0x
> (http://packages.ubuntu.com/raring/python3-matplotlib). I have all the
> build-dependencies (I backported python3-dateutil and python3-tz too
> from latest Debian) but when I run dpkg-buildpackage, it terminates
> with the errors:
>

If it helps, I have a PPA with builds of the development version of
matplotlib, including Python 3 builds on precise. I use those packages
myself, and it's been pretty stable.

https://code.launchpad.net/~takluyver/+archive/matplotlib-daily

Best wishes,
Thomas

Hello. I don't use Matplotlib directly (yet) but I wish to use
https://github.com/scios/IPyXt which needs it, and I'm using IPython
and SymPy both with Python 3. (Current version on my Kubuntu Precise
system is 3.2.3.) So I'm trying to backport v1.2.0 from Kubuntu 13.0x
(http://packages.ubuntu.com/raring/python3-matplotlib). I have all the
build-dependencies (I backported python3-dateutil and python3-tz too
from latest Debian) but when I run dpkg-buildpackage, it terminates
with the errors:

python-dbg: ../Objects/object.c:65: _Py_AddToAllObjects: Assertion
`(op->_ob_prev == ((void *)0)) == (op->_ob_next == ((void *)0))'
failed.
Aborted (core dumped)
make: *** [clean] Error 134
dpkg-buildpackage: error: fakeroot debian/rules clean gave error exit status 2

The complete runlog is attached. Can anyone please help me out? Thanks.

-- 
Shriramana Sharma

On Sun, Dec 23, 2012 at 2:24 PM, Nathaniel Smith <nj...@po...> wrote:
> On Sun, Dec 23, 2012 at 12:42 AM, Damon McDougall
> <dam...@gm...> wrote:
>> On Mon, Dec 17, 2012 at 8:10 AM, Michael Droettboom <md...@st...> wrote:
>>> On 12/16/2012 03:44 PM, Eric Firing wrote:
>>>> On 2012/12/16 9:21 AM, Damon McDougall wrote:
>>>>> On Sat, Dec 15, 2012 at 8:25 PM, Jason Grout
>>>>> <jas...@cr...> wrote:
>>>>>> On 12/14/12 10:55 AM, Nathaniel Smith wrote:
>>>>>>> sourceforge's horror of an interface.
>>>>>> I'll second that.  Every time I go to Sourceforge, I have to figure out
>>>>>> how in the world to download what I want (and I have to figure out which
>>>>>> things *not* to click on too).
>>>>> Ok sounds like there is a reasonable amount of resistance towards Sourceforge.
>>>>>
>>>>> Eric, when you suggest that NumFocus could 'provide hosting directly',
>>>>> do you mean they would have the physical hardware to host the files,
>>>>> or are you suggesting they provide the finances to seek hosting
>>>>> elsewhere?
>>>> I was thinking that perhaps NumFocus would be running a server that
>>>> could provide the hosting.  Funding for an external service is also
>>>> possible, though, and might make more sense.
>>>
>>> I'll definitely walk down the hall and talk to my local Numfocus board
>>> member ;)
>>
>> At the 6th Annual Scientific Software Day here at UT Austin, I met and
>> spoke to Travis Oliphant regarding funding for hosting our binaries.
>> Travis has links with NumFOCUS and was eager to help the matplotlib
>> community host binaries should we choose to not go with sourceforge or
>> another free option.
>>
>> I'll need touch base with him again to get specifics, but I thought
>> I'd just let everyone here know that that's still an option.
>>
>> To be honest with you, I'm thinking that if we only want to link to
>> binaries from the matplotlib web page then sourceforge really doesn't
>> sound like a bad option at all.
>
> Or -- I'll just point this out one more time then leave the dead horse
> alone :-) -- you could just register a project called
> 'matplotlib-downloads' on google code hosting, and have static URLs
> that look like e.g.
>   https://apa6e.googlecode.com/files/apa6e-v0.3.zip
> and let Google foot the bill for reliable high-bandwidth CDN hosting.
>
> -n

Thanks for reminding us about the google code option. Since there are
quite a lot of suggestions I've used this opportunity to write a short
summary of the options presented in this thread:

Google Code: Free. Interface is better than sourceforge.

Sourceforge: Free. Ugly. May not need interface if hotlinking to
binaries from the website.

PyPI: Free. Size quota too small. Will link to external files.

S3: Free and not free. We're over the free tier quota. NumFOCUS can
potentially fund the non-free tier (cost is circa $200/mo). I thought
it was $200/yr. IMO $200/mo is very expensive.

gh-pages: We're already over the size limit for this branch.

new gh repo: Free. 1GB of space.
https://help.github.com/articles/what-is-my-disk-quota. Will hold
about 4 releases?

Dropbox: Free 2GB account. Can hotlink to binaries.

-- 
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

On Wed, Dec 12, 2012 at 11:57 AM, Marcel Oliver <
m.o...@ja...> wrote:

> Paul Hobson writes:
>  > On Wed, Dec 12, 2012 at 5:59 AM, Michael Droettboom <md...@st...>
> wrote:
>  >
>  >     http://matplotlib.org/examples/api/clippath_demo.html
>  >
>  >     It's perfectly reasonable code, but seems strange that it's
>  >     clipped off to the corner which I think makes it less useful as
>  >     an example.
>  >
>  > If I understand correctly, you're proposing that that such an
>  > example would be deleted after a more practical example was
>  > available to replace it?
>  >
>  > While I'm 100% in favor of "cleaning out the closet", I used this exact
>  > example two days ago to clip a depressed groundwater surface to a
> landfill
>  > boundary :)
>
> There is one small issue with the example: If one is using imshow on
> random pixel data, there is not reason to use bilinear interpolation
> (which is what imshow defaults to).  "nearest" is the only reasonable
> choice.
>
> I know this sounds pedantic, but it irks me as a numerical analyst...
>
> --Marcel
>

Marcel: I agree examples like this should be changed. Actually, I think the
default interpolation should be changed to "nearest", but that's a whole
different can of worms ;)

For those who are interested, I just posted a pull request for this gallery
clean up <https://github.com/matplotlib/matplotlib/pull/1623>. It's
definitely a work in progress. In particular, the section names are less
than ideal, but I think github's PR interface is a better place for that
discussion than the wiki or mailing list. The new section names can be
found below:

https://github.com/matplotlib/matplotlib/pull/1623/files#L0R86

Also, I moved a number of examples to the new sections in order to
demonstrate some of the clean up guidelines. BTW, one of the examples I
cleaned up is the clippath demo discussed earlier in this thread.

Best,
-Tony

Marcel: Sorry for the second email, I forgot to "reply all" the first time.

When I sent the email, github itself was up (for me), but the MPL site was
down -- which is why I thought it odd.

I'm glad to hear that this appears to be just me. Thanks for letting me
know!


Patrick
---
Patrick Marsh
Ph.D. Candidate / Liaison to the HWT
School of Meteorology / University of Oklahoma
Cooperative Institute for Mesoscale Meteorological Studies
National Severe Storms Laboratory
http://www.patricktmarsh.com


On Sun, Dec 23, 2012 at 10:39 PM, Paul Hobson <pmh...@gm...> wrote:

> On Sat, Dec 22, 2012 at 3:25 PM, Patrick Marsh <pat...@gm...>wrote:
>
>> The matplotlib.org website is down right now -- at least for me.
>>
>> Also, over the last 2-3 months or so, I'd guess that about 50% of the
>> time I go to the website it's down. Granted, I don't go daily, so I'm
>> guessing that part of this is just awesome timing on my part, but my
>> general feeling is that this is becoming a more frequent occurrence. Is it
>> just me?
>>
>>
> I can't say for sure. But I think there was a github outage that caused
> this. As for the site being down, I'm probably looking at examples or the
> gallery 3x a day and haven't noticed it being down once. Who knows?
>  -p
>
>

On Sat, Dec 22, 2012 at 3:25 PM, Patrick Marsh <pat...@gm...>wrote:

> The matplotlib.org website is down right now -- at least for me.
>
> Also, over the last 2-3 months or so, I'd guess that about 50% of the time
> I go to the website it's down. Granted, I don't go daily, so I'm guessing
> that part of this is just awesome timing on my part, but my general feeling
> is that this is becoming a more frequent occurrence. Is it just me?
>
>
I can't say for sure. But I think there was a github outage that caused
this. As for the site being down, I'm probably looking at examples or the
gallery 3x a day and haven't noticed it being down once. Who knows?
-p

On Sun, Dec 23, 2012 at 12:42 AM, Damon McDougall
<dam...@gm...> wrote:
> On Mon, Dec 17, 2012 at 8:10 AM, Michael Droettboom <md...@st...> wrote:
>> On 12/16/2012 03:44 PM, Eric Firing wrote:
>>> On 2012/12/16 9:21 AM, Damon McDougall wrote:
>>>> On Sat, Dec 15, 2012 at 8:25 PM, Jason Grout
>>>> <jas...@cr...> wrote:
>>>>> On 12/14/12 10:55 AM, Nathaniel Smith wrote:
>>>>>> sourceforge's horror of an interface.
>>>>> I'll second that.  Every time I go to Sourceforge, I have to figure out
>>>>> how in the world to download what I want (and I have to figure out which
>>>>> things *not* to click on too).
>>>> Ok sounds like there is a reasonable amount of resistance towards Sourceforge.
>>>>
>>>> Eric, when you suggest that NumFocus could 'provide hosting directly',
>>>> do you mean they would have the physical hardware to host the files,
>>>> or are you suggesting they provide the finances to seek hosting
>>>> elsewhere?
>>> I was thinking that perhaps NumFocus would be running a server that
>>> could provide the hosting.  Funding for an external service is also
>>> possible, though, and might make more sense.
>>
>> I'll definitely walk down the hall and talk to my local Numfocus board
>> member ;)
>
> At the 6th Annual Scientific Software Day here at UT Austin, I met and
> spoke to Travis Oliphant regarding funding for hosting our binaries.
> Travis has links with NumFOCUS and was eager to help the matplotlib
> community host binaries should we choose to not go with sourceforge or
> another free option.
>
> I'll need touch base with him again to get specifics, but I thought
> I'd just let everyone here know that that's still an option.
>
> To be honest with you, I'm thinking that if we only want to link to
> binaries from the matplotlib web page then sourceforge really doesn't
> sound like a bad option at all.

Or -- I'll just point this out one more time then leave the dead horse
alone :-) -- you could just register a project called
'matplotlib-downloads' on google code hosting, and have static URLs
that look like e.g.
  https://apa6e.googlecode.com/files/apa6e-v0.3.zip
and let Google foot the bill for reliable high-bandwidth CDN hosting.

-n

On Mon, Dec 17, 2012 at 8:10 AM, Michael Droettboom <md...@st...> wrote:
> On 12/16/2012 03:44 PM, Eric Firing wrote:
>> On 2012/12/16 9:21 AM, Damon McDougall wrote:
>>> On Sat, Dec 15, 2012 at 8:25 PM, Jason Grout
>>> <jas...@cr...> wrote:
>>>> On 12/14/12 10:55 AM, Nathaniel Smith wrote:
>>>>> sourceforge's horror of an interface.
>>>> I'll second that.  Every time I go to Sourceforge, I have to figure out
>>>> how in the world to download what I want (and I have to figure out which
>>>> things *not* to click on too).
>>> Ok sounds like there is a reasonable amount of resistance towards Sourceforge.
>>>
>>> Eric, when you suggest that NumFocus could 'provide hosting directly',
>>> do you mean they would have the physical hardware to host the files,
>>> or are you suggesting they provide the finances to seek hosting
>>> elsewhere?
>> I was thinking that perhaps NumFocus would be running a server that
>> could provide the hosting.  Funding for an external service is also
>> possible, though, and might make more sense.
>
> I'll definitely walk down the hall and talk to my local Numfocus board
> member ;)

At the 6th Annual Scientific Software Day here at UT Austin, I met and
spoke to Travis Oliphant regarding funding for hosting our binaries.
Travis has links with NumFOCUS and was eager to help the matplotlib
community host binaries should we choose to not go with sourceforge or
another free option.

I'll need touch base with him again to get specifics, but I thought
I'd just let everyone here know that that's still an option.

To be honest with you, I'm thinking that if we only want to link to
binaries from the matplotlib web page then sourceforge really doesn't
sound like a bad option at all.

-- 
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

The matplotlib.org website is down right now -- at least for me.

Also, over the last 2-3 months or so, I'd guess that about 50% of the time
I go to the website it's down. Granted, I don't go daily, so I'm guessing
that part of this is just awesome timing on my part, but my general feeling
is that this is becoming a more frequent occurrence. Is it just me?


PTM
---
Patrick Marsh
Ph.D. Candidate / Liaison to the HWT
School of Meteorology / University of Oklahoma
Cooperative Institute for Mesoscale Meteorological Studies
National Severe Storms Laboratory
http://www.patricktmarsh.com

Has  anyone talk to the pandas people about underwriting the hosting?


--
Thomas Caswell
tca...@gm...

https://github.com/tacaswell/matplotlib/compare/master...issue1504

I jumped the gun and there is already a pull request as well (
https://github.com/matplotlib/matplotlib/pull/1505 )

Tom

--
Thomas Caswell
tca...@gm...

The discussion of this thread was continued off the mailing list as the
example script/data were too large to post to the list.  Here is a
simplified summary so that others who find this thread will benefit too.

The original poster had a script like the following:

for i in range(48):
    fig = figure(...)
    ax = fig.add_subplot()
    ax.tripcolor(lons, lats, triangles, values, ...)
    # other plotting of coastlines, text, changing fig/ax settings, etc.
    fig.savefig(...)
    close(fig)

Three areas of performance improvement were identified:

1) As lons, lats and triangles do not change between tripcolor calls, use a
single Triangulation object initialised before the start of the loop.  This
avoids unnecessarily creating, using once and deleting 48 separate
Triangulation objects.

triangulation = matplotlib.tri.Triangulation(lons, lats, triangles)
for i in range(48):
    ....
    ax.tripcolor(triangulation, values, ...)
    ....

2) Use tricontourf instead of tripcolor.  It is often significantly faster,
and you can control the tradeoff of speed verses output quality by changing
the number of contour levels used.

3) Reuse the Figure and Axes within the loop.  Create them before the loop
and do all plotting that is common to all iterations of the loop.  Then in
the loop do the single tricontourf call.  You must make sure you delete the
result of the previous iteration's tricontourf call or they will all stack
up with each other as you repeat the loop.

This is the new pseudocode incorporating all 3 changes:

triangulation = matplotlib.tri.Triangulation(lons, lats, triangles)  # One
and only Triangulation.
fig = figure(...)  # One and only Figure.
ax = fig.add_subplot()  # One and only Axes.
# plotting of coastlines, text, changing fig/ax settings, etc.
ncollections = len(ax.collections)  # Number of collections added to Axes
so far.
for i in range(48):
    del ax.collections[ncollections:]  # Remove collections added in the
last loop iteration.
    ax.tricontourf(triangulation, values, ...)
    fig.savefig(...)
close(fig)

These changes reduced the execution time of the OP's code by about a factor
of 10 when using 16 levels for the tricontourf call.

Ian Thomas

On 17 December 2012 14:04, campbell.kb <cam...@gm...> wrote:

> I am an environmental computer modeler. I've created scripts that use
> tripcolor() to do contour plots of various output variables (with values at
> the centers of triangles and at the corners of the triangles). The grids I
> am working on are rather large (I am involved in modeling the United
> States'
> Great Lakes). The issue is that tripcolor() is exceedingly slow to use
> because I am producing contour plots for 3 variables, for all 48 time steps
> in a 48 hour forecast. Is there any chance that the tripcolor() function
> can
> be sped up, or is there anything I should do (in generating the arguments
> passed to it) that would speed up its execution time?
>

There is probably something you can do to speed up your execution time, but
for us to help we will need to see exactly what you are doing.  Can you
post an example of one of your slow scripts with the appropriate data?  As
you are dealing with large datasets you may want to simplify the example
script/data you send.  If the data is still too large and/or you don't want
to post it to a public mailing list, you can email it to me directly.

Ian Thomas

I thought I would just throw this out there, and I'm not entirely sure it
would work, but here goes...

What about using something like Dropbox?  Create a matplotlib account, put
the binaries in a Public folder and then use that as the storage...I don't
know how this would work in practice, but I do know that people have been
using Dropbox to host things for websites in other venues (persontal blogs,
etc)

It's probably best to talk to Dropbox first, but since they are built on
Python, they might be willing to work with open source Python projects.


PTM

---
Patrick Marsh
Ph.D. Candidate / Liaison to the HWT
School of Meteorology / University of Oklahoma
Cooperative Institute for Mesoscale Meteorological Studies
National Severe Storms Laboratory
http://www.patricktmarsh.com



On Mon, Dec 17, 2012 at 7:33 AM, Maximilian Albert <
max...@gm...> wrote:

> 2012/12/16 Thomas Kluyver <th...@kl...>:
> > On 15 December 2012 23:38, Damon McDougall <dam...@gm...>
> > wrote:
> >>
> >> Maybe the best thing is to host the binaries on Sourceforge.
> >
> >
> > Having recently tried to do it, Sourceforge tries really hard to avoid
> > giving you a direct link that can repeatably be used to download a file
> > automatically, i.e. without a browser. In the case I was after it for, I
> > ended up downloading the file (a PyWin32 binary) with a browser, and
> storing
> > it on the CI server that I wanted to install it.
>
> I haven't followed this thread in detail, so not sure if it's really
> relevant: I agree that it's quite messy, but it's definitely possible
> to find stable download links for automated downloads from
> sourceforge. I sometimes need this because I use a ports-like system
> for installing certain packages from source on top of my regular
> Ubuntu system, and it does work quite well. So if this is what's
> keeping people from Sourceforge then there are definitely workarounds
> (give me a shout if you need more specific information; I'm likely to
> have very sporadic internet access over the next few weeks though, so
> replies may take a while). However, I also think Sourceforge's
> interface is really ugly so if there are better alternatives then
> that's great. :)
>
> Cheers,
> Max
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

I am an environmental computer modeler. I've created scripts that use
tripcolor() to do contour plots of various output variables (with values at
the centers of triangles and at the corners of the triangles). The grids I
am working on are rather large (I am involved in modeling the United States'
Great Lakes). The issue is that tripcolor() is exceedingly slow to use
because I am producing contour plots for 3 variables, for all 48 time steps
in a 48 hour forecast. Is there any chance that the tripcolor() function can
be sped up, or is there anything I should do (in generating the arguments
passed to it) that would speed up its execution time?




--
View this message in context: http://matplotlib.1069221.n5.nabble.com/tripcolor-sluggish-execution-tp40054.html
Sent from the matplotlib - devel mailing list archive at Nabble.com.

2012/12/16 Thomas Kluyver <th...@kl...>:
> On 15 December 2012 23:38, Damon McDougall <dam...@gm...>
> wrote:
>>
>> Maybe the best thing is to host the binaries on Sourceforge.
>
>
> Having recently tried to do it, Sourceforge tries really hard to avoid
> giving you a direct link that can repeatably be used to download a file
> automatically, i.e. without a browser. In the case I was after it for, I
> ended up downloading the file (a PyWin32 binary) with a browser, and storing
> it on the CI server that I wanted to install it.

I haven't followed this thread in detail, so not sure if it's really
relevant: I agree that it's quite messy, but it's definitely possible
to find stable download links for automated downloads from
sourceforge. I sometimes need this because I use a ports-like system
for installing certain packages from source on top of my regular
Ubuntu system, and it does work quite well. So if this is what's
keeping people from Sourceforge then there are definitely workarounds
(give me a shout if you need more specific information; I'm likely to
have very sporadic internet access over the next few weeks though, so
replies may take a while). However, I also think Sourceforge's
interface is really ugly so if there are better alternatives then
that's great. :)

Cheers,
Max

On 12/16/2012 03:44 PM, Eric Firing wrote:
> On 2012/12/16 9:21 AM, Damon McDougall wrote:
>> On Sat, Dec 15, 2012 at 8:25 PM, Jason Grout
>> <jas...@cr...> wrote:
>>> On 12/14/12 10:55 AM, Nathaniel Smith wrote:
>>>> sourceforge's horror of an interface.
>>> I'll second that.  Every time I go to Sourceforge, I have to figure out
>>> how in the world to download what I want (and I have to figure out which
>>> things *not* to click on too).
>> Ok sounds like there is a reasonable amount of resistance towards Sourceforge.
>>
>> Eric, when you suggest that NumFocus could 'provide hosting directly',
>> do you mean they would have the physical hardware to host the files,
>> or are you suggesting they provide the finances to seek hosting
>> elsewhere?
> I was thinking that perhaps NumFocus would be running a server that
> could provide the hosting.  Funding for an external service is also
> possible, though, and might make more sense.

I'll definitely walk down the hall and talk to my local Numfocus board 
member ;)

As for S3, their free tier is clearly insufficient -- I thought I'd get 
a hold on the cost of paying for it.

Our storage requirements are ~150MB per release (for all of the 
different binaries) (though likely to grow over time), so saying it's 
useful to have the last ~6 or so releases live at any given time, that's 
~900MB -- let's call it 1GB.

Our transfer requirements after a release seem to peak at around 
1.7TB/mo, about 67,000 requests, after a release (though presumably 
amortized over time, it's lower).  This is also likely to grow over 
time, of course.

So, I think we've got:

1,700 GB transfer @ $0.12/GB = $204.00
70,000 requests $0.01/10,000 = $0.07
1GB storage @ $0.095 = $0.10

That's a bit steep to cover with donations alone.

In any event, this is a useful assessment of what this is really worth 
(and what our eyeballs are worth to advertisers on Sourceforge ;)

Perhaps, as Skipper suggests, with a little investment in automation 
tools for SourceForge that may remain the best option.  The Sourceforge 
download experience isn't terrible if we provide direct links to the 
downloads on our own website.  wxPython has done this for years, and 
AFAIK they have not run in to any problems.  The links even work without 
a web browser (though, for example, wget and curl).

Cheers,
Mike

On 12/17/2012 07:39 AM, Michael Droettboom wrote:
> On 12/17/2012 07:36 AM, Michael Droettboom wrote:
>> On 12/16/2012 02:50 PM, Damon McDougall wrote:
>>> On Sun, Dec 16, 2012 at 1:38 PM, Todd<tod...@gm...>  wrote:
>>>> On Sun, Dec 16, 2012 at 8:21 PM, Damon McDougall<dam...@gm...>
>>>> wrote:
>>>>> On Sat, Dec 15, 2012 at 8:25 PM, Jason Grout
>>>>> <jas...@cr...>  wrote:
>>>>>> On 12/14/12 10:55 AM, Nathaniel Smith wrote:
>>>>>>> sourceforge's horror of an interface.
>>>>>> I'll second that.  Every time I go to Sourceforge, I have to figure out
>>>>>> how in the world to download what I want (and I have to figure out which
>>>>>> things *not* to click on too).
>>>>> Ok sounds like there is a reasonable amount of resistance towards
>>>>> Sourceforge.
>>>>>
>>>>> Eric, when you suggest that NumFocus could 'provide hosting directly',
>>>>> do you mean they would have the physical hardware to host the files,
>>>>> or are you suggesting they provide the finances to seek hosting
>>>>> elsewhere?
>>>>>
>>>>> In the GitHub blog post, they suggest using S3. We could try that.
>>>>> It's fairly inexpensive and the first year is free (within monthly
>>>>> bandwidth limits). We could try it for a year and see how that pans
>>>>> out? I'm not entirely sure how the Amazon stuff works but I've heard
>>>>> good things about it.
>>>>>
>>>>>
>>>> Are you sure the monthly bandwidth limits are sufficient?
>>>>
>>>> Also, have you talked to the pypi people about making exceptions for really
>>>> popular projects?  If critical packages like numpy, scipy, and matplotlib
>>>> cannot use pypi, that seems like a major failing of the system.
>>> Here's the pricing:http://aws.amazon.com/s3/#pricing. The free tier
>>> programme limits are on there too. Unfortunately, I do not have the
>>> knowledge to be able to say whether we would hit that or not.
>>
>> Since Nov 3, when 1.2.0 was released, we've used 1.7 GB of transfer 
>> from the github download site.  The S3 "free tier" limit of 1.5 
>> GB/month is awfully close to that.
> Oops -- I totally misread the S3 requirements: it's 15GB/month, so 
> we're fine there, but as Eric pointed out, there's also a 20,000 
> request limit per month, which we're well over (we've have 67,500 
> requests since Nov 3's 1.2.0 release).
And once again, writing e-mails before coffee is a bad idea ;)

We've used about 1.7TB in the approx six weeks since the 1.2.0 release.

Mike

On 12/16/2012 02:38 PM, Todd wrote:
> On Sun, Dec 16, 2012 at 8:21 PM, Damon McDougall 
> <dam...@gm... <mailto:dam...@gm...>> wrote:
>
>     On Sat, Dec 15, 2012 at 8:25 PM, Jason Grout
>     <jas...@cr... <mailto:jas...@cr...>>
>     wrote:
>     > On 12/14/12 10:55 AM, Nathaniel Smith wrote:
>     >> sourceforge's horror of an interface.
>     >
>     > I'll second that.  Every time I go to Sourceforge, I have to
>     figure out
>     > how in the world to download what I want (and I have to figure
>     out which
>     > things *not* to click on too).
>
>     Ok sounds like there is a reasonable amount of resistance towards
>     Sourceforge.
>
>     Eric, when you suggest that NumFocus could 'provide hosting directly',
>     do you mean they would have the physical hardware to host the files,
>     or are you suggesting they provide the finances to seek hosting
>     elsewhere?
>
>     In the GitHub blog post, they suggest using S3. We could try that.
>     It's fairly inexpensive and the first year is free (within monthly
>     bandwidth limits). We could try it for a year and see how that pans
>     out? I'm not entirely sure how the Amazon stuff works but I've heard
>     good things about it.
>
>
> Are you sure the monthly bandwidth limits are sufficient?
>
> Also, have you talked to the pypi people about making exceptions for 
> really popular projects?  If critical packages like numpy, scipy, and 
> matplotlib cannot use pypi, that seems like a major failing of the system.
>
>
I don't know if this is still the case, but having talked to one of the 
people involved with PyPI a couple of years ago, my understanding is 
that their infrastructure simply doesn't support it.  They provide the 
ability to link to external files (which is what matplotlib does now), 
and that is the standard solution to that problem.

Cheers,
Mike

On 12/15/2012 09:25 PM, Jason Grout wrote:
> On 12/14/12 10:55 AM, Nathaniel Smith wrote:
>> sourceforge's horror of an interface.
> I'll second that.  Every time I go to Sourceforge, I have to figure out
> how in the world to download what I want (and I have to figure out which
> things *not* to click on too).
>
That's to say nothing of the upload interface, which is 10 times worse! :)

Mike

On 12/17/2012 07:36 AM, Michael Droettboom wrote:
> On 12/16/2012 02:50 PM, Damon McDougall wrote:
>> On Sun, Dec 16, 2012 at 1:38 PM, Todd<tod...@gm...>  wrote:
>>> On Sun, Dec 16, 2012 at 8:21 PM, Damon McDougall<dam...@gm...>
>>> wrote:
>>>> On Sat, Dec 15, 2012 at 8:25 PM, Jason Grout
>>>> <jas...@cr...>  wrote:
>>>>> On 12/14/12 10:55 AM, Nathaniel Smith wrote:
>>>>>> sourceforge's horror of an interface.
>>>>> I'll second that.  Every time I go to Sourceforge, I have to figure out
>>>>> how in the world to download what I want (and I have to figure out which
>>>>> things *not* to click on too).
>>>> Ok sounds like there is a reasonable amount of resistance towards
>>>> Sourceforge.
>>>>
>>>> Eric, when you suggest that NumFocus could 'provide hosting directly',
>>>> do you mean they would have the physical hardware to host the files,
>>>> or are you suggesting they provide the finances to seek hosting
>>>> elsewhere?
>>>>
>>>> In the GitHub blog post, they suggest using S3. We could try that.
>>>> It's fairly inexpensive and the first year is free (within monthly
>>>> bandwidth limits). We could try it for a year and see how that pans
>>>> out? I'm not entirely sure how the Amazon stuff works but I've heard
>>>> good things about it.
>>>>
>>>>
>>> Are you sure the monthly bandwidth limits are sufficient?
>>>
>>> Also, have you talked to the pypi people about making exceptions for really
>>> popular projects?  If critical packages like numpy, scipy, and matplotlib
>>> cannot use pypi, that seems like a major failing of the system.
>> Here's the pricing:http://aws.amazon.com/s3/#pricing. The free tier
>> programme limits are on there too. Unfortunately, I do not have the
>> knowledge to be able to say whether we would hit that or not.
>
> Since Nov 3, when 1.2.0 was released, we've used 1.7 GB of transfer 
> from the github download site.  The S3 "free tier" limit of 1.5 
> GB/month is awfully close to that.
Oops -- I totally misread the S3 requirements: it's 15GB/month, so we're 
fine there, but as Eric pointed out, there's also a 20,000 request limit 
per month, which we're well over (we've have 67,500 requests since Nov 
3's 1.2.0 release).

Cheers,
Mike

On 12/16/2012 02:50 PM, Damon McDougall wrote:
> On Sun, Dec 16, 2012 at 1:38 PM, Todd <tod...@gm...> wrote:
>> On Sun, Dec 16, 2012 at 8:21 PM, Damon McDougall <dam...@gm...>
>> wrote:
>>> On Sat, Dec 15, 2012 at 8:25 PM, Jason Grout
>>> <jas...@cr...> wrote:
>>>> On 12/14/12 10:55 AM, Nathaniel Smith wrote:
>>>>> sourceforge's horror of an interface.
>>>> I'll second that.  Every time I go to Sourceforge, I have to figure out
>>>> how in the world to download what I want (and I have to figure out which
>>>> things *not* to click on too).
>>> Ok sounds like there is a reasonable amount of resistance towards
>>> Sourceforge.
>>>
>>> Eric, when you suggest that NumFocus could 'provide hosting directly',
>>> do you mean they would have the physical hardware to host the files,
>>> or are you suggesting they provide the finances to seek hosting
>>> elsewhere?
>>>
>>> In the GitHub blog post, they suggest using S3. We could try that.
>>> It's fairly inexpensive and the first year is free (within monthly
>>> bandwidth limits). We could try it for a year and see how that pans
>>> out? I'm not entirely sure how the Amazon stuff works but I've heard
>>> good things about it.
>>>
>>>
>> Are you sure the monthly bandwidth limits are sufficient?
>>
>> Also, have you talked to the pypi people about making exceptions for really
>> popular projects?  If critical packages like numpy, scipy, and matplotlib
>> cannot use pypi, that seems like a major failing of the system.
> Here's the pricing: http://aws.amazon.com/s3/#pricing. The free tier
> programme limits are on there too. Unfortunately, I do not have the
> knowledge to be able to say whether we would hit that or not.

Since Nov 3, when 1.2.0 was released, we've used 1.7 GB of transfer from 
the github download site.  The S3 "free tier" limit of 1.5 GB/month is 
awfully close to that.

>
> Matt, have you had experienced comitting binaries to the gh-pages
> branch? Are there size limits?
>
There used to be limits on github-pages specifically, but they seem to 
have silently removed information about them.  As for any github 
repository, the limit is 1GB per repository.

https://help.github.com/articles/what-is-my-disk-quota

We already have 375MB in our documentation (pages) repository.  We use 
about 150MB for all of the binaries for each release.  So we'd be able 
to squeeze about 3-4 releases in there before needing to explicitly 
prune stuff.  Additionally, the link above seems to discourage hosting 
very large files in the pages repository -- I don't know if that means 
they intend to not support them.

Mike

On 12/15/2012 08:10 PM, Matt Newville wrote:
> Hi,
>
>> Github has removed the ability to host binaries.  They've removed this
>> feature without any apparent notification except on their blog saying
>> "it's gone today".  And the suggested alternative is to use paid services.
>>
>> https://github.com/blog/1302-goodbye-uploads
>>
>> I had planned to complete our set of 1.2.0 binaries with a Python 3.2
>> from Russell Owen in the near future.  So much for that.
>>
>> Any thoughts?  Do we go back to Sourceforge for our download hosting?
>> Is anyone familiar with any other services?  Do we try to piggy-back on
>> what other scipy projects are doing?
> Why not use a gh-pages branch to host both docs and binary files from github?
>
>
We are already over the size limits that the used to state for their 
documentation.  I have been personally assured that this won't remain a 
problem, but I do wonder how long until they start cracking down on that 
for free projects as well.  I'd rather not additionally put our binary 
downloads in that basket as well.

Mike