Thread: [matplotlib-devel] v1.0.x branch seems confused

I am testing the doc builds on v1.0.x when I noticed that it seems to think
that it is version 1.1.0.  Sure enough, somehow lib/matplotlib/__init__.py
has __version__ defined as '1.1.0'.  Meanwhile, I can't seem to finish a doc
build on the v1.0.x branch, but I could with my docfix/smalltypos branch.

Anybody have any clues what went wrong?

Thanks,
Ben Root

Yes, it seems that the v1.0.x got hosed somehow back in early May.  Eric 
Firing did some spelunking and traced it to a push I made, but I'm not 
sure what I did wrong, and I'm even less sure how to fix it.  If someone 
with more git-fu wants to investigate and repair it, that would be 
fantastic, but I'm afraid to touch it myself.

Mike

On 06/01/2011 02:10 PM, Benjamin Root wrote:
> I am testing the doc builds on v1.0.x when I noticed that it seems to 
> think that it is version 1.1.0.  Sure enough, somehow 
> lib/matplotlib/__init__.py has __version__ defined as '1.1.0'.  
> Meanwhile, I can't seem to finish a doc build on the v1.0.x branch, 
> but I could with my docfix/smalltypos branch.
>
> Anybody have any clues what went wrong?
>
> Thanks,
> Ben Root
>
>
> ------------------------------------------------------------------------------
> Simplify data backup and recovery for your virtual environment with vRanger.
> Installation's a snap, and flexible recovery options mean your data is safe,
> secure and there when you need it. Data protection magic?
> Nope - It's vRanger. Get your free trial download today.
> http://p.sf.net/sfu/quest-sfdev2dev
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Mike,

On Wed, Jun 1, 2011 at 2:26 PM, Michael Droettboom <md...@st...> wrote:
> Yes, it seems that the v1.0.x got hosed somehow back in early May.  Eric
> Firing did some spelunking and traced it to a push I made, but I'm not sure
> what I did wrong, and I'm even less sure how to fix it.  If someone with
> more git-fu wants to investigate and repair it, that would be fantastic, but
> I'm afraid to touch it myself.

I'm taking a guess here: gtk_crash was branched off of master, but
perhaps the pull request was registered against the v1.0.x branch. In
any case, gtk_crash was merged with v1.0.x. I've done something
similar myself, but I always merge into a clean local copy of v1.0.x
or master and inspect the history graph, so I noticed my mistake
before I actually pushed the changes.

I have a little time now, maybe something can be done to correct it.

On 06/01/2011 08:26 AM, Michael Droettboom wrote:
> Yes, it seems that the v1.0.x got hosed somehow back in early May. Eric
> Firing did some spelunking and traced it to a push I made, but I'm not
> sure what I did wrong, and I'm even less sure how to fix it. If someone
> with more git-fu wants to investigate and repair it, that would be
> fantastic, but I'm afraid to touch it myself.
>
> Mike

Mike, all:

Suggestion:  Let's just abandon the v1.0.x branch, stabilize master, and 
get a release out.  (Easy for me to say--I have never contributed to the 
actual release process.)  I think that we really need to get releases 
out more frequently--that needs to become a higher priority.  I don't 
think it is even worth the trouble to maintain a maintenance branch at 
all.  It adds quite a bit of complexity to the development and release 
process--every time a bug is found, the fix has to be developed on 
maintenance, committed and pushed, checked on master, and propagated to 
master.  It's not worth it. The differences between master and 
maintenance are normally not large enough to justify keeping them 
separate, given our very constrained people-time resources.

Note that a release doesn't have to be made from master HEAD; git 
branching is extremely flexible, so at any time, one can make a branch 
from any point on the tree, do some checking and adjustment, and release 
it.  Where we get into difficulty and waste time is in trying to 
maintain a separate maintenance branch for a long period.  We just don't 
have the resources to do this well; and we don't really need to do it at 
all.

Eric

>
> On 06/01/2011 02:10 PM, Benjamin Root wrote:
>> I am testing the doc builds on v1.0.x when I noticed that it seems to
>> think that it is version 1.1.0. Sure enough, somehow
>> lib/matplotlib/__init__.py has __version__ defined as '1.1.0'.
>> Meanwhile, I can't seem to finish a doc build on the v1.0.x branch,
>> but I could with my docfix/smalltypos branch.
>>
>> Anybody have any clues what went wrong?
>>
>> Thanks,
>> Ben Root

On Wed, Jun 1, 2011 at 1:47 PM, Eric Firing <ef...@ha...> wrote:

> On 06/01/2011 08:26 AM, Michael Droettboom wrote:
> > Yes, it seems that the v1.0.x got hosed somehow back in early May. Eric
> > Firing did some spelunking and traced it to a push I made, but I'm not
> > sure what I did wrong, and I'm even less sure how to fix it. If someone
> > with more git-fu wants to investigate and repair it, that would be
> > fantastic, but I'm afraid to touch it myself.
> >
> > Mike
>
> Mike, all:
>
> Suggestion:  Let's just abandon the v1.0.x branch, stabilize master, and
> get a release out.  (Easy for me to say--I have never contributed to the
> actual release process.)  I think that we really need to get releases
> out more frequently--that needs to become a higher priority.  I don't
> think it is even worth the trouble to maintain a maintenance branch at
> all.  It adds quite a bit of complexity to the development and release
> process--every time a bug is found, the fix has to be developed on
> maintenance, committed and pushed, checked on master, and propagated to
> master.  It's not worth it. The differences between master and
> maintenance are normally not large enough to justify keeping them
> separate, given our very constrained people-time resources.
>
> Note that a release doesn't have to be made from master HEAD; git
> branching is extremely flexible, so at any time, one can make a branch
> from any point on the tree, do some checking and adjustment, and release
> it.  Where we get into difficulty and waste time is in trying to
> maintain a separate maintenance branch for a long period.  We just don't
> have the resources to do this well; and we don't really need to do it at
> all.
>
> Eric
>
>
Actually, there are plenty of differences between v1.0.x and master.  We
have a number of new features that are baking right now (animations, for
example).  I have personally made a number of changes with mplot3d that, in
some cases, were too risky to apply to v1.0.x and I just wanted them to sit
in the official development branch rather than in the stable branch.  Also,
because there is not much of downstream activity to the repos, I think many
of the packagers depend upon the bugfixes we apply to the maintenance
branches.  I am hesitant to change development policies without a clear
consideration for downstream.

The current practice worked very nicely with SVN (IMHO), and I think it
still works fine.  Most of the issues is really limited to svn --> git
transitions and understanding how git works.  I will agree that it is
possible that the v1.0.x branch might be damaged beyond repair.  I am not
enough of a git expert to know one way or another.

As for the comment about getting releases out faster, this contradicts your
assertion that we don't have the manpower to take care of the maintenance
branch.  We do need better planning for milestones and goals, but I don't
want to push out a release until it is ready.  In particular, I think a good
rule of thumb for the v1.1.0 release should be to have *all* the backends
behaving the same (::cough:: macosx ::cough::), and to officially deprecate
any backends that we can not continue to support (::cough:: Cairo
::cough::).

But that's just my opinion.

Ben Root

On 06/01/2011 09:07 AM, Benjamin Root wrote:
>
>
> On Wed, Jun 1, 2011 at 1:47 PM, Eric Firing <ef...@ha...
> <mailto:ef...@ha...>> wrote:
>
>     On 06/01/2011 08:26 AM, Michael Droettboom wrote:
>      > Yes, it seems that the v1.0.x got hosed somehow back in early
>     May. Eric
>      > Firing did some spelunking and traced it to a push I made, but
>     I'm not
>      > sure what I did wrong, and I'm even less sure how to fix it. If
>     someone
>      > with more git-fu wants to investigate and repair it, that would be
>      > fantastic, but I'm afraid to touch it myself.
>      >
>      > Mike
>
>     Mike, all:
>
>     Suggestion:  Let's just abandon the v1.0.x branch, stabilize master, and
>     get a release out.  (Easy for me to say--I have never contributed to the
>     actual release process.)  I think that we really need to get releases
>     out more frequently--that needs to become a higher priority.  I don't
>     think it is even worth the trouble to maintain a maintenance branch at
>     all.  It adds quite a bit of complexity to the development and release
>     process--every time a bug is found, the fix has to be developed on
>     maintenance, committed and pushed, checked on master, and propagated to
>     master.  It's not worth it. The differences between master and
>     maintenance are normally not large enough to justify keeping them
>     separate, given our very constrained people-time resources.
>
>     Note that a release doesn't have to be made from master HEAD; git
>     branching is extremely flexible, so at any time, one can make a branch
>     from any point on the tree, do some checking and adjustment, and release
>     it.  Where we get into difficulty and waste time is in trying to
>     maintain a separate maintenance branch for a long period.  We just don't
>     have the resources to do this well; and we don't really need to do it at
>     all.
>
>     Eric
>
>
> Actually, there are plenty of differences between v1.0.x and master.  We
> have a number of new features that are baking right now (animations, for
> example).  I have personally made a number of changes with mplot3d that,

Are you referring to animations.py?  The last change in that file on 
master was 9 months ago.


> in some cases, were too risky to apply to v1.0.x and I just wanted them
> to sit in the official development branch rather than in the stable
> branch.  Also, because there is not much of downstream activity to the
> repos, I think many of the packagers depend upon the bugfixes we apply
> to the maintenance branches.  I am hesitant to change development
> policies without a clear consideration for downstream.
>

Do the packagers use the tip of the maintenance branch, or do they use 
the most recent release?  If the former, then that bumps up the priority 
of keeping such a branch.  If the latter, it bumps up the priority of 
having frequent high-quality releases, regardless of what they are called.


> The current practice worked very nicely with SVN (IMHO), and I think it

(I recall that Mike had to rescue us more than once from svnmerge 
confusions, at least during the earlier days.)

> still works fine.  Most of the issues is really limited to svn --> git
> transitions and understanding how git works.  I will agree that it is
> possible that the v1.0.x branch might be damaged beyond repair.  I am
> not enough of a git expert to know one way or another.
>
> As for the comment about getting releases out faster, this contradicts
> your assertion that we don't have the manpower to take care of the
> maintenance branch.  We do need better planning for milestones and

I don't see the contradiction or inconsistency.  I am not saying we 
*can't* keep a maintenance branch; I am saying it is not clear to me 
that it is worth the fuss to do so indefinitely, especially when it is 
infrequently released.

> goals, but I don't want to push out a release until it is ready.  In
> particular, I think a good rule of thumb for the v1.1.0 release should
> be to have *all* the backends behaving the same (::cough:: macosx
> ::cough::), and to officially deprecate any backends that we can not

So, all progress should be held hostage?  This might work if we were a 
company, and could assign a programmer to a task.  Given that we rely on 
volunteers, I don't think it is practical.

> continue to support (::cough:: Cairo ::cough::).

And emf, and plain wx, and plain gtk, and fltkagg. (Many months ago I 
posted a query to matplotlib-users as to whether anyone was actually 
*using* fltkagg; there was no reply.  Regarding Cairo: I don't know what 
its status is, but it always seemed like a potentially useful backup in 
case of an Agg meltdown.) But that would be a big change in mpl de facto 
policy, which has always been very liberal with respect to leaving 
decaying code in place (like mplot3d) in case someone eventually picks 
it up and pumps some life back into it.  I have mixed thoughts about 
that; my general instinct would be to rip out such things, but John's 
liberal approach has actually worked quite well.
>
> But that's just my opinion.
>
It's good to get some opinions aired, to see if we can figure out ways 
to improve mpl and its development process.

Eric

> Ben Root

On Wed, Jun 1, 2011 at 2:58 PM, Eric Firing <ef...@ha...> wrote:

> On 06/01/2011 09:07 AM, Benjamin Root wrote:
> >
> >
> > On Wed, Jun 1, 2011 at 1:47 PM, Eric Firing <ef...@ha...
> > <mailto:ef...@ha...>> wrote:
> >
> >     On 06/01/2011 08:26 AM, Michael Droettboom wrote:
> >      > Yes, it seems that the v1.0.x got hosed somehow back in early
> >     May. Eric
> >      > Firing did some spelunking and traced it to a push I made, but
> >     I'm not
> >      > sure what I did wrong, and I'm even less sure how to fix it. If
> >     someone
> >      > with more git-fu wants to investigate and repair it, that would be
> >      > fantastic, but I'm afraid to touch it myself.
> >      >
> >      > Mike
> >
> >     Mike, all:
> >
> >     Suggestion:  Let's just abandon the v1.0.x branch, stabilize master,
> and
> >     get a release out.  (Easy for me to say--I have never contributed to
> the
> >     actual release process.)  I think that we really need to get releases
> >     out more frequently--that needs to become a higher priority.  I don't
> >     think it is even worth the trouble to maintain a maintenance branch
> at
> >     all.  It adds quite a bit of complexity to the development and
> release
> >     process--every time a bug is found, the fix has to be developed on
> >     maintenance, committed and pushed, checked on master, and propagated
> to
> >     master.  It's not worth it. The differences between master and
> >     maintenance are normally not large enough to justify keeping them
> >     separate, given our very constrained people-time resources.
> >
> >     Note that a release doesn't have to be made from master HEAD; git
> >     branching is extremely flexible, so at any time, one can make a
> branch
> >     from any point on the tree, do some checking and adjustment, and
> release
> >     it.  Where we get into difficulty and waste time is in trying to
> >     maintain a separate maintenance branch for a long period.  We just
> don't
> >     have the resources to do this well; and we don't really need to do it
> at
> >     all.
> >
> >     Eric
> >
> >
> > Actually, there are plenty of differences between v1.0.x and master.  We
> > have a number of new features that are baking right now (animations, for
> > example).  I have personally made a number of changes with mplot3d that,
>
> Are you referring to animations.py?  The last change in that file on
> master was 9 months ago.
>
>
Yeah... I guess we really haven't exercised it.  However, I know some of my
existing animation code based on it is broken and I have spotted a few
potential issues in parts of the code.  However, Ryan May is currently on
the finishing end of his Dissertation work (while I am only at the start of
it), and I doubt we will get much more out of him for a few more months.


>
> > in some cases, were too risky to apply to v1.0.x and I just wanted them
> > to sit in the official development branch rather than in the stable
> > branch.  Also, because there is not much of downstream activity to the
> > repos, I think many of the packagers depend upon the bugfixes we apply
> > to the maintenance branches.  I am hesitant to change development
> > policies without a clear consideration for downstream.
> >
>
> Do the packagers use the tip of the maintenance branch, or do they use
> the most recent release?  If the former, then that bumps up the priority
> of keeping such a branch.  If the latter, it bumps up the priority of
> having frequent high-quality releases, regardless of what they are called.
>
>
>
I think that is dependent upon the repo.  It seems that Ubuntu tends to stay
near the release, while I have seen some other repos stay very up to date (I
forget which one I saw).  However, many of the changes we make are bugfixes
and then transition to feature changes.


> > The current practice worked very nicely with SVN (IMHO), and I think it
>
> (I recall that Mike had to rescue us more than once from svnmerge
> confusions, at least during the earlier days.)
>
>
So, it appears we have difficulties with new technologies?


>  > still works fine.  Most of the issues is really limited to svn --> git
> > transitions and understanding how git works.  I will agree that it is
> > possible that the v1.0.x branch might be damaged beyond repair.  I am
> > not enough of a git expert to know one way or another.
> >
> > As for the comment about getting releases out faster, this contradicts
> > your assertion that we don't have the manpower to take care of the
> > maintenance branch.  We do need better planning for milestones and
>
> I don't see the contradiction or inconsistency.  I am not saying we
> *can't* keep a maintenance branch; I am saying it is not clear to me
> that it is worth the fuss to do so indefinitely, especially when it is
> infrequently released.
>
>
Well, I never really saw it as indefinite.  I thought it was defined as
maintaining the current release.  So, whenever we release v1.1.0, then we no
longer work on v1.0.x and bugfixes go to v1.1.x branch.  Seems pretty clear
to me.  So long as there is no snafus with the maintenance branch, then we
should be fine with this procedure.


> > goals, but I don't want to push out a release until it is ready.  In
> > particular, I think a good rule of thumb for the v1.1.0 release should
> > be to have *all* the backends behaving the same (::cough:: macosx
> > ::cough::), and to officially deprecate any backends that we can not
>
> So, all progress should be held hostage?  This might work if we were a
> company, and could assign a programmer to a task.  Given that we rely on
> volunteers, I don't think it is practical.
>
>
The bug fixes are still getting pushed out, and the next releases of the
distros will have updated versions.  I don't want to put out half-working
features that ultimately gets rejected by users because it wasn't ready for
primetime.  However, I do see your point and I think this is even more
evidence for a release manager.  And to answer the next question -- no, I
simply will not have the time to do take that role (I am trying to get my
contributions done now before I go into hermit mode again).

But that would be a big change in mpl de facto
> policy, which has always been very liberal with respect to leaving
> decaying code in place (like mplot3d)


Hey!


> in case someone eventually picks
> it up and pumps some life back into it.


Ah, ok, nevermind...


>  I have mixed thoughts about
> that; my general instinct would be to rip out such things, but John's
> liberal approach has actually worked quite well.
>

I don't like cruft either, but what I really don't like is redundant cruft.
For example, if tight_layout turns out to be useful, then I think that the
mplsizer module should probably go.


> >
> > But that's just my opinion.
> >
> It's good to get some opinions aired, to see if we can figure out ways
> to improve mpl and its development process.
>
>
That's my hope too.

Ben

On Wed, Jun 1, 2011 at 2:58 PM, Eric Firing <ef...@ha...> wrote:

> Do the packagers use the tip of the maintenance branch, or do they use
> the most recent release?  If the former, then that bumps up the priority
> of keeping such a branch.  If the latter, it bumps up the priority of
> having frequent high-quality releases, regardless of what they are called.
>
>
debian has  been pretty clear that they want to build their packages from
tarballs which we have uploaded and announced.

JDH

Hi,

On Wed, Jun 1, 2011 at 12:58 PM, Eric Firing <ef...@ha...> wrote:
>> The current practice worked very nicely with SVN (IMHO), and I think it
>
> (I recall that Mike had to rescue us more than once from svnmerge
> confusions, at least during the earlier days.)

I was just idly looking at the matplotlib network graph:

https://github.com/matplotlib/matplotlib/network

There seem to be lots of branches and cross merges ; the history of
1.0.x is extremely confusing.

I wonder whether it would be worthwhile to review git workflow?

I like Pauli's edits to the numpy gitwash docs in numpy for this.
I've actually just merged these back into the gitwash main docs,
example build here:

http://matthew-brett.github.com/pydagogue/gitwash/git_development.html

Maybe the overall point is that git does require some thought to
history, and some rules-of-work, to avoid confusion.

I've been managing a maintenance branch for my much smaller nibabel
project without much trouble; I've just been doing the occasional
cherry-pick and rebase from trunk for bugfixes.

Cheers,

Matthew

On Wed, Jun 1, 2011 at 6:38 PM, Matthew Brett <mat...@gm...> wrote:
> Hi,
>
> On Wed, Jun 1, 2011 at 12:58 PM, Eric Firing <ef...@ha...> wrote:
>>> The current practice worked very nicely with SVN (IMHO), and I think it
>>
>> (I recall that Mike had to rescue us more than once from svnmerge
>> confusions, at least during the earlier days.)
>
> I was just idly looking at the matplotlib network graph:
>
> https://github.com/matplotlib/matplotlib/network
>
> There seem to be lots of branches and cross merges ; the history of
> 1.0.x is extremely confusing.
>
> I wonder whether it would be worthwhile to review git workflow?
>
> I like Pauli's edits to the numpy gitwash docs in numpy for this.
> I've actually just merged these back into the gitwash main docs,
> example build here:
>
> http://matthew-brett.github.com/pydagogue/gitwash/git_development.html
>
> Maybe the overall point is that git does require some thought to
> history, and some rules-of-work, to avoid confusion.
>
> I've been managing a maintenance branch for my much smaller nibabel
> project without much trouble; I've just been doing the occasional
> cherry-pick and rebase from trunk for bugfixes.

I have a simpler rule of thumb. When merging work to push to the
matplotlib repository: inspect the history graph before the merge,
perform the merge locally, and inspect the graph after the merge but
before the push. Inspecting the history graph doesn't take long. If
the graph doesn't look the way you anticipated (unexplained or
unexpected complexity), don't push to the matplotlib repo. If you are
unsure or want help, push to your personal fork and post to the
mailing list. If you don't know how the history graph should look
after the merge, you aren't ready to push that merge to the matplotlib
repo.

Darren

Hi,

On Wed, Jun 1, 2011 at 4:05 PM, Darren Dale <dsd...@gm...> wrote:
> On Wed, Jun 1, 2011 at 6:38 PM, Matthew Brett <mat...@gm...> wrote:
>> Hi,
>>
>> On Wed, Jun 1, 2011 at 12:58 PM, Eric Firing <ef...@ha...> wrote:
>>>> The current practice worked very nicely with SVN (IMHO), and I think it
>>>
>>> (I recall that Mike had to rescue us more than once from svnmerge
>>> confusions, at least during the earlier days.)
>>
>> I was just idly looking at the matplotlib network graph:
>>
>> https://github.com/matplotlib/matplotlib/network
>>
>> There seem to be lots of branches and cross merges ; the history of
>> 1.0.x is extremely confusing.
>>
>> I wonder whether it would be worthwhile to review git workflow?
>>
>> I like Pauli's edits to the numpy gitwash docs in numpy for this.
>> I've actually just merged these back into the gitwash main docs,
>> example build here:
>>
>> http://matthew-brett.github.com/pydagogue/gitwash/git_development.html
>>
>> Maybe the overall point is that git does require some thought to
>> history, and some rules-of-work, to avoid confusion.
>>
>> I've been managing a maintenance branch for my much smaller nibabel
>> project without much trouble; I've just been doing the occasional
>> cherry-pick and rebase from trunk for bugfixes.
>
> I have a simpler rule of thumb. When merging work to push to the
> matplotlib repository: inspect the history graph before the merge,
> perform the merge locally, and inspect the graph after the merge but
> before the push. Inspecting the history graph doesn't take long. If
> the graph doesn't look the way you anticipated (unexplained or
> unexpected complexity), don't push to the matplotlib repo. If you are
> unsure or want help, push to your personal fork and post to the
> mailing list. If you don't know how the history graph should look
> after the merge, you aren't ready to push that merge to the matplotlib
> repo.

Sounds right to me :)  Pauli actually has the same rule, I discovered:

http://matthew-brett.github.com/pydagogue/gitwash/maintainer_workflow.html#check-the-history

See you,

Matthew

On 06/01/2011 12:38 PM, Matthew Brett wrote:
> Hi,
>
> On Wed, Jun 1, 2011 at 12:58 PM, Eric Firing<ef...@ha...>  wrote:
>>> The current practice worked very nicely with SVN (IMHO), and I think it
>>
>> (I recall that Mike had to rescue us more than once from svnmerge
>> confusions, at least during the earlier days.)
>
> I was just idly looking at the matplotlib network graph:
>
> https://github.com/matplotlib/matplotlib/network
>
> There seem to be lots of branches and cross merges ; the history of
> 1.0.x is extremely confusing.

Agreed!

>
> I wonder whether it would be worthwhile to review git workflow?

Yes.

>
> I like Pauli's edits to the numpy gitwash docs in numpy for this.
> I've actually just merged these back into the gitwash main docs,
> example build here:

I will have to take a look; mpl did pull some version of gitwash into 
its doc build.

>
> http://matthew-brett.github.com/pydagogue/gitwash/git_development.html

Thanks, I will take a look.

>
> Maybe the overall point is that git does require some thought to
> history, and some rules-of-work, to avoid confusion.

I think one of the problems is that documentation such as the Git book 
and at least early versions of gitwash, if I remember correctly, 
emphasize workflow for people who do not access the central repo 
directly.  There has been much discussion on the lists of procedure for 
those who do push to central repos, but I am not sure to what extent it 
has gotten condensed down into a sufficiently simple set of rules and 
examples in the standard documentation.  Maybe you and Pauli have done 
that now.


>
> I've been managing a maintenance branch for my much smaller nibabel
> project without much trouble; I've just been doing the occasional
> cherry-pick and rebase from trunk for bugfixes.

In a way, this illustrates the difficulty: you describe a procedure for 
working with a maintenance branch that is completely different from the 
one we have been using (apart from the errors). What we have been doing 
is initiating bug fixes in the maintenance branch and then merging that 
branch to master.  I'm sure either way can work fine, if one doesn't 
make mistakes.  I'm not sure what the relative merits of the two methods 
are, in terms of simplicity, clarity, and robustness against errors. I 
think they result in very different graphs, correct?  With your 
approach, the maintenance branch and master are separate lines, while 
with our approach, the merges keep pulling the branches together in the 
graph, even though their contents are steadily getting farther apart.

Eric

>
> Cheers,
>
> Matthew

Hi,

On Wed, Jun 1, 2011 at 4:06 PM, Eric Firing <ef...@ha...> wrote:
> On 06/01/2011 12:38 PM, Matthew Brett wrote:
>> Hi,
>>
>> On Wed, Jun 1, 2011 at 12:58 PM, Eric Firing<ef...@ha...>  wrote:
>>>> The current practice worked very nicely with SVN (IMHO), and I think it
>>>
>>> (I recall that Mike had to rescue us more than once from svnmerge
>>> confusions, at least during the earlier days.)
>>
>> I was just idly looking at the matplotlib network graph:
>>
>> https://github.com/matplotlib/matplotlib/network
>>
>> There seem to be lots of branches and cross merges ; the history of
>> 1.0.x is extremely confusing.
>
> Agreed!
>
>>
>> I wonder whether it would be worthwhile to review git workflow?
>
> Yes.
>
>>
>> I like Pauli's edits to the numpy gitwash docs in numpy for this.
>> I've actually just merged these back into the gitwash main docs,
>> example build here:
>
> I will have to take a look; mpl did pull some version of gitwash into
> its doc build.
>
>>
>> http://matthew-brett.github.com/pydagogue/gitwash/git_development.html
>
> Thanks, I will take a look.

If you like what you see, then the 'gitwash_dumper.py' script will
pull a new copy into your repo...

If you don't, then I'd love suggestions for improvements.

>> Maybe the overall point is that git does require some thought to
>> history, and some rules-of-work, to avoid confusion.
>
> I think one of the problems is that documentation such as the Git book
> and at least early versions of gitwash, if I remember correctly,
> emphasize workflow for people who do not access the central repo
> directly.  There has been much discussion on the lists of procedure for
> those who do push to central repos, but I am not sure to what extent it
> has gotten condensed down into a sufficiently simple set of rules and
> examples in the standard documentation.  Maybe you and Pauli have done
> that now.

That's quite right, we did more or less assume that the maintainers
were git experts, and yes, Pauli did fix that to some extent.  The
result, as ported back by me, is this page, which is new, and needs
expanding:

http://matthew-brett.github.com/pydagogue/gitwash/maintainer_workflow.html

>>
>> I've been managing a maintenance branch for my much smaller nibabel
>> project without much trouble; I've just been doing the occasional
>> cherry-pick and rebase from trunk for bugfixes.
>
> In a way, this illustrates the difficulty: you describe a procedure for
> working with a maintenance branch that is completely different from the
> one we have been using (apart from the errors). What we have been doing
> is initiating bug fixes in the maintenance branch and then merging that
> branch to master.  I'm sure either way can work fine, if one doesn't
> make mistakes.  I'm not sure what the relative merits of the two methods
> are, in terms of simplicity, clarity, and robustness against errors. I
> think they result in very different graphs, correct?  With your
> approach, the maintenance branch and master are separate lines, while
> with our approach, the merges keep pulling the branches together in the
> graph, even though their contents are steadily getting farther apart.

I must confess that my git fu is not 10/10, but to me the idea of
_merging_ the maintenance branch into trunk is very confusing.    I
mean, the trees should increasingly diverge, surely, so there will be
more and more stuff you don't want to see back in trunk.  At the
moment, you have to trust git magic to correctly leave out the commits
you don't want...

See you,

Matthew

On Wed, Jun 1, 2011 at 6:20 PM, Matthew Brett <mat...@gm...>wrote:

> Hi,
>
> On Wed, Jun 1, 2011 at 4:06 PM, Eric Firing <ef...@ha...> wrote:
> > On 06/01/2011 12:38 PM, Matthew Brett wrote:
> >> Hi,
> >>
> >> On Wed, Jun 1, 2011 at 12:58 PM, Eric Firing<ef...@ha...>
>  wrote:
> >>>> The current practice worked very nicely with SVN (IMHO), and I think
> it
> >>>
> >>> (I recall that Mike had to rescue us more than once from svnmerge
> >>> confusions, at least during the earlier days.)
> >>
> >> I was just idly looking at the matplotlib network graph:
> >>
> >> https://github.com/matplotlib/matplotlib/network
> >>
> >> There seem to be lots of branches and cross merges ; the history of
> >> 1.0.x is extremely confusing.
> >
> > Agreed!
> >
> >>
> >> I wonder whether it would be worthwhile to review git workflow?
> >
> > Yes.
> >
> >>
> >> I like Pauli's edits to the numpy gitwash docs in numpy for this.
> >> I've actually just merged these back into the gitwash main docs,
> >> example build here:
> >
> > I will have to take a look; mpl did pull some version of gitwash into
> > its doc build.
> >
> >>
> >> http://matthew-brett.github.com/pydagogue/gitwash/git_development.html
> >
> > Thanks, I will take a look.
>
> If you like what you see, then the 'gitwash_dumper.py' script will
> pull a new copy into your repo...
>
> If you don't, then I'd love suggestions for improvements.
>
> >> Maybe the overall point is that git does require some thought to
> >> history, and some rules-of-work, to avoid confusion.
> >
> > I think one of the problems is that documentation such as the Git book
> > and at least early versions of gitwash, if I remember correctly,
> > emphasize workflow for people who do not access the central repo
> > directly.  There has been much discussion on the lists of procedure for
> > those who do push to central repos, but I am not sure to what extent it
> > has gotten condensed down into a sufficiently simple set of rules and
> > examples in the standard documentation.  Maybe you and Pauli have done
> > that now.
>
> That's quite right, we did more or less assume that the maintainers
> were git experts, and yes, Pauli did fix that to some extent.  The
> result, as ported back by me, is this page, which is new, and needs
> expanding:
>
> http://matthew-brett.github.com/pydagogue/gitwash/maintainer_workflow.html
>
> >>
> >> I've been managing a maintenance branch for my much smaller nibabel
> >> project without much trouble; I've just been doing the occasional
> >> cherry-pick and rebase from trunk for bugfixes.
> >
> > In a way, this illustrates the difficulty: you describe a procedure for
> > working with a maintenance branch that is completely different from the
> > one we have been using (apart from the errors). What we have been doing
> > is initiating bug fixes in the maintenance branch and then merging that
> > branch to master.  I'm sure either way can work fine, if one doesn't
> > make mistakes.  I'm not sure what the relative merits of the two methods
> > are, in terms of simplicity, clarity, and robustness against errors. I
> > think they result in very different graphs, correct?  With your
> > approach, the maintenance branch and master are separate lines, while
> > with our approach, the merges keep pulling the branches together in the
> > graph, even though their contents are steadily getting farther apart.
>
> I must confess that my git fu is not 10/10, but to me the idea of
> _merging_ the maintenance branch into trunk is very confusing.    I
> mean, the trees should increasingly diverge, surely, so there will be
> more and more stuff you don't want to see back in trunk.  At the
> moment, you have to trust git magic to correctly leave out the commits
> you don't want...
>
> See you,
>
> Matthew
>
>
While this is all very important and we should definitely come to an
agreement about this, this still doesn't solve my issue at hand.  I can not
build the docs in the v1.0.x branch, therefore we can not push out a
revision to the docs on sourceforge.  Meanwhile, our website still has bad
links and is pointing users to download version 1.0.0 instead of version
1.0.1 (which may explain why we still see some old bugs on the lists every
now and then).  What do we want to do about my pull request for the docs?

Ben Root

On Thu, Jun 2, 2011 at 4:46 PM, Benjamin Root <ben...@ou...> wrote:
>
>
> On Wed, Jun 1, 2011 at 6:20 PM, Matthew Brett <mat...@gm...>
> wrote:
>>
>> Hi,
>>
>> On Wed, Jun 1, 2011 at 4:06 PM, Eric Firing <ef...@ha...> wrote:
>> > On 06/01/2011 12:38 PM, Matthew Brett wrote:
>> >> Hi,
>> >>
>> >> On Wed, Jun 1, 2011 at 12:58 PM, Eric Firing<ef...@ha...>
>> >>  wrote:
>> >>>> The current practice worked very nicely with SVN (IMHO), and I think
>> >>>> it
>> >>>
>> >>> (I recall that Mike had to rescue us more than once from svnmerge
>> >>> confusions, at least during the earlier days.)
>> >>
>> >> I was just idly looking at the matplotlib network graph:
>> >>
>> >> https://github.com/matplotlib/matplotlib/network
>> >>
>> >> There seem to be lots of branches and cross merges ; the history of
>> >> 1.0.x is extremely confusing.
>> >
>> > Agreed!
>> >
>> >>
>> >> I wonder whether it would be worthwhile to review git workflow?
>> >
>> > Yes.
>> >
>> >>
>> >> I like Pauli's edits to the numpy gitwash docs in numpy for this.
>> >> I've actually just merged these back into the gitwash main docs,
>> >> example build here:
>> >
>> > I will have to take a look; mpl did pull some version of gitwash into
>> > its doc build.
>> >
>> >>
>> >> http://matthew-brett.github.com/pydagogue/gitwash/git_development.html
>> >
>> > Thanks, I will take a look.
>>
>> If you like what you see, then the 'gitwash_dumper.py' script will
>> pull a new copy into your repo...
>>
>> If you don't, then I'd love suggestions for improvements.
>>
>> >> Maybe the overall point is that git does require some thought to
>> >> history, and some rules-of-work, to avoid confusion.
>> >
>> > I think one of the problems is that documentation such as the Git book
>> > and at least early versions of gitwash, if I remember correctly,
>> > emphasize workflow for people who do not access the central repo
>> > directly.  There has been much discussion on the lists of procedure for
>> > those who do push to central repos, but I am not sure to what extent it
>> > has gotten condensed down into a sufficiently simple set of rules and
>> > examples in the standard documentation.  Maybe you and Pauli have done
>> > that now.
>>
>> That's quite right, we did more or less assume that the maintainers
>> were git experts, and yes, Pauli did fix that to some extent.  The
>> result, as ported back by me, is this page, which is new, and needs
>> expanding:
>>
>> http://matthew-brett.github.com/pydagogue/gitwash/maintainer_workflow.html
>>
>> >>
>> >> I've been managing a maintenance branch for my much smaller nibabel
>> >> project without much trouble; I've just been doing the occasional
>> >> cherry-pick and rebase from trunk for bugfixes.
>> >
>> > In a way, this illustrates the difficulty: you describe a procedure for
>> > working with a maintenance branch that is completely different from the
>> > one we have been using (apart from the errors). What we have been doing
>> > is initiating bug fixes in the maintenance branch and then merging that
>> > branch to master.  I'm sure either way can work fine, if one doesn't
>> > make mistakes.  I'm not sure what the relative merits of the two methods
>> > are, in terms of simplicity, clarity, and robustness against errors. I
>> > think they result in very different graphs, correct?  With your
>> > approach, the maintenance branch and master are separate lines, while
>> > with our approach, the merges keep pulling the branches together in the
>> > graph, even though their contents are steadily getting farther apart.
>>
>> I must confess that my git fu is not 10/10, but to me the idea of
>> _merging_ the maintenance branch into trunk is very confusing.    I
>> mean, the trees should increasingly diverge, surely, so there will be
>> more and more stuff you don't want to see back in trunk.  At the
>> moment, you have to trust git magic to correctly leave out the commits
>> you don't want...
>>
>> See you,
>>
>> Matthew
>>
>
> While this is all very important and we should definitely come to an
> agreement about this, this still doesn't solve my issue at hand.  I can not
> build the docs in the v1.0.x branch, therefore we can not push out a
> revision to the docs on sourceforge.  Meanwhile, our website still has bad
> links and is pointing users to download version 1.0.0 instead of version
> 1.0.1 (which may explain why we still see some old bugs on the lists every
> now and then).  What do we want to do about my pull request for the docs?

Hold tight, lets see if anything can be done to back out the change.

On Thu, Jun 2, 2011 at 5:00 PM, Darren Dale <dsd...@gm...> wrote:
> On Thu, Jun 2, 2011 at 4:46 PM, Benjamin Root <ben...@ou...> wrote:
>>
>>
>> On Wed, Jun 1, 2011 at 6:20 PM, Matthew Brett <mat...@gm...>
>> wrote:
>>>
>>> Hi,
>>>
>>> On Wed, Jun 1, 2011 at 4:06 PM, Eric Firing <ef...@ha...> wrote:
>>> > On 06/01/2011 12:38 PM, Matthew Brett wrote:
>>> >> Hi,
>>> >>
>>> >> On Wed, Jun 1, 2011 at 12:58 PM, Eric Firing<ef...@ha...>
>>> >>  wrote:
>>> >>>> The current practice worked very nicely with SVN (IMHO), and I think
>>> >>>> it
>>> >>>
>>> >>> (I recall that Mike had to rescue us more than once from svnmerge
>>> >>> confusions, at least during the earlier days.)
>>> >>
>>> >> I was just idly looking at the matplotlib network graph:
>>> >>
>>> >> https://github.com/matplotlib/matplotlib/network
>>> >>
>>> >> There seem to be lots of branches and cross merges ; the history of
>>> >> 1.0.x is extremely confusing.
>>> >
>>> > Agreed!
>>> >
>>> >>
>>> >> I wonder whether it would be worthwhile to review git workflow?
>>> >
>>> > Yes.
>>> >
>>> >>
>>> >> I like Pauli's edits to the numpy gitwash docs in numpy for this.
>>> >> I've actually just merged these back into the gitwash main docs,
>>> >> example build here:
>>> >
>>> > I will have to take a look; mpl did pull some version of gitwash into
>>> > its doc build.
>>> >
>>> >>
>>> >> http://matthew-brett.github.com/pydagogue/gitwash/git_development.html
>>> >
>>> > Thanks, I will take a look.
>>>
>>> If you like what you see, then the 'gitwash_dumper.py' script will
>>> pull a new copy into your repo...
>>>
>>> If you don't, then I'd love suggestions for improvements.
>>>
>>> >> Maybe the overall point is that git does require some thought to
>>> >> history, and some rules-of-work, to avoid confusion.
>>> >
>>> > I think one of the problems is that documentation such as the Git book
>>> > and at least early versions of gitwash, if I remember correctly,
>>> > emphasize workflow for people who do not access the central repo
>>> > directly.  There has been much discussion on the lists of procedure for
>>> > those who do push to central repos, but I am not sure to what extent it
>>> > has gotten condensed down into a sufficiently simple set of rules and
>>> > examples in the standard documentation.  Maybe you and Pauli have done
>>> > that now.
>>>
>>> That's quite right, we did more or less assume that the maintainers
>>> were git experts, and yes, Pauli did fix that to some extent.  The
>>> result, as ported back by me, is this page, which is new, and needs
>>> expanding:
>>>
>>> http://matthew-brett.github.com/pydagogue/gitwash/maintainer_workflow.html
>>>
>>> >>
>>> >> I've been managing a maintenance branch for my much smaller nibabel
>>> >> project without much trouble; I've just been doing the occasional
>>> >> cherry-pick and rebase from trunk for bugfixes.
>>> >
>>> > In a way, this illustrates the difficulty: you describe a procedure for
>>> > working with a maintenance branch that is completely different from the
>>> > one we have been using (apart from the errors). What we have been doing
>>> > is initiating bug fixes in the maintenance branch and then merging that
>>> > branch to master.  I'm sure either way can work fine, if one doesn't
>>> > make mistakes.  I'm not sure what the relative merits of the two methods
>>> > are, in terms of simplicity, clarity, and robustness against errors. I
>>> > think they result in very different graphs, correct?  With your
>>> > approach, the maintenance branch and master are separate lines, while
>>> > with our approach, the merges keep pulling the branches together in the
>>> > graph, even though their contents are steadily getting farther apart.
>>>
>>> I must confess that my git fu is not 10/10, but to me the idea of
>>> _merging_ the maintenance branch into trunk is very confusing.    I
>>> mean, the trees should increasingly diverge, surely, so there will be
>>> more and more stuff you don't want to see back in trunk.  At the
>>> moment, you have to trust git magic to correctly leave out the commits
>>> you don't want...
>>>
>>> See you,
>>>
>>> Matthew
>>>
>>
>> While this is all very important and we should definitely come to an
>> agreement about this, this still doesn't solve my issue at hand.  I can not
>> build the docs in the v1.0.x branch, therefore we can not push out a
>> revision to the docs on sourceforge.  Meanwhile, our website still has bad
>> links and is pointing users to download version 1.0.0 instead of version
>> 1.0.1 (which may explain why we still see some old bugs on the lists every
>> now and then).  What do we want to do about my pull request for the docs?
>
> Hold tight, lets see if anything can be done to back out the change.

I had another look at the history after rereading Pauli's email. I'm
going to try the following on a temporary v1.0.x-cleanup branch:

* "git reset --hard 0e6dad5230"
* redo pull request 103
* cherry-pick the following commits off of the v1.0.x branch:
  - 069c21d
  - 53f8139e
  - de18d9ab2
  - 91e7d980
  - 0cc213b4fa
  - e7f1e83ace
  - 5c968a0ecdd

That should bring the v1.0.x-cleanup branch back to where we thought
it would be. I'll post the result in my fork as soon as it is ready,
and request comment. At that point, we should decide if we want to
rename it v1.0.x and force push, or rename it v1.0.x-maint (or
whatever) and delete the current v1.0.x branch.

Pauli, Jouni, any comments?

Darren

On 06/02/2011 11:48 AM, Darren Dale wrote:
[...]
>
> I had another look at the history after rereading Pauli's email. I'm
> going to try the following on a temporary v1.0.x-cleanup branch:
>
> * "git reset --hard 0e6dad5230"
> * redo pull request 103
> * cherry-pick the following commits off of the v1.0.x branch:
>    - 069c21d
>    - 53f8139e
>    - de18d9ab2
>    - 91e7d980
>    - 0cc213b4fa
>    - e7f1e83ace
>    - 5c968a0ecdd
>
> That should bring the v1.0.x-cleanup branch back to where we thought
> it would be. I'll post the result in my fork as soon as it is ready,
> and request comment. At that point, we should decide if we want to
> rename it v1.0.x and force push, or rename it v1.0.x-maint (or
> whatever) and delete the current v1.0.x branch.
>
> Pauli, Jouni, any comments?
>
> Darren

Darren,

That sounds very encouraging.  If it is successful, I think we should 
use a different name and obliterate the current v1.0.x.  If I understand 
correctly, doing a forced push instead would leave us open to having the 
repair undone.  I don't see any advantage to it; but my git-ability is 
not high.

Going forward, is there any good reason to retain all the old branches 
(transforms, 0.91.x etc.)?  Don't the release tags provide adequate 
access to those branches?  My sense is that merging them into master was 
not good from the standpoint of being able to read the graph and see 
what is really derived from what; but I don't know exactly what can be 
done about it.  They certainly clutter up the output of "git branch -a" 
to no useful effect.

Following along this line, does it perhaps make sense in the future to 
use cherry-picking instead of merging for propagating bug fixes between 
a maintenance or release branch and master?  My uneducated sense is that 
it would leave a less confusing graph, and be less likely to result in 
errors.

Eric

On Thu, Jun 2, 2011 at 6:07 PM, Eric Firing <ef...@ha...> wrote:
> On 06/02/2011 11:48 AM, Darren Dale wrote:
> [...]
>>
>> I had another look at the history after rereading Pauli's email. I'm
>> going to try the following on a temporary v1.0.x-cleanup branch:
>>
>> * "git reset --hard 0e6dad5230"
>> * redo pull request 103
>> * cherry-pick the following commits off of the v1.0.x branch:
>>    - 069c21d
>>    - 53f8139e
>>    - de18d9ab2
>>    - 91e7d980
>>    - 0cc213b4fa
>>    - e7f1e83ace
>>    - 5c968a0ecdd
>>
>> That should bring the v1.0.x-cleanup branch back to where we thought
>> it would be. I'll post the result in my fork as soon as it is ready,
>> and request comment. At that point, we should decide if we want to
>> rename it v1.0.x and force push, or rename it v1.0.x-maint (or
>> whatever) and delete the current v1.0.x branch.
>>
>> Pauli, Jouni, any comments?
>>
>> Darren
>
> Darren,
>
> That sounds very encouraging.  If it is successful, I think we should
> use a different name and obliterate the current v1.0.x.  If I understand
> correctly, doing a forced push instead would leave us open to having the
> repair undone.  I don't see any advantage to it; but my git-ability is
> not high.

I just pushed a v1.0.x-cleanup branch to
https://github.com/ddale/matplotlib . The next step would be to push
it to v1.0.x-maint, and then merge v1.0.x-maint into master with
--strategy=ours. At that point, I think we could delete the old v1.0.x
branch, but I'd like to get a harrumph from another dev first.

> Going forward, is there any good reason to retain all the old branches
> (transforms, 0.91.x etc.)?

I don't think we need the transforms branch. I kept it just for the
sake of completeness during the svn->git migration.

> Don't the release tags provide adequate
> access to those branches?  My sense is that merging them into master was
> not good from the standpoint of being able to read the graph and see
> what is really derived from what; but I don't know exactly what can be
> done about it.  They certainly clutter up the output of "git branch -a"
> to no useful effect.

There was actually a good reason for doing it this way. Each older
maintenance branch was merged into the next newer one, and it was done
with --strategy=ours (which basically means that any changes were
ignored during the merge). We did this so that if someone applied a
critical set of patches to an earlier maintenance branch, say 0.99.x,
and wanted to merge it into v1.0.x, and then into master, it would be
easy to do so without unintentionally pulling all of the other changes
between 0.99.x and 1.0.x (for example) along with it.

> Following along this line, does it perhaps make sense in the future to
> use cherry-picking instead of merging for propagating bug fixes between
> a maintenance or release branch and master?  My uneducated sense is that
> it would leave a less confusing graph, and be less likely to result in
> errors.

I would prefer to continue merging. I think its just a matter of
getting in the habit of inspecting the history graph. But that's just
my opinion.

Thanks by the way for the pointer to qgit, I think it is much nicer than gitk.

Darren

On Thu, Jun 2, 2011 at 6:35 PM, Darren Dale <dsd...@gm...> wrote:
> On Thu, Jun 2, 2011 at 6:07 PM, Eric Firing <ef...@ha...> wrote:
>> On 06/02/2011 11:48 AM, Darren Dale wrote:
>> [...]
>>>
>>> I had another look at the history after rereading Pauli's email. I'm
>>> going to try the following on a temporary v1.0.x-cleanup branch:
>>>
>>> * "git reset --hard 0e6dad5230"
>>> * redo pull request 103
>>> * cherry-pick the following commits off of the v1.0.x branch:
>>>    - 069c21d
>>>    - 53f8139e
>>>    - de18d9ab2
>>>    - 91e7d980
>>>    - 0cc213b4fa
>>>    - e7f1e83ace
>>>    - 5c968a0ecdd
>>>
>>> That should bring the v1.0.x-cleanup branch back to where we thought
>>> it would be. I'll post the result in my fork as soon as it is ready,
>>> and request comment. At that point, we should decide if we want to
>>> rename it v1.0.x and force push, or rename it v1.0.x-maint (or
>>> whatever) and delete the current v1.0.x branch.
>>>
>>> Pauli, Jouni, any comments?
>>>
>>> Darren
>>
>> Darren,
>>
>> That sounds very encouraging.  If it is successful, I think we should
>> use a different name and obliterate the current v1.0.x.  If I understand
>> correctly, doing a forced push instead would leave us open to having the
>> repair undone.  I don't see any advantage to it; but my git-ability is
>> not high.
>
> I just pushed a v1.0.x-cleanup branch to
> https://github.com/ddale/matplotlib . The next step would be to push
> it to v1.0.x-maint, and then merge v1.0.x-maint into master with
> --strategy=ours. At that point, I think we could delete the old v1.0.x
> branch, but I'd like to get a harrumph from another dev first.
>
>> Going forward, is there any good reason to retain all the old branches
>> (transforms, 0.91.x etc.)?
>
> I don't think we need the transforms branch. I kept it just for the
> sake of completeness during the svn->git migration.
>
>> Don't the release tags provide adequate
>> access to those branches?  My sense is that merging them into master was
>> not good from the standpoint of being able to read the graph and see
>> what is really derived from what; but I don't know exactly what can be
>> done about it.  They certainly clutter up the output of "git branch -a"
>> to no useful effect.
>
> There was actually a good reason for doing it this way. Each older
> maintenance branch was merged into the next newer one, and it was done
> with --strategy=ours (which basically means that any changes were
> ignored during the merge). We did this so that if someone applied a
> critical set of patches to an earlier maintenance branch, say 0.99.x,
> and wanted to merge it into v1.0.x, and then into master, it would be
> easy to do so without unintentionally pulling all of the other changes
> between 0.99.x and 1.0.x (for example) along with it.
>
>> Following along this line, does it perhaps make sense in the future to
>> use cherry-picking instead of merging for propagating bug fixes between
>> a maintenance or release branch and master?  My uneducated sense is that
>> it would leave a less confusing graph, and be less likely to result in
>> errors.
>
> I would prefer to continue merging. I think its just a matter of
> getting in the habit of inspecting the history graph. But that's just
> my opinion.
>
> Thanks by the way for the pointer to qgit, I think it is much nicer than gitk.

Actually, I feel fairly confident that this is a reasonable fix. I
just pushed v1.0.x-maint, I pushed v1.0.x to my personal clone as a
backup, and I deleted v1.0.x from the matplotlib repository.

Ben, would you please rebase your mplot3d/minor_errors branch onto the
new v1.0.x-maint branch?

I'll post an announcement on mpl-dev about the change in maintenance branch.

Darren

On 06/02/2011 12:35 PM, Darren Dale wrote:
> On Thu, Jun 2, 2011 at 6:07 PM, Eric Firing<ef...@ha...>  wrote:
>> On 06/02/2011 11:48 AM, Darren Dale wrote:
>> [...]
>>>
>>> I had another look at the history after rereading Pauli's email. I'm
>>> going to try the following on a temporary v1.0.x-cleanup branch:
>>>
>>> * "git reset --hard 0e6dad5230"
>>> * redo pull request 103
>>> * cherry-pick the following commits off of the v1.0.x branch:
>>>     - 069c21d
>>>     - 53f8139e
>>>     - de18d9ab2
>>>     - 91e7d980
>>>     - 0cc213b4fa
>>>     - e7f1e83ace
>>>     - 5c968a0ecdd
>>>
>>> That should bring the v1.0.x-cleanup branch back to where we thought
>>> it would be. I'll post the result in my fork as soon as it is ready,
>>> and request comment. At that point, we should decide if we want to
>>> rename it v1.0.x and force push, or rename it v1.0.x-maint (or
>>> whatever) and delete the current v1.0.x branch.
>>>
>>> Pauli, Jouni, any comments?
>>>
>>> Darren
>>
>> Darren,
>>
>> That sounds very encouraging.  If it is successful, I think we should
>> use a different name and obliterate the current v1.0.x.  If I understand
>> correctly, doing a forced push instead would leave us open to having the
>> repair undone.  I don't see any advantage to it; but my git-ability is
>> not high.
>
> I just pushed a v1.0.x-cleanup branch to
> https://github.com/ddale/matplotlib . The next step would be to push
> it to v1.0.x-maint, and then merge v1.0.x-maint into master with
> --strategy=ours. At that point, I think we could delete the old v1.0.x
> branch, but I'd like to get a harrumph from another dev first.

Dale,

Thank you.
>
>> Going forward, is there any good reason to retain all the old branches
>> (transforms, 0.91.x etc.)?
>
> I don't think we need the transforms branch. I kept it just for the
> sake of completeness during the svn->git migration.
>
>> Don't the release tags provide adequate
>> access to those branches?  My sense is that merging them into master was
>> not good from the standpoint of being able to read the graph and see
>> what is really derived from what; but I don't know exactly what can be
>> done about it.  They certainly clutter up the output of "git branch -a"
>> to no useful effect.
>

> There was actually a good reason for doing it this way. Each older

I understand the rationale, but...

> maintenance branch was merged into the next newer one, and it was done
> with --strategy=ours (which basically means that any changes were
> ignored during the merge). We did this so that if someone applied a

which means that it is a fundamentally misleading merge--a merge in name 
only, not an indicator of what is in a given branch.

> critical set of patches to an earlier maintenance branch, say 0.99.x,
> and wanted to merge it into v1.0.x, and then into master, it would be
> easy to do so without unintentionally pulling all of the other changes
> between 0.99.x and 1.0.x (for example) along with it.

I think the likelihood of anyone ever actually doing this is near zilch; 
we don't maintain old branches.  And for the single current maintenance 
branch at any given time, cherry-picking bug fixes from maintenance to 
master or the reverse seems to me more explicit, less mysterious, and 
less likely to have unintended consequences than merging.

>
>> Following along this line, does it perhaps make sense in the future to
>> use cherry-picking instead of merging for propagating bug fixes between
>> a maintenance or release branch and master?  My uneducated sense is that
>> it would leave a less confusing graph, and be less likely to result in
>> errors.
>
> I would prefer to continue merging. I think its just a matter of
> getting in the habit of inspecting the history graph. But that's just
> my opinion.

I still don't agree.  It looks to me like numpy is using the cherry-pick 
approach, not the merge approach, and the result is that each branch has 
a reasonably linear history that one can actually follow by eye, and 
easily see exactly what has been done.  Compare numpy:

http://currents.soest.hawaii.edu/hgstage/numpy_from_git/graph/9264?revcount=240

to mpl:

http://currents.soest.hawaii.edu/hgstage/mpl_from_git/graph/6855?revcount=240

Even without the foulup, I think you would see that the merges from 
maintenance branches into subsequent branches and into master make it 
very hard to figure out what has actually been done on any given branch. 
Undoubtedly one *can* figure it out, as apparently you have just done; 
but it is not immediately clear from the graph.

Eric

>
> Thanks by the way for the pointer to qgit, I think it is much nicer than gitk.
>
> Darren

On Thu, Jun 2, 2011 at 7:46 PM, Eric Firing <ef...@ha...> wrote:
> On 06/02/2011 12:35 PM, Darren Dale wrote:
>> On Thu, Jun 2, 2011 at 6:07 PM, Eric Firing<ef...@ha...>  wrote:
>>> Going forward, is there any good reason to retain all the old branches
>>> (transforms, 0.91.x etc.)?
>>
>> I don't think we need the transforms branch. I kept it just for the
>> sake of completeness during the svn->git migration.
>>
>>> Don't the release tags provide adequate
>>> access to those branches?  My sense is that merging them into master was
>>> not good from the standpoint of being able to read the graph and see
>>> what is really derived from what; but I don't know exactly what can be
>>> done about it.  They certainly clutter up the output of "git branch -a"
>>> to no useful effect.
>>
>
>> There was actually a good reason for doing it this way. Each older
>
> I understand the rationale, but...
>
>> maintenance branch was merged into the next newer one, and it was done
>> with --strategy=ours (which basically means that any changes were
>> ignored during the merge). We did this so that if someone applied a
>
> which means that it is a fundamentally misleading merge--a merge in name
> only, not an indicator of what is in a given branch.

No, it meant that all of the actual merges that we intended to do had
already been done using svnmerge. But svn2git wasn't able to capture
all of those relationships, so instead we did what we could at the
time of the conversion.

>> critical set of patches to an earlier maintenance branch, say 0.99.x,
>> and wanted to merge it into v1.0.x, and then into master, it would be
>> easy to do so without unintentionally pulling all of the other changes
>> between 0.99.x and 1.0.x (for example) along with it.
>
> I think the likelihood of anyone ever actually doing this is near zilch;
> we don't maintain old branches.

aside from 1.0.x...

> And for the single current maintenance
> branch at any given time, cherry-picking bug fixes from maintenance to
> master or the reverse seems to me more explicit, less mysterious, and
> less likely to have unintended consequences than merging.

I understand your position.

>>
>>> Following along this line, does it perhaps make sense in the future to
>>> use cherry-picking instead of merging for propagating bug fixes between
>>> a maintenance or release branch and master?  My uneducated sense is that
>>> it would leave a less confusing graph, and be less likely to result in
>>> errors.
>>
>> I would prefer to continue merging. I think its just a matter of
>> getting in the habit of inspecting the history graph. But that's just
>> my opinion.
>
> I still don't agree.  It looks to me like numpy is using the cherry-pick
> approach, not the merge approach, and the result is that each branch has
> a reasonably linear history that one can actually follow by eye, and
> easily see exactly what has been done.  Compare numpy:
>
> http://currents.soest.hawaii.edu/hgstage/numpy_from_git/graph/9264?revcount=240
>
> to mpl:
>
> http://currents.soest.hawaii.edu/hgstage/mpl_from_git/graph/6855?revcount=240

Numpy appears to be applying all of their changes on master, and then
cherry picking changes to apply to the maintenance branch.

> Even without the foulup, I think you would see that the merges from
> maintenance branches into subsequent branches and into master make it
> very hard to figure out what has actually been done on any given branch.

I strongly disagree. The only reason you get cleaner history graphs
with cherry picking is because it doesn't graph the cherry picks! If
you want to know what has been merged, you have to inspect the commit
message in one branch and match it up with the commit message in
another branch. How does that make it easier to figure out what has
been done on any given branch?

> Undoubtedly one *can* figure it out, as apparently you have just done;
> but it is not immediately clear from the graph.

I have another suggestion, relating to somebody's (your?) suggestion
that we make more frequent releases. Development would be more robust
if we were all branching from a commonly acknowledged reference point.
For example, that might be v1.0.1 on the maintenance branch. The
history graph would probably be cleaner as well. Then when a branch X
is merged into maint, maybe we should be merging branch X into master
as well, rather than merging v1.0.x into master. That would make it
clearer what was actually being merged. Then, when v1.0.2 was tagged,
we merge that into master (perhaps with strategy=ours if appropriate),
and that becomes the new commonly acknowledged reference point on the
master branch.

Darren

Hi,

On Thu, Jun 2, 2011 at 5:07 PM, Darren Dale <dsd...@gm...> wrote:
> On Thu, Jun 2, 2011 at 7:46 PM, Eric Firing <ef...@ha...> wrote:
...
>> Even without the foulup, I think you would see that the merges from
>> maintenance branches into subsequent branches and into master make it
>> very hard to figure out what has actually been done on any given branch.
>
> I strongly disagree. The only reason you get cleaner history graphs
> with cherry picking is because it doesn't graph the cherry picks! If
> you want to know what has been merged, you have to inspect the commit
> message in one branch and match it up with the commit message in
> another branch. How does that make it easier to figure out what has
> been done on any given branch?

I think Eric's point is that it kind of feels (and looks) wrong to
merge maintenance into master, rather than backporting fixes from
master with cherry-picks.  Maybe 'feels wrong' might be translatable
as 'harder to think about' and therefore 'more error prone'?  I can
see the argument for doing it though.  It is a common workflow?

See you,

Matthew

Matthew,

On Thu, Jun 2, 2011 at 8:17 PM, Matthew Brett <mat...@gm...> wrote:
> Hi,
>
> On Thu, Jun 2, 2011 at 5:07 PM, Darren Dale <dsd...@gm...> wrote:
>> On Thu, Jun 2, 2011 at 7:46 PM, Eric Firing <ef...@ha...> wrote:
> ...
>>> Even without the foulup, I think you would see that the merges from
>>> maintenance branches into subsequent branches and into master make it
>>> very hard to figure out what has actually been done on any given branch.
>>
>> I strongly disagree. The only reason you get cleaner history graphs
>> with cherry picking is because it doesn't graph the cherry picks! If
>> you want to know what has been merged, you have to inspect the commit
>> message in one branch and match it up with the commit message in
>> another branch. How does that make it easier to figure out what has
>> been done on any given branch?
>
> I think Eric's point is that it kind of feels (and looks) wrong to
> merge maintenance into master, rather than backporting fixes from
> master with cherry-picks.  Maybe 'feels wrong' might be translatable
> as 'harder to think about' and therefore 'more error prone'?

Maybe "feels wrong" translates to "unfamiliar", and has nothing to do
with difficulty or potential for error. Git is a powerful tool. If we
used a cherry-picking workflow (which I would not advocate), there
would still be chances for error by inappropriately specifying hash
ranges, for example. I think there is much more potential for error
using cherry-picking. There are other advantages as well, see
http://stackoverflow.com/questions/1241720/git-cherry-pick-vs-merge-workflow
(note the warnings about rebasing, which are not relevant to this
discussion)

> I can
> see the argument for doing it though.  It is a common workflow?

Yes, I believe it is a very common workflow.

Before we made the git transition, I read about various workflows.
What we are doing now is somewhat similar to what used to be done with
svnmerge. I just googled "git workflow", and found
http://www.kernel.org/pub/software/scm/git/docs/gitworkflows.html .
See the section on "Merging Upwards":

---
The "downwards graduation" discussed above cannot be done by actually
merging downwards, however, since that would merge all changes on the
unstable branch into the stable one. Hence the following:

Rule: Merge upwards
Always commit your fixes to the oldest supported branch that require
them. Then (periodically) merge the integration branches upwards into
each other.

This gives a very controlled flow of fixes. If you notice that you
have applied a fix to e.g. master that is also required in maint, you
will need to cherry-pick it (using git-cherry-pick(1)) downwards. This
will happen a few times and is nothing to worry about unless you do it
very frequently.
---

With cherry picking, imagine we get ready to cut a v1.0.2 release, and
we want to know if all of the bug fixes that should be applied to the
maintenance branch have been applied. How do we verify? Much easier is
to apply them like the docs at kernel.org suggest, to the oldest
supported branch that require them and then merge upwards. Then the
history graph can tell you that the bug fixes in older versions have
been applied to newer branches.

Merging is going to happen anyway, whenever someone files a pull
request. We have a good workflow, we just had a small mistake and have
now overcome it.

Darren

Hi,

On Thu, Jun 2, 2011 at 7:06 PM, Darren Dale <dsd...@gm...> wrote:
> Matthew,
>
> On Thu, Jun 2, 2011 at 8:17 PM, Matthew Brett <mat...@gm...> wrote:
>> Hi,
>>
>> On Thu, Jun 2, 2011 at 5:07 PM, Darren Dale <dsd...@gm...> wrote:
>>> On Thu, Jun 2, 2011 at 7:46 PM, Eric Firing <ef...@ha...> wrote:
>> ...
>>>> Even without the foulup, I think you would see that the merges from
>>>> maintenance branches into subsequent branches and into master make it
>>>> very hard to figure out what has actually been done on any given branch.
>>>
>>> I strongly disagree. The only reason you get cleaner history graphs
>>> with cherry picking is because it doesn't graph the cherry picks! If
>>> you want to know what has been merged, you have to inspect the commit
>>> message in one branch and match it up with the commit message in
>>> another branch. How does that make it easier to figure out what has
>>> been done on any given branch?
>>
>> I think Eric's point is that it kind of feels (and looks) wrong to
>> merge maintenance into master, rather than backporting fixes from
>> master with cherry-picks.  Maybe 'feels wrong' might be translatable
>> as 'harder to think about' and therefore 'more error prone'?
>
> Maybe "feels wrong" translates to "unfamiliar", and has nothing to do
> with difficulty or potential for error.

Ah no - I mean that the way I think most of think of bugfix workflow
is that we fix in trunk and backport the fix to the maintenance
branch.   Here though you are fixing in the maintenance branch and
_merging_ to trunk.  The counter-argument is 'well think of it the
other way round and it will be fine'.  It's a little difficult to know
if that's true I suppose.

It's a funny kind of thing too, because it puts an extra constraint on
the bugfix.  For example, imagine a bugfix in maintenance, in some
code that has crazy-changed in trunk.   Now you have to decide if you
try and merge it anyway, and basically do a rewrite as you fix the
merge conflict.  Or you could merge-ours on that commit and write a
completely different fix on trunk.  But that's getting a bit dark and
magic.   And so it imposes some slight what-will-happen brain overhead
on the person writing the fix.

> Git is a powerful tool. If we
> used a cherry-picking workflow (which I would not advocate), there
> would still be chances for error by inappropriately specifying hash
> ranges, for example. I think there is much more potential for error
> using cherry-picking. There are other advantages as well, see
> http://stackoverflow.com/questions/1241720/git-cherry-pick-vs-merge-workflow
> (note the warnings about rebasing, which are not relevant to this
> discussion)
>
>> I can
>> see the argument for doing it though.  It is a common workflow?
>
> Yes, I believe it is a very common workflow.

Ah - OK, I guess I had not seen it before.

> Before we made the git transition, I read about various workflows.
> What we are doing now is somewhat similar to what used to be done with
> svnmerge. I just googled "git workflow", and found
> http://www.kernel.org/pub/software/scm/git/docs/gitworkflows.html .
> See the section on "Merging Upwards":

Yes, I think I see why it would be attractive.

> With cherry picking, imagine we get ready to cut a v1.0.2 release, and
> we want to know if all of the bug fixes that should be applied to the
> maintenance branch have been applied. How do we verify? Much easier is
> to apply them like the docs at kernel.org suggest, to the oldest
> supported branch that require them and then merge upwards. Then the
> history graph can tell you that the bug fixes in older versions have
> been applied to newer branches.

I believe that the git log --cherry etc machinery is designed to deal
with this case, but I haven't used it myself.  I have to emphasize,
the projects I've been involved with have fewer developers and smaller
code-bases,

Cheers,

Matthew

Thu, 02 Jun 2011 17:48:55 -0400, Darren Dale wrote:
[clip]
> * "git reset --hard 0e6dad5230"
> * redo pull request 103
> * cherry-pick the following commits off of the v1.0.x branch:
>   - 069c21d
>   - 53f8139e
>   - de18d9ab2
>   - 91e7d980
>   - 0cc213b4fa
>   - e7f1e83ace
>   - 5c968a0ecdd
> 
> That should bring the v1.0.x-cleanup branch back to where we thought it
> would be. I'll post the result in my fork as soon as it is ready, and
> request comment. At that point, we should decide if we want to rename it
> v1.0.x and force push, or rename it v1.0.x-maint (or whatever) and
> delete the current v1.0.x branch.
> 
> Pauli, Jouni, any comments?

Seems OK to me.

	Pauli