Thread: [matplotlib-devel] A new KNOWNFAIL, pcolormesh.png ?

Hi,
I was playing with test suite and I noticed that :

morph@zion:~/deb/build-area/matplotlib-1.0.1$
PYTHONPATH=build/lib.linux-x86_64-2.6 python -c "import matplotlib as
m ; m.test(verbosity=1)"
/usr/lib/pymodules/python2.6/nose/plugins/manager.py:391: UserWarning:
Module matplotlib was already imported from
/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/__init__.pyc,
but /usr/lib/pymodules/python2.6 is being added to sys.path
  import pkg_resources
..K........K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..KE.K..K..K..K..K..K/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/axes.py:2381:
UserWarning: Attempting to set identical left==right results
in singular transformations; automatically expanding.
left=730139.0, right=730139.0
  + 'left=%s, right=%s') % (left, right))
...K..KK..K..K.....K..K..K..K..K....K..K..K....K..K..K..K..K
======================================================================
ERROR: matplotlib.tests.test_axes.test_pcolormesh
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/testing/decorators.py",
line 32, in failer
    result = f(*args, **kwargs)
  File "/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/testing/decorators.py",
line 126, in decorated_compare_images
    '(RMS %(rms).3f)'%err)
ImageComparisonFailure: images not close:
/home/morph/deb/build-area/matplotlib-1.0.1/result_images/test_axes/pcolormesh.png
vs. /home/morph/deb/build-area/matplotlib-1.0.1/result_images/test_axes/expected-pcolormesh.png
(RMS 116.512)

----------------------------------------------------------------------
Ran 150 tests in 107.607s

FAILED (KNOWNFAIL=46, errors=1)

Except for the UserWarning, there is an error in the suite: is someone
else able to replicate it? is it another KNOWNFAIL?

Cheers,
-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

On Mon, Jan 17, 2011 at 1:28 PM, Sandro Tosi <mo...@de...> wrote:

> Hi,
> I was playing with test suite and I noticed that :
>
> morph@zion:~/deb/build-area/matplotlib-1.0.1$
> PYTHONPATH=build/lib.linux-x86_64-2.6 python -c "import matplotlib as
> m ; m.test(verbosity=1)"
> /usr/lib/pymodules/python2.6/nose/plugins/manager.py:391: UserWarning:
> Module matplotlib was already imported from
>
> /home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/__init__.pyc,
> but /usr/lib/pymodules/python2.6 is being added to sys.path
>  import pkg_resources
>
> ..K........K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..KE.K..K..K..K..K..K/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/axes.py:2381:
> UserWarning: Attempting to set identical left==right results
> in singular transformations; automatically expanding.
> left=730139.0, right=730139.0
>  + 'left=%s, right=%s') % (left, right))
> ...K..KK..K..K.....K..K..K..K..K....K..K..K....K..K..K..K..K
> ======================================================================
> ERROR: matplotlib.tests.test_axes.test_pcolormesh
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>  File "/usr/lib/pymodules/python2.6/nose/case.py", line 183, in runTest
>    self.test(*self.arg)
>  File
> "/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/testing/decorators.py",
> line 32, in failer
>    result = f(*args, **kwargs)
>  File
> "/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/testing/decorators.py",
> line 126, in decorated_compare_images
>    '(RMS %(rms).3f)'%err)
> ImageComparisonFailure: images not close:
>
> /home/morph/deb/build-area/matplotlib-1.0.1/result_images/test_axes/pcolormesh.png
> vs.
> /home/morph/deb/build-area/matplotlib-1.0.1/result_images/test_axes/expected-pcolormesh.png
> (RMS 116.512)
>
> ----------------------------------------------------------------------
> Ran 150 tests in 107.607s
>
> FAILED (KNOWNFAIL=46, errors=1)
>
> Except for the UserWarning, there is an error in the suite: is someone
> else able to replicate it? is it another KNOWNFAIL?
>
> Cheers,
> --
> Sandro Tosi (aka morph, morpheus, matrixhasu)
> My website: http://matrixhasu.altervista.org/
> Me at Debian: http://wiki.debian.org/SandroTosi
>
>
I have seen this before, and I think it was discussed once before.
Visually, there is very little difference, but supposedly there is some sort
of issue with different PIL versions.  What version of PIL do you have?

Ben Root

Hi Ben,
thanks for the fast reply!

On Mon, Jan 17, 2011 at 20:35, Benjamin Root <ben...@ou...> wrote:
> I have seen this before, and I think it was discussed once before.
> Visually, there is very little difference,

indeed, looking at the 2 images, I can't see any difference.

> but supposedly there is some sort
> of issue with different PIL versions.  What version of PIL do you have?

1.1.7

Cheers,
-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

I'm not sure PIL enters into it -- there shouldn't be any code path 
involving PIL in that case.

I think this a case where the image comparison tolerance needs to be 
increased.  You would do this be passing a "tol" parameter to the 
image_comparison decorator on the pcolormesh test.  The default is 1e-3, 
but it should be conservatively increased until the test passes.  You 
can perform this experiment yourself, or attach the result image for the 
test to this list and I can experiment to find a correct value.

Mike

On 01/17/2011 02:44 PM, Sandro Tosi wrote:
> Hi Ben,
> thanks for the fast reply!
>
> On Mon, Jan 17, 2011 at 20:35, Benjamin Root<ben...@ou...>  wrote:
>> I have seen this before, and I think it was discussed once before.
>> Visually, there is very little difference,
> indeed, looking at the 2 images, I can't see any difference.
>
>> but supposedly there is some sort
>> of issue with different PIL versions.  What version of PIL do you have?
> 1.1.7
>
> Cheers,

Hi,

On Tue, Jan 18, 2011 at 15:00, Michael Droettboom <md...@st...> wrote:
> I'm not sure PIL enters into it -- there shouldn't be any code path
> involving PIL in that case.
>
> I think this a case where the image comparison tolerance needs to be
> increased.  You would do this be passing a "tol" parameter to the
> image_comparison decorator on the pcolormesh test.  The default is 1e-3,
> but it should be conservatively increased until the test passes.  You
> can perform this experiment yourself, or attach the result image for the
> test to this list and I can experiment to find a correct value.

I'm attaching the images, just for reference; as you can see, the
difference is very tiny and with a tolerance of 0.02 I was able to
pass the test (RMS Value: 0.0116511801977).

Cheers,
-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

Thanks.  I have increased the tolerance in the SVN repository in r 
8926/r8927.

Cheers,
Mike

On 01/18/2011 01:29 PM, Sandro Tosi wrote:
> Hi,
>
> On Tue, Jan 18, 2011 at 15:00, Michael Droettboom<md...@st...>  wrote:
>> I'm not sure PIL enters into it -- there shouldn't be any code path
>> involving PIL in that case.
>>
>> I think this a case where the image comparison tolerance needs to be
>> increased.  You would do this be passing a "tol" parameter to the
>> image_comparison decorator on the pcolormesh test.  The default is 1e-3,
>> but it should be conservatively increased until the test passes.  You
>> can perform this experiment yourself, or attach the result image for the
>> test to this list and I can experiment to find a correct value.
> I'm attaching the images, just for reference; as you can see, the
> difference is very tiny and with a tolerance of 0.02 I was able to
> pass the test (RMS Value: 0.0116511801977).
>
> Cheers,

Sandro Tosi, on 2011-01-18 19:29,  wrote:
> Hi,
> 
> On Tue, Jan 18, 2011 at 15:00, Michael Droettboom <md...@st...> wrote:
> > I'm not sure PIL enters into it -- there shouldn't be any code path
> > involving PIL in that case.
> >
> > I think this a case where the image comparison tolerance needs to be
> > increased.  You would do this be passing a "tol" parameter to the
> > image_comparison decorator on the pcolormesh test.  The default is 1e-3,
> > but it should be conservatively increased until the test passes.  You
> > can perform this experiment yourself, or attach the result image for the
> > test to this list and I can experiment to find a correct value.
> 
> I'm attaching the images, just for reference; as you can see, the
> difference is very tiny and with a tolerance of 0.02 I was able to
> pass the test (RMS Value: 0.0116511801977).

I just wanted to chime in that there *is* structure in the
differences between the images - which don't show up on screens
which don't have linearized gamma (the difference between 0 and 1
are much smaller in brightness than the difference between 127
and 128, for example).

A quick way to get around this is to just invert the colors.

in X11, I do this with 'xcalib -a -i' which toggles back to
normal if you run the command twice (Compiz also has it's own
version of this via some keyboard shortcut, IIRC, but I don't use
it). ImageMagick's convert can do this with the -negate flag. On
OS X, there's something like command-F8 to do this.  

For completeness, if you're interested in doing this in
matplotlib, ;) just subtract your (floating point 0.0-1.0) color
from (1.,1.,1.) to get the new color.

I'm attaching the difference image, with its colors inverted.

best,
-- 
Paul Ivanov
314 address only used for lists,  off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7