Skip to content

Conversation

gfyoung
Copy link
Member

@gfyoung gfyoung commented Nov 14, 2018

False criterion was causing errors when specified headers appeared to capture
a seemingly unnamed row, just because they had the string "Unnamed" in it.

Setup:

from pandas import read_csv
from pandas.compat import StringIO

data = "Unnamed,NotUnnamed\n0,1\n2,3\n4,5"
read_csv(StringIO(data), header=[0, 1])

Previously, this would error:

...
ValueError : Passed header=[0,1] are too many rows for this multi_index of columns

Now, it nicely returns a DataFrame:

  Unnamed NotUnnamed
        0          1
0       2          3
1       4          5

Leverages the patch used in #23484 of self.unnamed_cols.

@pep8speaks
Copy link

Hello @gfyoung! Thanks for submitting the PR.

@gfyoung gfyoung force-pushed the unnamed-unrobust-criterion branch from ec41e07 to 7bcca7b Compare November 14, 2018 06:31
@gfyoung gfyoung added Bug IO CSV read_csv, to_csv labels Nov 14, 2018
@gfyoung gfyoung added this to the 0.24.0 milestone Nov 14, 2018
@codecov
Copy link

codecov bot commented Nov 14, 2018

Codecov Report

Merging #23687 into master will decrease coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #23687      +/-   ##
==========================================
- Coverage   92.25%   92.25%   -0.01%     
==========================================
  Files         161      161              
  Lines       51383    51381       -2     
==========================================
- Hits        47404    47402       -2     
  Misses       3979     3979
Flag Coverage Δ
#multiple 90.64% <100%> (-0.01%) ⬇️
#single 42.32% <0%> (ø) ⬆️
Impacted Files Coverage Δ
pandas/io/parsers.py 95.54% <100%> (-0.01%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a23f901...ff68dbd. Read the comment docs.

@gfyoung gfyoung force-pushed the unnamed-unrobust-criterion branch from 7bcca7b to 8e92e4c Compare November 14, 2018 18:15
@gfyoung
Copy link
Member Author

gfyoung commented Nov 14, 2018

@jreback : Address the comment, and all is still green. PTAL.

@jreback
Copy link
Contributor

jreback commented Nov 14, 2018

conflict in the whatsnew, can you merge master

@gfyoung gfyoung force-pushed the unnamed-unrobust-criterion branch from 8e92e4c to 52cc5c9 Compare November 14, 2018 21:57
@gfyoung
Copy link
Member Author

gfyoung commented Nov 15, 2018

@jreback : Rebased my changes, and all is green. PTAL.

False criterion was causing errors when
specified headers appeared to capture
a seemingly unnamed row, just because
they had the string "Unnamed" in it.
@gfyoung gfyoung force-pushed the unnamed-unrobust-criterion branch from 52cc5c9 to ff68dbd Compare November 15, 2018 18:15
@gfyoung
Copy link
Member Author

gfyoung commented Nov 15, 2018

@jreback : Addressed all comments, all is still green. PTAL.

@jreback jreback merged commit 4a6a918 into pandas-dev:master Nov 16, 2018
@jreback
Copy link
Contributor

jreback commented Nov 16, 2018

thanks!

@gfyoung gfyoung deleted the unnamed-unrobust-criterion branch November 16, 2018 21:27
tm9k1 pushed a commit to tm9k1/pandas that referenced this pull request Nov 19, 2018
False criterion was causing errors when
specified headers appeared to capture
a seemingly unnamed row, just because
they had the string "Unnamed" in it.
Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019
False criterion was causing errors when
specified headers appeared to capture
a seemingly unnamed row, just because
they had the string "Unnamed" in it.
Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019
False criterion was causing errors when
specified headers appeared to capture
a seemingly unnamed row, just because
they had the string "Unnamed" in it.
@SamLevinSE
Copy link

I'm getting bunch of these "unnamed ..." where my column is empty. Is this related to the same bug?
ex:

Unnamed: 1_level_0 Baseline Features Unnamed: 3_level_0 Unnamed: 4_level_0 Unnamed: 5_level_0 Unnamed: 6_level_0 Unnamed: 7_level_0

@gfyoung
Copy link
Member Author

gfyoung commented Feb 27, 2020

@SamLevinSE : I don't believe so. However, if you are welcome to open a new issue to provide more details, as it could still be a problem with pandas (can't tell from this).

@SamLevinSE
Copy link

@gfyoung What if I give you some more Information? could you please check these two screenshots and let me know if I need to create a new bug/issue or if I'm doing something wrong?
image

image
Thank you!

@gfyoung
Copy link
Member Author

gfyoung commented Feb 27, 2020

This looks expected as far as I can see.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug IO CSV read_csv, to_csv
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants