read_csv(<file>, nrows=x) raises an StopIteration exception when given file doesn't contain any newlines

[vbmaarten]

The problem occurs when using read_csv with nrows specified on a file that only contains a row of column names, but no newlines after that. An empty DataFrame would be expected but a StopIteration error is raised instead.

Code:

import pandas
pandas.read_csv(one_line_file, nrows=0)

INSTALLED VERSIONS

commit: None
python: 3.4.2.final.0
python-bits: 64
OS: Linux
OS-release: 3.16.0-28-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.14.1
nose: 1.3.4
Cython: None
numpy: 1.8.2
scipy: 0.14.0
statsmodels: None
IPython: None
sphinx: None
patsy: None
scikits.timeseries: None
dateutil: 2.0
pytz: 2012c
bottleneck: None
tables: 3.1.1
numexpr: 2.4
matplotlib: 1.3.1
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: 3.3.6
bs4: 4.3.2
html5lib: 0.999
httplib2: 0.9
apiclient: None
rpy2: None
sqlalchemy: None
pymysql: None
psycopg2: None

[brimcfadden]

I ran into this today, too, so it hasn't been fixed.

[jreback]

well that's what the Open designation means. This needs a reproducible example (might be as simple as

pd.read_csv(StringIO("foo\n\n"),nrows=1)

but an actual example would be helpful.

[brimcfadden]

Sorry, Jeff. Here is a fully-contained program with several string examples. At this time I can only run it under Python 2.7 on Windows, but I think it will run under Python 3.x as well, and I think the issue probably happens on all platforms.

import tempfile

import pandas as pd


for s in ('foo,bar', 'foo,bar\n', 'foo,bar\n\n', 'foo,bar\n1,2'):
    with tempfile.TemporaryFile(mode='w+b') as f:
        f.write(s)
        f.seek(0)  # Seek so that the file can be read
        try:
            df = pd.read_csv(f, nrows=1)
        except StopIteration:
            print("Bad: {!r} caused StopIteration.".format(s))
        else:
            print("Good: {!r} works fine.".format(s))

I get the following output:

Bad: 'foo,bar' caused StopIteration.
Bad: 'foo,bar\n' caused StopIteration.
Bad: 'foo,bar\n\n' caused StopIteration.
Good: 'foo,bar\n1,2' works fine.

@vbmaarten's original statement that the lack of newlines is causing this seems to be inaccurate, though it is definitely related to the presence of nrows. If I change the above example to use nrows=0, or a larger number, the results do not change. However, if I remove that argument entirely, I get:

Good: 'foo,bar' works fine.
Good: 'foo,bar\n' works fine.
Good: 'foo,bar\n\n' works fine.
Good: 'foo,bar\n1,2' works fine.

[jreback]

closed by #10827