Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preserve Complex Data Types for to_csv #61157

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open
Prev Previous commit
Next Next commit
Basic test
  • Loading branch information
Jaspvr committed Mar 13, 2025
commit b44af37f575bc6dced2c1f1816af17e1e4f824c0
26 changes: 23 additions & 3 deletions scripts/tests/test_csv.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,26 @@
import pandas as pd
import numpy as np

print("Pandas version:", pd.__version__)
# Create a DataFrame with NumPy arrays
df = pd.DataFrame({
'id': [1, 2],
'embedding': [np.array([0.1, 0.2, 0.3]), np.array([0.4, 0.5, 0.6])]
})

df = pd.DataFrame({"Name": ["Alice", "Bob"], "Age": [25, 30]})
print(df)
# Save to CSV
csv_file = "test_numpy_array.csv"
df.to_csv(csv_file, index=False)
print(f"Saved CSV:\n{open(csv_file).read()}")

# Read back the CSV
df_loaded = pd.read_csv(csv_file)

# Print results
print("\nLoaded DataFrame:")
print(df_loaded)

# Check if numpy arrays were converted to strings
if isinstance(df_loaded["embedding"][0], str):
print("\nTest Passed: NumPy arrays were converted to strings in CSV")
else:
print("\nTest Failed: NumPy arrays were not converted as expected")
3 changes: 3 additions & 0 deletions scripts/tests/test_numpy_array.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
id,embedding
1,[0.1 0.2 0.3]
2,[0.4 0.5 0.6]