04/12/2024, 00:18 How to write bash script to insert the records from file my_records.txt into PostgreSQL DB table?
reSQL DB table? - DEV Community
Dmitry Romanoff
Posted on 11 de abr. de 2023 • Edited on 14 de mai.
6 1 1
How to write bash script to insert the records
from file my_records.txt into PostgreSQL DB
table?
#script #bash #database #postgres
One | 1 | 2021-01-01 01:01:01
Two | 2 | 2022-02-02 02:02:02
Three | 3 | 2023-03-03 03:03:03
Four | 4 | 2024-04-04 04:04:04
Five | 5 | 2025-05-05 05:05:05
First step, create the table of the corresponding structure in the PostgreSQL DB.
postgres=# create table your_table_name("name" varchar(100), "number" smallint, "dat
CREATE TABLE
Here's a bash script to insert the records from a file called "my_records.txt" into a
PostgreSQL database table:
#!/bin/bash
# Set the database host, port, name, username and password
export DB_HOST="your_db_host"
export DB_PORT="your_db_port"
export DB_NAME="your_db_name"
export DB_USER="your_db_user"
export PGPASSWORD='your_db_password'
# Loop through the lines in the file
while read -r line
do
# Split the line by '|' delimiter
IFS='|' read -ra values <<< "$line"
https://dev.to/dm8ry/how-to-write-bash-script-to-insert-the-records-from-file-myrecordstxt-into-postgresql-db-table-45cd 1/6
04/12/2024, 00:18 How to write bash script to insert the records from file my_records.txt into PostgreSQL DB table? - DEV Community
# Extract the values
name="${values[0]}"
number="${values[1]}"
date="${values[2]}"
# Insert the values into the database
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -c "INSERT INTO your_
done < "my_records.txt"
Note that you need to replace "your_db_host", "your_db_port", "your_db_name",
"your_db_user", "your_db_password", "your_table_name" with your own database
details and table name.
dmi@dmi-laptop:~$ cat populate_file_recs_into_pg_table.sh
#!/bin/bash
# Set the database host, port, name, username and password
export DB_HOST="localhost"
export DB_PORT="5442"
export DB_NAME="postgres"
export DB_USER="myuser"
export PGPASSWORD='mypwd'
# Loop through the lines in the file
while read -r line
do
# Split the line by '|' delimiter
IFS='|' read -ra values <<< "$line"
# Extract the values
name="${values[0]}"
number="${values[1]}"
date="${values[2]}"
# Insert the values into the database
psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -c "INSERT INTO your_
done < "my_records.txt"
dmi@dmi-laptop:~$ ./populate_file_recs_into_pg_table.sh
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
https://dev.to/dm8ry/how-to-write-bash-script-to-insert-the-records-from-file-myrecordstxt-into-postgresql-db-table-45cd 2/6
04/12/2024, 00:18 How to write bash script to insert the records from file my_records.txt into PostgreSQL DB table? - DEV Community
INSERT 0 1
dmi@dmi-laptop:~$
Here's an updated script that adds a check to ensure that the number of inserted
records matches the number of records in the table:
#!/bin/bash
export DB_HOST="localhost"
export DB_PORT="5442"
export DB_NAME="postgres"
export DB_USER="myuser"
export PGPASSWORD='mypwd'
export DB_NAME="postgres"
export TABLE_NAME="your_table_name"
export RECORDS_FILE="my_records.txt"
# Count the number of records in the file
count_num_of_recs_in_the_file=$(wc -l < $RECORDS_FILE)
echo "count_num_of_recs_in_the_file: $count_num_of_recs_in_the_file"
# Count the number of records in the table before the insert
count_before=$(psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_NAME -c "SELECT COUNT
# Insert the records from the file
psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_NAME -c "\copy $TABLE_NAME FROM $REC
# Count the number of records in the table after the insert
count_after=$(psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_NAME -c "SELECT COUNT(
# Compare the counts to ensure that the correct number of records were inserted
if [ "$count_after" -eq $(( "$count_before" + "$count_num_of_recs_in_the_file" )) ];
echo "All records were inserted successfully."
else
echo "Error: Incorrect number of records inserted."
fi
This script adds two SELECT COUNT(*) queries to count the number of records in the
table before and after the insert. It then compares the counts to ensure that the
correct number of records were inserted. The expected number of records in the
table is calculated by adding the number of records in the file to the count before the
insert.
In this code, I've used a copy approach to populate the table with the records.
https://dev.to/dm8ry/how-to-write-bash-script-to-insert-the-records-from-file-myrecordstxt-into-postgresql-db-table-45cd 3/6
04/12/2024, 00:18 How to write bash script to insert the records from file my_records.txt into PostgreSQL DB table? - DEV Community
[email protected]
AWS PROMOTED
Innovation streaming live from AWS re:Invent
Join hosts from AWS and AWS Partners as they discuss the latest news, trends,
and strategies for driving success on the cloud. Streaming live from AWS
re:Invent in Las Vegas.
Register now
https://dev.to/dm8ry/how-to-write-bash-script-to-insert-the-records-from-file-myrecordstxt-into-postgresql-db-table-45cd 4/6
04/12/2024, 00:18 How to write bash script to insert the records from file my_records.txt into PostgreSQL DB table? - DEV Community
Top comments (0)
Code of Conduct • Report abuse
Speechmatics PROMOTED
The Fastest, Most Accurate API for Voice AI
Power your Conversational AI with the most accurate real-time speech-to-text
API. Available in 50 languages and at <1-second latency. Perfect for building
seamless and emotionally intelligent voice experiences.
Start building today 🛠️
Dmitry Romanoff
JOINED
18 de set. de 2022
More from Dmitry Romanoff
How to Set Up Prometheus Exporters for PostgreSQL and MongoDB in Kubernetes
https://dev.to/dm8ry/how-to-write-bash-script-to-insert-the-records-from-file-myrecordstxt-into-postgresql-db-table-45cd 5/6
04/12/2024, 00:18 How to write bash script to insert the records from file my_records.txt into PostgreSQL DB table? - DEV Community
#monitoring #devops #database #kubernetes
Deploying PostgreSQL, MySQL, and MongoDB on Kubernetes: A Step-by-Step Guide
#kubernetes #devops #database #productivity
Getting Started with MongoDB Compass: A GUI for MongoDB
#mongodb #monitoring #database #devops
Neon PROMOTED
Top 3 Features in Postgres 17
Learn about the top 3 features in the latest version of Postgres.
See Article
https://dev.to/dm8ry/how-to-write-bash-script-to-insert-the-records-from-file-myrecordstxt-into-postgresql-db-table-45cd 6/6
