04/12/2024, 00:18 How to write bash script to insert the records from file my_records.txt into PostgreSQL DB table?

reSQL DB table? - DEV Community

Dmitry Romanoff
Posted on 11 de abr. de 2023 • Edited on 14 de mai.

6 1 1

How to write bash script to insert the records

from file my_records.txt into PostgreSQL DB
table?
#script #bash #database #postgres

One | 1 | 2021-01-01 01:01:01

Two | 2 | 2022-02-02 02:02:02
Three | 3 | 2023-03-03 03:03:03
Four | 4 | 2024-04-04 04:04:04
Five | 5 | 2025-05-05 05:05:05

First step, create the table of the corresponding structure in the PostgreSQL DB.

postgres=# create table your_table_name("name" varchar(100), "number" smallint, "dat

CREATE TABLE

Here's a bash script to insert the records from a file called "my_records.txt" into a
PostgreSQL database table:

#!/bin/bash

# Set the database host, port, name, username and password

export DB_HOST="your_db_host"
export DB_PORT="your_db_port"
export DB_NAME="your_db_name"
export DB_USER="your_db_user"
export PGPASSWORD='your_db_password'

# Loop through the lines in the file

while read -r line
do
# Split the line by '|' delimiter
IFS='|' read -ra values <<< "$line"

https://dev.to/dm8ry/how-to-write-bash-script-to-insert-the-records-from-file-myrecordstxt-into-postgresql-db-table-45cd 1/6
04/12/2024, 00:18 How to write bash script to insert the records from file my_records.txt into PostgreSQL DB table? - DEV Community

# Extract the values

name="${values[0]}"
number="${values[1]}"
date="${values[2]}"

# Insert the values into the database

psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -c "INSERT INTO your_
done < "my_records.txt"

Note that you need to replace "your_db_host", "your_db_port", "your_db_name",

"your_db_user", "your_db_password", "your_table_name" with your own database
details and table name.

dmi@dmi-laptop:~$ cat populate_file_recs_into_pg_table.sh

#!/bin/bash

# Set the database host, port, name, username and password

export DB_HOST="localhost"
export DB_PORT="5442"
export DB_NAME="postgres"
export DB_USER="myuser"
export PGPASSWORD='mypwd'

# Loop through the lines in the file

while read -r line
do
# Split the line by '|' delimiter
IFS='|' read -ra values <<< "$line"

# Extract the values

name="${values[0]}"
number="${values[1]}"
date="${values[2]}"

# Insert the values into the database

psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -c "INSERT INTO your_
done < "my_records.txt"

dmi@dmi-laptop:~$ ./populate_file_recs_into_pg_table.sh
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1

https://dev.to/dm8ry/how-to-write-bash-script-to-insert-the-records-from-file-myrecordstxt-into-postgresql-db-table-45cd 2/6
04/12/2024, 00:18 How to write bash script to insert the records from file my_records.txt into PostgreSQL DB table? - DEV Community

INSERT 0 1
dmi@dmi-laptop:~$

Here's an updated script that adds a check to ensure that the number of inserted
records matches the number of records in the table:

#!/bin/bash

export DB_HOST="localhost"
export DB_PORT="5442"
export DB_NAME="postgres"
export DB_USER="myuser"
export PGPASSWORD='mypwd'
export DB_NAME="postgres"
export TABLE_NAME="your_table_name"
export RECORDS_FILE="my_records.txt"

# Count the number of records in the file

count_num_of_recs_in_the_file=$(wc -l < $RECORDS_FILE)
echo "count_num_of_recs_in_the_file: $count_num_of_recs_in_the_file"

# Count the number of records in the table before the insert

count_before=$(psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_NAME -c "SELECT COUNT

# Insert the records from the file

psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_NAME -c "\copy $TABLE_NAME FROM $REC

# Count the number of records in the table after the insert

count_after=$(psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_NAME -c "SELECT COUNT(

# Compare the counts to ensure that the correct number of records were inserted
if [ "$count_after" -eq $(( "$count_before" + "$count_num_of_recs_in_the_file" )) ];
echo "All records were inserted successfully."
else
echo "Error: Incorrect number of records inserted."
fi

This script adds two SELECT COUNT(*) queries to count the number of records in the
table before and after the insert. It then compares the counts to ensure that the
correct number of records were inserted. The expected number of records in the
table is calculated by adding the number of records in the file to the count before the
insert.

In this code, I've used a copy approach to populate the table with the records.

https://dev.to/dm8ry/how-to-write-bash-script-to-insert-the-records-from-file-myrecordstxt-into-postgresql-db-table-45cd 3/6
04/12/2024, 00:18 How to write bash script to insert the records from file my_records.txt into PostgreSQL DB table? - DEV Community

[email protected]

AWS PROMOTED

Innovation streaming live from AWS re:Invent

Join hosts from AWS and AWS Partners as they discuss the latest news, trends,
and strategies for driving success on the cloud. Streaming live from AWS
re:Invent in Las Vegas.

Register now

https://dev.to/dm8ry/how-to-write-bash-script-to-insert-the-records-from-file-myrecordstxt-into-postgresql-db-table-45cd 4/6
04/12/2024, 00:18 How to write bash script to insert the records from file my_records.txt into PostgreSQL DB table? - DEV Community

Top comments (0)

Code of Conduct • Report abuse

Speechmatics PROMOTED

The Fastest, Most Accurate API for Voice AI

Power your Conversational AI with the most accurate real-time speech-to-text

API. Available in 50 languages and at <1-second latency. Perfect for building
seamless and emotionally intelligent voice experiences.

Start building today 🛠️

Dmitry Romanoff
JOINED
18 de set. de 2022

More from Dmitry Romanoff

How to Set Up Prometheus Exporters for PostgreSQL and MongoDB in Kubernetes

https://dev.to/dm8ry/how-to-write-bash-script-to-insert-the-records-from-file-myrecordstxt-into-postgresql-db-table-45cd 5/6
04/12/2024, 00:18 How to write bash script to insert the records from file my_records.txt into PostgreSQL DB table? - DEV Community

#monitoring #devops #database #kubernetes

Deploying PostgreSQL, MySQL, and MongoDB on Kubernetes: A Step-by-Step Guide

#kubernetes #devops #database #productivity

Getting Started with MongoDB Compass: A GUI for MongoDB

#mongodb #monitoring #database #devops

Neon PROMOTED

Top 3 Features in Postgres 17

Learn about the top 3 features in the latest version of Postgres.

See Article

https://dev.to/dm8ry/how-to-write-bash-script-to-insert-the-records-from-file-myrecordstxt-into-postgresql-db-table-45cd 6/6