Command Line File Downloader in Python
Last Updated :
24 Jan, 2021
Python is one of the most popular general-purpose programming languages with a wide range of use cases from general coding to complex fields like AI. One of the reasons for such popularity of python as a programming language is the availability of many built-in as well as third-party libraries and packages.
In this article, we are going to build a simple command-line file downloader, which you can download a file if you have the download link.
Approach:
We are going to download files over HTTP instead of FTP. Once we have made the request for the file, we will use the response as an input stream to write or save the file to the file system. While Downloading, details like download speed, time, and amount of file downloaded will be shown.
- Take the file URL via command line
- Open the file input or download stream over HTTP via requests.
- Download the file by writing the file in chunks from the download/input stream into the file system.
- Display the details
- Save the file.
Step-by-step implementation:
Step 1: Libraries required
Python3
import requests
import sys
import time
Step 2: a dictionary for detecting file size in proper units
While Downloading the file, parameters like speed, size, time will be shown. So let's see how to do this. Dictionary for getting file size and download speed unit depending on the amount downloaded takes sizes in bytes.
Python3
units = {
'B':{'size':1, 'speed':'B/s'},
'KB':{'size':1024, 'speed':'KB/s'},
'MB':{'size':1024*1024, 'speed':'MB/s'},
'GB':{'size':1024*1024*1024, 'speed':'GB/s'}
}
Function for checking units of the length of the file downloaded. Length is in bytes so corresponding units are checked.
Python3
def check_unit(length): # length in bytes
if length < units['KB']['size']:
return 'B'
elif length >= units['KB']['size'] and length <= units['MB']['size']:
return 'KB'
elif length >= units['MB']['size'] and length <= units['GB']['size']:
return 'MB'
elif length > units['GB']['size']:
return 'GB'
Step 3: Downloading file over HTTP
We will open a file stream over HTTP using requests and then save the data in chunks to the local file. Let's see how the code will look and then put it all together.
Python3
# Opening file stream
r = requests.get(link_to_file, stream = True)
# writing file data in chunks.
# for examples, A file of 10 MB written in
# chunk size of 8096 Bytes.
with open(file_name, 'wb') as f:
for chunk in r.iter_content(chunk_size):
f.write(chunk)
Step 4: Printing output
The output will consist of a dynamic progress bar with downloading details. For this we use stdout.write() and stdout.flush() methods
Python3
sys.stdout.write(format_string % (list_of_variables))
# for printing details in single
# line which updates without going to the next line
# once the details are updated.
sys.stdout.flush()
Below is the full implementation:
Python3
import requests
import sys
import time
units = {
'B' : {'size':1, 'speed':'B/s'},
'KB' : {'size':1024, 'speed':'KB/s'},
'MB' : {'size':1024*1024, 'speed':'MB/s'},
'GB' : {'size':1024*1024*1024, 'speed':'GB/s'}
}
def check_unit(length): # length in bytes
if length < units['KB']['size']:
return 'B'
elif length >= units['KB']['size'] and length <= units['MB']['size']:
return 'KB'
elif length >= units['MB']['size'] and length <= units['GB']['size']:
return 'MB'
elif length > units['GB']['size']:
return 'GB'
# takes download link and directory where file to be saved.
def downloadFile(url, directory) :
localFilename = url.split('/')[-1] # files name
with open(directory + '/' + localFilename, 'wb') as f:
print ("Downloading . . .\n")
start = time.time() # start time
r = requests.get(url, stream=True)
# total length in bytes of the file
total_length = float(r.headers.get('content-length'))
d = 0 # counter for amount downloaded
# when file is not available
if total_length is None:
f.write(r.content)
else:
for chunk in r.iter_content(8192):
d += float(len(chunk))
f.write(chunk) # writing the file in chunks of 8192 bytes
# amount downloaded in proper units
downloaded = d/units[check_unit(d)]['size']
# converting the unit of total length or size of file from bytes.
tl = total_length / units[check_unit(total_length)]['size']
trs = d // (time.time() - start) # speed in bytes per sec
#speed in proper unit
download_speed = trs/units[check_unit(trs)]['size']
speed_unit = units[check_unit(trs)]['speed'] # speed in proper units
done = 100 * d / total_length # percentage downloaded or done.
fmt_string = "\r%6.2f %s [%s%s] %7.2f%s / %4.2f %s %7.2f %s"
set_of_vars = ( float(done), '%',
'*' * int(done/2),
'_' * int(50-done/2),
downloaded, check_unit(d),
tl, check_unit(total_length),
download_speed, speed_unit)
sys.stdout.write(fmt_string % set_of_vars)
sys.stdout.flush()
return (time.time() - start) # total time taken for download
def main() :
directory = '.'
if len(sys.argv) > 1 :
url = sys.argv[1] # url from cmd line arg
if len(sys.argv) > 2:
directory = sys.argv[2]
total_time = downloadFile(url, directory)
print ('')
print ("Download complete...")
print ("\rTime Elapsed: %.2fs" % total_time)
else :
print("No link found!")
if __name__ == "__main__" :
main()
Save the code in a python file and use it as follows
python <program_name>.py <file_link> <save_location(by default '.')>
Output:
Similar Reads
Check if a File Exists in Python When working with files in Python, we often need to check if a file exists before performing any operations like reading or writing. by using some simple methods we can check if a file exists in Python without tackling any error. Using pathlib.Path.exists (Recommended Method)Starting with Python 3.4
3 min read
How Use Linux Command In Python Using System.Os Using the system module from the os library in Python allows you to interact with the Linux command line directly from your Python script. This module provides a way to execute system commands, enabling you to automate various tasks by integrating Linux commands seamlessly into your Python code. Whe
3 min read
Python Loop through Folders and Files in Directory File iteration is a crucial process of working with files in Python. The process of accessing and processing each item in any collection is called File iteration in Python, which involves looping through a folder and perform operation on each file. In this article, we will see how we can iterate ove
4 min read
Python Delete File When any large program is created, usually there are small files that we need to create to store some data that is needed for the large programs. when our program is completed, so we need to delete them. In this article, we will see how to delete a file in Python. Methods to Delete a File in Python
4 min read
Check If A File is Writable in Python When it comes to Python programming it is essential to work with files. One important aspect is making sure that a file can be written before you try to make any changes. In this article, we will see how we can check if a file is writable in Python. Check If a File Is Writable in PythonBelow are som
3 min read
Delete a directory or file using Python In this article, we will cover how to delete (remove) files and directories in Python. Python provides different methods and functions for removing files and directories. One can remove the file according to their need.Table of ContentUsing the os.remove() MethodDelete a FileRemove file with absolut
6 min read
Download Anything to Google Drive using Google colab When we download/upload something from a cloud server, it gives more transfer rate as compared to a normal server. We can use Google Drive for storage as well as fast speed download. The problem is how to upload something to G-Drive direct from Internet. So, Here we will see a solution to upload any
2 min read
Get the File Extension from a URL in Python Handling URLs in Python often involves extracting valuable information, such as file extensions, from the URL strings. However, this task requires careful consideration to ensure the safety and accuracy of the extracted data. In this article, we will explore four approaches to safely get the file ex
2 min read
Create a File Path with Variables in Python The task is to create a file path using variables in Python. Different methods we can use are string concatenation and os.path.join(), both of which allow us to build file paths dynamically and ensure compatibility across different platforms. For example, if you have a folder named Documents and a f
3 min read
Get File Size in Bytes, Kb, Mb, And Gb using Python Handling file sizes in Python is a common task, especially when working with data processing, file management, or simply understanding resource usage. Fortunately, Python provides several methods to obtain file sizes in various units, such as bytes, kilobytes, megabytes, and gigabytes. In this artic
2 min read