How to Upload File to Google Drive using Python Script?

Objective For ‘How To Upload File To Google Drive Using Python Script’

This step-by-step recipe will show you how to upload files to Google Drive Python.

Why Do Data Engineers Extract From/Upload To Google Drive Python?

When working on big data projects, extracting data from Google Drive into orchestration workflows to initially store the data in data lakes, followed by a series of operations like data validation, cleansing, and transformation, is widely used to gather business insights from the data. In addition to extracting data from Google Drive, data engineers also upload file to Drive Python scripts. This is useful for storing data in a central location where other users can easily access it. Python can also be used to automate the process of uploading data to Google Drive, saving data engineers a lot of time.

Steps For Uploading Files on Google Drive Using Python

In the following Python Sample Code, we will upload files to Google Drive using Python and use them in data flow orchestration processes.

Pre-Requisites For Google Drive Upload Python

• Install the pydrive python module as follows: pip install pydrive

• The below codes can be run in Jupyter notebook or any python console

Get Closer To Your Dream of Becoming a Data Scientist with 70+ Solved End-to-End ML Projects

Step 1: Import the Libraries

from pydrive.auth import GoogleAuth from pydrive.drive import GoogleDrive

Step 2: OAuth Made Easy

Follow the steps to Get Authentication for Google Service API in the below link: Get Authentication for Google Service API

Download client_secrets.json from Google API Console and OAuth2.0 is done in two lines. You can customize the behavior of OAuth2 in one settings file settings.yaml

gauth = GoogleAuth() drive = GoogleDrive(gauth)

Above steps together as follows :

from pydrive.auth import GoogleAuth from pydrive.drive import GoogleDrive gauth = GoogleAuth() drive = GoogleDrive(gauth)

Step 3: Upload files to your Google Drive

upload_file_list = ['1.jpg', '2.jpg'] for upload_file in upload_file_list: gfile = drive.CreateFile({'parents': [{'id': '1pzschX3uMbxU0lB5WZ6IlEEeAUE8MZ-t'}]}) # Read file and set it as the content of this instance. gfile.SetContentFile(upload_file) gfile.Upload() # Upload the file.

The output of the above code:

• The above code uploads my two local files, 1.jpg and 2.jpg, to my Google Drive folder test/. To do that, the pydrive library will create two files in Google Drive and then read and upload the two files to the corresponding folder.

• Note that we need to provide the id of the corresponding Google Drive folder. In this example, the test folder's ID is 1pzschX3uMbxU0lB5WZ6IlEEeAUE8MZ-t. You can get the Google Drive folder ID from the browser.

• For example: when we open the test folder in my Google Drive, the browser shows the address as https://drive.google.com/drive/folders/1cIMiqUDUNldxO6Nl-KVuS9SV-cWi9WLi. Then the corresponding ID for the test folder is the part after the last \ symbol, which is 1cIMiqUDUNldxO6Nl-KVuS9SV-cWi9WLi.

Step 4: List out files from Google Drive

We can also list all files from the specific folder in google drive as follows :

file_list = drive.ListFile({'q': "'{}' in parents and trashed=false".format('1cIMiqUDUNldxO6Nl-KVuS9SV-cWi9WLi')}).GetList() for file in file_list: print('title: %s, id: %s' % (file['title'], file['id']))

The output of the above code:

Step 5: Download the files from Google Drive

We can also download the files from Google Drive as follows. Note - after listing the files only we can download the file.

for i, file in enumerate(sorted(file_list, key = lambda x: x['title']), start=1): print('Downloading {} file from GDrive ({}/{})'.format(file['title'], i, len(file_list))) file.GetContentFile(file['title'])

The output of the above code:

In the above snapshot files are downloaded from the specific folder, Note here files will download where the code will be executed.

Step 6: Create the Text files in Google Drive

We can also write files directly to Google Drive using the following code:

# Create a GoogleDriveFile instance with title 'test.txt'. file1 = drive.CreateFile({'parents': [{'id': '1cIMiqUDUNldxO6Nl-KVuS9SV-cWi9WLi'}],'title': 'test.txt'}) # Set content of the file from the given string. file1.SetContentString('Hello World!') file1.Upload()

The output of the above code: test.txt file is created in google drive.

Explore More Data Science and Machine Learning Projects for Practice. Fast-Track Your Career Transition with ProjectPro

Step 7: Read the content of the text file directly from Google Drive

Also, we can read the file directly from Google Drive using the below code:

file2 = drive.CreateFile({'id': file1['id']}) file2.GetContentString('test.txt')

The output of the above code:

In the above snapshot reading, the file's content is "Hello world."

How Does Python write to Google Drive?

The first step is to install the pydrive library using the following command:

pip install pydrive

Once the pydrive library is installed, you can use it to write to Google Drive. The following code shows how to do this:

import pydrive

# Create a PyDrive client

client = pydrive.auth.GoogleAuth()

client.authenticate()

# Get the file to write to

file_path = '/path/to/file.txt'

# Create a Google Drive file object

file = pydrive.DriveFile(file_path)

# Open the file in write mode

with open(file_path, 'w') as f:

  f.write('This is a test.')

# Save the file to Google Drive

file.update()

How Does Python Send File to Google Drive?

Once you have successfully installed the pydrive library, you can use it to send a file to Google Drive, as shown in the following code:

import pydrive

# Create a PyDrive client

client = pydrive.auth.GoogleAuth()

client.authenticate()

# Get the file to upload

file_path = '/path/to/file.txt'

# Create a Google Drive file object

file = pydrive.DriveFile(file_path)

# Upload the file to Google Drive

file.upload()

How Does Python Save to Google Drive?

You can save a file to Python Google Drive using the google-cloud-storage library. The google-cloud-storage library is a Python library that provides a high-level API for Google Cloud Storage. To use the google-cloud-storage library, you first need to install it using the following command:

pip install google-cloud-storage

Once the google-cloud-storage library is installed, you can use it to save a file to Google Drive with the following code:

from google.cloud import storage

# Create a Google Cloud Storage client

client = storage.Client()

# Get the file to save

file_path = '/path/to/file.txt'

# Create a bucket

bucket = client.bucket('my-bucket')

# Create a blob

blob = bucket.blob('my_file.txt')

# Open the file in binary mode

with open(file_path, 'rb') as f:

  blob.upload_from_file(f)