How to Check if PySpark DataFrame is empty? Last Updated : 30 May, 2021 Comments Improve Suggest changes Like Article Like Report In this article, we are going to check if the Pyspark DataFrame or Dataset is Empty or Not. At first, let's create a dataframe Python3 # import modules from pyspark.sql import SparkSession from pyspark.sql.types import StructType, StructField, StringType # defining schema schema = StructType([ StructField('COUNTRY', StringType(), True), StructField('CITY', StringType(), True), StructField('CAPITAL', StringType(), True) ]) # Create Spark Object spark = SparkSession.builder.appName("TestApp").getOrCreate() # Create Empty DataFrame with Schema. df = spark.createDataFrame([], schema) # Show schema and data df.printSchema() df.show(truncate=False) Output: Checking dataframe is empty or not We have Multiple Ways by which we can Check : Method 1: isEmpty() The isEmpty function of the DataFrame or Dataset returns true when the DataFrame is empty and false when it’s not empty. If the dataframe is empty, invoking “isEmpty” might result in NullPointerException. Note : calling df.head() and df.first() on empty DataFrame returns java.util.NoSuchElementException: next on empty iterator exception. Python3 print(df.head(1).isEmpty) print(df.first(1).isEmpty) print(df.rdd.isEmpty()) Output: True True True Method 2: count() It calculates the count from all partitions from all nodes Code: Python3 print(df.count() > 0) print(df.count() == 0) False True Comment More infoAdvertise with us Next Article How to Check if PySpark DataFrame is empty? D deepanshu_rustagi Follow Improve Article Tags : Python Python-Pyspark Practice Tags : python Similar Reads How to check dataframe is empty in Scala? In this article, we will learn how to check dataframe is empty or not in Scala. we can check if a DataFrame is empty by using the isEmpty method or by checking the count of rows. Syntax: val isEmpty = dataframe.isEmpty OR, val isEmpty = dataframe.count() == 0 Here's how you can do it: Example #1: us 2 min read How To Check If Cell Is Empty In Pandas Dataframe An empty cell or missing value in the Pandas data frame is a cell that consists of no value, even a NaN or None. It is typically used to denote undefined or missing values in numerical arrays or DataFrames. Empty cells in a DataFrame can take several forms:NaN: Represents missing or undefined data.N 6 min read How to create an empty PySpark DataFrame ? In PySpark, an empty DataFrame is one that contains no data. You might need to create an empty DataFrame for various reasons such as setting up schemas for data processing or initializing structures for later appends. In this article, we’ll explore different ways to create an empty PySpark DataFrame 4 min read How to check the schema of PySpark DataFrame? In this article, we are going to check the schema of pyspark dataframe. We are going to use the below Dataframe for demonstration. Method 1: Using df.schema Schema is used to return the columns along with the type. Syntax: dataframe.schema Where, dataframe is the input dataframe Code: Python3 # impo 2 min read How to delete columns in PySpark dataframe ? In this article, we are going to delete columns in Pyspark dataframe. To do this we will be using the drop() function. This function can be used to remove values from the dataframe. Syntax: dataframe.drop('column name') Python code to create student dataframe with three columns: Python3 # importing 2 min read Like