How to check dataframe size in Scala? Last Updated : 27 Mar, 2024 Summarize Comments Improve Suggest changes Share Like Article Like Report In this article, we will learn how to check dataframe size in Scala. To check the size of a DataFrame in Scala, you can use the count() function, which returns the number of rows in the DataFrame. Here's how you can do it: Syntax: val size = dataframe.count() Example #1: Scala import org.apache.spark.sql.{DataFrame, SparkSession} object DataFrameSizeCheck { def main(args: Array[String]): Unit = { // Create SparkSession val spark = SparkSession.builder() .appName("DataFrameSizeCheck") .master("local[*]") .getOrCreate() // Sample DataFrame (replace this with your actual DataFrame) val dataframe: DataFrame = spark.emptyDataFrame // Get the size of DataFrame val size = dataframe.count() // Print the size println(s"DataFrame size: $size") // Stop SparkSession spark.stop() } } Output: DataFrame size: 0Explanation: We create a SparkSession.Define a sample DataFrame. You should replace this with your actual DataFrame.We use the count() function to get the size of the DataFrame, i.e., the number of rows it contains.Example #2: Scala import org.apache.spark.sql.{DataFrame, SparkSession} import org.apache.spark.sql.types.{IntegerType, StructField, StructType} object DataFrameSizeCheck { def main(args: Array[String]): Unit = { // Create SparkSession val spark = SparkSession.builder() .appName("DataFrameSizeCheck") .master("local[*]") .getOrCreate() // Sample data for DataFrame val data = Seq( (1, "John"), (2, "Alice"), (3, "Bob") ) // Define the schema val schema = StructType( Seq( StructField("ID", IntegerType, nullable = false), StructField("Name", StringType, nullable = false) ) ) // Create DataFrame val dataframe: DataFrame = spark.createDataFrame(spark.sparkContext.parallelize(data), schema) // Get the size of DataFrame val size = dataframe.count() // Print the size println(s"DataFrame size: $size") // Stop SparkSession spark.stop() } } Output: DataFrame size: 3Explanation: We create a SparkSession.Define some sample data in the form of tuples.Define the schema for the DataFrame, specifying the data types of each column.Create a DataFrame using the createDataFrame method and passing the sample data and schema.Use the count() function to get the size of the DataFrame.Print the size of the DataFrame.Finally, we stop the SparkSession. Comment More infoAdvertise with us Next Article How to Check the Schema of DataFrame in Scala? R raushanikuf9x7 Follow Improve Article Tags : Scala Similar Reads How to Check the Schema of DataFrame in Scala? With DataFrames in Apache Spark using Scala, you could check the schema of a DataFrame and get to know its structure with column types. The schema contains data types and names of columns that are available in a DataFrame. Apache Spark is a powerful distributed computing framework used for processin 3 min read How to check dataframe is empty in Scala? In this article, we will learn how to check dataframe is empty or not in Scala. we can check if a DataFrame is empty by using the isEmpty method or by checking the count of rows. Syntax: val isEmpty = dataframe.isEmpty OR, val isEmpty = dataframe.count() == 0 Here's how you can do it: Example #1: us 2 min read How to Check Datatype in Scala? In this article, we will learn to check data types in Scala. Data types in Scala represent the type of values that variables can hold, aiding in type safety and program correctness. Table of Content Checking Datatype in ScalaApproach 1: Use Pattern Matching in ScalaApproach 2: Use the getClass Metho 3 min read How to print dataframe in Scala? Scala stands for scalable language. It was developed in 2003 by Martin Odersky. It is an object-oriented language that provides support for functional programming approach as well. Everything in scala is an object e.g. - values like 1,2 can invoke functions like toString(). Scala is a statically typ 4 min read How to Join Two DataFrame in Scala? Scala stands for scalable language. It is a statically typed language although unlike other statically typed languages like C, C++, or Java, it doesn't require type information while writing the code. The type verification is done at the compile time. Static typing allows us to build safe systems by 4 min read How to create an empty dataframe in Scala? In this article, we will learn how to create an empty dataframe in Scala. We can create an empty dataframe in Scala by using the createDataFrame method provided by the SparkSession object. Syntax to create an empty DataFrame: val df = spark.emptyDataFrame Example of How to create an empty dataframe 2 min read Like