How to parse nested JSON using Scala Spark? Last Updated : 24 Apr, 2024 Summarize Comments Improve Suggest changes Share Like Article Like Report In this article, we will learn how to parse nested JSON using Scala Spark. To parse nested JSON using Scala Spark, you can follow these steps:Define the schema for your JSON data.Read the JSON data into a Datc aFrame.Select and manipulate the DataFrame columns to work with the nested structure.Scala Spark Program to parse nested JSON: Scala //Scala Spark Program to parse nested JSON : import org.apache.spark.sql.SparkSession import org.apache.spark.sql.functions // Step 1: Define the schema val schema = """ { "type": "struct", "fields": [ {"name": "id", "type": "integer", "nullable": false}, {"name": "name", "type": "string", "nullable": true}, {"name": "details", "type": { "type": "struct", "fields": [ {"name": "age", "type": "integer", "nullable": true}, {"name": "city", "type": "string", "nullable": true} ] }, "nullable": true } ] } """ // Step 2: Create SparkSession val spark = SparkSession.builder() .appName("Nested JSON Parsing") .master("local[*]") .getOrCreate() // Step 3: Read JSON data into DataFrame val df = spark.read.schema(schema).json("path_to_your_json_file") // Step 4: Select and manipulate DataFrame columns val parsedDF = df.select( col("id"), col("name"), col("details.age").as("age"), col("details.city").as("city") ) // Step 5: Show the result parsedDF.show() Output: +---+------+---+-----+| id| name|age| city|+---+------+---+-----+| 1| Alice| 30|Paris|| 2| Bob| 25|New York|| 3| Carol| 35|London| Comment More infoAdvertise with us Next Article How to Reverse keys and values in Scala Map R raushanikuf9x7 Follow Improve Article Tags : Scala Similar Reads Pyspark - Parse a Column of JSON Strings In this article, we are going to discuss how to parse a column of json strings into their own separate columns. Here we will parse or read json string present in a csv file and convert it into multiple dataframe columns using Python Pyspark. Example 1: Parse a Column of JSON Strings Using pyspark.sq 4 min read How to Create SQLContext in Spark Using Scala? Scala stands for scalable language. It was developed in 2003 by Martin Odersky. It is an object-oriented language that provides support for functional programming approach as well. Everything in Scala is an object. It is a statically typed language although unlike other statically typed languages li 5 min read How to Reverse keys and values in Scala Map In Scala, Map is same as dictionary which holds key:value pairs. In this article, we will learn how to reverse the keys and values in given Map in Scala. After reversing the pairs, keys will become values and values will become keys. We can reverse the keys and values of a map with a Scala for-compr 2 min read How to Reverse keys and values in Scala Map In Scala, Map is same as dictionary which holds key:value pairs. In this article, we will learn how to reverse the keys and values in given Map in Scala. After reversing the pairs, keys will become values and values will become keys. We can reverse the keys and values of a map with a Scala for-compr 2 min read Download and Parse JSON Using R In this article, we are going to learn how to download and parse JSON using the R programming language. JavaScript Object Notation is referred to as JSON. These files have the data in text form, which is readable by humans. The JSON files are open for reading and writing just like any other file. Th 4 min read How to Print RDD in scala? Scala stands for scalable language. It was developed in 2003 by Martin Odersky. It is an object-oriented language that provides support for functional programming approach as well. Everything in scala is an object e.g. - values like 1,2 can invoke functions like toString(). Scala is a statically typ 4 min read Like