How do you convert a Spark RDD into a DataFrame?

Spark - Interview Questions

There are 2 ways to convert a Spark RDD into a DataFrame:

Using the helper function - toDF :

import com.mapr.db.spark.sql._

val df = sc.loadFromMapRDB(<table-name>)

.where(field(“first_name”) === “Peter”)

.select(“_id”, “first_name”).toDF()

Using SparkSession.createDataFrame :

You can convert an RDD[Row] to a DataFrame by

calling createDataFrame on a SparkSession object

def createDataFrame(RDD, schema:StructType)