1 year ago

#298453

test-img

RemovedQuasar

SparkSession doesen't read database despite correct configuration

I'm trying running this simple function using Spark 3.0.0 using Zeppelin 0.9.0:

def getSession(uri: String) : SparkSession = {
return SparkSession
      .builder()
      .master("local[*]")
      .appName("TEST")
      .config( "spark.mongodb.input.database", uri)
      .config("spark.driver.extraJavaOptions", "-Duser.timezone=Europe/Rome")
      .config("spark.executor.extraJavaOptions", "-Duser.timezone=Europe/Rome")
      .getOrCreate()}

Despite the configuration it's correct, i receive this error:

java.lang.IllegalArgumentException: Missing database name. Set via the 'spark.mongodb.input.uri' or 'spark.mongodb.input.database' property

I really don't have idea why this is not working. Before a Zeppelin update, this function worked correctly. We have updated Zeppelin to 0.9.0 from 0.9.0preview but i don't know hoe this can influence this. Someone have any idea?

apache-spark

session

apache-spark-sql

apache-zeppelin

0 Answers

Your Answer

Accepted video resources