Questions about apache-spark
Read more about apache-spark
python (65.1k questions)
javascript (44.2k questions)
reactjs (22.7k questions)
java (20.8k questions)
c# (17.4k questions)
html (16.3k questions)
r (13.7k questions)
android (12.9k questions)
Azure Synapse Serverless SQL Pools - how to optimize transformations using notebooks and load tables into ADLSG2
We use Synapse Notebooks to perform data transformations and load the data into fact and dimension tables within our ADLSG2 data lake. We are disappointed with the time it takes to load the transform...
Phil
Votes: 0
Answers: 0
Scala: SPARK UDAF for polynomial curve fitting in Scala, got " type (char[]) cannot be converted to the string type" error
I am trying to do Polynomial curve-fitting on a Spark data frame similar to the one below (Using SPARK version 2.4.0.7.1.5, Scala version 2.11.12 (OpenJDK 64-Bit Server VM, Java 1.8.0_232)).
I wrote ...
conge
Votes: 0
Answers: 1
Why did optimizing my Spark config slow it down?
I'm running Spark 2.2 (legacy codebase constraints) on an AWS EMR Cluster (version 5.8), and run a system of both Spark and Hadoop jobs on a cluster daily. I noticed that the configs that are submitte...
NateH06
Votes: 0
Answers: 1
Apache spark, is it possible to have Google guice as dependency injection technique
Is it possible to use Google guice as dependency injection provider for a Apache spark Java application?
I am able to achieve this if the execution is happening at the driver but no control over when ...
Vissu
Votes: 0
Answers: 1