Questions about scala
Read more about scala
python (65.1k questions)
javascript (44.2k questions)
reactjs (22.7k questions)
java (20.8k questions)
c# (17.4k questions)
html (16.3k questions)
r (13.7k questions)
android (12.9k questions)
Scala: SPARK UDAF for polynomial curve fitting in Scala, got " type (char[]) cannot be converted to the string type" error
I am trying to do Polynomial curve-fitting on a Spark data frame similar to the one below (Using SPARK version 2.4.0.7.1.5, Scala version 2.11.12 (OpenJDK 64-Bit Server VM, Java 1.8.0_232)).
I wrote ...
conge
Votes: 0
Answers: 1
Why did optimizing my Spark config slow it down?
I'm running Spark 2.2 (legacy codebase constraints) on an AWS EMR Cluster (version 5.8), and run a system of both Spark and Hadoop jobs on a cluster daily. I noticed that the configs that are submitte...
NateH06
Votes: 0
Answers: 1
parquet4s not returning all records
I have a simple Scala application that uses parquet4s with fs2 to read a set of partitioned records (spread across directories, generated by a Spark job).
When I run the app, it only returns a fractio...
samirbajaj
Votes: 0
Answers: 1
HDFS file system, get latest folders using scala API
Our application reads data from several HDFS data folders, folders get updated weekly/daily/monthly so based on the updated period we need to find the latest path and then read the data.
We would like...
DataNoob
Votes: 0
Answers: 1