Questions about pyspark
Read more about pyspark
python (65.1k questions)
javascript (44.2k questions)
reactjs (22.7k questions)
java (20.8k questions)
c# (17.4k questions)
html (16.3k questions)
r (13.7k questions)
android (12.9k questions)
dynamically create a grand total row in pandas/pyspark?
I currently have a pandas dataframe that looks like this:
location
count
qty
approved_count
Phoenix
24
300
15
Dallas
18
403
14
I would like to append a row to the dataframe that iterates...
Alex Triece
Votes: 0
Answers: 2
use pyspark row as dataframe header
I have a pyspark data frame with just 2 records. Out of these 2 records, I have to extract latest record and use that as the header for another dataframe
ex:
df1
col_1 col_2 col_2
em_name mg_name 2020...
dragonachu
Votes: 0
Answers: 1
SparkSQL query using "PARTITION by" giving wrong output
I have a bunch of csv files for which I am using Pyspark for faster processing. However, am a total noob with Spark (Pyspark). So far I have been able to create a RDD, a subsequent data frame and a te...
Pankaj
Votes: 0
Answers: 0
Exception: Unable to find py4j in /content/spark-3.2.1-bin-hadoop3.2/python, your SPARK_HOME may not be configured correctly
While setting up PySpark to run in Google Colab, I got this error:
Exception: Unable to find py4j in /content/spark-3.2.1-bin-hadoop3.2/python, your SPARK_HOME may not be configured correctly.
Above t...
BeatrizGomes
Votes: 0
Answers: 2