python (65.1k questions)

javascript (44.2k questions)

reactjs (22.7k questions)

java (20.8k questions)

c# (17.4k questions)

html (16.3k questions)

r (13.7k questions)

android (12.9k questions)

Questions - sparkr

What is the issue with my pagerank algorithm code?

df1 \<- tbl(sc ,"databricks.tablename") g \<- graph_from_data_frame(df1, directed = TRUE, vertices = NULL) # Directed Igraph creation is_weighted(g) #Check for Weight pagerank \<...
test-img

Anand Salimath

r

igraph

sparkr

sparklyr

pagerank

Votes: 0

Answers: 1

Latest Answer

The "power" method was already deprecated, and it has been removed in igraph 1.3.0, as is made clear in the changelog: https://cran.r-project.org/web/packages/igraph/news/news.html Use one o...
test-img

Szabolcs

Java Heap Space Error in SparklyR &amp; Tidymodels

Hi, I am trying to model a large dataset (2000000+ rows, ~150 columns) using the tidymodels and the spark engine. However, I am receiving the following error: java.lang.OutOfMemoryError: Java heap spa...
test-img

Enes

r

apache-spark

sparkr

sparklyr

tidymodels

Votes: 0

Answers: 0

How to drop all NA columns in a SparkDataFrame with SparkR?

Once again, I'm facing a problem that I can't transcribe under SparkR. I have a SparkDataFrame which some columns contain only NAs, and I want to delete all these columns. I discovered SparkR recently...
test-img

Discus23

r

azure

databricks

sparkr

Votes: 0

Answers: 1

Latest Answer

You can consider the following approach library(SparkR) df <- data.frame(V1 = base::sample(1 : 10,5), V2 = base::rep(NA,5), V3 = base::sample(1 : 10,5), ...
test-img

Emmanuel Hamel

How to add multiple calculated columns to a SparkDataFrame using SparkR?

Now I'm stuck on a rather basic case, but I can't find a clever solution with SparkR ... From N columns in my SparkDataFrame, I need to create N new calculated columns. df <- data.frame(V1 = base::...
test-img

Discus23

r

databricks

azure-databricks

sparkr

Votes: 0

Answers: 2

Latest Answer

Maybe you can consider the following approach library(SparkR) df <- data.frame(V1 = base::sample(1:10,5), V2 = base::sample(1:10,5), V3 = base::sample(1:10,5), ...
test-img

Emmanuel Hamel

Posts

Questions

Blogs

Jobs