Questions about delta-lake
Read more about delta-lake
python (65.1k questions)
javascript (44.2k questions)
reactjs (22.7k questions)
java (20.8k questions)
c# (17.4k questions)
html (16.3k questions)
r (13.7k questions)
android (12.9k questions)
Databricks Delta tables from json files: Ignore initial load when running COPY INTO
I am working with Databricks on AWS. I have mounted an S3 bucket as /mnt/bucket-name/. This bucket contains json files under the prefix jsons. I create a Delta table from these json files as follows:
...
dwolfeu
Votes: 0
Answers: 0
Reading Databricks tables in Azure
Please clarify my confusion as I keep hearing we need read every Parquet file created by Databricks Delta tables to get to latest data in case of a SCD2 table. Is this true?
Can we simply use SQL and ...
teknik
Votes: 0
Answers: 1
trying to use johnsnow pretrained pipeline on spark dataframe but unable to read delta file in the same session
i am using the below code to read the spark dataframe from hdfs:
from delta import *
from pyspark.sql import SparkSession
builder= SparkSession.builder.appName("MyApp") \
.config(&quo...
Siddhant Ghungrudkar
Votes: 0
Answers: 1
Append the "_commit_timestamp" Column to the Latest Data Version When Reading from a DeltaTable
I have data in an delta lake WITHOUT a timestamp on each row to determine when that row was added/modified, but I only need rows that were created/modified after a specified date/time.
I want the late...
SCP
Votes: 0
Answers: 1