python (65.1k questions)

javascript (44.2k questions)

reactjs (22.7k questions)

java (20.8k questions)

c# (17.4k questions)

html (16.3k questions)

r (13.7k questions)

android (12.9k questions)

Questions - scd2

What should happen to a SCD 2 table record if the record becomes inactive?

Let's say we have a table that has these values: id| name| country| start_dt| end_dt| is_current| current_dt 1001| John| CA| 2001-01-10| 2012-06-01| TRUE| 2012-06-01 Next day, the same record comes f...
test-img

Parijat Bose

sql

etl

scd

scd2

Votes: 0

Answers: 1

Latest Answer

You leave the last version as IS_CURRENT=TRUE. The last-known-good version of an SCD row is sometimes used instead of the time-sensitive version. So you can write select d.Name, sum(f.someval) totalV...
test-img

David Browne - Microsoft

Python nan and None

I am trying to apply the SCD2 with Python. The source files are .xlsx files and will be loaded into PostgreSQL. I have applied the condition where I identify the changed rows in the merged dataframe b...
test-img

magji

python

pandas

scd2

Votes: 0

Answers: 1

Latest Answer

If think nan is a string and not NaN (see the case). Try: Source = ftp_invoices.replace('nan', None) Example: df = pd.DataFrame({'A': [None, np.NaN, 'nan']}) print(df) # Output A 0 None # &l...
test-img

Corralien

Logic to find out deltas/CDC records for capturing Just Inserts or Updates in Bigquery

I have some raw data sitting in Big-query tables which are truncate load and my daily ETL feed which runs on these raw BQ tables is a daily snapshot of agents which are the daily extracts below . To g...
test-img

cheapcoder

google-bigquery

insert-update

cdc

delta

scd2

Votes: 0

Answers: 1

Latest Answer

A couple comments on your question first: Daily extract on 2022-03-03 shows row_eff_ts as "2022-03-02" for Mark. I'm assuming this is a typo and should be "2022-03-03". I think yo...
test-img

Jeff

SQL Server: Slowly Changing Dimension Type 2 on historical records

I am trying to set up a SCD of Type 2 for historical records within my Customer table. Attached is how the Customer table is set up alongside the expected outcome. Note that the Customer table in prac...
test-img

Aiden Aslam

sql-server

scd

scd2

Votes: 0

Answers: 1

Latest Answer

I think a three step query is likely needed. Use LEAD and LAG, partitioned by Customer and ordered by date, to peek at the next row's values for both Date and Lifecycle. Use a CASE statement to emit ...
test-img

Daniel

Posts

Questions

Blogs

Jobs