As I got to know from Airflow documentation that Airflow runs jobs at the end of an interval, not the beginning. This means that the first run of your job is going to be after the first interval. However, My requirement is to run the job at the start of interval itself.

Now as per the recommendation from various documentation/public forums, I have tried to set start date “one day before the actual start date (For testing purpose, I have also tried by setting hardcoded start date to some past date)” and I am setting the “catchup=False” in python Dag definition.

And I have observed that, for the first-time dag is running for one extra job occurrence

Example: - On 2022-04-05 18:53 UTC (YYYY-MM-DD) I have created one DAG and scheduled it to run at 19:00UTC everyday with start date 2022-04-04 00:00:00 (one day before) and it had trigger first scheduled job just after it got rendered on AWS MWAA UI, i.e exactly at 2022-04-05 18:57 UTC (catchup=False was set) and job got trigger again at 19:00UTC (which was expected).

Hence it is looks like "catchup=False" is not working properly.

Please help me understanding the mistake here. Thanks.!!

DAG Code:-

import time
from airflow import DAG
from airflow.operators.python_operator import PythonOperator
from datetime import datetime

DEFAULT_ARGS = {
    'owner': 'airflow',
    'depends_on_past': True
}
def add_sleep(**kwargs):
    print("operator called")
    time.sleep(5)

with DAG(
        dag_id="airflow_scheduler_poc",
        description='Airflow scheduler Test',
        start_date=datetime.strptime("2022-04-04 00:00:00",'%Y-%m-%d %H:%M:%S'),
        end_date= None,
        schedule_interval="00 19 * * *",
        is_paused_upon_creation = False,
        catchup=False,
        tags=['emr'],
) as dag:

    job_flow_1 = PythonOperator(task_id="job_flow_1", python_callable=add_sleep, op_kwargs={})
    job_flow_2 = PythonOperator(task_id="job_flow_2", python_callable=add_sleep, op_kwargs={})
                                            
    job_flow_1 >> job_flow_2