Skip to content

Fix start_date, end_date dag level uses - #495

Merged
pankajastro merged 4 commits into
mainfrom
fix_start_date
Jul 25, 2025
Merged

Fix start_date, end_date dag level uses#495
pankajastro merged 4 commits into
mainfrom
fix_start_date

Conversation

@pankajastro

@pankajastro pankajastro commented Jul 24, 2025

Copy link
Copy Markdown
Contributor

closes: #286

This PR addresses issues related to the usage of start_date and end_date at the DAG level.

Before this PR:

  • Setting start_date in default_args had no effect
  • Defining start_date or end_date directly as DAG parameters could result in errors during DAG initialisation.

After this PR:

  • start_date and end_date passed via default_args are now properly recognized and applied at the DAG level.
  • Errors related to setting these parameters directly on the DAG have been resolved.
my_dag:
      schedule: "0 3 * * *"
      start_date: 2024-11-11
      end_date: 2025-11-11
      tasks:
        task_1:
          operator: airflow.operators.bash.BashOperator
          bash_command: "echo 1"

@pankajastro
pankajastro marked this pull request as ready for review July 24, 2025 15:59
@pankajastro
pankajastro requested a review from a team as a code owner July 24, 2025 15:59
pankajastro
pankajastro commented Jul 24, 2025
View reviewed changes
Comment thread tests/test_dagbuilder.py
@codecov-commenter

codecov-commenter commented Jul 24, 2025 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.81%. Comparing base (5ed0b48) to head (76746c1).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #495      +/-   ##
==========================================
+ Coverage   93.69%   93.81%   +0.12%     
==========================================
  Files          11       11              
  Lines        1078     1083       +5     
==========================================
+ Hits         1010     1016       +6     
+ Misses         68       67       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

tatiana
tatiana approved these changes Jul 25, 2025
View reviewed changes

@tatiana tatiana left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, @pankajastro , thank you - I faced this issue in the past, and it is a great improvement.

@pankajastro
pankajastro merged commit 33589d2 into main Jul 25, 2025
63 checks passed
@pankajastro
pankajastro deleted the fix_start_date branch July 25, 2025 11:18
@pankajastro pankajastro mentioned this pull request Jul 29, 2025
Merged
tatiana pushed a commit that referenced this pull request Sep 3, 2025
@pankajastro
Release 1.0.0 🎉 (#521)
0e34e4c 
### Breaking Changes

- Airflow providers are now optional dependencies by @pankajastro in
[#486](#486)
- Previously, `dag-factory` enforced the installation of
`apache-airflow-providers-http` and
`apache-airflow-providers-cncf-kubernetes`. These Airflow providers
dependencies are now optional. If your DAGs depend on these providers,
you must install them manually. Alternatively, you can install
`dag-factory` with extras like `dag-factory[all]`,
`dag-factory[kubernetes]`, etc.
- Removed `clean_dags` function by @pankajastro in
[#498](#498)
- You no longer need to call `example_dag_factory.clean_dags(globals())`
in your DAG files. DAG cleanup is now controlled via the Airflow config
setting `AIRFLOW__DAG_PROCESSOR__REFRESH_INTERVAL`.
- Remove `schedule_interval` parameter from DAG configuration YAML by
@viiccwen in [#503](#503)
  - Use `schedule` parameter instead of `schedule_interval`.
- Change `DagFactory` class access to private by @pankajastro in
[#509](#509)
- The import path `from dagfactory import DagFactory` has been removed.
  - The class `DagFactory` has been renamed to `_DagFactory`.
- The `generate_dags` method of `DagFactory` has been renamed to
`_generate_dags`.
- Remove Inconsistent Parameters for Airflow Consistent by @pankajastro
in [#512](#512)
  - Removed `dagrun_timeout_sec` from dag param.
  - Removed `retry_delay_sec`, `sla_secs` from default_args.
  - Removed accepting `execution_timeout` as integer.
- Removed `execution_timeout_secs`, `sla_secs` and
`execution_delta_secs` from task param.
- Remove custom parsing for Kubernetes object and refactor KPO to use
`__type__` syntax by @pankajastro in
[#523](#523)
- The custom parsing for Kubernetes objects has been removed. You can no
longer pass a custom YAML dictionary to DAG-Factory configuration unless
accepted by the KubernetesPodOperator. We suggest you to use `__type__`
syntax to supply Kubernetes object in your YAML DAG. For an example KPO
configuration, visit: [KubernetesPodOperator
Documentation](https://astronomer.github.io/dag-factory/dev/features/kpo/).
- Consolidate `!and`, `!or`, `!join`, `and` and `or` key in YAML DAG
configuration by @pankajastro in
[#525](#525)
  - Use `__and__`, `__or__` and `join__` instead
- Remove custom parsing for DAG parameter `timetable` by @pankajastro in
[#533](#533)
  - Use the `__type__` annotation for the `timetable` parameter.
- Rename parameter of `load_yaml_dags` and `_DagFactory` to reflect
behaviour by @pankajastro in
[#546](#546)
  - Rename `config` to `config_dict`
  - Rename `default_args_config_path` to `defaults_config_path`
  - Rename `default_args_config_dict`  to `defaults_config_dict`

### Added

- Support dag-level arguments in global defaults by @gyli in
[#480](#480)
- Support `*args` in custom Python object by @pankajastro in
[#484](#484)
- Support tasks and task_groups as lists by @pankajkoti in
[#487](#487)
- Support overriding `defaults.yml` based on the directory hierarchy by
@tatiana in [#500](#500)
- Introduced DAG Factory CLI by @tatiana in
[#510](#510)
- Added `lint` command to CLI by @tatiana in
[#513](#513)
- Add convert CLI command to migrate from af2 to af3 by @tatiana in
[539](#539)
- Allow yaml/yml suffix for shared defaults by @pankajastro in
[#567](#567)

### Fixed

- Fix the Airflow version condition check to parse inlets/outlets syntax
according to the dataset by @pankajastro in
[#485](#485)
- Ensure `dag_params` contain `schedule` before operating on it by
@pankajkoti in
[#488](#488)
- Fix `start_date`, `end_date` at the DAG level by @pankajastro in
[#495](#495)
- Allow `execution_timeout` in `default_args` by @pankajastro in
[#501](#501)
- Capture Telemetry DNS gaierror and handle it gracefully by @tatiana in
[#544](#544)
- Fix require response_check param in http sensor by @pankajastri in
[#576](#576)
- Add none check before evaluating lambda func in HttpSensor
@pankajastro in
[#577](#577)

### Docs

- Restore basic DAG example by @pankajastro in
[#483](#483)
- Replace the usages in example dags, tests and docs for tasks and
taskgroups to be list by @pankajkoti in
[#492](#492)
- Update default documentation based on #500 by @tatiana in
[#504](#504)
- Add more examples for Custom Python object by @pankajastro in
[#506](#506)
- Add documentation for DAG Factory CLI by @tatiana in
[#511](#511)
- Add documentation and example YAMLs for task and task_group
configuration formats by @pankajkoti in
[#530](#530)
- Add migration guide docs by @pankajastro in
[#532](#532)
- Docs: Fix rendering of note block by @pankajastro in
[#537](#537)
- Document the Asset example DAG by @pankajastro in
[#538](#538)
- Add docs for the CLI convert command by @tatiana in
[#541](#541)
- Add remaining breaking changes in migration guide by @pankajastro in
[#549](#549)
- Fix typos in scheduling and datasets docs by @viiccwen in
[#565](#565)
- docs: Make markdownlint happy (fix MD007 ul-indent in dev/README.md)
by @viiccwen in
[#566](#566)
- Update Index.md by @pankajastro in
[#570](#570)
- Document dag-factory reserved keys by @pankajastro in
[#571](#571)
- Add an introduction to the migration guide by @pankajastro in
[#572](#572)

### Other Changes

- Improve unit tests to disregard `$AIRFLOW_HOME` by @tatiana in
[#490](#490)
- Resolve unpinned action reference error alerts raised by Zizmor by
@pankajkoti in
[#493](#493)
- Resolve 'credential persistence through GitHub Actions artifacts'
warnings from Zizmor by @pankajkoti in
[#494](#494)
- Resolve 'overly broad permissions' warnings from Zizmor by @pankajkoti
in [#496](#496)
- CI: Add GitHub CodeQL analysis workflow (`codeql.yml`) by @pankajkoti
in [#497](#497)
- Fix deploy pages job missing credentials by @pankajkoti in
[#499](#499)
- Add the breaking changes to changelog by @pankajastro in
[#502](#502)
- Add pre-commit to update `uv.lock` by @pankajastro in
[#514](#514)
- Remove `clean_dags` usage from object storage DAG by @pankajastro in
[#515](#515)
- Remove broad exceptions and catch more specific exceptions by
@pankajastro in
[#519](#519)
- Add missing env in contributing doc by @pankajastro in
[#522](#522)
- Enhance PyPI Stats API error handling by @viiccwen in
[#535](#535)
- Update example to be Airflow 3 compatible by @tatiana in
[540](#540)
- Remove AUTO_CONVERT_TO_AF3 from tests by @tatiana in
[#543](#543)
- Bump actions/download-artifact from 4 to 5 by @dependabot in
[#548](#548),
[#558](#558),
[#559](#559),
[#560](#560),
[#562](#562),
[#653](#563) and
[#575](#575)
- Update pyproject.toml to Sync Test Versions with CI/CD Pipeline by
@viiccwen in [#553](#553)
- Fix file URI format in ObjectStoragePath example to prevent duplicate
slashes by @viiccwen in
[#556](#556)
- CI: Only build docs on PR push event and deploy for merge and release
by @pankajastro in
[#568](#568)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tatiana tatiana tatiana approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[bug] Unable to define start_date in the per DAG or task without using default_args

3 participants

@pankajastro @codecov-commenter @tatiana