Add YearlyPartitionsDefinition

[mkleinbort-ic]

What's the use case?

We have hourly, daily, weekly and monthly partition definitions... it feels natural to add yearly as well (I was looking for it today and could not find it).

The alternative is for users to use StaticPartitionDefinition (or Dynamic) which looses some of the benefits of a time-based partition.

Ideas of implementation

No response

Additional information

This is useful for large scale "backfills", often partitioned by entities in the context of a MultiPartition.

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

[ion-elgreco]

While at it, let's also have QuarterlyPartitionDefinition

[mmutso-boku]

The alternative is for users to use StaticPartitionDefinition (or Dynamic) which looses some of the benefits of a time-based partition.

I think the better alternative is to use TimeWindowPartitionsDefinition and specify the cron_schedule parameter as needed, for example:

quarterly_partitions_def = TimeWindowPartitionsDefinition(
    start=datetime(2015, 1, 1),
    cron_schedule="0 0 1 */3 *",
    fmt="%Y-%m")

half_year_partitions_def = TimeWindowPartitionsDefinition(
    start=datetime(2015, 1, 1),
    cron_schedule="0 0 1 */6 *",
    fmt="%Y-%m")

yearly_partitions_def = TimeWindowPartitionsDefinition(
    start=datetime(2015, 1, 1),
    cron_schedule="0 0 1 1 *",
    fmt="%Y")

At least that's what I've been using so far, and seems to have worked fine.