add custom granularities to mf timespine #6145
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
github-actions
bot
added
content
| time_spine: | ||
| standard_granularity_column: date_day | ||
| custom_granularities: | ||
| - name: fiscal_year |
There was a problem hiding this comment.
presume this is case sensitive, right?
There was a problem hiding this comment.
yea it should be lowercase
There was a problem hiding this comment.
Spec looks good but linking back to the source of truth just to be safe: dbt-labs/dbt-semantic-interfaces#338 (comment)
|
|
||
| To use a custom calendar model, ensure it meets the following requirements: | ||
|
|
||
| - The model must be at a daily grain and joinable to other date columns using `date_day`. |
There was a problem hiding this comment.
I don't think this is a requirement, you could use a dim date table at an hourly grain and define custom granularities for that model.
There was a problem hiding this comment.
++ you can use any of our standard granularities!
|
|
||
| - The model must be at a daily grain and joinable to other date columns using `date_day`. | ||
| - Columns must align with expected data types for a given granularity (for example, `quarter` should be a date). | ||
| - Support period-over-period calculations. |
There was a problem hiding this comment.
This isn't a requirement of the model, it's a requirement of for the functionality we support. We can remove it.
| - The model must be at a daily grain and joinable to other date columns using `date_day`. | ||
| - Columns must align with expected data types for a given granularity (for example, `quarter` should be a date). | ||
| - Support period-over-period calculations. | ||
| - Support cumulative metric windows using custom granularities (for example, cumulative fiscal year-to-date bookings). |
There was a problem hiding this comment.
Same comment as above, this is a requirement for us, not the underlying model
| ```yaml | ||
| models: | ||
| - name: my_time_spine | ||
| description: "my favorite time spine" |
There was a problem hiding this comment.
We don't need to wrap this in quotes
|
|
||
| #### Configuring time-spine | ||
| ### Configuring time-spine |
There was a problem hiding this comment.
Is there a reason we use "time-spine" instead of just "time spine"? Curious if @Jstein77 has an opinion on this
There was a problem hiding this comment.
i was originally using time spine but then saw it used with a dash and thought/assumed i had a wrong. happy with time spine too! @Jstein77 are we good with time spine?
|
|
||
| #### Understanding time-spine and granularity | ||
| The previous configuration demonstrates a time-spine model called `time_spine_daily`. It sets the time spine configurations under the `time_spine` key. The `standard_granularity_column` is the lowest grain of the table, in this case, it's hourly. It needs to reference a column defined under the columns key, in this case, `date_hour`. Use the `standard_granularity_column` as the join key for the time spine table when joining tables in MetricFlow. Here, the granularity of the `standard_granularity_column` is set at the column level, in this case, `hour`. Additionally, the `custom_granularities` field lets you specify non-standard time periods like `fiscal_year` or `retail_month` that your organization may use. |
There was a problem hiding this comment.
Instead of:
"The standard_granularity_column is the lowest grain of the table, in this case, it's hourly."
Can we say:
The standard_granularity_column is the column that maps to one of our standard granularities [list or link to options here]. The grain of this column must be finer or equal in size to the granularity of all custom granularity columns in the same model. In this case, it's hourly.
Slightly more accurate!
There was a problem hiding this comment.
not: when referencing the columns key, can we put that in quotes like the other keys referenced?
There was a problem hiding this comment.
Instead of:
"Use the standard_granularity_column as the join key for the time spine table when joining tables in MetricFlow."
"MetricFlow will use the standard_granularity_column as the join key when joining the time spine table to other source table."
There was a problem hiding this comment.
And one more thing - can we gate anything that references custom granularities on version 1.9+? Or add a note about upgrading like you did in the custom calendar section.
|
|
||
| To use a custom calendar model, ensure it meets the following requirements: | ||
|
|
||
| - The model must be at a daily grain and joinable to other date columns using `date_day`. |
There was a problem hiding this comment.
++ you can use any of our standard granularities!
| To use a custom calendar model, ensure it meets the following requirements: | ||
|
|
||
| - The model must be at a daily grain and joinable to other date columns using `date_day`. | ||
| - Columns must align with expected data types for a given granularity (for example, `quarter` should be a date). |
There was a problem hiding this comment.
Can we update this to be a little more specific:
"The standard_granularity_column must be a date time type."
(Or something similar - datetime/date/timestamp I think are all fine but not sure if I'm missing options from any specific engines)
Also could move that note to the section about standard_granularity_column.
The other columns can be any type so I don't think we need to add any detail about that.
|
Also wondering if we should add a not about features that are upcoming but not supported yet, like calculating offsets / period over period. Might be helpful to get ahead of those questions. @Jstein77 thoughts? |
this pr adds the new
custom_granularitiesfeature to the mf timespine doc. this also includes a release noteResolves #6075