Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define column names #6140

Open
wants to merge 4 commits into
base: current
Choose a base branch
from
Open

Define column names #6140

wants to merge 4 commits into from

Conversation

nataliefiann
Copy link
Contributor

@nataliefiann nataliefiann commented Sep 25, 2024

I've created this PR / ppdated Snapshot meta-fields section to highlight new config allows users to define column names following this Git issue: https://github.com/orgs/dbt-labs/projects/14/views/35?pane=issue&itemId=80581863

Resolves #6112

What are you changing in this pull request and why?

Checklist

  • I have reviewed the Content style guide so my content adheres to these guidelines.
  • The topic I'm writing about is for specific dbt version(s) and I have versioned it according to the version a whole page and/or version a block of content guidelines.
  • I have added checklist item(s) to this list for anything anything that needs to happen before this PR is merged, such as "needs technical review" or "change base branch."

@nataliefiann nataliefiann requested a review from a team as a code owner September 25, 2024 10:34
Copy link

vercel bot commented Sep 25, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
docs-getdbt-com ✅ Ready (Inspect) Visit Preview Sep 25, 2024 2:28pm

@github-actions github-actions bot added content Improvements or additions to content size: small This change will take 1 to 2 days to address labels Sep 25, 2024

### Define column names in snapshot table

In dbt Cloud, there's a config which will now allow you to define the names of the columns that are created in the [snapshot table](/docs/build/snapshots#snapshot-meta-fields). Users can now set this config to use different names.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm a little confused -- is this only for dbt cloud? i thought this was something avail to both cloud and core users since it relates to snapshots? maybe the diff is it's available to versionless users now and 1.9 users when 1.9 is launched?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry - that was a force if habit. I'll update this. Thanks for this


### Define column names in snapshot table

In dbt Cloud, there's a config which will now allow you to define the names of the columns that are created in the [snapshot table](/docs/build/snapshots#snapshot-meta-fields). Users can now set this config to use different names.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In dbt Cloud, there's a config which will now allow you to define the names of the columns that are created in the [snapshot table](/docs/build/snapshots#snapshot-meta-fields). Users can now set this config to use different names.
Define the names of the columns created in the [snapshot table](/docs/build/snapshots#snapshot-meta-fields) using custom configurations. The original columns were named `dbt_valid_to`, `dbt_valid_from`, `dbt_scd_id`, and `dbt_updated_at`, however, users can now customize these names as needed.

@mirnawong1
Copy link
Contributor

hey @nataliefiann , gave it a nice review and looks good! some small suggestions + an outstanding question.

also, should you add something about this here too?

<VersionBlock firstVersion="1.9">

### Define column names in snapshot table

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small sugg that's not a blocker: maybe have the header outside the versionblock and tell users on 1.8 and lower that this feature is only avail in 1.9 and higher or versionless so they know?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++ we need to do this for all new features!

@@ -587,6 +587,16 @@ Snapshot results:

</details>

<VersionBlock firstVersion="1.9">

### Define column names in snapshot table
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is the correct language.. something more like "customize the names of the dbt-generated meta fields in your snapshot tables"

<VersionBlock firstVersion="1.9">

### Define column names in snapshot table

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++ we need to do this for all new features!


### Define column names in snapshot table

In [Versionless](/docs/dbt-versions/upgrade-dbt-version-in-cloud#versionless), you can now define the names of the columns created in the [snapshot table](/docs/build/snapshots#snapshot-meta-fields) using custom configurations. The original columns were named `dbt_valid_to`, `dbt_valid_from`, `dbt_scd_id`, and `dbt_updated_at`, however, users can now customize these names as needed.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In [Versionless](/docs/dbt-versions/upgrade-dbt-version-in-cloud#versionless), you can now define the names of the columns created in the [snapshot table](/docs/build/snapshots#snapshot-meta-fields) using custom configurations. The original columns were named `dbt_valid_to`, `dbt_valid_from`, `dbt_scd_id`, and `dbt_updated_at`, however, users can now customize these names as needed.
In [Versionless](/docs/dbt-versions/upgrade-dbt-version-in-cloud#versionless), you can now customize the names of the meta columns created by dbt in the [snapshot table](/docs/build/snapshots#snapshot-meta-fields) using the `snapshot_meta_column_names` configuration.

### Define column names in snapshot table

In [Versionless](/docs/dbt-versions/upgrade-dbt-version-in-cloud#versionless), you can now define the names of the columns created in the [snapshot table](/docs/build/snapshots#snapshot-meta-fields) using custom configurations. The original columns were named `dbt_valid_to`, `dbt_valid_from`, `dbt_scd_id`, and `dbt_updated_at`, however, users can now customize these names as needed.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add an example?

In [Versionless](/docs/dbt-versions/upgrade-dbt-version-in-cloud#versionless), you can now define the names of the columns created in the [snapshot table](/docs/build/snapshots#snapshot-meta-fields) using custom configurations. The original columns were named `dbt_valid_to`, `dbt_valid_from`, `dbt_scd_id`, and `dbt_updated_at`, however, users can now customize these names as needed.

Note, if you rename snapshot columns without updating the existing table, it will cause an error. It's recommended to use this feature for new snapshots or update your existing tables before committing a column name change.
This feature will also be available in the upcoming 1.9 release.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This feature will also be available in the upcoming 1.9 release.
This feature will also be available in the upcoming 1.9 release of dbt Core.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
content Improvements or additions to content size: small This change will take 1 to 2 days to address
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Core] dbt-core Issue #10185 - new snapshot_meta_column_names config for dbt snapshots
3 participants