SqlDBM users can push project assets to a connected git repository. Currently, the following assets can be generated and pushed to git using this functionality:
Create scripts [Forward Engineering] - DDL
Alter scripts [Forward Engineering] - DDL
Alter scripts [Environments/Releases] - DDL
dbt source properties [Forward Engineering] - YAML
dbt model properties [Forward Engineering] - YAML
Pushing to a repository
As a prerequisite, please ensure that a User Connection with a configured git integration already exists. To configure a new User Connection, please refer to the related article at the end of this post.
Link the User Connection with the project.
To use the git integration of choice, set your (default) User Connection for the project you wish to push from.
Select the SqlDBM logo.
Select Settings and Conventions.
Select User Connections.
Select the desired user connection
Select Next
Select the default git repository.
Select Save.
Once the integration has been configured between the SqlDBM project and a git repository through a User Connection (see links below for further information), changes can be pushed to the repository in various ways:
Create and alter (DDL)
dbt source and model (YAML)
Releases compare alter (DDL)
Selecting target folders in git
Before proceeding to generate and push each of the available asset types (i.e., DDL, YAML), please get acquainted with their respective target folder settings.
Users can choose the target git folders for each type of content being pushed.
To configure this for each content type (i.e., create, alter, yaml), click on the triangle icon on the "Push to Git button" (1).
Next, click the "Settings" button (2).
Here, select edit the selection for each type of content by clicking on it (1).
To select a folder (2), single-click to click on one of the available options. To drill down into a subfolder, double-click or press the ">" icon to the right of the folder name. To return to the parent folder, click the "<-" (left arrow) at the top fo the folder list.
To preview the list of files that will be generated and pushed under the chosen settings, and their target folders, select the "Preview" tab at the top of the "Push to Git" settings.
Create and alter script DDL
In order to generate a Create DDL or an Alter DDL, select the corresponding option in Format in the Forward Engineering screen.
In format (1) select DDL and alter script radio button.
Generate a (2) DDL create or an (3) alter depending on the use case.
Create DDL
The Source DDL option is intended for pushing the complete (CREATE) object definition to git as separate files. This integration pushes DDL for generated objects as individual files to a destination in the connected repository.
This option should be used if wishing to keep a parallel object history in git in addition to the project history enabled through SqlDBM project revisions. Files are created in the "SqlDBM/src" (default) subdirectory within the git repository and can be further segmented into subfolders by object type.
The subdirectories can be configured from the settings options when pushing to git, as described earlier in this article.
Source files are named after their respective objects and do not follow a naming convention. Ex. the DWH.DIM_CUSTOMER table generates a "DWH.DIM_CUSTOMER.sql" file.
The full file list can be viewed using the "Preview" button.
Branches
Branches will be named using the following pattern:
Timestamp of date and time of creation: < yyyy/mm/dd_hh-mm-ss >
ex. "2022/11/29_16-41-20"
Alter DDL
This option will generate a single file (with multiple DML statements) within a branch.
Single-file (Migrations)
Please refer to the respective product websites for instructions on how to set up these tools to perform automated deployments. Both tools use the same naming conventions detailed below.
schemachange (open source)
Flyway (licensed)
Files
Files are created in the "Migrations/" subdirectory within the git repository and can be changed on Settings on the "Push to git" menu.
They follow the next naming pattern:
Migrations/<from_version>.<to_version>__<Project_Name>_<from_version>_vs_<to_version>.sql
ex. Migrations/V10.15__SampleProject_V10_vs_V15.sql
Branches
Branches will be named in the following pattern:
Timestamp of date and time of creation: < yyyy/mm/dd_hh-mm-ss >
ex. "2022/11/29_16-41-20"
dbt YAML
A dbt (data build tool) model or dbt source can be generated in YAML and exported to your git of choice.
In dbt a source refers to raw tables in your data warehouse that are ingested from external systems. These sources are defined in your dbt project to facilitate referencing and tracking data lineage. A model, on the other hand, is a dbt object that transforms this raw data into a more refined state through SQL queries. Models are built upon sources or other models and materialized as tables or views in your data warehouse.
In order to generate a model or source from the project, in the Forward Engineering screen select in Format (1) YAML, select your export choice (2) and generate the YAML (3).
YAML file is generated by default in the "yml" directory but can be configured in Settings, as described earlier in this article.
They follow the next naming pattern:
Migrations/SqlDBM.
<Project_Name>_<version>.yml
ex. Migrations/SampleProject_V10.yml
Branches
Branches will be named in the following pattern:
Timestamp of date and time of creation: < yyyy/mm/dd_hh-mm-ss >
ex. "2022/11/29_16-41-20"
Environment compare alter scripts
Users can generate alter scripts in the compare environments feature in the
Releases screen. The functionality is the same as previously covered when generating alter scripts in Forward Engineering.
To do this, select the Releases screen in the explorer menu. Next pick two environments to compare.
Click on the "Generate alter script" icon at the top of the screen.
Press push to git or the triangle icon to open the settings. The settings for this option are shared with the "Alter script" functionality on Forward Engineering.
Pull/merge requests
If the "Create pull request" option is enabled for the chosen git integration, a pull/merge request will also be generated after the branch is created. Pull/merge requests will be named in the following pattern:
Generated by SqlDBM: <Project_Name>_<version>_by <"user">
ex. Generated by SqlDBM: SampleProject_V10 by Dave@Yew.inc
Concurrent working
Push to git functionality is supported in Concurrent Working-enabled projects. This applies to the main branch and change branches alike. Concurrent Working projects follow the same rules as described in this article.