VOOZH about

URL: https://dev.to/ben_watson_data/deeper-into-dataform-2-other-api-features-174n

⇱ Deeper into Dataform 2: Other API features - DEV Community

In part 1 we looked at the Dataform CompilationResult and WorkflowInvocation objects. Let's follow up with a quick look at some other API endpoints that I find useful.

Workspace management - delete_workspace and create_workspace

delete_workspace is essential for Dataform instances used by large teams as it allows workspaces to be automatically deleted post-merge, saving the bi-annual "can everyone please let me know which workspaces they're finished with?" message. This can be integrated into CI/CD tools such as GitHub Actions.

from google.cloud import dataform_v1

client = dataform_v1.DataformClient()

workspace_path = client.workspace_path(
 PROJECT_ID,
 REGION,
 REPOSITORY_ID,
 WORKSPACE_ID,
)

try:
 operation = client.delete_workspace(name=workspace_path)

There's also create_workspace - very useful for automatically creating a workspace if a code change is made on a new branch outside of the Dataform UI.

Git operations - push_git_changes and pull_git_changes

Another set of useful endpoints that allow code changes to be automatically applied outside of the Dataform UI:

from google.cloud import dataform_v1

client = dataform_v1.DataformClient()

workspace_path = client.workspace_path(
 project_id,
 region,
 repository_id,
 workspace_id,
)

# ------------------------------------------------------------
# 1. Pull latest changes from Git into the workspace
# ------------------------------------------------------------
print("Pulling latest Git changes into workspace...")

pull_request = dataform_v1.PullGitChangesRequest(
 name=workspace_path
)

pull_operation = client.pull_git_changes(request=pull_request)
print("Pull initiated:", pull_operation.operation.name)

# Optional: wait for completion (simplified polling)
pull_result = pull_operation.result()
print("Pull completed successfully")

# ------------------------------------------------------------
# (Optional) 2. Make programmatic changes here
# ------------------------------------------------------------
# e.g. update config blocks, inject variables, generate models, format SQL, etc.

print("Applying automated workspace changes...")

# ------------------------------------------------------------
# 3. Push workspace changes back to Git
# ------------------------------------------------------------
print("Pushing workspace changes to Git...")

push_request = dataform_v1.PushGitChangesRequest(
 name=workspace_path
)

push_operation = client.push_git_changes(request=push_request)
push_result = push_operation.result()
print("Push completed successfully")
print("Workspace successfully synced with Git")