Developer Workflow

This page describes the end-to-end lifecycle for building features on the Smart Data Platform. Every change -- whether a new dbt model, semantic model, report, or ingestion handler -- follows the same flow: provision an isolated environment, develop locally, test, push, review, merge, and tear down.

Workflow Overview

graph TD
  START[New Feature Request] --> PROFILE{Choose Profile}
  PROFILE -->|Quick dbt fix| LOCAL[Profile: local]
  PROFILE -->|Reports on existing data| MODEL[Profile: model]
  PROFILE -->|dbt + semantic + reports| FULL[Profile: full]
  PROFILE -->|New Bronze source| BRONZE[Profile: bronze]
  PROFILE -->|+ Azure Functions| FUNC[Profile: +functions]
  LOCAL --> BRANCH[Create feature branch]
  MODEL --> BRANCH
  FULL --> BRANCH
  BRONZE --> BRANCH
  FUNC --> BRANCH
  BRANCH --> PROVISION[Provision feature env]
  PROVISION --> DEV[Develop locally with DuckDB]
  DEV --> TEST[Test: dbt build --target local]
  TEST --> PUSH[Push to feature branch]
  PUSH --> CI[CI validates]
  CI --> PR[Create PR to main]
  PR --> REVIEW[Code review]
  REVIEW --> MERGE[Squash merge to main]
  MERGE --> DEPLOY[Auto-deploy to DEV]
  DEPLOY --> TEARDOWN[Teardown feature env]

Step-by-Step Summary

1. Choose Your Profile

Every feature starts with a profile decision. The profile determines which Fabric resources are provisioned for your isolated environment. Choose the smallest profile that fits your work -- smaller profiles are faster to provision and cheaper to run. See Feature Provisioning Profiles for the full decision tree and provisioning commands.

2. Provision Feature Environment

Run the provisioning CLI to create isolated Fabric workspaces on a feature branch:

./scripts/fabric dev start --feature my-feature \
  --profile full \
  --seed-strategy bronze \
  --developers daan@geris.nl

This creates workspaces (FEAT-my-feature-Gold, FEAT-my-feature-Semantic, FEAT-my-feature-Reports), connects them to a feature/my-feature Git branch, and populates the feat Gold Warehouse via the local-first loop (Bronze Parquet → DuckDB → COPY INTO). Provisioning always does a full Bronze pull + full Gold build + full push; selective flags (--only / --select) are only available on the standalone refresh-bronze-local / build-gold-local / push-gold commands.

3. Develop Locally with DuckDB

Before touching Fabric, validate all dbt model changes locally. DuckDB provides sub-second builds with no cloud dependency:

fabric dev build-gold-local          # preferred (runs dbt build --target local)
# or directly:
cd dbt && dbt build --target local --profiles-dir .

A passing local build is a hard gate — no story is considered done until this succeeds. See Local Development Setup for prerequisites and troubleshooting.

Query Bronze + Gold locally

After fabric dev refresh-bronze-local + fabric dev build-gold-local, the project DuckDB holds Bronze (via Parquet sources) and Gold (as tables) side-by-side. Connect any DuckDB-aware client to the file to model and test joins.

Canonical path: dev-data/fabric_datalake.duckdb (override with DBT_DUCKDB_PATH).

• DBeaver — New Connection → DuckDB → path = <repo>/dev-data/fabric_datalake.duckdb
• Azure Data Studio — Install the DuckDB extension, same path
• duckdb CLI — duckdb dev-data/fabric_datalake.duckdb

Example cross-layer query:

SELECT b.product_id, g.kpi_name, g.value
FROM bronze.dim_product b
JOIN gold.fact_kpi g USING (product_id);

See Local dbt Workflow for the full DBeaver walkthrough and debugging tips.

4. Test on Feature Warehouse

Once local tests pass, run against your isolated Fabric Gold Warehouse to catch dialect-specific issues:

cd dbt && dbt build --target feat-my-feature --profiles-dir .

This target only exists if your profile includes a Gold Warehouse. The model profile shares the DEV Gold Warehouse.

5. Build Semantic Models and Reports in Fabric UI

Semantic models and reports are created in the Fabric portal, not in code:

1. Open FEAT-my-feature-Semantic workspace and create/modify semantic models
2. Open FEAT-my-feature-Reports workspace and create/modify reports
3. Use Source control > Commit to save changes to the feature branch

Auto-binding ensures semantic models connect to the correct data source based on your profile.

6. Open a PR to Main

When your feature is ready:

az repos pr create \
  --source-branch feature/my-feature \
  --target-branch main \
  --title "Add new trading dashboard"

CI gates run automatically: DuckDB smoke test (all models compile) and Fabric slim CI (modified models build on CI Warehouse). See PR & Code Review for review requirements and what happens after merge.

7. Tear Down

After your PR merges, run teardown manually to clean up the feature environment:

./scripts/fabric dev teardown --feature my-feature

Idle feature environments consume zero Fabric CUs (warehouses auto-pause after 30 minutes), so teardown is about cleanliness, not cost. Inactive environments are also caught by the daily activity-check pipeline (feature-env-cleanup) after a configurable grace period.

Promotion After Merge

Once changes land on main, they auto-deploy to DEV. Promotion to UAT and PROD uses cherry-pick PRs:

feature/* --> PR to main --> auto-deploy to DEV
                main --> cherry-pick PR to release/uat --> auto-deploy to UAT
                              release/uat --> cherry-pick PR to release/prod --> auto-deploy to PROD (with approval gate)

Each environment has its own deployment/ENV.yml with connection overrides that automatically rebind semantic models to the correct warehouse. No manual connection editing is needed at any stage.

Quick Reference

Related Pages

• Feature Provisioning Profiles -- Profile decision tree and provisioning details
• Local Development Setup -- Prerequisites, installation, and DuckDB walkthrough
• Coding Conventions -- Naming, structure, and verification checklist
• PR & Code Review -- CI gates, review permissions, and post-merge pipeline chain
• Dual-Dialect Patterns -- Writing SQL that works on both DuckDB and Fabric
• AI-Guided Power BI Workflow -- Conversational variant of this loop for Power BI / semantic-model work (Claude drives provision → edit↔deploy → PR → teardown via skills)