Feature Provisioning Profiles

Every feature environment uses a profile that determines which Fabric resources are provisioned. The profile system exists to minimize cost and provisioning time: if you only need to build a report on existing data, there is no reason to provision a full Gold Warehouse and Azure Function App.

Auth model (how this works for developers)

Any profile that provisions Fabric resources (everything except local) needs Azure + Fabric write access. Developers authenticate as themselves (az login) for identity/RBAC checks; under the hood the provisioning scripts fetch the platform SPN credentials from Key Vault and use those to run Terraform and call the Fabric API.

Prereq (one-time per developer or group): your account — or the FP GER Fabric Developers group — must have Get Secrets on the kv-fabric-dbt-keys Key Vault. A Platform Team member grants this once:

group_id=$(az ad group show --group 'FP GER Fabric Developers' --query id -o tsv)
az keyvault set-policy --name kv-fabric-dbt-keys \
  --object-id "$group_id" --secret-permissions get

If this is missing you'll see a clear "Failed to read platform SPN secrets" error when you run fabric dev start. The local profile doesn't need this.

Decision Tree

graph TD
  Q1{What are you building?}
  Q1 -->|Report or semantic model on existing data| MODEL[Profile: model]
  Q1 -->|dbt models + semantic + reports| FULL[Profile: full]
  Q1 -->|Register a NEW Bronze source<br/>shortcuts, raw files, new ingestion| BRONZE[Profile: bronze]
  Q1 -->|Quick dbt fix / local-only iteration<br/>bronze→silver→gold in DuckDB| LOCAL[Profile: local]
  Q1 -->|Azure Function for data ingestion| FUNC{Which base?}
  FUNC -->|Function + full stack| FULLFUNC[Profile: full+functions]
  FUNC -->|Function + Bronze lakehouse only| BRONZEFUNC[Profile: bronze+functions]

Start from the top and follow the path that matches your work. Always choose the smallest profile that covers your needs.

Profile Comparison

Note: bronze is for ingestion work only — adding a new source into Bronze. Bronze→Silver→Gold dbt transformations belong in local (fast iteration in DuckDB) or full (against a dedicated Fabric Warehouse). bronze has no warehouse and no dbt target.

Provisioning Commands

Profile: local

For dbt hotfixes, export query changes, or access/permission IaC changes. Works entirely on your machine with DuckDB -- no Fabric workspaces needed.

Prerequisites:

• Python 3.12 installed (winget install Python.Python.3.12 on Windows)
• Git Bash (comes with Git for Windows)

Provision and start working:

./scripts/fabric dev start --feature my-fix \
  --profile local \
  --developers daan@geris.nl

This creates the branch and automatically sets up the local dev environment (.venv with dbt + DuckDB + stub data) if it doesn't exist yet.

Build and test:

source .venv/Scripts/activate
cd dbt && dbt build --target local --profiles-dir .

If the build passes, push and create a PR:

cd ..
git push
# Create PR to main via Azure DevOps or: az repos pr create ...

After merge to main, CI auto-deploys to DEV. No manual deploy needed.

Profile: bronze

For registering a new Bronze source: shortcuts to external OneLake lakehouses, raw Parquet/CSV uploads, end-to-end ingestion testing.

./scripts/fabric dev start --feature new-source \
  --profile bronze \
  --developers daan@geris.nl

Creates FEAT-new-source-Bronze workspace with a git-synced Lakehouse_Bronze. Your FP GER Fabric Developers group already has Contributor on shared DEV-Bronze, so you can later promote working shortcuts/sources to the DEV lakehouse.

What you can do with this profile:

Upload a raw file (Parquet/CSV) to test ingestion:

./scripts/fabric dev upload --feature new-source \
  --file dev-data/new_vendor.parquet \
  --dest Files/new_vendor/2024-01-01/data.parquet

Create a shortcut to another Fabric lakehouse (e.g., Dataverse mirror or an existing source):

./scripts/fabric dev add-shortcut --feature new-source \
  --name MY_TABLE \
  --path Tables/dataverse \
  --target-workspace-id <source-ws-id> \
  --target-item-id <source-lakehouse-id> \
  --target-path Tables/my_table

Add an ingestion Function (combines with +functions add-on):

./scripts/fabric dev start --feature new-source \
  --profile bronze+functions \
  --developers daan@geris.nl

Creates the Bronze lakehouse plus an Azure Function App that writes ingestion data directly into FEAT-new-source-Bronze/LH_Bronze/Files/ — a fully isolated end-to-end test environment.

This profile does NOT include: a Gold Warehouse, dbt target, semantic models, or reports. For bronze→silver→gold transformations, use local (DuckDB) or full (dedicated Fabric Warehouse).

Profile: model

For building reports and semantic models on existing Gold data. No dbt build needed.

./scripts/fabric dev start --feature dashboard-v2 \
  --profile model \
  --developers daan@geris.nl

Creates FEAT-dashboard-v2-Semantic and FEAT-dashboard-v2-Reports workspaces. Semantic models bind to the shared DEV Gold Warehouse.

Profile: full

For adding new dbt models that transform existing Bronze data into new mart tables, plus semantic models and reports.

The full profile is local-first: Bronze data is pulled to your laptop as Parquet, dbt transforms run in DuckDB (zero Warehouse CU), and only the results are pushed to the feat Gold Warehouse.

./scripts/fabric dev start --feature new-logistics \
  --profile full \
  --seed-strategy bronze \
  --developers daan@geris.nl

Creates FEAT-new-logistics-Gold, FEAT-new-logistics-Semantic, and FEAT-new-logistics-Reports. No FEAT-X-Bronze workspace and no Bronze Lakehouse inside the Gold workspace. A feat-new-logistics dbt target is added to dbt/profiles.yml.

Seed strategies (--seed-strategy)

Controls which shared Bronze workspace is pulled from. Default: bronze.

Developers have Viewer access on DEV-Bronze, UAT-Bronze, and PROD-Bronze — no Contributor needed for the local-first flow.

Provisioning always does a full Bronze pull + full Gold build + full push. Selective flags (--only / --select) are available on the standalone refresh commands, not on dev start.

The three commands

The canonical iteration loop uses three separate commands (also composed automatically by dev start):

• fabric dev refresh-bronze-local [--scope gold|all-sources] [--only <tables>] [--all] [--parallel N] — pull Bronze → dev-data/*.parquet (skips unchanged tables via manifest).
  • Default is --scope gold: only Bronze sources consumed by marts (i.e. dbt ls --select +marts --resource-type source). This is the minimum needed to build Gold end-to-end and is what dev start --profile full uses.
  • --scope all-sources adds sources referenced only by staging/intermediate models (sources that don't feed a mart). Use when you're working on a staging model that isn't yet wired into a mart.
  • --only a,b,c pulls an explicit bare-name list, ignoring --scope.
  • --all pulls every Delta table physically present in the Bronze Lakehouse, including monitoring tables and UAT mirrors that aren't declared as dbt sources. Slow; use for rare "what's actually in there" audits.
  • --parallel N — number of tables to download concurrently. Default 6; each worker uses its own thread-local DuckDB connection. Raise for fatter pipes; drop to 1 for deterministic logging when debugging.
• fabric dev build-gold-local [--select <selector>] — run dbt build --target local in DuckDB; zero Warehouse CU. --select restricts to a dbt selector. Pre-flight verifies that every Bronze source needed by the selected models is present in dev-data/; if anything is missing it prints the exact refresh-bronze-local --only ... command to run. Failing data-quality tests surface loudly but do NOT fail provisioning — only failing models/seeds/snapshots do. This matches the DevOps dbt-dev-build pipeline contract so the two paths behave identically.
• fabric dev push-gold --feature <name> [--select <selector>] [--only <tables>] — stage DuckDB Gold Parquet into the feat Bronze Lakehouse's Files/_refresh_staging/ folder (feat Gold has no Lakehouse) and COPY INTO the feat Gold Warehouse. The staging Lakehouse ID is resolved at push time via Fabric REST (git-synced, not in yml). --select / --only restrict which tables are pushed.

See Local dbt Workflow for the full daily loop.

Daily loop

# 1. Refresh local Bronze Parquet (Gold-required sources only, ~once a day)
fabric dev refresh-bronze-local

# Variants (opt-in, not the daily default):
#   fabric dev refresh-bronze-local --scope all-sources     # include staging-only sources
#   fabric dev refresh-bronze-local --only salestable,custtable  # just these
#   fabric dev refresh-bronze-local --all                   # every table in the Lakehouse

# 2. Build Gold locally in DuckDB — free, instant, no Warehouse CU
fabric dev build-gold-local

# 3. Push Gold to the feat Warehouse via COPY INTO
fabric dev push-gold --feature <name>

Use fabric dev refresh-gold --feature <name> to run steps 2 + 3 together.

refresh-bronze-local source matrix

IDs are resolved from deployment/<env>.yml automatically. Missing/unprovisioned source fails fast with a clear message — nothing is partially written.

Profile: full+functions

For developing new Azure Functions that ingest data from external APIs, with the full stack behind them.

./scripts/fabric dev start --feature new-api \
  --profile full+functions \
  --seed-strategy bronze \
  --developers daan@geris.nl

Creates all full resources plus a FEAT-X-Bronze workspace and an Azure Function App. The Bronze workspace is mandatory for any +functions profile: the Function App writes ingestion data into FEAT-X-Bronze/Lakehouse_Bronze/Files/ and the Terraform module references module.workspace_bronze[0].id directly. Without a Bronze workspace the Function App resources cannot be created (see terraform/main.tf:49 has_function_app).

The +functions add-on therefore always sets create_bronze_workspace: true on top of the base profile.

Shared identity (mi-fabric-functions) — one-time platform setup

Function Apps don't get a system-assigned MI. Every Function App across DEV / UAT / PROD / feat-* attaches the same user-assigned managed identity: mi-fabric-functions in rg-fabric-dbt-platform. That identity is granted Key Vault Secrets User on kv-fabric-dbt-keys exactly once. Every subsequent dev start --profile *+functions provisioning needs zero RBAC writes — neither sp-fabric-data-worker nor sp-fabric-platform-admin requires elevated Key Vault permissions.

If mi-fabric-functions does not yet exist (clean tenant), Platform Team performs this one-time setup:

1. Create the user-assigned MI: portal → Managed Identities → + Create → resource group rg-fabric-dbt-platform, region West Europe, name mi-fabric-functions.
2. Open kv-fabric-dbt-keys → Access control (IAM) → + Add → Add role assignment → role Key Vault Secrets User → member: User-assigned managed identity → mi-fabric-functions → assign.
3. Copy the MI's Resource ID, principal (object) ID, and client ID into deployment/dev.yml under functions.user_assigned_identity_* (and into UAT/PROD when those environments are cut over).

Standalone Deploy

Deploy is always manual -- never auto-deployed on push to feature branches.

./scripts/fabric dev deploy --feature my-report                   # Deploy all
./scripts/fabric dev deploy --feature my-report --items semantic  # Semantic only
./scripts/fabric dev deploy --feature my-report --items reports   # Reports only

Deploy uses a two-pass strategy: semantic models first (for GUID resolution), then reports.

UI ↔ Git Sync (model / full profiles)

Semantic AND Reports workspaces are both git-connected on model / full feature envs. You can edit in the Fabric UI, in code, or both — but never simultaneously. Fabric has no three-way merge: if both sides have changes you must pick one side wholesale (Commit OR Discard+Update).

Two helpers enforce the rule:

Both refuse if the other side has pending work, telling you exactly which command to run first. pull --force-discard drops any cosmetic workspace drift before updating.

Cosmetic drift: After any commit Fabric re-serialises items and flags them as "Modified" even though nothing meaningful changed. provision_feature_env.py runs flush automatically at the end of start to clear this noise; after that, every diff you see in Source Control reflects a real change.

PR gate — automatic, no developer action

Every PR into main triggers pipelines/feature-sync-check.yml, which runs fabric dev pr-check against the source branch. If any connected feature workspace still has uncommitted UI edits, incoming git changes not yet pulled, or a .pbir referencing a semantic model GUID that doesn't exist in the feature Semantic workspace, the build fails and (once the pipeline is wired as a required Branch Policy on main) the PR cannot merge. The PR page shows exactly what's blocking.

Optional manual run at any time:

./scripts/fabric dev pr-check --feature X

Same checks, same output — useful for debugging before opening the PR.

--seed-strategy

Optional for the full profile (default: bronze). Selects which shared Bronze workspace Parquet is pulled from. The full matrix is in Seed strategies above.

Not needed for local, bronze, or model profiles.

RBAC Validation

The provisioning script validates your FP GER group membership before provisioning any resources.

Consultants do not run the provisioning CLI. A Platform Team member provisions a model workspace for them, then shares the workspace link. If a Consultant tries to run the script, they get a clear message explaining this.

If the Graph API is unavailable, validation falls back gracefully and logs a warning.

Cascade Detection

Before pushing changes that touch SQL or TMDL files, run cascade detection to trace the impact across the stack:

python scripts/check_cascade.py Revenue

This traces column and measure names across dbt models, TMDL definitions, reports, RLS rules, and exports. An advisory .githooks/pre-push hook warns about changed SQL/TMDL files automatically.

The tool is also available interactively in the developer portal under the Tools tab (/tools).

Multiple Developers

All developers on a feature share the same workspaces and branch:

./scripts/fabric dev start --feature shared-feature \
  --profile full \
  --seed-strategy bronze \
  --developers daan@geris.nl,alice@geris.nl

Coordinate Fabric UI commits -- one developer at a time should commit from Source control. Use "Update" (pull) before "Commit" (push) to pick up changes from other developers.

Teardown

When done with a feature, destroy the environment:

# Preview what would be destroyed
./scripts/fabric dev teardown --feature my-feature --dry-run

# Destroy workspaces and delete the remote + local Git branch (default)
./scripts/fabric dev teardown --feature my-feature

# Destroy workspaces but preserve the Git branch (e.g. work-in-progress review)
./scripts/fabric dev teardown --feature my-feature --keep-branch

What gets destroyed: All FEAT workspaces and contents, role assignments, Git connections, dbt target from profiles.yml, deployment config, Terraform state blob, and the feature/<name> branch (both local and remote).

What is preserved: All committed code remains in git history on whichever branches merged it. Pass --keep-branch to preserve the feature/<name> ref itself (e.g. to leave an open PR in place).

Automatic Cleanup on Failed Provisioning

If ./scripts/fabric dev start fails part-way (Terraform error, post-apply step crash, etc.), the wrapper invokes teardown_feature_env.sh <name> --force --keep-branch automatically. Orphan feat-<name>-* workspaces are removed and the next dev start <name> has a clean slate to work with — no manual teardown needed for the common case.

Safety guard — same-name collisions do NOT destroy live environments. The auto-teardown is gated on a sentinel file (terraform/environments/feat-<name>/.provision-incomplete) that the provisioning orchestrator writes only after the workspace-name pre-flight confirms no feat-<name>-* workspaces exist in the tenant. If you run dev start X while a live environment X already exists, the pre-flight aborts before the sentinel is written, and the failure-cleanup path does not touch the live environment. The sentinel's presence is proof that THIS run created the workspaces in question.

The sentinel is removed on successful provisioning so a later, manually triggered teardown is not mistaken for post-failure cleanup.

Provisioning step order

fabric dev start --profile full runs, in order:

1. Terraform apply (workspaces + warehouse + role assignments)
2. fabric-cicd deploys (Bronze + Semantic + Reports)
3. Platform .platform logicalId refresh
4. Bronze seed (feat Bronze Lakehouse data)
5. Generate dbt target (profiles.yml feat block)
6. Generate deployment/feat-<name>.yml — must run before step 6c because push-gold reads that file
7. 6c. Populate Gold (local-first) — refresh-bronze-local → sync-cloud-parquets → build-gold-local → push-gold. The sync-cloud-parquets sub-step (6c.1b) pulls every model tagged cloud_only (currently fact_inventory_snapshot) from the source env's Gold Warehouse into dev-data/*.parquet, so build-gold-local can read the pre-computed result on DuckDB instead of recomputing it. Source env defaults to dev; --seed-strategy uat switches it to uat.
8. Git sync / function-app / docs-site / post-deploy reports refresh

Until the commit that moved gold-populate after deploy-config, step 6c ran before the feat yml existed and push-gold died with FileNotFoundError: deployment/feat-<name>.yml. That ordering bug is fixed.

Self-Service Sandbox

For business users who need to explore data with Power BI. Three modes:

Mode A: Grant access to existing model

No new infrastructure -- users create reports in their own personal workspace.

./scripts/fabric sandbox grant \
  --model LH_Gold_Full \
  --workspace-id <semantic-workspace-id> \
  --users analyst@geris.nl,user2@geris.nl

Mode B: Deploy simplified model

Creates a SELF-SERVICE-<name> workspace with a purpose-built semantic model.

./scripts/fabric sandbox create \
  --name "Trade Overview" \
  --model workspaces/semantic/Trade_Simplified.SemanticModel

Mode C: Starter kit (model + report)

Same as Mode B plus a starter report for users to clone and customize.

./scripts/fabric sandbox create \
  --name "Finance Starter" \
  --model workspaces/semantic/Finance_Basic.SemanticModel \
  --report workspaces/reports/Finance_Starter.Report

Sandbox workspaces are permanent (not feature-bound). SQL-level security (RLS, column restrictions) still applies because DirectLake reads from the same Gold Warehouse.

Command Quick Reference

Auto-Pause and Capacity

Fabric Warehouses auto-pause after 30 minutes of inactivity. This means:

• Idle feature environments consume zero Fabric CUs
• 10+ feature environments can safely coexist without capacity contention
• Resume is automatic when a query is sent (cold start takes a few seconds)
• Teardown is recommended for cleanliness, not cost savings

Inactive environments are automatically flagged by the auto-teardown system after a configurable period of inactivity.

Claude / AI assistants

Claude sessions that need to edit workspaces/** (Power BI reports or semantic models) use the fabric-feature-* skills, which wrap fabric dev start / deploy / pr-check / teardown with the right preconditions:

• fabric-feature-start — provisions the feature workspaces (--profile model), creates the worktree + feat/<slug> branch, and prints the Fabric URLs Daan will review in.
• fabric-feature-deploy — runs the validator, pushes TMDL + report.json to the feature workspace, and renders each report to PDF in .dev/screenshots/<short-sha>/ for Claude to read.
• fabric-feature-pr — opens the integration PR with the Fabric URLs + screenshot list in the description. Never sets autocomplete; Daan reviews the rendered report in Fabric before approving.
• fabric-feature-teardown — destroys the feature workspaces after the PR is Completed or Abandoned. An ADO service hook (pipelines/feature-env-teardown.yml) automates this on PR close — see .mex/patterns/feature-env-auto-teardown.md.

The validator (scripts/validate_powerbi.py) gates the loop in three places: pre-commit (per-file, --quick), CI on diff vs origin/main (pipelines/dbt-ci-smoke.yml, --quick), and fabric-feature-deploy (--deep, before pushing to Fabric).

Pattern doc with the full loop lives in the repo at .mex/patterns/claude-feature-env-loop.md (not published to this docs site — it's a Claude-facing brief).

Related Pages

• Developer Workflow -- Full feature development lifecycle
• Local Development Setup -- Prerequisites and DuckDB setup
• Local dbt Workflow -- Bronze → Silver → Gold with DBeaver
• PR & Code Review -- What happens after you push