config.yml¶

Each workspace folder carries a fabric-cicd config.yml. allez runs it through fab deploy as a configuration deployment. The dag in the root allez.yml decides which configs deploy and in what order.

Put it beside that workspace's item folders and parameter.yml, as shown in the repo structure.

A sub-folder without config.yml is not a deploy unit. The file names the workspace for each environment, sets the item scope, and holds the publish and unpublish rules:

core:
  workspace:              # display names by environment
    dev:  data-dev
    test: data-test
    prod: data-prod
  repository_directory: "."
  item_types_in_scope: [Notebook, Warehouse, DataBuildToolJob]
  parameter: "parameter.yml"

publish:
  skip: {feature: false, dev: false, test: false, prod: false}

unpublish:
  skip:
    feature: true       # never delete a developer's uncommitted items
    dev: false          # dev mirrors main; orphaned items are removed
    test: false
    prod: false
  exclude_regex: "^DEBUG.*"

core.parameter names the parameter file that rewrites environment-specific values during the publish. core.repository_directory is the folder the item folders sit under, relative to this file; . is the folder holding config.yml. allez discovers the node's items there.

To split one workspace's publish into ordered steps, add overlays under .deploy/config-overlay/ and name them as dag nodes (Overlays).

Per-environment values¶

Any setting may be written once for every environment, or as a mapping by environment. allez reads both the way fabric-cicd 1.4.0 does: a mapping yields its entry for the environment, and a mapping with no entry for the environment is the setting absent. This applies to every key below, core.item_types_in_scope and core.repository_directory included:

core:
  item_types_in_scope:
    dev: [Notebook, Lakehouse]
    prod: [Notebook]       # prod never publishes the lakehouse

publish:
  exclude_regex:
    dev: "^DEBUG"          # no entry for prod: nothing is excluded there

allez deploy --plan prints the entry for --env on the Item types: line, and allez document shows the mapping as written in the node's effective config (Reading the plan).

Publish filters¶

fab deploy publishes an item when every filter under publish lets it through. allez computes the same answer offline, in fabric-cicd's order, so the plan, allez document, the Deployment view and allez audit say what the deploy will do. The first filter that fails is the reason the item is excluded, named by its key.

The folder path is the item's folder relative to core.repository_directory, with a leading slash and the item folder dropped: /ingest for ingest/Load.Notebook, /ingest/raw for ingest/raw/Land.Notebook, and empty for an item at the top. The two folder filters never touch an item at the top. A KQL database under an Eventhouse's .children/ takes the path above the Eventhouse.

features:
  - enable_exclude_folder

publish:
  exclude_regex: "^DEBUG"            # a name that starts with DEBUG
  folder_exclude_regex: "^/legacy"   # everything under legacy/, however deep

With that config, allez deploy --plan prints Excluded: DEBUG_Notes.Notebook (publish.exclude_regex), legacy/Old.Report (publish.folder_exclude_regex) under the node, --since does not list either item, the Deployment view dims them with the key, and the audit does not scan their files (Reading the plan, allez audit).

publish.shortcut_exclude_regex filters shortcuts inside a lakehouse, not items. allez prints it with the node's config and nothing else changes.

Refused at load¶

Three states fabric-cicd refuses at fab deploy are refused when allez loads the config, before any fab call, naming the file, the key and the fix: a filter whose flag is missing from features: (enable_items_to_include for publish.items_to_include or unpublish.items_to_include, enable_exclude_folder, enable_include_folder); publish.folder_exclude_regex and publish.folder_path_to_include both set for one environment; a regex that does not compile. Every verb that reads the config fails the same way, --plan, document and audit included.

Unpublish rules¶

After the publish, fab deploy deletes orphans: items in the workspace whose type is in scope and which have no folder in the repository. An orphan is unpublished when unpublish.skip is off for the environment and either unpublish.items_to_include is set and lists its <name>.<Type> (the regex is then ignored), or the list is unset and its name does not match unpublish.exclude_regex from the start. An orphan of a type outside core.item_types_in_scope is never considered. allez deploy --plan --since prints the verdict on each - line (changes).

In the sample¶

The shipped solution uses one filter. fabric/allez-data/ keeps Scratch.Notebook, a scratch notebook committed for reference, and its config.yml sets publish: exclude_regex: "^Scratch", so the notebook stays in the repo and is never published. The plan for allez-data prints Excluded: Scratch.Notebook (publish.exclude_regex) and the audit ignores its ids.