The deployment dag¶

The deploy order, declared once in the root allez.yml. It orders config files, so a workspace can be visited more than once. allez deploy runs one fab deploy per node, in this order, with the node's hooks before and after it.

Every id resolved at deploy time is looked up in what is already published. Inside one workspace, fabric-cicd orders the publish itself. Between workspaces it guarantees nothing: a $workspace.<Name>.$items.* reference or a $allez.* token names whatever happens to be published when that node runs. The dag makes sure the item it names is already published.

The dag is required

Required. Without dag:, allez deploy refuses to run.
Only listed configs deploy; only named hooks run. A config.yml, overlay or hooks file that no node mentions is ignored and reported once: not in the dag, ignored: …. It is not an error, so a file can stay on disk, unused, while a change is in progress. A folder deployed through overlays still uses its config.yml as the base, so that one is not reported.
Single-folder runs. allez deploy fabric/reporting --env dev filters the dag to that folder's nodes and keeps their relative order; other workspaces' nodes are skipped, not re-ordered.
--plan draws it: a tree. Each node sits under the node it depends on, numbered in deploy order, with hooks as leaves and out-of-scope nodes dimmed. Reading the plan.

Rejected before any fab call

A dag: anywhere but the root file; a config path listed twice, of an illegal shape, or not on disk; depends_on naming a path that is not a node; a cycle (members named); a pre_deploy_hooks / post_deploy_hooks entry not at <sub-folder>/.deploy/hooks/<purpose>.hooks.yml or not on disk; an overlay that sets core.workspace*; an unknown node key.

Nodes¶

A node is one fab deploy. Two config shapes are valid:

<sub-folder>/config.yml: the folder's whole publish;
<sub-folder>/.deploy/config-overlay/<name>.config.yml: one overlay, part of the folder's publish.

Key	Required	Meaning
`config`	yes	The config path, relative to the solution root. Each path at most once.
`depends_on`	no	Config paths (one, or a list) that must publish first.
`pre_deploy_hooks`	no	One or more hooks files run before this node's publish, each at `<sub-folder>/.deploy/hooks/<purpose>.hooks.yml`, relative to the solution root (Hooks files).
`post_deploy_hooks`	no	Likewise, run after a successful publish. A hooks file has no phase of its own: the key it is named under is the phase.
`environments`	no	Inclusion Filter: the node deploys, and its hooks run, only for these environments. Otherwise it is reported (`not for --env …, skipped: …`) and keeps its place in the order, so the nodes that depend on it are ordered as before.

Order follows depends_on: a node runs after every node it depends on. Nodes with no dependency between them run in the order they are declared. Filenames never order anything.

fabric/allez.yml

dag:
  - config: data/config.yml
    post_deploy_hooks: [data/.deploy/hooks/ingest.hooks.yml]

  - config: reporting/config.yml
    # the semantic model depends on the data workspace
    depends_on: [data/config.yml]
    pre_deploy_hooks: [reporting/.deploy/hooks/takeover.hooks.yml]

Every path in that dag is a file on disk, relative to fabric/. Nothing else in the tree is a dag input (Repo structure).

Overlays¶

fabric-cicd publishes the items in a workspace in a fixed order it does not let you change. To change the order, split the folder's deploy into parts with partial configs in <sub-folder>/.deploy/config-overlay/.

An example: a notebook in reporting/ refreshes the Sales model, so its parameter.yml rewrites the model id with $items.SemanticModel.Sales.$id. fabric-cicd publishes Notebook before SemanticModel, so in an environment where the model has never been published, there is no id to resolve. Publish the model first, then everything else.

📁 fabric/reporting/
├── 📄 config.yml                       # (1)!
└── 📁 .deploy/
    └── 📁 config-overlay/
        ├── 📄 model.config.yml         # (2)!
        └── 📄 rest.config.yml          # (3)!

The folder's config.yml. Never edited to change a run.
Narrows the publish to the model.
Everything else, in fabric-cicd's own order.

An overlay is only what differs from config.yml:

fabric/reporting/.deploy/config-overlay/model.config.yml

core:
  item_types_in_scope: [SemanticModel]

fabric/reporting/.deploy/config-overlay/rest.config.yml

core:
  item_types_in_scope: [Notebook, Report]

Each is a dag node, ordered by depends_on.

fabric/allez.yml

dag:
  - config: data/config.yml

  - config: reporting/.deploy/config-overlay/model.config.yml
    depends_on: [data/config.yml]

  - config: reporting/.deploy/config-overlay/rest.config.yml
    depends_on: [reporting/.deploy/config-overlay/model.config.yml]
    post_deploy_hooks: [reporting/.deploy/hooks/refresh.hooks.yml]

For each node, allez merges the overlay over the folder's config.yml, key by key, and passes the result to fab deploy as params. The committed config is never copied or rewritten:

fab deploy --config fabric/reporting/config.yml --target_env dev --force \
  --params 'config_override={"core": {"item_types_in_scope": ["SemanticModel"]}}'

An overlay may narrow any key, though usually only core.item_types_in_scope, and never core.workspace or core.workspace_id.

Each node still unpublishes

Every node is a full fab deploy: it publishes, then unpublishes, unless unpublish.skip is set for the environment. fabric-cicd unpublishes an item in the workspace, of a type in that node's scope, with no folder under the config's repository_directory. So:

Never narrow core.repository_directory in an overlay. Items outside the narrower folder count as missing from the repo, and that node unpublishes them.
Item types no node scopes are never unpublished. Between them, the overlays should cover every type in config.yml's item_types_in_scope.
publish filters are safe. exclude_regex, folder_exclude_regex, folder_path_to_include and items_to_include under publish limit what is published, not what is unpublished. The plan, allez document, the Deployment view and allez audit honour them, so an excluded item is listed with the key that excludes it and its files are not scanned (publish filters).

Revisiting a workspace is the case folder-level ordering cannot express, because the dependency crosses workspaces:

fabric/allez.yml

dag:
  - config: data/.deploy/config-overlay/lakehouse.config.yml
  - config: reporting/.deploy/config-overlay/model.config.yml
    depends_on: [data/.deploy/config-overlay/lakehouse.config.yml]
  - config: data/.deploy/config-overlay/notebook.config.yml
    depends_on: [reporting/.deploy/config-overlay/model.config.yml]
    post_deploy_hooks: [data/.deploy/hooks/ingest.hooks.yml]

Hooks belong to a node, and the node says when they run, so each visit to a workspace can run different hooks. Put a refresh on the workspace's last node, once everything in it is published, rather than on an earlier node while it is only half published.

$allez.* tokens follow the visits too. A rule names the items it is for, with item_type or with the item folder in file_path, and a node resolves only the tokens of the rules for the types it publishes. The notebook overlay above resolves the notebook's rules; the lakehouse node prints them as deferred and leaves them as written. A rule with neither filter is resolved on every visit. The sample does this: allez-orchestration/config.yml publishes the control database first of everything, and allez-orchestration/.deploy/config-overlay/pipeline.config.yml publishes the pipeline last, after the notebook and the model its activities name ($allez.* variables). allez audit judges each rule at the visit that publishes its type, and scans the items every visit publishes together, publish filters included.