Skip to main content

Infra Plane

The infra plane covers the resources, identities, and deployment steps that make the data plane runnable.

Shared concerns​

  • Subscription and resource group ownership.
  • Storage accounts, containers, and path conventions.
  • Secret and credential distribution.
  • CI/CD permissions and deployment identities.
  • Environment-specific parameterization.

Resource categories​

The reusable infra model typically includes these categories:

  • Primary data-platform resource group.
  • Managed platform resource group where the platform creates dependent assets.
  • Landing storage for ingestion.
  • Catalog or metastore storage for governance-related objects.
  • Key Vault for secret distribution and recovery-safe lifecycle management.
  • Platform workspace plus managed identity or access connector resources.

Shared repository surfaces​

  • infra/ contains the reusable infrastructure assets that support the platform baseline.
  • The example implementations carry the platform-specific deployment inputs your team will usually adjust.
  • Parameter files and environment values should stay close to the example implementation they belong to.

Identity and RBAC model​

  • CI/CD uses a non-personal deployment identity.
  • Databricks or platform-managed identities are granted storage access rather than embedding secrets in notebooks.
  • Key Vault access should be RBAC-based.
  • Role assignment logic belongs in versioned templates and deployment steps, not in undocumented manual setup.

Parameter and environment layout​

  • Reusable templates stay in framework-owned infra directories.
  • Environment-specific values live in parameter directories under the client's implementations.

Where client teams usually make infra changes​

  • Update environment-specific parameter files.
  • Keep deployment wiring, naming, and identity values consistent with that example's structure.
  • Escalate shared-template changes only when the client requirement cannot be handled through configuration.

Fabric-specific infra concerns​

  • Workspace and lakehouse lifecycle.
  • Capacity planning and assignment.
  • Notebook library availability and execution permissions.
  • Storage shortcuts and lakehouse object placement.

Deployment guardrails​

  1. Validate Bicep before pushing large infra changes.
  2. Treat Key Vault recovery behavior as part of the deployment design, especially in dev environments.
  3. Keep tags, naming, and environment boundaries consistent across parameter files.

Deployment guidance​

  1. Keep infra provisioning separate from notebook logic changes.
  2. Validate identities and storage access before debugging data-plane behavior.