# Azure Machine Configuration Deep Dive: PowerShell DSC, Configuration Drift, Azure Policy, and Auto-Remediation

![Neon Azure Machine Configuration architecture showing two Windows servers, PowerShell DSC, configuration drift, Azure Policy, and automatic remediation.](assets/azure-machine-configuration-deep-dive-cover.png)

Azure Automation State Configuration is approaching retirement. In this hands-on Azure Workshopz lab, we replace it with Azure Machine Configuration: PowerShell DSC packaged as immutable artifacts, delivered privately with managed identity, assigned through Azure Policy, and used to detect and continuously repair configuration drift on Windows Server.

You will build the lab twice in spirit. First, you operate in `Audit` mode and prove that drift is visible but untouched. Then you promote an immutable `AuditAndSet` package, create remediation tasks for existing VMs, and prove that the local Machine Configuration agent corrects a second drift without another remediation task.

> Guided workshop time: 35–40 minutes, excluding Azure wait periods. Live execution normally takes 90–150 minutes because Azure Policy, RBAC, extension, and Machine Configuration evaluation are asynchronous.

## What you will build

- Two private Windows VMs in West Europe: Windows Server 2022 and Windows Server 2025.
- No public VM IPs and no inbound RDP rule.
- A NAT Gateway for controlled outbound servicing.
- Private Blob Storage reached through a Private Endpoint and Private DNS.
- A user-assigned managed identity with container-scoped Blob Reader access.
- Machine Configuration and Azure Monitor Agent extensions.
- A Log Analytics workspace and DCR for operating-system and workshop drift events.
- Role-aware custom Azure Policy definitions and versioned audit/enforcement initiatives.

The desired state is intentionally straightforward but operationally meaningful:

- Every Windows Firewall profile is enabled.
- SMBv1 is absent.
- Windows Event Log is running and starts automatically.
- A controlled registry baseline exists under `HKLM:\SOFTWARE\AzureWorkshopz\MachineConfiguration`.
- A proof directory and marker file exist.
- IIS is absent on the baseline server and present on the application server.

## Why Machine Configuration now?

Microsoft will retire Azure Automation State Configuration on September 30, 2027. Machine Configuration is the current implementation for auditing and applying operating-system configuration across Azure VMs and Azure Arc-enabled servers. It combines a local agent, DSC-based packages, guest assignments, Azure Policy, and native compliance reporting.

The important separation is:

1. The DSC package defines what the operating system should look like.
2. A guest assignment connects an immutable package to one machine.
3. Azure Policy selects machines at scale and creates those assignments.
4. A remediation task deploys the assignment to existing resources.
5. `ApplyAndAutoCorrect` lets the local agent repair later drift at its next evaluation.

Microsoft documents that a remediation task is required once for existing non-compliant machines in `ApplyAndAutoCorrect` mode; subsequent correction is local and continuous. See [Machine Configuration remediation options](https://learn.microsoft.com/en-us/azure/governance/machine-configuration/concepts/remediation-options).

### The control loop from policy to Windows

Machine Configuration spans several Azure and guest layers. Troubleshooting becomes much easier when you know which layer owns each responsibility.

The Azure Resource Manager control plane stores policy definitions, initiatives, assignments, remediation tasks, extensions, and guest-assignment child resources. Azure Policy evaluates the VM resource—its type, location, tags, and the presence or state of a guest assignment. It does not directly edit the Windows registry or install IIS. When a `DeployIfNotExists` member is non-compliant, a remediation deployment creates or updates the guest assignment beneath the VM.

The Machine Configuration extension bootstraps and updates the local agent. The agent reads the guest assignment, acquires a managed-identity token, downloads the package, validates its hash, expands the bundled modules, and evaluates the MOF. This is the operating-system data plane. The agent returns a report containing assignment state and per-resource reasons, which Machine Configuration exposes through ARM and Azure Resource Graph. Azure Policy later ingests that state into its own compliance model.

Those layers are eventually consistent. A policy assignment can exist while its remediation is still evaluating. A remediation can succeed while the guest assignment is still `Pending`. The guest can already be repaired while Azure Policy displays the preceding non-compliant snapshot. The workshop therefore polls each boundary separately instead of treating one green status as proof of the whole path.

The useful mental model is a chain:

1. **Policy selection** decides whether the VM belongs to the ring and role.
2. **Remediation deployment** creates the guest-assignment contract for an existing VM.
3. **Extension and agent** execute that contract locally.
4. **Guest report** describes the latest DSC evaluation.
5. **Policy ingestion** turns the report into governance compliance.

When a VM remains pending, start at the earliest unproven link. Check tag selection before reinstalling an extension. Check the extension before changing Storage. Check private DNS and RBAC before rebuilding a valid package. This prevents random troubleshooting changes from hiding the real fault.

### Audit, AuditAndSet, and remediation are different controls

`Audit` is deliberately non-mutating. The package can run `Get` and `Test` logic and report drift, but it must not call DSC `Set` methods. It is ideal for discovery, change-review evidence, and safe pilot expansion.

`AuditAndSet` makes the package capable of changing the guest. The policy-generated assignment type controls how that capability is used. `ApplyAndMonitor` applies once and continues reporting. `ApplyAndAutoCorrect` applies and keeps correcting future drift at the configured consistency interval.

A remediation task is an Azure Policy operation, not the recurring Windows correction engine. It is needed to deploy `DeployIfNotExists` assignments to resources that existed before the enforcement assignment. After the guest assignment is present, the local `ApplyAndAutoCorrect` cycle performs later repairs without another Azure Policy remediation task.

## Prerequisites

- PowerShell 7.
- Azure CLI authenticated to the intended subscription.
- Permission to create VMs, networking, Private Endpoints, role assignments, custom policy definitions, initiatives, assignments, and remediations.
- Four free B-series vCPUs in West Europe.
- Permission to let the script generate a strong disposable administrator password in memory. It is passed only to VM creation, never printed, and cleared immediately afterward.

The authoring script pins:

```powershell
GuestConfiguration        4.12.0
PSDesiredStateConfiguration 2.0.7
PSDscResources            2.12.0.0
```

`PSDscResources` is deliberate. GuestConfiguration 4.12.0 rejects deprecated in-box DSC engine resources, so the configuration imports supported resources explicitly.

![PowerShell and Azure CLI preflight.](screenshots/01-preflight.png)

Run the safe preflight:

```powershell
$subscriptionId = az account show --query id -o tsv

.\scripts\Start-AzureMachineConfigurationWorkshop.ps1 `
  -SubscriptionId $subscriptionId `
  -Location westeurope `
  -ResourceGroupName rg-machine-configuration-deep-dive-we `
  -Stage Preflight
```

The script binds every operation to the immutable active subscription ID, validates provider registration and quota, and rejects an overlapping `10.160.0.0/16` address space.

> Live API note: the planned `Microsoft.Authorization` API `2026-06-01` was rejected by the Azure control plane during this lab. The newest stable version accepted by the live service was `2025-11-01`, so the verified scripts use that version. Never keep a future API pin merely because it appeared in a design document—test it against the target cloud.

## Step 1 — Deploy a private Windows foundation

```powershell
.\scripts\Start-AzureMachineConfigurationWorkshop.ps1 `
  -SubscriptionId $subscriptionId `
  -Location westeurope `
  -ResourceGroupName rg-machine-configuration-deep-dive-we `
  -Stage Foundation
```

The foundation stage creates:

- `vm-mc-ws2022-we` with `ConfigurationRing=Pilot` and `ServerRole=Baseline`.
- `vm-mc-ws2025-we` with the intentional typo `ConfigurationRing=Pliot` and `ServerRole=Application`.
- Workload and Private Endpoint subnets.
- An NSG with no custom inbound rules.
- A NAT Gateway and one Standard public IP used by the NAT resource—not by either VM NIC.
- Private Storage, a Blob Private Endpoint, a zone group, and a linked `privatelink.blob.core.windows.net` zone.

![Private Windows Server 2022 and 2025 infrastructure.](screenshots/02-private-windows-infrastructure.png)

The storage account disables shared-key authorization and anonymous Blob access. Public network access is disabled after authoring, while the VM NICs resolve the normal Blob hostname to the Private Endpoint address.

![Storage public access disabled.](screenshots/03-storage-private-networking.png)

![Approved Storage Blob Private Endpoint.](screenshots/03-storage-private-endpoint.png)

The guest initialization runs through Azure VM Run Command. It sets a known compliant baseline without opening RDP, and it installs IIS only on the Windows Server 2025 application VM.

> Run Command compatibility note: `RunPowerShellScript` executes Windows PowerShell 5.1 on these Windows Server images. Guest helper scripts therefore declare `#requires -Version 5.1`; the local orchestration and authoring scripts use PowerShell 7. A guest helper that declared PowerShell 7.2 returned a successful extension envelope but placed the real version error in stderr—always inspect both streams.

### Follow the private package path

The private delivery path still uses the normal Blob hostname. Windows resolves `<account>.blob.core.windows.net`; Azure Private DNS follows the platform CNAME into `privatelink.blob.core.windows.net` and returns the Private Endpoint address from `10.160.2.0/24`. The package URI in policy remains a regular HTTPS Blob URL. Hard-coding a private IP or using the `privatelink` hostname directly would bypass the supported resolution model and break TLS hostname validation.

The Private Endpoint protects the Storage data path, but it does not grant authorization. Network reachability, authentication, authorization, and integrity remain independent gates:

- Private DNS and the endpoint decide whether traffic reaches Storage privately.
- The attached user-assigned identity supplies the OAuth identity.
- `Storage Blob Data Reader` on the one package container authorizes the read.
- The policy's content hash proves that the downloaded bytes are the reviewed package.

This separation is why the deliberate 403 is useful. It proves private name resolution and TCP/TLS reachability can work while the data-plane role is absent. Granting Contributor on the resource group would not solve that 403 because Contributor does not include Blob data access.

The NAT Gateway serves a different purpose: explicit outbound connectivity for Windows servicing and Azure platform dependencies. It is not used to reach the Blob package after public Storage access is disabled. Neither VM NIC has a public IP, and the NSG exposes no inbound administration rule. Run Command supplies temporary management without opening RDP.

### Why use a user-assigned identity?

A user-assigned identity decouples package-read authorization from either VM lifecycle. Both servers share the same narrowly scoped reader identity, and the policy embeds one stable identity resource ID. A system-assigned VM identity would also work, but every VM would require a separate Blob role assignment. At scale that creates more role assignments and makes package-access reviews harder.

The runtime identity has no write permission. The authoring user receives temporary Blob Contributor only while uploading, and that grant is removed in `finally`. The VMs retain read-only access to `machine-configuration`, not to the entire Storage account or resource group.

## Step 2 — Author one DSC source for two roles

The configuration is parameterized rather than duplicated:

```powershell
Configuration WorkshopWindowsBaseline {
    param(
        [ValidateSet('Present', 'Absent')]
        [string]$IisEnsure
    )

    Import-DscResource -ModuleName PSDscResources -ModuleVersion 2.12.0.0
    Import-DscResource -ModuleName WorkshopMachineConfiguration -ModuleVersion 1.0.0

    Node localhost {
        Registry WorkshopBaselineVersion {
            Key       = 'HKLM:\SOFTWARE\AzureWorkshopz\MachineConfiguration'
            ValueName = 'BaselineVersion'
            ValueData = '1.1.0'
            ValueType = 'String'
            Ensure    = 'Present'
            Force     = $true
        }

        WorkshopWindowsOptionalFeature Smb1Protocol {
            Name   = 'SMB1Protocol'
            Ensure = 'Absent'
        }

        WorkshopWindowsFeature IisWebServer {
            Name   = 'Web-Server'
            Ensure = $IisEnsure
        }

        # File, service and firewall resources follow.
    }
}
```

![PowerShell DSC source configuration.](screenshots/04-dsc-source.png)

Compile the parameterized MOF and inspect it before packaging:

The authoring script compiles one MOF containing an `IisEnsure` package parameter. The generated baseline and application policies supply `Absent` or `Present` when they create the guest assignment. This keeps one configuration source and one package per operating mode while still producing role-aware assignments.

![Compiled MOF files.](screenshots/05-compiled-mof.png)

### Deliberate failure: package name and MOF name do not match

Machine Configuration expects the package name to match the configuration name embedded in the MOF. The workshop first uses a deliberately incorrect name and preserves the expected authoring error.

![Expected package-name mismatch.](screenshots/06-package-name-mismatch.png)

Correct the name and create two immutable packages:

Re-run the `Author` stage after correcting the name. It compiles, packages, hashes, tests, and uploads the two final artifacts as one controlled workflow.

| Artifact | Version | Package type |
|---|---:|---|
| `WorkshopWindowsBaseline-1.0.0-Audit.zip` | 1.0.0 | `Audit` |
| `WorkshopWindowsBaseline-1.1.0-AuditAndSet.zip` | 1.1.0 | `AuditAndSet` |

The package type is embedded in the archive. Promoting from audit to enforcement therefore creates a new artifact; it does not mutate version 1.0.0.

```powershell
Get-FileHash .\packages\WorkshopWindowsBaseline-1.0.0-Audit.zip -Algorithm SHA256
Get-FileHash .\packages\WorkshopWindowsBaseline-1.1.0-AuditAndSet.zip -Algorithm SHA256
```

![Correct package contents and local audit.](screenshots/07-package-and-local-test.png)

The current GuestConfiguration 4.12.0 module exposes `Get-GuestConfigurationPackageComplianceStatus` for the local non-mutating audit. Older examples that use `Test-GuestConfigurationPackage` do not match this installed module. The elevated local test validates the same custom feature bridge that the Azure guest agent receives in the package.

### What is inside the package?

The ZIP is a self-contained execution artifact. It includes the compiled `WorkshopWindowsBaseline.mof`, metaconfiguration, checksum, and the DSC modules required by the MOF. That includes `WorkshopMachineConfiguration`, a small class-based DSC module used for Windows feature state. Bundling modules avoids depending on whatever happens to be installed globally on the target server. It also makes execution reproducible: one package version always contains the same resource implementations.

The MOF is declarative. Each resource provides enough information for the engine to retrieve current state, test it against desired state, and—only for an enforcement package—set desired state. Dependencies are explicit. Registry values depend on the root key, and the marker script creates its parent directory before it writes deterministic content.

`Force = $true` on controlled registry resources is important. Without it, `PSDscResources` refuses to replace an existing value. The live drift test found that behavior before publication: the agent repaired the marker and service startup but stopped at the deliberately changed registry value. Rebuilding with `Force = $true` made the intended ownership boundary explicit. That is better than suppressing the error or using a broad script resource to overwrite anything under the key.

### Live troubleshooting: feature resources under the PowerShell 7 worker

The first enforcement run found a second, more subtle compatibility boundary. Machine Configuration executes DSC with PowerShell 7, while the inbox `Get-WindowsFeature` and DISM feature modules are Windows PowerShell components. Microsoft explicitly warns not to use the `WindowsFeature`, `WindowsFeatureSet`, `WindowsOptionalFeature`, or `WindowsOptionalFeatureSet` resources from `PSDscResources` in Machine Configuration because of a known module-loading issue.

The original package reached the agent but failed while testing `MSFT_WindowsOptionalFeature` with `This resource must run as an Administrator`. Treating that as propagation would have hidden a deterministic package defect. The repair follows Microsoft's [Windows PowerShell bridge guidance](https://learn.microsoft.com/en-us/azure/governance/machine-configuration/whats-new/psdsc-in-machine-configuration#running-commands-in-windows-powershell): a custom class-based resource launches the inbox Windows PowerShell executable in the agent's protected local context, returns standardized reasons, and exposes `Ensure` as a real package parameter. The corrected package is rebuilt, rehashed, uploaded through the same private path, and redeployed before compliance evidence is accepted.

The bridge also drains standard output and standard error asynchronously. Reading one redirected stream completely before the other can deadlock a child process when Server Manager emits enough progress or warning output to fill the second pipe. A 15-minute process timeout prevents a blocked Windows feature operation from leaving the Machine Configuration worker pending forever. The live lab deliberately caught this during IIS installation; the final module uses concurrent stream reads and verifies that no workshop bridge process remains after the consistency cycle.

This keeps the role contract intact: one MOF still exposes `[WorkshopWindowsFeature]IisWebServer;Ensure`, the baseline policy supplies `Absent`, and the application policy supplies `Present`.

Package type and version are part of the release contract. Version 1.0.0 is `Audit`; version 1.1.0 is `AuditAndSet`. The workshop never replaces 1.0.0 bytes with enforcement bytes. Immutability matters for rollback, evidence, and incident investigation. If source, modules, or the parameter contract changes, create a new version, a new hash, and a reviewed policy definition.

### Local testing has boundaries

Local compliance testing catches malformed archives, missing modules, resource-loading failures, and basic DSC behavior before Azure is involved. It does not prove managed-identity access, Private DNS, policy targeting, remediation permissions, or cloud reporting. Conversely, a successful remediation deployment does not prove every DSC resource applied. This workshop requires both local validation and a real guest evaluation.

Run the authoring shell as administrator for the full local test because feature state is privileged operating-system data. If elevation is unavailable, keep construction deterministic and perform final execution validation on the disposable Azure VMs. Local validation cannot reproduce the Azure extension identity, so the live agent report remains the final authority.

## Step 3 — Upload without a key or SAS token

```powershell
.\scripts\Start-AzureMachineConfigurationWorkshop.ps1 `
  -SubscriptionId $subscriptionId `
  -Location westeurope `
  -ResourceGroupName rg-machine-configuration-deep-dive-we `
  -Stage Author
```

The authoring workflow temporarily allows only the current client IP, grants the signed-in author `Storage Blob Data Contributor`, polls until the data-plane role is usable, and uploads with:

```powershell
az storage blob upload `
  --auth-mode login `
  --account-name $storageAccountName `
  --container-name machine-configuration `
  --file $packagePath `
  --name $packageName `
  --overwrite true
```

In `finally`, it removes the exact temporary role assignment and firewall rule, then disables public Storage access. No account key or SAS token enters the policy JSON.

![Microsoft Entra package upload.](screenshots/08-package-upload.png)

For VM access, the policy carries the plain Blob URL plus the resource ID of the attached user-assigned identity. Microsoft recommends granting that identity `Storage Blob Data Reader` at Blob-container scope. See [secure package access with managed identity](https://learn.microsoft.com/en-us/azure/governance/machine-configuration/how-to/develop-custom-package/5-access-package).

## Step 4 — Break checksum and package authorization

Deploy the audit initiative with two intentional defects:

```powershell
.\scripts\Start-AzureMachineConfigurationWorkshop.ps1 `
  -SubscriptionId $subscriptionId `
  -Location westeurope `
  -ResourceGroupName rg-machine-configuration-deep-dive-we `
  -Stage BrokenAccess
```

The baseline policy references a wrong SHA-256 value, and the package identity still lacks Blob data access.

![Incorrect checksum failure.](screenshots/09-incorrect-checksum.png)

The VM can resolve the private Blob endpoint but receives `HTTP 403` when its managed identity requests the package. A token is acquired through IMDS and is never printed.

![Package-download authorization failure.](screenshots/10-package-access-403.png)

Repair both issues:

```powershell
.\scripts\Start-AzureMachineConfigurationWorkshop.ps1 `
  -SubscriptionId $subscriptionId `
  -Location westeurope `
  -ResourceGroupName rg-machine-configuration-deep-dive-we `
  -Stage Audit
```

The script republishes the definitions with the real file hashes and grants only `Storage Blob Data Reader` on:

```text
.../blobServices/default/containers/machine-configuration
```

![Correct container-scoped managed-identity RBAC.](screenshots/11-container-rbac.png)

## Step 5 — Observe with Azure Policy

The generated definitions are role-aware:

- Baseline definition: `ServerRole=Baseline`, `IisEnsure=Absent`.
- Application definition: `ServerRole=Application`, `IisEnsure=Present`.
- Both also require `ConfigurationRing=Pilot`.

They are grouped into a versioned 1.0.0 audit initiative and assigned at the disposable resource-group scope.

![Custom audit policies and initiative.](screenshots/12-audit-policy-initiative.png)

Initially, only the Windows Server 2022 VM enters scope. The 2025 VM has `Pliot`, not `Pilot`.

![The application VM is deliberately excluded by a mistyped ring.](screenshots/13-excluded-vm-tag-typo.png)

The Machine Configuration extension is installed on the 2022 VM but intentionally absent from the 2025 VM. This separates policy scope from guest readiness: a resource can match a policy and still remain pending when its required agent is missing.

![Machine Configuration extension state.](screenshots/14-extension-state.png)

### How the initiative selects one policy per VM

An initiative is a collection, not a sequence. Azure Policy evaluates each member independently. The baseline member checks for `ConfigurationRing=Pilot` and `ServerRole=Baseline`; the application member checks for the same ring and `ServerRole=Application`. Correct tags ensure exactly one role member applies to each VM.

This is more maintainable than copying an initiative for every rollout ring. In production, keep role and velocity separate: `ServerRole` describes the durable workload contract, while `ConfigurationRing` describes deployment pace. A baseline server can move from `Observe` to `Pilot` to `Broad` without becoming an application server.

The typo proves tags are exact strings. Azure Policy does not infer that `Pliot` means `Pilot`, and it does not warn that the value looks suspicious. The excluded VM is not a Machine Configuration failure—it is outside the policy `if` condition. Resource Graph and a tag inventory are the first checks when the expected resource count is wrong.

Assignment scope is the disposable resource group, while custom definitions and initiatives are subscription resources. Scope answers where evaluation occurs; it does not change where definitions are stored. Cleanup must therefore remove the resource-group assignment and the subscription-level custom artifacts.

### Read compliance at the correct layer

The Portal policy view answers governance questions: which resources match, which initiative member applies, and whether the latest ingested state is compliant. Guest-assignment REST answers agent questions: package version, assignment type, content hash, provisioning state, last check, and resource reasons. Resource Graph is best for fleet queries. Run Command verifies actual Windows state while cloud reports catch up.

Do not use one layer to impersonate another. `Succeeded` remediation means its ARM deployment succeeded; it does not mean IIS is already installed. `Succeeded` extension provisioning means the handler is ready; it does not prove the package was downloadable. A guest report is strong evidence of its last evaluation, but the before-and-after Run Command output is clearer for demonstrating automatic-correction timing.

## Step 6 — Introduce safe configuration drift

```powershell
.\scripts\Start-AzureMachineConfigurationWorkshop.ps1 `
  -SubscriptionId $subscriptionId `
  -Location westeurope `
  -ResourceGroupName rg-machine-configuration-deep-dive-we `
  -Stage Drift
```

The drift script uses Run Command and never disables Windows Firewall. It makes four reversible changes:

```powershell
Set-ItemProperty HKLM:\SOFTWARE\AzureWorkshopz\MachineConfiguration `
  -Name BaselineVersion -Value DRIFTED

Remove-Item C:\AzureWorkshopz\MachineConfiguration\desired-state.txt -Force
Set-Service EventLog -StartupType Manual
Remove-WindowsFeature Web-Server -Remove:$false -Restart:$false
```

![Deliberately introduced Windows configuration drift.](screenshots/15-introduced-drift.png)

Request a new scan and poll rather than assuming instant results:

```powershell
az policy state trigger-scan `
  --resource-group rg-machine-configuration-deep-dive-we
```

In Audit mode, Machine Configuration reports exactly which resources failed but does not call their DSC `Set` methods. The registry value remains `Drifted`, the proof file stays absent, Event Log startup remains Manual, and IIS remains absent.

![Non-compliance summary and detailed drift reasons.](screenshots/16-drift-noncompliance.png)

## Step 7 — Expand the pilot ring

Correct the typo and install the missing extension:

```powershell
.\scripts\Start-AzureMachineConfigurationWorkshop.ps1 `
  -SubscriptionId $subscriptionId `
  -Location westeurope `
  -ResourceGroupName rg-machine-configuration-deep-dive-we `
  -Stage ExpandRing
```

Now both role-specific policy members match exactly one VM. The application VM first appears as pending, then exposes IIS and other DSC drift details after the extension is ready.

![Corrected tag and both VMs in policy scope.](screenshots/17-expanded-pilot-ring.png)

## Step 8 — Promote to AuditAndSet

```powershell
.\scripts\Start-AzureMachineConfigurationWorkshop.ps1 `
  -SubscriptionId $subscriptionId `
  -Location westeurope `
  -ResourceGroupName rg-machine-configuration-deep-dive-we `
  -Stage Enforce
```

This stage:

1. Removes the audit assignment.
2. Assigns the 1.1.0 enforcement initiative with a system-assigned identity.
3. Sets both `assignmentType=ApplyAndAutoCorrect` and LCM `configurationMode=ApplyAndAutoCorrect`.
4. Grants the assignment identity `Guest Configuration Resource Contributor` only on the lab resource group.
5. Creates separate remediation tasks for the baseline and application initiative members.

![AuditAndSet initiative and remediation tasks.](screenshots/18-enforcement-remediations.png)

Remediation deploys guest assignments to the existing VMs. The agent then returns each role to its desired state:

- Registry marker restored.
- Proof file recreated.
- Event Log startup restored to Automatic.
- IIS remains absent on Server 2022.
- IIS is reinstalled and the proof site is restored on Server 2025.

![Successful DSC repair.](screenshots/19-successful-repair.png)

## Step 9 — Prove continuous autocorrection

Run the drift stage again, but do **not** create another remediation task. Record the drift timestamp, wait for the next 15-minute evaluation, and validate the guest again.

`ApplyAndAutoCorrect` can repair drift quickly enough that the reported cloud state remains compliant. For that reason, the workshop uses three independent evidence sources:

- Run Command timestamps before and after drift.
- Workshop events in Log Analytics.
- Native guest-assignment and Azure Policy compliance.

```powershell
.\scripts\Get-AzureMachineConfigurationEvidence.ps1 `
  -SubscriptionId $subscriptionId `
  -ResourceGroupName rg-machine-configuration-deep-dive-we
```

Useful Resource Graph query:

```kusto
guestconfigurationresources
| where id contains '/resourceGroups/rg-machine-configuration-deep-dive-we/'
| project machine = tostring(properties.targetResourceId),
          assignment = name,
          status = tostring(properties.complianceStatus),
          checked = todatetime(properties.lastComplianceStatusChecked)
```

![Repeated drift, automatic correction, and CLI/Resource Graph evidence.](screenshots/20-autocorrection-evidence.png)

### Correlate the evidence timeline

The second drift test is deliberately different from the initial remediation. The script records Application event 4101 when drift is introduced. No new Azure Policy remediation is created. At the next local consistency cycle, the agent runs the `AuditAndSet` package and restores the registry value, marker, service startup type, and role-specific IIS state. The verification helper writes event 4102 only after it observes the full desired state.

That produces a defensible sequence:

1. Run Command reports drift and supplies the introduction timestamp.
2. The remediation list remains unchanged.
3. The guest assignment remains `ApplyAndAutoCorrect` on package 1.1.0.
4. A later Run Command reports every selected setting compliant.
5. Log Analytics contains the drift and verification marker events.
6. Resource Graph and Azure Policy converge on compliant state afterward.

Continuous correction can be faster than centralized reporting. You may never see a durable cloud `NonCompliant` snapshot for the second drift if the local agent fixes it before the next report is ingested. That is not missing enforcement. The before-and-after guest state, unchanged remediation count, and event timestamps prove the local control loop.

### Interpret detailed DSC reasons

An assignment report identifies resources by DSC type and resource ID. Registry failure should name `WorkshopBaselineVersion`; marker failure should name `WorkshopMarker`; service drift should identify `WindowsEventLog`; and role drift should identify `IisWebServer`. Use those IDs to map cloud evidence back to source.

If the report says the package could not be downloaded, DSC never started—focus on hash, identity, DNS, and Storage access. If the package was downloaded but a resource throws during `Set`, focus on the module and desired-state declaration. If every resource is compliant but policy remains stale, wait for ingestion or trigger a scan; do not rebuild a working package.

## Troubleshooting map

| Symptom | Likely cause | Check |
|---|---|---|
| Package is never evaluated | Package name does not match the MOF configuration name | Inspect the MOF instance/document name and package metadata |
| `Pending` never changes | Machine Configuration extension missing or unhealthy | VM Extensions + applications; require a supported version |
| Run Command returns an empty stdout | Guest helper requires PowerShell 7 but Azure invoked Windows PowerShell 5.1 | Keep guest scripts 5.1-compatible and inspect the stderr component |
| Package download returns 403 | UAMI lacks Blob data-plane access | Container IAM, role propagation, VM identity attachment |
| Download fails with public access off | Private DNS or Private Endpoint problem | Resolve the normal Blob hostname from the VM and inspect the zone group |
| Hash mismatch | Policy `contentHash` differs from the uploaded bytes | Recalculate SHA-256 after the final ZIP is created |
| Registry repair fails with “specify Force” | `PSDscResources` refuses to replace an existing registry value | Set `Force = $true` on controlled registry resources, rebuild the package, and publish the new hash |
| Feature resource says it must run as Administrator | Incompatible `PSDscResources` Windows feature resource is running in the PowerShell 7 worker | Use the packaged class-based Windows PowerShell bridge and inspect the guest-worker log |
| Feature changed but the assignment stays Pending | Redirected child-process output deadlocked or timed out | Drain stdout/stderr concurrently, bound the child lifetime, and verify no bridge process remains |
| Assignment says autocorrect but Windows only audits | LCM `configurationMode` remained `MonitorOnly` | Set both the assignment type and configuration setting to `ApplyAndAutoCorrect`; verify them through REST |
| Correct assignment remains Pending while another worker is active | Another guest assignment is ahead in the extension's local evaluation queue | Inspect `gc_worker.exe` assignment names; wait for unrelated evaluation rather than deleting it |
| Audit detects drift but changes nothing | Expected `Audit` behavior | Promote a new `AuditAndSet` artifact and use remediation |
| Existing VM does not receive the enforcement assignment | DeployIfNotExists has not run for existing resources | Create a remediation task for the correct initiative member |
| Auto-correction appears to have no drift history | Local correction completed before the next cloud report | Correlate Run Command timestamps and workshop events in Log Analytics |

## Production guidance

Treat the package, policies, and initiative as one release. Put the DSC source, module lock information, generated JSON, hashes, and test evidence in version control. A pipeline should compile and test the package, upload a new version, verify the final hash, generate definitions, and require review before assignment. Use workload identity federation/OIDC so the pipeline stores no client secret.

Promote by ring. Begin with audit on a small representative population. Review detailed drift reasons and identify settings another management system owns. Move to `AuditAndSet` only after change approval and a rollback decision. Expand from pilot to broad rings through assignment or tag strategy, never by silently replacing package bytes.

Define ownership carefully. A DSC resource with `Force = $true` declares that Machine Configuration owns that value. Avoid managing the same registry value, service, or Windows feature with Group Policy, an image-hardening script, another DSC engine, and Machine Configuration simultaneously. Competing controllers create oscillation and noisy compliance.

Keep runtime identity read-only and scoped to a dedicated package container. Separate authoring permission from consumption permission. Use Private Endpoints and centrally managed Private DNS, and monitor for public access being re-enabled. If multiple regions consume packages, design DNS, endpoint, and Storage resilience deliberately instead of assuming one regional endpoint is sufficient.

At scale, use Microsoft's prerequisite policies or initiatives to deploy the supported extension and managed-identity prerequisites. Track extension versions, failed provisioning, assignments that remain pending, stale `lastComplianceStatusChecked` timestamps, download failures, remediation failures, and role-assignment propagation errors.

Use native Machine Configuration and Azure Policy state as the compliance system of record. Log Analytics provides supporting evidence—agent-adjacent events, drift markers, and operational correlation—but it does not replace the guest report. Export compliance into the central governance workflow and alert on stale or suddenly expanding non-compliant populations.

Finally, plan removal as carefully as rollout. Stop or delete enforcement assignments before deleting packages or identities. Remove guest assignments and remediation tasks, allow agents to converge, and only then retire definitions or Storage content. This avoids stranded assignments repeatedly requesting unavailable packages.

## Cleanup

Preserve the sanitized ZIP first. Then remove exact workshop scope:

```powershell
.\scripts\Remove-AzureMachineConfigurationWorkshop.ps1 `
  -SubscriptionId $subscriptionId `
  -ResourceGroupName rg-machine-configuration-deep-dive-we `
  -Execute
```

The cleanup script removes remediation tasks and policy assignments before guest-assignment child resources, custom initiatives and definitions, role assignments, and finally the lab resource group. It never targets another resource group.

![Zero-resource cleanup verification.](screenshots/21-cleanup-verification.png)

## What you proved

- Private Windows VMs can consume a custom configuration package without a password, account key, or SAS token.
- Azure Policy ring selection is deterministic and sensitive to tag values.
- Audit reports Windows drift without repairing it.
- A missing extension prevents guest evaluation even when policy scope is correct.
- A versioned `AuditAndSet` package plus remediation brings existing machines under control.
- `ApplyAndAutoCorrect` repairs later drift locally without another remediation task.
- Azure Policy, Machine Configuration, Resource Graph, REST, CLI, and Log Analytics can be correlated without exposing secrets or identity values.

## Download the lab

The downloadable bundle contains the article source, PowerShell scripts, generated policy JSON, immutable package hashes, sanitized validation transcript, publishing brief, cover, and ordered screenshots. It excludes Azure profiles, credentials, tokens, client IPs, subscription/tenant IDs, identity GUIDs, raw ARM IDs, temporary role assignments, and unsanitized API responses.

## References

- [Azure Machine Configuration overview](https://learn.microsoft.com/en-us/azure/governance/machine-configuration/)
- [Create a custom Machine Configuration package](https://learn.microsoft.com/en-us/azure/governance/machine-configuration/how-to/develop-custom-package/2-create-package)
- [Secure package access with managed identity](https://learn.microsoft.com/en-us/azure/governance/machine-configuration/how-to/develop-custom-package/5-access-package)
- [Machine Configuration remediation options](https://learn.microsoft.com/en-us/azure/governance/machine-configuration/concepts/remediation-options)
- [Migration from Azure Automation State Configuration](https://learn.microsoft.com/en-us/azure/governance/machine-configuration/whats-new/migrating-from-azure-automation)
