# Application Gateway WAF — Windows IIS troubleshooting workshop

Start with [article.html](article.html) for the illustrated workshop. [ARTICLE.md](ARTICLE.md) is its editable source. Use this complete extracted package: the main PowerShell controller requires the adjacent infrastructure and helper files.

## Release status

**Live Azure validation passed on 6 November 2026.** All three fault/recovery cycles, final log correlation, no-drift checks and repeated cleanup passed. All 27 pre-existing resources were preserved. The release contains 20 genuine Portal/CLI captures. See [VALIDATION.md](VALIDATION.md), the [complete sanitized acceptance export](docs/evidence/sanitized-evidence.json), and [seven curated request/log examples](docs/evidence/request-log-correlations.json). Archive integrity is recorded separately after packaging in the external receipt.

## Environment

- Windows operator workstation with PowerShell 7.6.9, Azure CLI 2.88.0, Terraform 1.15.8, and Windows curl.exe 7.70 or newer.
- AzureRM 5.8.0 is pinned; retain the committed provider lockfile.
- West Europe; one dedicated disposable resource group.
- WAF_v2 Application Gateway, one private Standard_B2s Windows Server 2025 IIS VM, explicit NAT egress and a dedicated workspace.
- Fresh frontend/backend certificates issued by the workshop CA. Tests trust that CA explicitly and verify hostnames.
- Microsoft Default Rule Set 2.2, WAF Prevention, resource-specific access/firewall logs.
- Operator rights to manage the isolated deployment, invoke Run Command, and remove its resources. The scripts do not elevate the operator or silently register providers.

Authenticate using your approved process before running the workshop. Explicitly select your intended isolated subscription and inspect preflight before applying changes. A new Preflight records the supplied full subscription and tenant; later stages remain bound to that exact context.

## Time and cost

Allow 3–5 hours, including asynchronous provisioning and ingestion. Total operating allowance is €20, including €5 reserved for cleanup. Preflight must obtain a conservative six-hour estimate of no more than €15 for active resources. Gateway fixed and capacity charges, Windows VM, disk, NAT, public IP, logs and expected traffic belong in the estimate.

Autoscale minimum zero does not remove the gateway's fixed charge. These checks are an operational limit rather than an Azure-enforced spending cap. Start cleanup by 4½ hours and verify removal before six hours from the new Preflight. This clock starts before any paid resources are created and is not reset by rerunning Preflight. Never leave the deployment running while debugging documentation or packaging.

## Private state

Open PowerShell in this folder. State must be outside the public package and any Git repository:

~~~powershell
$private = Join-Path $env:LOCALAPPDATA ('AzureWorkshopz-Waf-' + [guid]::NewGuid().ToString('N'))
$state = Join-Path $private 'state.json'
az login
az account set --subscription '<your-subscription-id>'
$subscriptionId = (az account show --query id --output tsv).Trim()
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Preflight -StatePath $state -SubscriptionId $subscriptionId
~~~

Keep the private ledger, plans, Terraform state, generated passwords, certificate private keys, raw logs and unredacted screenshots outside the package and any Git repository. The example uses LocalAppData for that reason. Terraform output redaction does not encrypt state or saved plans. Publish only reviewed sanitized evidence.

## Guided stage sequence

~~~powershell
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Plan -StatePath $state
# Inspect the exact saved plan before continuing.
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Deploy -StatePath $state -Execute -AcknowledgeCost -ReviewedPlan
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Bootstrap -StatePath $state -Execute
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Validate -StatePath $state -Execute
~~~

The published fixtures/waf-exclusions.json contains the live-calibrated group-to-rule map, and fixtures/waf-fixture.json contains its request payload. Plan automatically loads these packaged defaults for a new run. Do not substitute legacy rule-group names or assume a rule from another version will match. Authoring can override the map with Plan -WafExclusionRulesJson and the payload with -WafFixture; the ledger retains the exact chosen values. The legitimate request must succeed before the WAF fault cycle starts.

The exclusion belongs to the gateway-associated WAF policy and applies across every path inspected by that policy, not only /docs. Its boundary is the exact field name labSnippet plus the three selected XSS rules; it is not a path restriction or an allowlist of specific snippet contents. Other fields and other rules, including aggregation rule 949110, remain inspected. The static documentation endpoint supplies the teaching request; it does not limit the policy exclusion to that path.

Run one scenario at a time. Always begin with a fresh healthy validation:

~~~powershell
$scenario = 'Probe' # Then repeat for Tls and Waf.
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Validate -StatePath $state -Execute
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Fault -Scenario $scenario -StatePath $state -Execute
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Validate -Scenario $scenario -StatePath $state -Execute
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Repair -Scenario $scenario -StatePath $state -Execute
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Validate -Scenario $scenario -StatePath $state -Execute
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Repair -Scenario $scenario -StatePath $state -Execute
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Validate -Scenario $scenario -StatePath $state -Execute
~~~

| Scenario | Owned change | Expected fault | Required repair |
|---|---|---|---|
| Probe | Exercise probe path | Target route 502, missing-path health reason; control 200 | Restore /health.txt; both routes healthy |
| Tls | Exercise backend hostname | Target route 502, certificate/name health reason; control 200 | Restore backend.lab.test; validation stays enabled |
| Waf | Exact owned managed-rule exclusion | Legitimate labSnippet request 403; control 200 | Restore exact field/rule exclusion; legitimate request 200 and untrustedSnippet still 403 |

The harmless custom marker labMarker=awz-block-me is a separate enforcement control. Its 403 cannot establish that a managed rule matched the legitimate fixture.

## Final validation and cleanup

~~~powershell
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Validate -StatePath $state -Execute -CollectLogs
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Plan -StatePath $state
# Require no unexpected changes after all repairs.
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Cleanup -StatePath $state
# Inspect the exact cleanup plan.
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Cleanup -StatePath $state -Execute -ReviewedPlan
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Cleanup -StatePath $state -Execute -ReviewedPlan
./Invoke-AppGatewayWafWorkshop.ps1 -Stage Export -StatePath $state
~~~

The group and exact recorded resources must be absent. Repeated cleanup must create nothing. Keep the ledger until residual checks finish. If an unexpected group member or ownership mismatch blocks deletion, inspect and reconcile it rather than broadening the delete target.

## Interrupted operation

Do not start another apply, erase state, clear a live lock, or rerun a competing installer. Confirm the original process has ended; inspect the private journal and the exact live operation. Use -Resume only for the recorded interrupted stage after reconciliation. A local timeout does not cancel an operation already accepted by Azure.

## Private CA and Windows curl

The disposable workshop CA has no CRL distribution service. Windows Schannel curl uses --ssl-revoke-best-effort (introduced in curl 7.70) so unavailable revocation-list distribution points do not prevent the lab baseline. Explicit CA trust and hostname verification remain active. The option applies only to the Windows Schannel client; it is not the intentional backend TLS fault. See the [curl documentation](https://curl.se/docs/manpage.html#--ssl-revoke-best-effort). Production PKI needs an appropriate revocation service and lifecycle.

## Capture execution context

The local operator versions above drive the deployment. Some genuine CLI captures use Azure Cloud Shell, whose own Azure CLI/PowerShell versions are recorded in capture provenance. Captures 01 and 16 show retained local preflight/plan and final-validation/Terraform reports displayed in real Cloud Shell. Recovery captures 08, 10 and 15 combine labelled retained local controller reports with newly executed Cloud Shell curl checks. Captions preserve original execution times and distinguish the later display from fresh commands; no retained report is described as a new Terraform/controller execution in Cloud Shell.

## Downloaded automation files

| File | Purpose |
|---|---|
| [Invoke-AppGatewayWafWorkshop.ps1](Invoke-AppGatewayWafWorkshop.ps1) | Staged deployment, validation, fault/repair, cleanup and evidence export |
| [Workshop.Common.psm1](scripts/Workshop.Common.psm1) | Private state/context, bounded native commands, locking and cost helpers |
| [New-WorkshopCertificates.ps1](scripts/New-WorkshopCertificates.ps1) | Fresh CA and frontend/backend certificates kept in the private directory |
| [New-IisBootstrapScript.ps1](scripts/New-IisBootstrapScript.ps1) | Generates the Windows PowerShell guest bootstrap, including embedded static IIS content for all four routes, consumed by the controller |
| [Export-BackendHealthEvidence.ps1](scripts/Export-BackendHealthEvidence.ps1) | Reads protected native Azure CLI journals and exports allowlisted real fault/recovery health observations with source hashes |

The backend-health helper takes -PrivateLogDirectory and -OutputPath. It reads retained observations; it does not query Azure or create test results. Keep the source journal directory private and review the allowed output before sharing it.

## Learning and evidence

- [Challenge](CHALLENGE.md), [hints](HINTS.md), [worked solution](SOLUTION.md), and [eight knowledge questions](KNOWLEDGE-CHECK.md).
- [Capture guide](docs/screenshot-guide.md), [capture inventory: 18 required plus two supplemental](docs/capture-plan.json), and [primary source map](docs/source-map.md).
- [Access logs](queries/01-access-diagnostics.kql), [WAF decisions](queries/02-waf-decisions.kql), [transaction correlation](queries/03-correlate-transactions.kql).

To rebuild the HTML after editing documentation, run python docs/build_article.py. This authoring helper needs Python's markdown package. It is not needed to run the Azure lab.

This is independent community training, not an official Microsoft lab or exam endorsement.