What If Tool to Test Conditional Access Policies in Entra ID

Conditional Access policies in Entra ID operate as simple “if-then” statements designed to manage and secure user access within your organization. For example, a policy might state, “If a user tries to sign in from an unknown device, then block access.” While these Conditional Access policies are powerful, they can sometimes cause unexpected issues. Therefore, it’s essential to test Conditional Access policies and validate them thoroughly before applying them broadly. Traditionally, administrators would test these policies manually by signing in as users to see how the policies would work in real-life scenarios. To simplify this, Microsoft rolled out the Conditional Access What If tool.

This tool helps you test Conditional Access policies in Azure AD. It gives you a clear picture of how your policies will work and enables you to deploy them effectively.

The Conditional Access What If tool in Microsoft Entra ID is a built-in feature that lets you simulate sign-in scenarios to see how Conditional Access policies will affect a specific user in real-time. It allows you to create a test scenario where you can evaluate how the system will respond if a user attempts to sign in. This is useful for testing authentication processes, security measures, and user experience without involving a real user.

By using this What If tool in Azure AD, you can pinpoint which policies apply to a specific user under various conditions and understand why certain policies might not apply. It validates policies that are both in production and those set to ‘report only’ mode. Simply put,

“The What If tool helps you see which Conditional Access policies are in play and why others aren’t, so you can better understand and troubleshoot access issues.”

To use the Conditional Access policies What If feature, follow these steps:

• Go to the Microsoft Entra admin center. Select Protection → Conditional Access → Policies → ‘What If’ in the top bar.

• On the What If page, select the user or service principal you want to test. Note that you can only test one user or service principal at a time.
• If needed, you can also specify the cloud apps you want to test for that user.
• Other conditions, such as client apps or device platforms, are optional and can be configured as needed.
• After selecting the user and configuring any optional conditions, click the ‘What If’ button.

The evaluation results will be displayed under two tabs:

• Policies that will apply: Lists the policy names, grant controls, session controls, and policy states that will affect the selected user.
• Policies that will not apply: Shows policies that won’t affect the user, along with the reasons why, policy names, and states.

Now, from the evaluation report, we can understand that the selected user is impacted by only two Conditional Access policies configured in your organization. This insight enables you to analyze these policies further and monitor changes to you CA policies over time.

Your organization needs to ensure that multi-factor authentication (MFA) is enforced for external users accessing SharePoint Online from untrusted or high-risk locations. Since we cannot manually sign in with an external user account to test whether the Conditional Access policy is working as expected, we can use the “What If” tool to virtually simulate the scenario. This allows us to test how the policy would be applied and verify if it enforces the desired conditions correctly.

• User or Workload Identity: Select the external user type you want to include in the test. Note that you cannot add specific external users to the test. Instead, you must provide the tenant ID.
• Country: Brazil (assumed to be an untrusted location for this scenario)
• IP Address: 190.25.30.45 (representing an untrusted or high-risk IP address)
• Device Platform: Windows
• Sign-In Risk: High
• User Risk: Medium

On running the test, the ‘What If’ tool will return the Conditional Access policies that are applied based on the provided parameters. You should verify whether MFA is required for the external user accessing SharePoint Online from Brazil with a high sign-in risk and medium user risk. This helps ensure that your policies are correctly configured to enforce MFA for high-risk scenarios involving external users.

Similarly, you can also execute Conditional Access What If tests with Maester’s “Test-MtConditionalAccessWhatIf” cmdlet.

The new What If experience leverages Microsoft Graph to offer enhanced evaluation capabilities and is currently in preview.

Continuous Access Evaluation (CAE) for Workload Identities

CAE is supported only for access requests sent to Microsoft Graph as a resource provider. While Continuous Access Evaluation in Azure AD manages real-time enforcement, the enhanced What If tool allows you to simulate and understand how these policies would function under various scenarios, giving you a clearer view of their potential impact.

Improved Insights

The new What If tool includes detailed evaluation capabilities for advanced conditions, enabling you to see how policies involving insider risk and authentication flows affect user sign-ins. Note that these preview features need to be toggled on for taking effect.

Wrap Up

The Conditional Access What If tests are a powerful way to test and understand the impact of your policies before they go live. With the new enhancements powered by Microsoft Graph, you can now evaluate more complex scenarios and ensure that your Conditional Access policies are both effective and secure. You can also leverage other tools like report-only mode and Conditional Access gap analyzer workbook to further evaluate and monitor the effects of your policies.

We hope this blog has provided valuable insights on how to troubleshoot conditional access using the What If Tool in Entra ID. Thanks for reading! If you have any questions or need further assistance, please leave a comment below.