Executing M365 CIS Assessment

• Nirmal@DynamicPacks.net
• May 10, 2024
• No Comments

SmartProfiler is a Desktop Application designed to perform security, health and risk assessment of Microsoft 365, Active Directory and Azure Virtual Desktop tenants. When performing an assessment of technologies, the SmartProfiler requires necessary permissions to the Assessment target. SmartProfiler can also be used for executing M365 CIS Assessment. There are various options/tools available with SmartProfiler Microsoft 365 as shown in the screenshot below:

• ASSESSMENT CONSOLE: Assessment Console under Microsoft 365 Security is used to execute assessment.
• ASSESSMENT SUMMARY: Assessment Summary is used to show assessment summary for each test.
• GENERATE REPORT: If you need to generate a report, please use Generate Report console.

Microsoft 365 CIS Assessment Requirements

Below table provides a guidance on what permissions to be made available when running assessment for Microsoft 365:

Microsoft 365 Tenant

• Global Admin Account
• Global Admin Account to be a Non-MFA Account
• Global Admin Account to be part of following Microsoft 365 Roles:

1. Global Reader
2. Message Center Privacy Reader
3. Message Center Reader
4. Reports Reader
5. Security Reader
6. Usages Summary Report Reader

Note: It is recommended that Global Admin Account is part of all Microsoft 365 Reader Roles.

Mobile Device Management Category

Admin Consent for Microsoft.Graph module for below Read Permissions:

• DeviceManagementApps.Read.All
• DeviceManagementConfiguration.Read.All
• DeviceManagementServiceConfig.Read.All
• Directory.Read.All

Registering Microsoft 365 Tenant

To use Microsoft 365 CIS Assessment with SmartProfiler, please log on to SmartProfiler using a Microsoft 365 Tenant which is already registered. If the Microsoft 365 Tenant is not registered, then you are required to register the Tenant.

If you are registering the first Tenant in the SmartProfiler, then click on the “Register Tenant” button on the login screen. The Managed Tenants screen will be shown:

Click on the “Add New Tenant” button as shown in the red circle in the screenshot above, which, in turn, will show registration screen:

To register M365 Tenant, click M365 Tenant Tab. Registering M365 Tenant in SmartProfiler requires below inputs:

• Office 365 Domain: Enter the Microsoft 365 onmicrosoft.com domain name. For example, DynamicPacksnet.OnMicrosoft.com
• Global Reader: We have recently changed the way Microsoft 365 works. Instead of Global Reader a Global Admin account is required.
• Password: Enter the password for Global Admin account.

Client ID, Certification Thumbprint and Tenant ID are optional unless you are using SmartProfiler Scheduler for Microsoft 365.

Note that the registration process for a Microsoft 365 Tenant will actually check the status of the Microsoft 365 domain entered and global admin account to ensure domain is verified and global admin account has access to the tenant.

Once done, click on “Update/Register” button to add tenant under the management of SmartProfiler.

Once the Microsoft 365 Tenant is registered with SmartProfiler, apply the license file. To apply license file, click on the Microsoft 365 Tenant name that you just registered in the list of registered tenants.

And then click on “Browse” button to browse the license file. After selecting the current license file, you will see a message asking if the license file needs to be applied to the select Microsoft 365 Tenant:

Click “Yes” to apply for the licenses to selected Microsoft 365 Tenant. If the license file is applied successfully, you will see “REGISTERED” in green color in front of the Microsoft 365 Tenant in the list.

Executing M365 CIS Assessment

Once the Tenant has been registered with SmartProfiler, you need to open the M365 Tenant. After opening Microsoft 365 Tenant, click on “Execute Assessment” button found under Microsoft 365 Security section in left pane.

Note that SmartProfiler for Microsoft 365 not just supports executing CIS Tests, it also supports executing various tests designed by our Microsoft 365 Expert Team. To give you an overview of the tests that we have included in the SmartProfiler are explained below:

• 114 CIS Tests: These tests are from CIS Version 3.1 for Microsoft 365 Foundation Benchmark.
• 131 SmartProfiler Tests: These tests are available to execute as part of the execution process and are designed by learning from our clients around the world.

Note that SmartProfiler execution console provides executing below tests:

• CIS V3.0 Tests: If you select this, SmartProfiler execution process will only execute the tests from CIS.
• SmartProfiler V1.0 Tests: If you select this option, the SmartProfiler will only execute SmartProfiler Tests. All CIS Tests will not be executed.
• CIS and SmartProfiler Tests: If youselect this radio button, execution process will execute both CIS and SmartProfiler tests.
• Selected M365 Tests: If you select this option, only the tests selected in the grid will be executed.

Note that if the Admin Consent is not granted, the below screen will show “Not Granted” in red color.

It is important to grant admin consent to run some of the tests which require permissions to connect to Microsoft 365 Tenant.

Granting Admin Consent

Granting Admin Consent to Microsoft.Graph module is required to perform assessment of Mobile Device Management. There are 22 tests executed as part of Mobile Device Management category which requires “READ-ONLY” permissions to Microsoft.Graph App. The following list of permissions are needed for assessment of Mobile Device Management:

• DeviceManagementApps.Read.All
• DeviceManagementConfiguration.Read.All
• DeviceManagementServiceConfig.Read.All
• Directory.Read.All

There are three ways to grant Admin Consent so assessment can cover MDM category as well as listed below:

• Using the SmartProfiler Admin Consent Process.
• By manually running PowerShell commands from SmartProfiler computer
• Using Azure AD Portal

Note: Admin Consent process is one-time.

Using the SmartProfiler Admin Consent Process

SmartProfiler supports granting Admin Consent for InTune by executing a series of PowerShell commands. To grant Admin Consent using SmartProfiler, click on Play icon shown below:

And then click on “Grant” button:

In the above window, click on “Grant”. When you click on the button, you will be presented with a Microsoft Login Prompt to enter Global Administrator credentials and then will be asked to grant “READ-ONLY” permissions to Microsoft.Graph app.

By manually running PowerShell commands from SmartProfiler computer

If you do not want to use SmartProfiler-way to grant Admin Consent to Microsoft-Graph app then you can execute below PowerShell commands on the SmartProfiler computer.

1. Open a PowerShell window in elevated prompt.
2. Execute Import-Module “C:\Users\Public\SmartProfiler\SmartProfilerAssessment\Modules\microsoft.graph.intune.6.1907.1\Microsoft.Graph.Intune.psd1”
3. Once modules have been imported, execute:

Connect-MSGraph

Update-MSGraphEnvironment -SchemaVersion Beta

• In the next step the process will check if the Admin Consent has already been granted to Microsoft.Graph. If not granted, then you will be presented with a prompt as shown below:

You need to check the box “Consent on behalf of your organization” and then click on “Accept” button to continue.

Once the Admin Consent has been granted to Microsoft.Graph all tests including tests in Mobile Device Management category can be executed using a Global Reader Account.

Using Azure AD Portal

If you would like to use Azure AD Portal to grant permissions for Microsoft.Graph, please follow steps outlined here.

Once the Microsoft 365 Tests have been loaded, you can click on “Execute Assessment” button to start the assessment. While the assessment is in progress, you can see the assessment progress in the top bar. After the assessment has been completed you will see a message indicating the assessment has been completed successfully.

Microsoft 365 CIS Assessment Summary

In the Assessment Summary window, you can see the issues that were detected for Microsoft 365 Tenant and impact and recommendation for each issue. When clicked on “Assessment Summary” button the summary window will populate all issues in each Microsoft 365 Assessment category as shown below:

As you can see in the screenshot above, SmartProfiler has reported high, medium, low and non-compliance issues. It has also reported the items that have been passed successfully. If you need to see data for a test, click on test.

Here are the columns that are displayed as part of the Microsoft 365 Assessment and their meaning:

• No.: Shows test number.
• Category/Test: Showstest name that was executed.
• Severity: Shows severity of the test. The severity can be Critical, High, Medium, Low, Passed, Completed, and Not Executed.
• Control: This column shows if the test belongs to CIS Control or not. You will see SmartProfiler Control 1.0 or CIS Control Version 1.5

Note: SmartProfiler for Microsoft 365 includes all CIS Control tests for Microsoft 365 foundation. However, some of the tests have been designed by our Microsoft 365 experts to ensure every aspect of Microsoft 365 Subscription is checked and reported as part of the report.

• CIS Profile: Shows CIS Profile type for CIS Test.
• Compliance: Shows compliance status for the test. The reported data is “Compliant” or “Non-Compliant”.
• Affected Objects: This column shows the number of affected objects for each test. If showed “0 or NA” that means the test doesn’t have affected objects or affected objects is not applicable to the current test.
• Impact: Associated impact text with the test.
• Recommendation: Associated recommendation text with the test.

Generating Report

To generate a report for Microsoft 365 Tenant, click on “Generate Report” button found under Microsoft 365 Security section in left pane.

Note that SmartProfiler for Microsoft 365 is capable of generating a Microsoft Word Report and an excel summary which contains the affected objects list for each test.

Click on the Browse button to specify the report location and check/uncheck various other options as explained below:

• Include Reference Link and Default Value: If you check this option, SmartProfiler is going to include reference link for each test and its default values. The option is checked by default.
• Include SmartProfiler Tests: If you would like to include SmartProfiler tests in the report, please check this option.
• Include “Not Executed” and “Passed” tests in same section: If you check this option all not executed and passed items will be executed in the same section of each category. Unchecking this option will instruct reporting process to include all passed and not executed items in a separate section in word document.

Once you have checked/unchecked required options, please click on “Generate Report” button. The process will take some time and progress will be shown on the screen.

All CIS Tests that are covered by SmartProfiler can be found on CIS website. Note that SmartProfiler supports latest CIS Version 3.0.