Executing M365 CIS Assessment

• Nirmal@DynamicPacks.net
• May 10, 2024
• No Comments

SmartProfiler is a Desktop Application designed to perform security, health, risk and compliance assessment of Microsoft 365, Active Directory, Azure Virtual Desktop, and other technologies supported by SmartProfiler-SecID. When performing an assessment of technologies, the SmartProfiler requires necessary permissions to the Assessment target.

Microsoft 365 CIS Assessment Requirements

M365 CIS Assessment support two connection methods; Executing Assessment by Entering Credentials in MS Authentication Prompt and STORED-CRED. There are two types of apps that SmartProfiler can utilize to make connections with M365 Services; Built-In Microsoft Graph PowerShell or Custom Entra App.

SmartProfiler cannot generate an Entra App automatically with necessary permissions – we need you to create a custom App and then apply those permissions depending on the authentication method you select.

M365 Assessment supports two connection methods:

• Entering Credentials in MS Login Prompt
• STORED-CRED

Entering Credentials in MS Login Prompt

• This method shows you Microsoft Login prompt for connecting to all M365 Services and require you to enter credentials 6-7 times.
• It uses Connect-MGGraph -Scope “<necessary permissions>”, which, in turn, applies permissions to Built-In Microsoft Graph PowerShell.
• The connection to the rest of M365 Services will be made using Global Reader account.
• The disadvantage with this connection method is that you cannot use other functions of SmartProfiler such as “Health Check” and “Assessment Scheduler” for M365.

STORED-CRED

• In STORED-CRED connection method, you create a Custom Entra App and apply those required Application Permissions.
• Here Connect-MGGraph will connect to Custom Entra App.
  • In this case, Built-In Microsoft Graph PowerShell will not be touched.
• The connection to the rest of M365 Services will be made using Global Reader account.
• The Advantage here is that you can run unattended assessment and also use “Health Check” and “Assessment Scheduler” for M365.

Executing Assessment by Entering Credentials in MS Login Prompt

The MS Authentication Prompt connection method can be used when you want to execute M365 CIS Assessment using a Global Reader or Global Admin Account. Please note that not all tests can be executed using the Global Reader account unless Global Reader account is part of certain M365 Roles.

Using a Global Reader account

If you plan to execute M365 CIS Assessment using a Global Reader account, then make sure Global Reader is part of below Microsoft 365 roles:

• Compliance Administrator
• Compliance Data Administrator
• Global Reader
• SharePoint Administrator

If you are using a Global Admin for M365 CIS Assessment, then no need to add user to any of the M365 roles. Note that you will be prompted to enter credentials 6-7 times if you use “Entering Credentials in MS Authentication Prompt” connection method.

Executing Assessment using STORED-CRED Connection Method

The STORED-CRED Connection method can be used if you:

• Plan to execute Unattended Assessment – You will not be prompted to enter credentials.
• Plan to use Health Monitoring feature of SmartProfiler-SecID for Microsoft 365 Tenants
• Plan to use M365 Assessment Scheduler.

Note that you will still be required to use a Global Admin or Global Reader account if you plan to use STORED-CRED option and Global Reader or Admin account must be a non-MFA account.

To use the STORED-CRED option you are required to create an Entra App and assign the following permissions to the Entra App:

• AccessReview.Read.All
• AuditLog.Read.All
• AuthenticationContext.Read.All
• Directory.Read.All
• DirectoryRecommendations.Read.All
• IdentityProvider.Read.All
• IdentityRiskyUser.Read.All
• Mail.Read
• MailboxSettings.Read
• MultiTenantOrganization.Read.All
• MultiTenantOrganization.ReadBasic.All
• OnPremDirectorySynchronization.Read.All
• Organization.Read.All
• Policy.Read.All
• Policy.Read.ConditionalAccess
• RoleManagement.Read.All
• SecurityActions.Read.All
• SecurityEvents.Read.All
• SharePointTenantSettings.Read.All
• ThreatIndicators.Read.All
• User.Read
• User.Read.All
• UserAuthenticationMethod.Read.All
• RoleAssignmentSchedule.Read.Directory

And “Reader” IAM role has been assigned to Entra App to the subscription.

Once you have meet above requirements, you can proceed with next steps.

Registering Microsoft 365 Tenant

To register Microsoft 365 Tenant with SmartProfiler-SecID, please expand Tenants and Settings Section in left pane and then click on “Add New Tenant” button as shown below:

Then in the register a new tenant, select Microsoft 365 in the list of available technologies and provide the Tenant details as below:

SmartProfiler requires below inputs for M365 Tenants:

• M365 Domain: Enter the Microsoft 365 onmicrosoft.com domain name. For example, DynamicPacksnet.OnMicrosoft.com.
• Global Reader: We have recently changed the way Microsoft 365 works. Instead of Global Reader a Global Admin account is required.
• Password: Enter the password for Global Admin account.
• Is MFA Enabled?: Type Yes or No
• Azure Tenant ID: Enter the Azure Tenant ID here if you plan to use STORED-CRED option.
• Azure App ID: Enter the Azure App ID here if you plan to use STORED-CRED option.
• Azure App Secret: Please note to enter the Secret you need to enter the secret in box below the grid and then click on Apply button to apply the secret.

• Azure Subscription ID: Enter the Azure Subscription ID here if you plan to use STORED-CRED option.
• Is a GOV Cloud? : Type Yes or No.

If you have purchased M365 CIS Assessment license, then click on Browse button to apply the license codes from the license file.

Note that the registration process for a Microsoft 365 Tenant will actually check the status of the Microsoft 365 domain entered and global admin account to ensure domain is verified and global admin account has access to the tenant.

Once done, click on “Add Tenant” button to add tenant under the management of SmartProfiler.

Creating an Assessment View

Note that you need to create an assessment view before the assessment can be executed. To create an Assessment View, expand Tenants and Settings section in left pane and then click on “Manage Settings” button:

And then click on “Create View” button. In the Create a New View Window, provide the following inputs:

• View Name: The Assessment View name can be any name that you would like to provide.
• Select Assessment Technology: Select Microsoft 365 CIS v3.1.0 in the list of technologies.
• Select Tenant: Select the Tenant that you added recently.
• Select Test Template: Leave it default

Once you have provided details for a new Assessment View, click on “Create View” button to create the view.

Executing M365 CIS Assessment

Once the Tenant has been registered with SmartProfiler and you have create an Assessment View, you can expand “Assessment Views” section in the left pane to see your view, expand the Assessment View and then click on “Assessment Console” to open the assessment console as shown below:

After clicking on Assessment Console, you will see list of tests available in console.

Admin Consent

Note that if the Admin Consent is not granted, the below screen will show “Not Granted” in red color. Please note that Admin Consent is only required if you plan to use “Entering Credentials in MS Authentication Prompt” connection method.

It is important to grant admin consent to run some of the tests which require permissions to connect to Microsoft 365 Tenant.

Granting Admin Consent

There are three ways to grant Admin Consent:

• Using the SmartProfiler Admin Consent Process.
• By manually running PowerShell commands from SmartProfiler computer
• Using Azure AD Portal and assigning necessary permissions to Microsoft.Graph Enterprise Application.

Note: Admin Consent process is a one-time process.

Using the SmartProfiler Admin Consent Process

SmartProfiler supports granting Admin Consent by executing a series of PowerShell commands. To grant Admin Consent using SmartProfiler, click on “GRANT” button shown below:

When you click on the button, you will be presented with a Microsoft Login Prompt to enter Global Administrator credentials and then will be asked to grant “READ-ONLY” permissions to Microsoft.Graph app.

By manually running PowerShell commands from SmartProfiler computer

If you do not want to use SmartProfiler-way to grant Admin Consent to Microsoft-Graph app then you can execute below PowerShell commands on the SmartProfiler computer.

1. Open a PowerShell window in elevated prompt.
2. Execute Import-Module “C:\Users\Public\SmartProfiler\SmartProfilerAssessment\Modules\microsoft.graph.intune.6.1907.1\Microsoft.Graph.Intune.psd1”
3. Once modules have been imported, execute:
4. Connect-MgGraph -ContextScope Process -Scopes “AccessReview.Read.All”, “AuditLog.Read.All”, “AuthenticationContext.Read.All”, “Directory.Read.All”, “DirectoryRecommendations.Read.All”, “IdentityProvider.Read.All”, “IdentityRiskyUser.Read.All”, “Mail.Read”, “MailboxSettings.Read”, “MultiTenantOrganization.Read.All”, “MultiTenantOrganization.ReadBasic.All”, “OnPremDirectorySynchronization.Read.All”, “Organization.Read.All”, “Policy.Read.All”, “Policy.Read.ConditionalAccess”, “RoleManagement.Read.All”, “SecurityActions.Read.All”, “SecurityEvents.Read.All”, “SharePointTenantSettings.Read.All”, “ThreatIndicators.Read.All”, “User.Read”, “User.Read.All”, “UserAuthenticationMethod.Read.All”, “RoleAssignmentSchedule.Read.Directory”, “Application.Read.All”

• In the next step the process will check if the Admin Consent has already been granted to Microsoft.Graph. If not granted, then you will be presented with a prompt as shown below:

You need to check the box “Consent on behalf of your organization” and then click on “Accept” button to continue.

Once the Admin Consent has been granted to Microsoft.Graph all tests including tests in Mobile Device Management category can be executed using a Global Reader Account.

Using Azure AD Portal

If you would like to use Azure AD Portal to grant permissions for Microsoft.Graph, please go to Enterprise Applications in Azure portal and then apply those permissions to Microsoft.Graph PowerShell app.

Executing CIS Assessment

To start the execution, select the credential from the credential dropdown:

And then click on “Execute Assessment” button to start the M365 CIS Assessment.

Note that SmartProfiler for Microsoft 365 not just supports executing CIS Tests, it also supports executing various tests designed by our Microsoft 365 Expert Team. To give you an overview of the tests that we have included in the SmartProfiler are explained below:

• 114 CIS Tests: These tests are from CIS Version 3.1 for Microsoft 365 Foundation Benchmark.
• 101 SmartProfiler Tests: These tests are available to execute as part of the execution process and are designed by learning from our clients around the world.

Note that SmartProfiler execution console provides executing below tests:

• CIS V3.0 Tests: If you select this, SmartProfiler execution process will only execute the tests from CIS.
• SmartProfiler V1.0 Tests: If you select this option, the SmartProfiler will only execute SmartProfiler Tests. All CIS Tests will not be executed.
• CIS and SmartProfiler Tests: If youselect this radio button, execution process will execute both CIS and SmartProfiler tests.
• Selected M365 Tests: If you select this option, only the tests selected in the grid will be executed.

While the assessment is in progress, you can see the assessment progress in the top bar.

Microsoft 365 CIS Assessment Summary

In the Assessment Summary window, you can see the issues that were detected for Microsoft 365 Tenant and impact and recommendation for each issue. When clicked on “Assessment Dashboard” tree node found under the Assessment View, the summary window will populate all issues in each Microsoft 365 Assessment category as shown below:

As you can see in the screenshot above, SmartProfiler has reported high, medium, low and non-compliance issues. It has also reported the items that have been passed successfully. If you need to see data for a test, click on test.

Here are the columns that are displayed as part of the Microsoft 365 Assessment and their meaning:

• No.: Shows test number.
• Category/Test: Showstest name that was executed.
• Severity: Shows severity of the test. The severity can be Critical, High, Medium, Low, Passed, Completed, and Not Executed.
• Control: This column shows if the test belongs to CIS Control or not. You will see SmartProfiler Control 1.0 or CIS Control Version 1.5

Note: SmartProfiler for Microsoft 365 includes all CIS Control tests for Microsoft 365 foundation. However, some of the tests have been designed by our Microsoft 365 experts to ensure every aspect of Microsoft 365 Subscription is checked and reported as part of the report.

• CIS Profile: Shows CIS Profile type for CIS Test.
• Compliance: Shows compliance status for the test. The reported data is “Compliant” or “Non-Compliant”.
• Affected Objects: This column shows the number of affected objects for each test. If showed “0 or NA” that means the test doesn’t have affected objects or affected objects is not applicable to the current test.
• Impact: Associated impact text with the test.
• Recommendation: Associated recommendation text with the test.

Generating Report

To generate a report for Microsoft 365 Tenant, click on “Generate Report” node found under Assessment View in left pane.

Note that SmartProfiler for Microsoft 365 is capable of generating a Microsoft Word Report and an excel summary which contains the affected objects list for each test.

Click on the Browse button to specify the report location and check/uncheck various other options as explained below:

• Include Reference Link and Default Value: If you check this option, SmartProfiler is going to include reference link for each test and its default values. The option is checked by default.
• Include SmartProfiler Tests: If you would like to include SmartProfiler tests in the report, please check this option.
• Include “Not Executed” and “Passed” tests in same section: If you check this option all not executed and passed items will be executed in the same section of each category. Unchecking this option will instruct reporting process to include all passed and not executed items in a separate section in word document.

Once you have checked/unchecked required options, please click on “Generate Report” button. The process will take some time and progress will be shown on the screen.

All CIS Tests that are covered by SmartProfiler can be found on CIS website. Note that SmartProfiler supports latest CIS Version 3.1.0.