Dealing with False Positive Breaches in Universal Audit Log Search in Office 365

Modern cyber security threats have today mutated into a new class that is immune to detection and prevention solutions offered by the security industry. We are looking at this new generation of hackers that master zero-day exploits, credential thefts, fake identities, and developing stealthy malware.  These threats have kept the security personnel on their toes, figuring out what the next attack would look like. One of these challenges includes identifying false positive and false negative alerts. In this article, we shall learn more about false positive alerts in Microsoft Office 365, how they appear and what you can do about them.

What is False Positive in Cybersecurity?

False positive or f/p as commonly written, is a term in cybersecurity that refers to the mislabeled threats that actually do not exist. Seem harmless, doesn’t it? Well, it is not. These false alerts end up burdening your security teams, who already have a lot on their plate. This results in exhausting your organisation’s resources and time, thereby decreasing its productivity.

According to a survey conducted with SOC professionals, Managed Security Services Providers(MSSP) and Managed Detection & Response (MDR), it was found that 70% of respondents investigate more than ten alerts each day and 78% state that it takes them more than ten minutes to investigate each alert.

Similar to false positives, there are false negatives in cybersecurity as well. False negative are the undetected threats that are overlooked due to their latent nature. These sophisticated threats often escape preventive software such as endpoint detection and response. These software specialise in identifying ‘familiar’ threats, and on many instances miss the advanced/dormant cyber threats.

My Experience with False Positive Breaches in Universal Audit Log Search in Office 365

The Universal Audit Log (UAL) search is a useful tool for checking or auditing for events such as logins and the creation of rules on accounts that might have been breached.  However, at times, it can be somewhat of a blunt instrument. This is because on the surface it can report supposedly risky events that would alarm you even though the underlying reasons can be benign.

Such a false positive recently caught my worried attention when I noticed seemly successful login attempts in our UAL search from users that I did not recognize.  It started when I did a search for the “User Logged In” event on my tenant.  I noticed that a user called htht@octiga.com had logged in and what’s more, a quick check of the source IP address showed it came from Microsoft Corporation.  

Expanding the details pane and further expanding the “More Information” section I could see to my alarm that the “ResultStatus” was “Succeeded”.  Now Mr/Ms “hthr” is neither a mailbox, azure user, or guest user on our Microsoft 365 tenant. I quickly checked that such a user had indeed never been created on our tenant. For example, this could happen if one of our Administrator accounts had been breached.  

I immediately expanded the search to see all events in and around the same time and noticed, in each instance of the suspicious login, a failed login attempt immediately after by the same user but from a different IP address. The IP address of the failed login was always associated with a more sinister location.

Seeing the second and seemly associated/paired “UserLoginFailed” did calm my nerves however I wanted to get to the bottom of the “User Logged In”. Looking again at the expanded “More Information” section for the event, I could see that it also had a parameter “LogonError” set to “FaultDomainRedirect”.

Some googling later I found that the “ResultStatus” can be very misleading in this very case of logon events.  It reads

 “Important: Different workloads may overwrite the value of the ResultStatus property. For example, for Azure Active Directory STS logon events, a value of Succeeded for ResultStatus indicates only that the HTTP operation was successful; it doesn't mean the logon was successful. To determine if the actual logon was successful or not, see the LogonError property in the Azure Active Directory STS Logon schema. If the logon failed, the value of this property will contain the reason for the failed logon attempt.”

Beware of the False Positives

Calling these events “false positives” is slightly disingenuous to Microsoft given my above findings, however it did take up over an hour of my time. I even raised a support case with Microsoft.  Even the support case contact at Microsoft was confused by this at first, but nonetheless they were very helpful.  

I do call these false positives on the surface however because one would think that searching for “User Logged In” events should not show me events where users did not, in effect log in.  

Tips for Threat prevention: False Positive Mitigation  

It is impossible for security teams to detect all the threats 100% right. However, there are ways to align the threat indicators more contextually while dealing with false positive threats. You can:

1. Add more elaborate details of the threat indicator, for example use specific arguments to go with each threat

2. Analyse the relationship between different indicators in the form of interconnected activities, linked actions or a group of tactics and techniques

3. Include specifics of the environment or conditions where the indicator should be analysed

4. Consider the lifecycle of indicators in terms of the magnitude of the threat behaviour in the present scenario. For example, a threat that was malicious last year might not show the same behavior this year

5. Change to a proactive security approach, using tools that allow you to implement incident response against hidden attacks

Summing Up

If your organisation lacks affordable and automated cybersecurity, we can help address your needs. Octiga Microsoft Office 365 Security Automation, aims to simplify and automate Office 365 security configuration, to ensure you have ample time to focus on other important things. Its features filter out false positive events and also give a timely detection of possible threats. Schedule a demo with us to learn more about our services.