Showing posts with label Azure. Show all posts
Showing posts with label Azure. Show all posts

Thursday, 10 September 2020

Using DLP Policy to Block Passwords in Microsoft Teams Chats and Channels

You may have heard that the massive Twitter hack recently came from an admin credentials found in a Slack channel. This raises the question: how can you avoid people putting passwords into Microsoft Teams channels and chats to avoid this kind of situation happening to your business? In the case of Microsoft we are talking about Office 365 passwords being passed around in chats by help desk personnel or staff that didn't realise the potential implications of leaving this data lying around. I had a look through the Microsoft Sensitive Information Types and noticed that there was not a built-in policy for AD and AAD password formats. You may be excused from thinking that the built in DLP policies are the only ones that are available to you. However, you have the ability to create your own DLP policies and, with some fine tuning, make them block things like passwords from being posted in Teams. In this blog post I’m going to give an example of how you could use a custom AD/AAD password Sensitive Information Type to create a DLP policy in Teams for stopping people from sending around Office 365 passwords in Teams chat and channel messages. You could also use the Sensitive Information Type created here with Communications Compliance policies, etc, but we are going to focus on DLP policies for this example. 

Data Loss Prevention Licencing


First things first, let's talk about the licensing that you need to do DLP within Teams. I find the licensing for these security features to be a bit of a minefield, and this is a good example. If you’re talking about DLP for Exchange and SharePoint then you can do that with a Office/Microsoft E3 licence, so of course you would expect the same for Teams. Unfortunately that’s not the case. You need Office/Microsoft E5 licensing (or a handful of different add-on licences) in Teams to take advantage of DLP policies. Here’s a spiel from Microsoft docs that explains it for you:

“Data loss prevention capabilities were recently added to Microsoft Teams chat and channel messages for users licensed for Office 365 Advanced Compliance, which is available as a standalone option and is included in Office 365 E5 and Microsoft 365 E5 Compliance. Office 365 and Microsoft 365 E3 include DLP protection for SharePoint Online, OneDrive, and Exchange Online. This also includes files that are shared through Teams because Teams uses SharePoint Online and OneDrive to share files. Support for DLP protection in Teams Chat requires E5.”

You can read more here: docs

Sensitive Information Types


The concept of a Sensitive Information Type usually relates to matching data that might be considered private or sensitive. This is especially the case when dealing with organisations that might store customer data and have compliance around keeping this data safe and not sending it off site, etc. Microsoft has built many data matching policies for things like passport numbers, US Social Security numbers, Australian Tax File numbers, etc. If you would like to see a list of these pre-built rules and how they match data, there is some good documentation here: docs

Something that’s not on this list is AD or AAD passwords, which in some ways I find to be a bit weird considering this is what’s used for Office 365 sign-ins. 

A Sensitive Information Type is made up of the following things (as described by Microsoft Docs):

  • Primary pattern: employee ID numbers, project numbers, etc. This is typically identified by a regular expression (RegEx), but it can also be a list of keywords.
  • Additional evidence: Suppose you're looking for a nine-digit employee ID number. Not all nine-digit numbers are employee ID numbers, so you can look for additional text: keywords like "employee", "badge", "ID", or other text patterns based on additional regular expressions. This supporting evidence (also known as supporting or corroborative evidence) increases the likelihood that nine-digit number found in content is really an employee ID number.
  • Character proximity: It makes sense that the closer the primary pattern and the supporting evidence are to each other, the more likely the detected content is going to be what you're looking for. You can specify the character distance between the primary pattern and the supporting evidence (also known as the proximity window) as shown in the following diagram:

  • Confidence level: The more supporting evidence you have, the higher the likelihood that a match contains the sensitive information you're looking for. You can assign higher levels of confidence for matches that are detected by using more evidence.

So when you build your own Sensitive Information Type, the main things that you need to worry about are being able to capture the sensitive information using a regular expression, and making a list of words that are likely to be used in close proximity to the text captured by the regular expression. This is what we are going to focus on next.

Primary Regex Pattern


As previously mentioned the primary matching rule is a regular expression. Something that’s not really documented as clearly in the Microsoft docs is that they police the regular expression rules for Sensitive Information Types so you can’t go and put in some super CPU-intensive rule that will affect the performance of the platform. If you put in a Regex expression that doesn’t adhere to these rules, then you will get an “Invalid Regex” error that shows up when you enter your regex. I hit this in the first Password check rules that I created and I had to do some massaging of the regex to get it into a format that passed these rules. Here’s a list of the rules for your reference:

When entering the regex value for a new Sensitive Information Type, the following rules are used to check the regular expression:

1. Cannot begin or end with alternator "|", which matches everything because it's considered an empty match.
For example, "|a" or "b|" will not pass validation.

2. Cannot begin or end with a ".{0,m}" pattern, which has no functional purpose and only impairs performance.
For example, ".{0,50}ASDF" or "ASDF.{0,50}" will not pass validation.

3. Cannot have ".{0,m}" or ".{1,m}" in groups, and cannot have ".*" or ".+" in groups.
For example, "(.{0,50000})" will not pass validation.

4. Cannot have any character with "{0,m}" or "{1,m}" repeaters in groups.
For example, "(a*)" will not pass validation.

5. Cannot begin or end with ".{1,m}"; instead, use just "."
For example, ".{1,m}asdf" will not pass validation; instead, use just ".asdf".

6. Cannot have an unbounded repeater (such as "*" or "+") on a group.
For example, "(xx)*" and "(xx)+" will not pass validation.

More information available here: docs.

Now let's consider the rules that govern what an Active Directory and Azure Active Directory complex password will look like. These are the rules that describe a complex password for both platforms:

Passwords must be a minimum of 8 characters and a maximum of 256 characters.
Passwords require at least include characters that match three out of four of these:
  • Lowercase characters
  • Uppercase characters
  • Numbers (0-9)
  • Symbols
In order for us to capture passwords using a Sensitive Information Type policy, we need to be able to describe the item as a regular expression. To do this you will need to have some RegEx skills and understanding. Teaching RegEx is a bit outside of the scope of this article so rather than doing that I'm going to give one that I prepared earlier. The following Regex fulfills Microsoft’s regex limitations and will match AD/AAD complex password formats as described above. Here is is:

((?=[\S]*?[A-Z])(?=[\S]*?[a-z])(?=[\S]*?\d)|(?=[\S]*?[A-Z])(?=[\S]*?[a-z])(?=[\S]*?[^a-zA-Z0-9])|(?=[\S]*?[A-Z])(?=[\S]*?\d)(?=[\S]*?[^a-zA-Z0-9])|(?=[\S]*?[a-z])(?=[\S]*?\d)(?=[\S]*?[^a-zA-Z0-9]))[^\s]{8,256}

Yep, that's quite a hefty regular expression. Feel free to break it down and try to figure out how it works. Or just test it out and see if it works the way you want it to :)


Additional evidence


In addition to just having a Regex rule, we also need to have some additional evidence in the Sensitive Information Type configuration. The additional evidence usually comes in the form of words that might be found near the RegEx matched password. In this case is fairly likely that the word “password” will be included in the same message in which the actual password will be sent. The words that you include in the list will be match any case type (upper or lower or combination) so you only need to put the words in once. For this example, I am going to keep this simple and match some common password variations:

"password","pass","pwd"



Create the Sensitive Information Policy


Log into the https://compliance.microsoft.com/ and open the Data Classification > Sensitive Info Types (tab) and click the "+ Create Info Type" button.

Step 1: Give it a name.


Step 2: Add and element.


Step 3: Enter the regular expression that I detailed earlier. Add the supporting element key word list I mentioned earlier. The minimum count in this case is 1 because we only need to see one of these key words to make a match.

Note: Be careful when you're cutting and pasting quote marks in the key word list. There are different kinds of quote marks that look almost the same but are not the same. The keywords text box is looking for the default vertical quote marks and not the fancier angled quote marks. If you get this wrong then the rule may not match!

Step 4: Set the confidence level. In this case I have set a 100% confidence level because we don't have multiple levels of matching rules and it won't be used in this case. For character proximity I have used 300 characters, which means the key words have to be within 300 characters of the regex pattern match.


Step 5: That's it for creating the sensitivity label.


Step 6: Test the sensitivity label by making a txt document and putting example text in it and uploading it to the test dialog:

Now on to creating the DLP Policy...



Let’s Create the DLP Policy


Log into the https://compliance.microsoft.com/ and open the Data Loss Prevention section and click the "+ Create Policy".

Step 1: I've elected to do this in the new compliance portal. Open the Data Loss Protection section (you may need to select Show All.. at the bottom of the list to see this option) and Create a new policy. Select Custom categories and Custom policy Template.


Step 2: Give it a name:

Step 3: We are just going to use the policy in Teams to stop people sending passwords. On this screen you could also elect to limit the scope of this policy to just help desk and admin staff members to avoid false positives for general users that don't deal with passwords. However in this case, I just left it as All.

Step 4: Select "Create or customise advanced DLP rules"


Step 5: Click "Create rule"


Step 6a: Give it a name and select our previously created Sensitive Info Type for detecting passwords:


Step 6b: Select the Action that you would like taken if a password is found. In this case I am blocking the user from sending the message (what actually happens practically is that the message is sent for a couple of seconds and then replaced with an error message).


Step 6c: I added a policy tip here, however, these don't show up when Teams messages are blocked at the moment. You never know, they might add this in the future so it's worth adding some text here.


Step 6d: I also selected to send an email alert to an angry administrator that will be likely to have some harsh words with the person sending around passwords willy-nilly:



Step 7: Almost finished:

Step 8: Even more almost finished:

Step 9: Yep, finished now:




Let’s Test it Out


What's it look like in practice? Well check this out:



If the user clicks on the "What can I do?" link in the message, they will be able to see details of why the message was blocked and depending on what settings you chose, the DLP policy may have the options to report back that it was incorrectly flagged or to provide a business justification for sending the message:


The user receiving the message will see:


When do DLP Rules Not Work?


Something that you need to be aware of with Teams is that it won’t check for the regex and additional evidence across multiple Teams chat messages. So if someone was to send the password in a separate line, like shown below, it would still send this:


Based on this, don't expect a DLP policy to police people that are trying to get around the system. The expectation is really that your internal users are not deliberately trying to do the wrong thing, but rather, that they have unknowingly done something that they should have tried to avoid.

There is also the potential to block messages where someone was having a legitimate conversation and was not sending a password to someone at all. Here’s an example of people having a conversation that could get blocked by these rules:



The Wrap Up


There it is, if you run a multi-billion dollar social network you too can use DLP to stop your passwords being stored in your Teams channels or chat for eternity by stopping it ever being allowed to be written in the first place. Keep your passwords safe everyone. Till next time.





Read more →

Monday, 26 March 2018

Building an Edge Server Port Monitor with Azure Function Apps – Part 3

This blog is an expansion on the previous Part 1 and Part 2 posts found here and here. The process of setting up the Function App for this part 3 section is the same as was documented in Part 1 and 2.  I suggest you head over to the first two parts and give them a good read before moving onto this post.

In part 3 we will be adding to the Function App so it can save data over time that we can use to graph and manipulate in Power Bi. To do this we will expand the application to do the following:
  • Save all port test results to Azure Storage tables for analysis.
  • Use Power Bi to connect to Azure Storage Tables and create nice graphs showing the status of Edge servers over a period of time.

Just like in Part 2 we will use the Azure Storage Tables module for Powershell to allow our application to keep both a short term memory for logging errors as well as a long term memory for logging all port testing attempts.

Step 1
To begin with follow Steps 1 through 5 of Part 2.

Step 2 – Download a copy of the script
You can grab a copy of the script I wrote for part 3 from here:



Step 3 - Update variables

Update the Storage Account details in the Powershell Script.

IMPORTANT: These settings are slightly different than in Part 2. In this case we have 2 different storage tables: one of them stores the current state of edge servers (same as part 2) and the other one stores all attempts into a separate table (which we will use for analysis in Power Bi):
#AZURE STORAGE VARIABLES######
#SETTINGS ARE FOUND UNDER PLATFORM FEATURES TAB -> PROPERTIES
$subscriptionName = "Visual Studio Premium with MSDN"  #SUBSCRIPTION NAME
$resourceGroup = "EdgePortTester-Part003"      #RESOURCE GROUP
$storageAccount = "edgeporttesterp8dbf" #STORAGE ACCOUNT NAME
$tableName = "EdgeTesterTablePart3Current"      #CHOOSE A NAME 1
$tableName2 = "EdgeTesterTablePart3Results"      #CHOOSE A NAME 2
$partitionKey = "EdgeTesterStoragePart3Current"      #CHOOSE A NAME 1
$partitionKey2 = "EdgeTesterStoragePart3Results"      #CHOOSE A NAME 2
$storageAccountKey = "7asdkjhasd7KHDKJHAS0dsflasdnnlasd099asdpncsdlknclLJSDLjbadksdjbfa9su9duhoasivRqXA615jQ=="             #STORAGE ACCOUNT > ACCESS KEYS
#AZURE STORAGE VARIABLE END######

Don’t forget to fill in your Mail Jet email account information (as you did in Part 1) and add your Skype for Business Edge server's details. See Part 1 for more details. Enter your Mail Jet API Key (Username) and Secret Key (Password) and paste them into the following section of the script:

#MAIL JET USERNAME/PASSWORD#######
$emailUsername = "kjh3k23h4kjhkj37573f8f020879dff7"     
$emailPassword = "9898f98fhdjkkdjh46cd418100075a3b"
#EMAIL ADDRESS TO SEND ERRORS FROM
$SENDEREMAIL = "YourRealEmailAddress@domain.com"
#EMAIL ADDRESS TO SEND ERRORS TO
$RECIPIENTEMAIL = "YourRealEmailAddress@domain.com"
################################## 

Edit the Skype for Business Edge server details as required. These are entered as an array of hash tables. The sections highlighted in yellow can be changed. In this case the application is monitoring 2 Edge servers, one in Melbourne and one in Sydney.

Location
ServerName
ServerRole
DestinationPort
Protocol
Melbourne
147.70.50.10
Federation
5061
TCP
Melbourne
147.70.50.10
Access Edge
443
TCP
Melbourne
147.70.50.11
Web Conferencing
443
TCP
Melbourne
147.70.50.12
AV Edge
443
TCP
Sydney
147.70.60.20
Federation
5061
TCP
Sydney
147.70.60.20
Access Edge
443
TCP
Sydney
147.70.60.21
Web Conferencing
443
TCP
Sydney
147.70.60.22
AV Edge
443
TCP
Note: The script only supports testing TCP ports at this time.

#SETUP EACH SERVER
$Records = @(@{"Location" = "Melbourne"; "ServerName" = "147.70.50.10"; "ServerRole" = "Federation"; "DestinationPort" = "5061"; "Protocol" = "TCP"})
$Records += @(@{"Location" = "Melbourne"; "ServerName" = "147.70.50.10"; "ServerRole" = "Access Edge"; "DestinationPort" = "443"; "Protocol" = "TCP"})
$Records += @(@{"Location" = "Melbourne"; "ServerName" = "147.70.50.11"; "ServerRole" = "Web Conferencing"; "DestinationPort" = "443"; "Protocol" = "TCP"})
$Records += @(@{"Location" = "Melbourne"; "ServerName" = "147.70.50.12"; "ServerRole" = "AV Edge"; "DestinationPort" = "443"; "Protocol" = "TCP"})

$Records += @(@{"Location" = "Sydney"; "ServerName" = "147.70.60.20"; "ServerRole" = "Federation"; "DestinationPort" = "5061"; "Protocol" = "TCP"})
$Records += @(@{"Location" = "Sydney"; "ServerName" = "147.70.60.20"; "ServerRole" = "Access Edge"; "DestinationPort" = "443"; "Protocol" = "TCP"})
$Records += @(@{"Location" = "Sydney"; "ServerName" = "147.70.60.21"; "ServerRole" = "Web Conferencing"; "DestinationPort" = "443"; "Protocol" = "TCP"})
$Records += @(@{"Location" = "Sydney"; "ServerName" = "147.70.60.22"; "ServerRole" = "AV Edge"; "DestinationPort" = "443"; "Protocol" = "TCP"})


Step 4 – Parameter Tweaking
This version of the script like part 2 has a few settings that you can tweak. These are how many failures on each port is required before an email gets sent ($RequiredNumberOfFailuresBeforeEmail). There is also a setting for consolidating multiple errors or recoveries into a single email ($consolidateEmailsOnError and $consolidateEmailsOnError). Set these as you like:

#This is the number of required port check failures before an email is sent out
$RequiredNumberOfFailuresBeforeEmail = 3

#Send 1 email rather than one per record
$consolidateEmailsOnError = $true
$consolidateEmailsOnRecover = $true

Step 5 - Download Azure Storage Explorer
Now let your Function Application run for a while and gather some data. At any time you can look into your Function Apps Table Storage using Azure Storage Explorer. This application will show you all of the rows in your storage tables and allow you to see and edit as you see fit. You can download your free copy from here:




Once you have logged into your Azure Account within Azure Storage Explorer you can dig into your storage tables by selecting Storage Accounts > (Storage Resource Group Name) > Tables to see your table data. Note, there will be no table or data until you actually start running the Function App.

Step 6 - Download Power Bi
Now download a copy of Power BI for desktop:




Install the downloaded Power Bi on your PC.

Step 7 - Open Power Bi
Open Power Bi Desktop and you will be greeted with a splash screen and dialog. Click on the “Get Data” button:




Step 8 - Import Data
The Get Data dialog will then be displayed. Select “Azure Table Storage” from the list and click the “Connect” button:



Step 8 - Account URL dialog
Power Bi will now request an Account Name or URL to connect to:



Step 9 - Find Account URL
The account name for the dialog above can be found in the Azure Portal under the storage account Overview > Tables section:



The “Table service endpoint” is the value you will need to fill in the dialog with:



Step 10 - Paste in URL
Enter the Table service endpoint into the “Azure Table Storage” dialog and click “OK”:



Step 11 - Enter Account Key
You will now be asked to enter your “Account Key”:



This can be found under the Storage Account > Access Keys section in the Azure Portal:



Enter the Account Key and click “Connect”:



Step 12 - Load Data
Power Bi will now connect to your Table Storage and list up all of the Tables in there. Select the Results table and click “Load”:



Step 13 - Not all data is displayed
Power Bi will now download your data into the application. You may notice though that all of the columns that you can see in Azure Storage Explorer will not be displayed:



Step 14 - Edit Query
To be able to see all of the columns you need to do a little extra work. On the right hand side of the screen, Right Click on Table name at the top of the top of the column names listed and select “Edit Query”:



Step 15 - Expand content column
You will now see an extended view of the data that includes a “Content” Column:



On the top right of the Column click on the double arrow “expand” button:




You will now get a full list of all of the additional columns available that are stored in Table Storage as a Json blob. Tick the Columns that you want to include in your graphs and data analysis and click OK:



Step 16 - All columns are now available
You will now see the extra data columns:



 Click the “Close and Apply” button from the Home Tab:



The full array of data is now available for you to do as you please with:



Step 18 - Make charts
Back on the "Report" tab you can now put together some nice looking graphs of your data. Here is an example of a Pie Chart and a Bar Chart showing information about the number of errors for each role:



To create these graphs you use the following settings:




From here you can play with the data in Power Bi and make whatever graph you like (I included location in the data so you can even plot your Edge servers on a map). This is what makes Power Bi so powerful!

The Wrap Up

This post ends my series on creating an Edge port monitor with Azure Function Apps. I hope that in addition to helping you monitor your edge servers, this has been informative and taught you some new skills that might help in the future when making your own Function Apps.



Read more →

Popular Posts