What’s the Difference Between Microsoft Copilot and ChatGPT?

Introduction

In this post I go into some detail of how the different Copilots in Microsoft 365 operate in practice and show that not all the Copilots are created equal. This information could be both useful from a technical perspective but also useful from a staff training perspective. When you rollout Microsoft Copilot, people within the organisation need to understand that all the Copilots within the Office applications are not the same and are all tuned in different ways.

 

A Copilot is a Copilot is a Copilot?

When I first heard about Microsoft Copilot and saw the similar looking Copilot frame on the right side of the screen, I figured it was probably just a common interface that could access the data from the application you had open at the time. However, after actually getting the opportunity of playing with Microsoft Copilot in the various apps it has becomes clear that it is actually a lot more complex than that. Each of the Copilots within the apps has been tailored to respond in a context that makes sense for the type of application that you’re using. This has been achieved by the engineers at Microsoft, using various methods of prompt engineering and orchestration in the background.

I thought it would be useful to demonstrate the differences in the way the various Copilots in different apps respond to the exact same prompt. For this demo I have chosen an innocuous query that is not explicit and could be interpreted in different ways to see what happens. The query I chose was “Tell me about the weather in Melbourne”. This is not the kind of prompt you would really use in practice but is instead something that I’ve chosen to highlight the differences in the way each Copilot responds to the prompt.

Let's start by querying the OpenAI ChatGPT 3.5 model and see how this foundation model interprets this request. This will offer a comparison to see the difference that the exact same prompt will give when asking it of the various Copilots.

 

1. ChatGPT 3.5

You will see here that the ChatGPT foundational model has interpreted this question as a request to know the specific temperature in Melbourne right now. This is because I wasn’t explicit enough in what I had asked the model and so I didn’t get back any general information about expected temperature ranges in Melbourne. 

In setting up the ChatGPT model the OpenAI team appear to have created the system to fail gracefully in these cases where it thinks it's getting asked for data that's more current than it knows about. This is an unfortunate trait of the foundation models, they only know information up to when they were finished being trained. It is interesting that it did not respond with some more generic information about what the expected temperatures are throughout the year or historical information about the weather though (keep this in mind when we get to the Word Copilot example).

 

2. Bing Chat

Bing Chat is geared to behave much more like a web search engine. You can see in the example above that it reached out to the web and pulled back information from various websites about what the current temperature, and upcoming temperatures, will be in Melbourne. It also gave references to websites that it got this information from.

The method used here is called a Retrieval Augmented Generation (RAG) framework, where it doesn't ask the foundation model for the answer to the question directly. Instead, Bing will first retrieve some reputable sources for the kind of information being requested and provide that data as part of the prompt to the foundation model (also often referred to as Grounding the model with data). The foundation model here has been used to interpret the retrieved data instead of using its own “knowledge” from the data it was trained on. In this case, Bing is functioning as an orchestration engine that retrieves data which it compiles into an expanded prompt that will be sent to the ChatGPT mode in addition to your original query.

 

3. M365 Chat

When I asked the M365 Chat interface within Teams this question, it responded that it couldn’t find the answer to the question and recommended that I use a web search. This is because the M365 Chat Copilot uses a similar Retrieval Augmented Generation (RAG) framework to Bing. Rather than searching the Internet for information on the weather in Melbourne, it attempted a Semantic Index search (Reference: https://learn.microsoft.com/en-us/microsoftsearch/semantic-index-for-copilot) across the documents, emails, chats and other data within my Office 365 tenant. I didn’t actually have any information within my tenancy on this topic at the time. As a result, M365 Chat was unable to get any information to the pass onto the foundation model to provide an answer. What is interesting to me here, is that it didn’t just ask the foundation model to have a go at telling me about the weather in Melbourne, but instead apologised for not being able to find any documents about this.

Note: In this case, the Microsoft 365 Chat Copilot was configured to only have access to internal documents and was not enabled for searching the Internet for data. This is a setting that administrators have control over: https://learn.microsoft.com/en-us/microsoft-365-copilot/manage-public-web-access

Of course, had I have had documents that contained information on the weather in Melbourne it would have been able to answer me. Below is an example of the output when there is a document containing information about the weather in Melbourne. You will see here that the RAG model has been used to retrieve the data and the document is referenced below the response:

What is also interesting about the previous response is that this information was actually generated in Word from a later example that I ran for this blog post. The data being displayed here is actually an interpretation of information previously generated by the model. I find this to be an interesting, because when data like this keeps getting recycled through these models over time, will there start to be degradation of the quality of the information? Like a photocopy of a photocopy. Here’s an interesting article that goes into some more detail on what could be the result of this in the long term: (reference: https://cosmosmagazine.com/technology/ai/training-ai-models-on-machine-generated-data-leads-to-model-collapse/). Always take care to check the information the Copilot outputs before using the information.

 

4. Microsoft Word

Microsoft Word is usually used to create longer form documents, as a result, Microsoft has tuned the way the foundation model is prompted when you ask it questions in Word. In the example of asking it about the weather in Melbourne, the model responded with more of a Wikipedia style response, where it attempts to go into depth about what the climate is like in Melbourne throughout the year.

This is a stark difference to the way the ChatGPT foundation model tried to answer this question. This happens by design, as Microsoft realises that this is more likely what you want in a Word document rather than the wanting to know the temperature right now. The way they do this is by taking the original query and then adding additional (“system prompt”) information to it before sending it to the foundation model. This allows them to change the output to be more like what you might want in a Word document. It’s not clear exactly what Microsoft is including in the prompt that it sends to the foundation model, as you never get to see this additional information. If you play around enough with ChatGPT you can see that adding additional text like “provide an extended response similar to a reference encyclopaedia” will cause the model to give outputs more like this. I don’t believe it’s documented anywhere exactly what Microsoft add to the prompts to get these responses as the prompt engineering is a bit of secret sauce.

  

5. PowerPoint

The PowerPoint Copilot is an even more interesting topic as it doesn’t just produce text, it will also add pictures and make design choices when producing its output. You can see that for our example weather query it produced a nice picture of Melbourne’s botanical gardens and skyline, creates a meaningful heading and some dot points about the weather in Melbourne. It looks pretty impressive as an output to such a basic query:

This is all the more impressive when you have some understanding of what’s going on in the background for the PowerPoint Copilot. There is an interesting paper that I found which is produced by some of the research staff at Microsoft about how this works. It can be found here: https://arxiv.org/abs/2306.03460

TLDR: For apps like PowerPoint the Copilot needs to be able to tell the application itself how to style the page in addition to just generating text. This kind of thing can be done with scripting languages which the foundation model could be used to produce (like Github Copilot), however, this method is prone to syntax errors. The researchers at Microsoft found that it was safer to create a specialised domain specific language for describing the layout of a document (more like a declarative language like is used for Terraform or PowerShell Desired State Configuration). The language, in this case, is called Office Domain Specific Language (ODSL) and is designed to use a minimal number of tokens (words) and be easily describable as an input to a foundation model. Here’s an example of the language:

1 # Inserts new "Title and Content" slides after provided ones.

2 slides = insert_slides(precededBy=slides, layout="Title and Content")

When the prompt is sent to the model it will include schema information about what the ODSL language and what the format of the desired response. The model will then respond with a description of what each slide should look like in the desired ODSL format. The response is thoroughly checked and validated to have the right format and then translated into a lower-level language by an interpreter program which then gets executed by PowerPoint. This is both very cool and crazy that the foundation models are powerful enough to do these kinds of things.

 

6. Outlook

When you write an email your colleagues you don’t really want to be known as the person that writes the dreaded War and Peace novel length emails. Fortunately, Microsoft are aware of this and when designing the Outlook Copilot, they took this into account. The output of this Copilot is designed to produce output that looks like, in both format and content, like an email. You can see below that the simple weather in Melbourne prompt actually created what looks and reads like an email. I must admit it did take a bit of artistic licence and go on a bit more of a ramble than I would have liked in this case though:

 

7. Excel

The Excel Copilot is once again quite different than the other Copilots. Asking it the weather is not exactly what it’s supposed to be used for, but I asked it anyway, because, why not?:

In excel, the Copilot is more for creating formulas and reasoning over the data that is in your spreadsheets. In the current preview version, the Copilot will only work on data that is in a defined table. This is likely to do with the fact that the data needs to be ordered in such a way to be sent as a prompt to the foundation model. In doing this the data needs to retain all the column and row information but also keeps the token count low enough to be processed. I’m not sure if it’s clear how Microsoft could process an entire very large spreadsheet (with the potential complexity of multiple pages, and scattered data, etc) through the foundation models give their token limits currently. Until they figure this out, we may be stuck with only processing data that is in defined smaller tables for the time being.

If you are wondering what the Excel Copilot can actually do though, here’s an example of how you could ask the Excel Copilot to reason over the data in a table and give you an answer:

Also, here’s an example of how you can ask the Excel Copilot for a formula for producing a Fahrenheit column from a Celsius column:

 

 

8. Microsoft Whiteboard

The Microsoft Whiteboard Copilot has another take on what it produces based on our modest weather question. It produced a bunch of sticky notes for various things that the weather could be in Melbourne. This is more contextualized toward a brainstorming type of session which is be common when using a Whiteboard:

 

This is once again, a fun and different take on how a foundation model can be used to produce a more context aware output for the application at hand.

 

The Wrap Up

As you can see, all these Copilots across the Microsoft Office apps are all very different beasts, and this is something that people within your organisation should understand in order to get the most out of Copilot product set. This is certainly something to keep in mind when training staff on the potential use cases and determining which Copilot is right for the task at hand. Cheers!

Read more →

Microsoft Teams Phone Configuration and Licensing (2022 Edition)

There have been quite a few changes recently to feature sets and licensing of Microsoft Teams Phones. This post digs into the details of the various phone modes and licensing options in 2022.  Note, this post is being released just after Ignite 2022 when it was announced that the Common Area Phone licence is being renamed to the Teams Shared Device licence in "a couple of months". In this post I am going to be sticking with the Common Area Phone licence name until we have further information on any new capabilities that will be included with the Teams Shared Device licence for phones.

User Interface Modes

There has been three user interface available on Teams phone devices for a while now, which allows you to tailor the experience to different scenarios. In the past these required PowerShell configuration, however, the Teams will now try to auto-magically choose the interface for the phone based on the licence the user account has been assigned.

For regular users logging in with E3-E5 licensing, the system assumes that the User Mode interface should be used. When a Common Area Phone licence (announced at Ignite 2022 to be changing names to the Teams Shared Device licence) is assigned the Common Area Phone interface will be selected by default. Lastly, when a Teams Meeting Room licence is assigned to the phone, the Meeting Room interface will be shown. When testing this I have found that in most cases the licence based auto assignment works, but for some reason this sometimes doesn't work and the phone falls back to User Mode. As a result, I would still recommend using the PowerShell command to select the interface mode for more consistent outcomes.

There are also now two different versions of the Common Area Phone mode. The original mode is now called Common Area Phone Basic and an extra special Common Area Phone Premium mode. The premium mode take into account a bunch of new capabilities in the updated Common Area Phone licence updates that came in mid-2022. Without further ado - let's have a more detailed look at the modes...

Common Area Phone Mode (Basic)

Common Area Phone (Basic) mode is used for devices that have their own user accounts and are (usually) assigned a Common Area Phone licence. The default Common Area Phone user interface has a basic keypad layout for dialing phone numbers. All the other bells and whistles like calendar, voicemail, etc at nowhere to be seen in this interface. Below is an example of the default Common Area Phone user interface:

 

   

Note: The About screen on the phone will show Common Area Phone (Basic) when you're in this mode.

For Common Area Phone Basic mode there is another setting where you can turn on or off the user search capabilities on the device. You may want to use this if you have a phone sitting in a lobby of a building and you don’t want people to be able to find out people that work there using the search function.

SearchOnCommonAreaPhoneMode = Enabled

It is recommended that if you want to use a phone in Common Area Mode that you create a Teams IP Phone Policy and assign it to the Common Area Phone account. A Common Area Phone account should be a regular user account. You should not be using room accounts for these devices. Here’s an example of setting up the Phone Policy:

New-CsTeamsIPPhonePolicy -Identity CommonAreaPhone -SignInMode CommonAreaPhoneSignin

 

Grant-CsTeamsIPPhonePolicy –Identity 'testuser@contoso.com' –PolicyName ‘CommonAreaPhone’

 Note: The quickest way to get this change to take effect is to sign the phone out and back in again.

Common Area Phone Mode (Premium)

Since mid-2022 (Teams app version 1449/1.0.94.2022061702) the licensing for Common Area devices was expanded to include some new features (Voicemail, Call Forward settings, Call Park, Call Queues, Auto Attendants, Intune enrollment into Endpoint Manager). As you can probably imagine, the very basic default Common Area Phone user interface is not going to give you the flexibility to use all these features. So Microsoft went for a user interface that looks mostly like the User Mode interface, only without the meetings section.

The Common Area Phone Advanced Calling interface looks like this: 

 

    

Note: In the about page you will be able to tell that it's in this mode because it says "License Details: Common Area Phone (Premium)".

These devices are still geared towards being an admin controlled device, so if you want to access the Call Forwarding settings on the device you need to access the Admin level of settings in the phone - for example:

Note: Wow, cool animated GIF… If you want to make your own, download my Teams Phone Screen Capture Tool script: https://www.myteamslab.com/2020/10/teams-phone-screen-capture-tool.html

The new features are called Advanced Calling features and are controlled by a setting within Configuration Profiles in Device Configuration in the Teams Admin Portal. Personally, I would have preferred this setting be a Teams IP Phone Policy rather than an individual device profile… however, Microsoft sometimes enjoy making things a little more complicated.

To enable a device for Common Area Advanced Calling features you need to create a new Configuration Policy and turn on the "Advanced Calling" Setting:

After this you have to assign the Configuration Policy to the devices that you want to use it:

After doing this you need to wait a while for the configuration policy to be pushed to the phone. You will have to have the phone registered to the cloud and signed in for this to happen.

As for the Common Area Phone Basic mode, it is recommended that you create a Teams IP Phone Policy and assign it to the user account. Here’s an example:

New-CsTeamsIPPhonePolicy -Identity CommonAreaPhone -SignInMode CommonAreaPhoneSignin

 

Grant-CsTeamsIPPhonePolicy –Identity 'testuseradvanced@contoso.com' –PolicyName ‘CommonAreaPhone’

 Note: The quickest way to get this change to take effect is to sign the phone out and back in again.

Meeting Room Mode

Meeting Room mode is for devices that you want to put in Meeting Rooms (Surprise!). Most commonly this would be for a conference room style phone, however, it could also be for small rooms where you might have a desk phone running the meeting room interface.

The Meeting mode was recently updated to have a new look which is more in line with the other Microsoft Teams Meeting Room device interfaces. This is great when you have a combination of video meeting rooms and basic conference phone rooms in your organisation. The similar interface allows for a consistent user experience for booking the rooms in both cases and having a one-touch join button can make meeting more likely to start on time and be much more streamlined.

Here is an example of the new Meeting Room interface:

 

  

When creating Meeting Room accounts it’s usually a better idea to create the phone account as a Room account rather than a User account. This gives you the option to have the room auto accept meeting invites and be shown as a room in Teams meeting invites. Here is an example of creating the room account:

Step 1: Add A Room Resource In Microsoft 365 Admin Centre. Click the "Add resource" button:

 Step 2a: Fill in the details of the account you will be creating:

 Step 2b: Optionally you can edit the booking options:

Step 2c: Ensure that you have “Auto accept meeting requests” ticked for your device so it will display the meetings properly:

Step 3a: Open the User Information pane in Microsoft 365 Admin Centre:

Step 3b: In the Licenses and Apps tab, assign a Microsoft Teams Room Basic (I would recommend also adding a Teams Phone licence for PSTN calling capabilities) or Teams Room Pro license:

Step 3c: Reset the account Password by clicking on the Reset Password button at the top this panel. This will allow you to sign into the Room account:

After setting up the account you can also force the Meeting Room mode by run the following Team Module PowerShell commands:

New-CsTeamsIPPhonePolicy -Identity MeetingRoomPhone -SignInMode MeetingSignIn

 

Grant-CsTeamsIPPhonePolicy –Identity 'testuser@contoso.com' –PolicyName ‘MeetingRoomPhone’

 Note: The quickest way to get this change to take effect is to sign the phone out and back in again.

As a bonus you can also add the Teams room to the list of locations that show up in the Teams client meeting invite screen:

How to create a Room Location (Distribution Group Room List):

Connect-ExchangeOnline

 

New-DistributionGroup -Name “City Office” -RoomList

 

#Add the affected room mailbox to that distribution group

Add-DistributionGroupMember -Identity “City Office” -Member TeamsRoomPhone01@contoso.com

 

As an additional super bonus, you can help to indicate that this is an Audio only room in Outlook when scheduling the meeting by configuring Place information for the room (see the little Audio symbol, wheelchair symbol and room capacity value. Subtle but useful).

 

Here's some example PowerShell for doing this:

Set-Place -identity TeamsRoomPhone01@M365x52543986.onmicrosoft.com -IsWheelChairAccessible $true -AudioDeviceName "Poly C60 Conference Phone" -Capacity 4 -Building “City Office”

User Mode

User Mode is the mode that any regular user that signs into a Teams Phone will see. It has the full suite of features available including Calls, Calendar, Voicemail, People, etc. Here is an example of the User mode interface:

 

     

Usually you shouldn’t have to set Teams IP Phone policy to achieve user mode. By default, when a user has E3-E5 licencing SKUs the phone will default to this mode (unless you have the IP Phone Policy set to one of the other modes for the user). If you want to force User Mode, you can do this with the following PowerShell commands:

New-CsTeamsIPPhonePolicy -Identity UserPhone -SignInMode UserSignIn

 

Grant-CsTeamsIPPhonePolicy –Identity 'testuser@contoso.com' –PolicyName ‘UserPhone’

Note: The quickest way to get this change to take effect is to sign the phone out and back in again.

Licencing Microsoft Teams Phones

As previously mentioned, the licensing of Teams Phones can control what the default interface shown on the phone will look like. However, it begs the question: if you do choose to override the default interface can you choose any of the UI modes with any licence combination?

Firstly, in all cases you need to ensure that your phone software is up to date. The testing for this post was with version 1449/1.0.94.2022090705 of the Teams Phone App. There have been a few iterations of the way licensing works on the phones, so older versions may behave differently. I've also found that when you do a full config reset on a phone it will actually drop the Teams Phone version back to the version that shipped with the base firmware. After doing a config reset on the phone you will likely have to upgrade the Teams Phone App again - so be sure to check this.

After extensively searching the Internet, I found a docs page that does the bare minimum to describe what happens when you use Teams IP Phone Policy to override the license based automatic configuration: https://learn.microsoft.com/en-us/microsoftteams/devices/teams-android-devices-user-interface

The important note on this page being:

"The license assigned to the user account must have at least the same license entitlements as the desired user interface. The Common Area Phone licence allows the Common Area phone user interface. The meeting room license allows meeting room and common area phone user interfaces. An E3 or E5 license supports all sign-in modes."

Note: This statement doesn't include E1 licensing. E1 only has Plan 1 Exchange licensing, and doesn't include Voicemail -  which may be the reason for this omission..? It has been left out of this support statement, so use E1 at your own risk... 

I have created the table below to try to give you a quick idea of the options for licensing Team Phones for the various modes (including some important Add-On licences):

 

Base Licence	Supported  Interface  Modes	Teams Phone  (Direct Routing /  Operator Connect)	Conditional Access	Intune	Calling Plans (Microsoft PSTN,  i.e. Not Direct  Routing)
Common Area Phone Licence	Common Area Phone (Basic / Premium)	Included	Included	Included	Add-On Required
Teams Room Basic	Meeting Mode, Common Area Mode	Teams Phone Add-On	Add-On Required	Add On Required	Add-On Required
Teams Room Pro	Meeting Mode, Common Area Mode	Included	Included	Included	Add-On Required
M365 E3	User Mode, Meeting Mode, Common Area Mode	Teams Phone Add-On	Included	Included	Add-On Required
M365 E5	User Mode, Meeting Mode, Common Area Mode	Included	Included	Included	Add-On Required
Office 365 E3	User Mode, Meeting Mode, Common Area Mode	Teams Phone Add-On	Add-On Required	Add-On Required	Add-On Required
Office 365 E5	User Mode, Meeting Mode, Common Area Mode	Included	Add-On Required	Add-On Required	Add-On Required

 Note: There are various ways to get the Add-On licences including individual or bundled licences. Speak to your local licensing expert for more details on these.

When talking about licensing for a Teams phone devices, in most cases a Common Area Phone licence (to be renamed to the Teams Shared Device licence) will do if it’s a phone that is out in a public space and users just need to dial PSTN numbers. The updates recently to the Common Area Phone licence that expanded capabilities (allowing Common Area Phone Premium mode) have made the supportable scenarios for common area devices even broader. 

If it’s a meeting room based device (e.g. a dedicated conference phone device) for which you want to use the Meeting Room Mode user interface, then you must use a Meeting Room licence (https://learn.microsoft.com/en-us/microsoftteams/rooms/rooms-licensing) or an E3-E5 licence for the device. Here I would like to note that Microsoft did announce that MTR devices will not support E3-E5 licencing as of 2023. However, it doesn't appear at this stage that this includes phone devices. With the pricing updates to the Meeting Room Pro licensing, this could end up being more expensive than you anticipated. You can use Meeting Room Basic licences for the first 25 meeting room phones but you will have to pair this with a Team Phone add-on licence for PSTN calling (and potentially Azure P1 and Intune licensing if this functionality is required for your deployment). When you run out of Teams Meeting Room Basic licences, you will need to step up to Teams Room Pro licences (which are $40US each per month) or fall back to E3-E5 licensing. If this is too rich for your blood, then you could choose to deploy your meeting room phones as basic Common Area Phone devices (i.e. with the Common Area Phone user interface and Common Area Phone Licence). This would make for a lesser user experience though, and have you trading the One-Touch meeting join button for users having to do audio conference dial-in using the Audio Conferencing number and Phone Conference ID for every call. 

Microsoft at this point have announced that the Teams Shared Device licence update in 2023 will bring support for Teams Displays which can use this licence to connect to Teams for shared device usage: (https://techcommunity.microsoft.com/t5/microsoft-teams-blog/microsoft-inspire-2022-innovations-coming-to-teams/ba-p/3559351). It feels like Microsoft really should be considering including the Meeting Room interface scenario for phone device with the new Teams Shared Device licence as well... However, there is nothing confirming this at the moment. Until further information comes to light we will have to hold our breath on this one. When more information becomes available, I will be back to update this post... Stay tuned.

Update 16/11/2022: According to a response from Microsoft on their latest post on Tech Community about the Shared Device Licence, it will not be supporting one touch join (i.e. I take this to mean the Meeting Room UI experience). Comment: "One touch meeting join will be available only with Microsoft Teams Rooms licenses. It will not be supported on the Teams Shared Device license." 

The Wrap Up

Hopefully that has given you some insights into what configuration and licensing you need to achieve your desired Teams Phone outcomes! If you have any comments or corrections, feel free to send them through. I plan to keep this post up to date with information on the new Teams Shared Device licence, so check back later for updates or follow me on Twitter to stay in the know.

Read more →