Spectralink 8400 Edge Issue

I found an interesting issue when testing some Spectralink 8440 handsets the other day and I didn’t see the answer documented anywhere, so I thought I'd change that situation by blogging about it.

Note: The firmware that I was using when I ran into this issue was 5.4.4.1156. I also rolled back to a 5.3 and 5.2 version and had the same issue, so this problem may have existed for a while.

The Symptom

Outbound Call

After setting the phones up and logging in using PIN Authentication successfully when making an outbound call, the call would hang after the number was dialled. The screen would sit at the calling screen forever and nothing would happen (i.e. the dialled phone wouldn’t ring).

Outbound Call

Inbound Call

For an inbound call, the calling party (i.e. a Skype for Business client) would receive ringback tone and the call would not get displayed on the Spectralink handset. After the call forwarded timeout was reached, the call forwarded to voicemail and a missed call would appear on the Spectralink screen.

  

Inbound Call

Troubleshooting

The symptoms described above were consistent and it was clear that the SIP Signalling was working well enough to allow the device to sign in and receive information from the system. However, both inbound and outbound calls were not setting up correctly. The next port of call was to look into the logs. For an outbound call there wasn’t much information in the logs to dig into. However, for an inbound call I could see the inbound INVITE message arrive and the phone would respond with a TRYING message. At this point I could see that the inbound call was not progressing past the point of parsing the candidates from the inbound INVITE message. From this, I suspected that this was something to do with the Spectralink not being about to process the candidates correctly. I then realised that the lab system I was testing the phone on didn’t actually have an Edge server associated with it. This, of course, meant that the phone would not be getting or receiving any reflexive or relay candidates… Hhhhmmmm, interesting...

Inbound Call Log

1205143558|sip  |0|00|<<< Data received TLS

1205143558|sip  |0|00|    INVITE sip:Terry.Adams@10.22.2.11:33207;transport=tls;ms-received-cid=781300 SIP/2.0

1205143558|sip  |0|00|    Record-Route: <sip:SFB001.sfbdomain.com:5061;transport=tls;opaque=state:T:F:Ci.R781300:Ieh.Lvw6TSBMlsUoTJbFhkstQInxeS6wHgXRHMHiO5ioDln-fhNvUO1qLucUD4gO1tXpzhB_8cegAA;lr;ms-route-sig=hkx-TWiKYeik4c0hYMHXqBiJ4op9gbnpuTatBwsuD7moptXpzhB_8cegAA>;tag=B12630CFFCD2C61F74BDE5EB8A4C611A

1205143558|sip  |0|00|    Via: SIP/2.0/TLS 10.20.1.223:5061;branch=z9hG4bK8BA0D60F.E19C5E6D7DAA886D;branched=TRUE;ms-internal-info="dgFJqKHJ0AXHZY0sy2IaYnenMPK5auGe9rVA0q4_Ro3qxtXpzhVO8OYQAA"

1205143558|sip  |0|00|    Authentication-Info: TLS-DSK qop="auth", opaque="4E02C6ED", srand="4D31E5BD", snum="17", rspauth="420755234a2dbcecd21a249213ceb92d8ba9cfc5", targetname="SFB001.sfbdomain.com", realm="SIP Communications Service", version=4

1205143558|sip  |0|00|    Max-Forwards: 69

1205143558|sip  |0|00|    Content-Length: 890

1205143558|sip  |0|00|    Via: SIP/2.0/TLS 10.22.0.22:42727;branch=z9hG4bK1d5487849DAF07;ms-received-port=42727;ms-received-cid=77D300

1205143558|sip  |0|00|    P-Asserted-Identity: "Terry Adams"<sip:Terry.Adams@sfbdomain.com>,<tel:+61266591007;ext=1007>

1205143558|sip  |0|00|    From: "Terry Adams" <sip:Terry.Adams@sfbdomain.com>;tag=5C30CCBE-7E228105;epid=64167f23b38c

1205143558|sip  |0|00|    To: <sip:+61266591007@sfbdomain.com;user=phone>;epid=00907a0f09d8

1205143558|sip  |0|00|    CSeq: 1 INVITE

1205143558|sip  |0|00|    Call-ID: 0888c5534fafd0fc37bc94f7a523b38c

1205143558|sip  |0|00|    Contact: <sip:Terry.Adams@sfbdomain.com;opaque=user:epid:F5iF87H531yKmJ1oYYBMjwAA;gruu>

1205143558|sip  |0|00|    Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, INFO, MESSAGE, SUBSCRIBE, NOTIFY, PRACK, UPDATE, REFER

1205143558|sip  |0|00|    User-Agent: Polycom/5.6.0.17325 PolycomVVX-VVX_311-UA/5.6.0.17325_64167f23b38c

1205143558|sip  |0|00|    Accept-Language: en

1205143558|sip  |0|00|    ms-subnet: 10.22.0.0

1205143558|sip  |0|00|    Allow-Events: conference,talk,hold

1205143558|sip  |0|00|    Supported: replaces

1205143558|sip  |0|00|    Supported: ms-safe-transfer

1205143558|sip  |0|00|    Supported: ms-bypass

1205143558|sip  |0|00|    Supported: ms-dialog-route-set-update

1205143558|sip  |0|00|    Supported: timer

1205143558|sip  |0|00|    Supported: 100rel

1205143558|sip  |0|00|    Supported: gruu-10

1205143558|sip  |0|00|    MS-Conversation-ID: AdNteidtM2YxYWFkN2ZiZjM0ODVhZQ==

1205143558|sip  |0|00|    Content-Type: application/sdp

1205143558|sip  |0|00|    history-info: <sip:Terry.Adams@sfbdomain.com>;index=1;ms-target-phone="tel:+61266591007;ext=1007"

1205143558|sip  |0|00|   

1205143558|sip  |0|00|    v=0

1205143558|sip  |0|00|    o=- 1512444953 1512444953 IN IP4 10.22.0.22

1205143558|sip  |0|00|    s=Polycom IP Phone

1205143558|sip  |0|00|    c=IN IP4 10.22.0.22

1205143558|sip  |0|00|    t=0 0

1205143558|sip  |0|00|    a=sendrecv

1205143558|sip  |0|00|    m=audio 5350 RTP/S

1205143558|sip  |0|00|<<< Data received TLS

1205143558|sip  |0|00|    AVP 9 112 8 0 18 101

1205143558|sip  |0|00|    a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:/u2wztfJizjOcLml2T1uN/3qiNEZpAiiPqCcyOhn|2^31|1:1

1205143558|sip  |0|00|    a=crypto:2 AES_CM_128_HMAC_SHA1_80 inline:gXAfAu4Z6VZ/1ZQYLJe3g4HAGcgwY9w4vDdlLSkJ|2^31

1205143558|sip  |0|00|    a=rtpmap:9 G722/8000

1205143558|sip  |0|00|    a=rtpmap:112 G7221/16000

1205143558|sip  |0|00|    a=fmtp:112 bitrate=24000

1205143558|sip  |0|00|    a=rtpmap:8 PCMA/8000

1205143558|sip  |0|00|    a=rtpmap:0 PCMU/8000

1205143558|sip  |0|00|    a=rtpmap:18 G729/8000

1205143558|sip  |0|00|    a=fmtp:18 annexb=no

1205143558|sip  |0|00|    a=rtpmap:101 telephone-event/8000

1205143558|sip  |0|00|    a=ice-pwd:EK5jmhsUk5xntwoCmEIEAmfT

1205143558|sip  |0|00|    a=ice-ufrag:pbdW

1205143558|sip  |0|00|    a=rtcp:5351

1205143558|sip  |0|00|    a=candidate:1 1 UDP 2130706431 10.22.0.22 5350 typ host

1205143558|sip  |0|00|    a=candidate:1 2 UDP 2130706430 10.22.0.22 5351 typ host

1205143558|sip  |0|00|    a=candidate:2 1 TCP-ACT 1684733951 10.22.0.22 5350 typ srflx raddr 10.22.0.22 rport 5350

1205143558|sip  |0|00|    a=candidate:2 2 TCP-ACT 1684733950 10.22.0.22 5350 typ srflx raddr 10.22.0.22 rport 5350

1205143558|sip  |1|00|MsgSipTcpPacket

1205143558|sip  |1|00|MsgSipTcpPacket

1205143558|sip  |3|00|CStkDialog::CreateRouteSet: transport set to top route 'TLS'

1205143558|sip  |3|00|CStkDialog::SetAddressLocal localTag set to ''

1205143558|sip  |3|00|CStkDialog::SetAddressLocal new address added of 1

1205143558|sip  |2|00|CStkDialog::CStkDialog SetAddressLocal from pRequest To: 'Terry Adams' <+61266591007@sfbdomain.com:0>

1205143558|sip  |2|00|CStkDialog::CStkDialog SetAddressLocal Config 'Terry Adams' <Terry.Adams@sfbdomain.com:0>

1205143558|sip  |2|00|CStkDialog::CStkDialog TAG 'B640E4F5-9C530CE' generated

1205143558|sip  |2|00|CStkDialog::CStkDialog local addr 'Terry Adams' <+61266591007@sfbdomain.com:0> Tag 'B640E4F5-9C530CE'

1205143558|sip  |2|00|CStkDialog::CStkDialog exit 0x9d5bf0 local list size 1

1205143558|sip  |2|00|CCallBase::IsChallenged COPIED Dialog Tag to pRequest 'B640E4F5-9C530CE'

1205143558|sip  |2|00|CCallBase::IsChallenged 'INVITE' Dialog Tag 'B640E4F5-9C530CE' pRequest Tag 'B640E4F5-9C530CE' state 'Trying'

1205143558|sip  |2|00|new UA Server INVITE trans state 'proceeding', timeout=0 (0x40d40ea8)

1205143558|sip  |3|00|CStateInviteServer::CStateInviteServer

1205143558|sip  |3|00|CStateInviteServer::CStateInviteServerHandler

1205143558|sip  |3|00|CStateInviteServer::CStateInviteServerHandler - pReplace (0x0)

1205143558|sip  |*|00|CSdp::operator << - setting m_bIsSecured == true

1205143558|sip  |3|00|CStateInviteServer::CStateInviteServerHandler - calling RemoteSdpOffer

1205143558|sip  |3|00|CStkCall::RemoteSdpOffer m_nMediaRTPPort=0

1205143558|sip  |1|00|Dialog 'id9cd7e4d4' State 'Trying'->'Early'

1205143558|sip  |1|00|signatureBuffer: <TLS-DSK><D0E0949F><11><SIP Communications Service><SFB001.sfbdomain.com><0888c5534fafd0fc37bc94f7a523b38c><1><INVITE><sip:Terry.Adams@sfbdomain.com><5C30CCBE-7E228105><sip:+61266591007@sfbdomain.com;user=phone><B640E4F5-9C530CE><sip:Terry.Adams@sfbdomain.com><tel:+61266591007;ext=1007><><100>

1205143558|sip  |1|00|doDnsListLookup(tls): doDnsSrvLookupForARecordList for '10.20.1.223' port 5061 returned 1 results

1205143558|sip  |1|00|doDnsListLookup(tls): result 0 host '' IP '10.20.1.223' port 5061 isInBound 0

1205143558|sip  |1|00|CTcp::Send(TLS) entry for address 10.20.1.223 port 5061 can Connect 0 canFailOver 1

1205143558|sip  |0|00|>>> Data Send TLS 10.20.1.223:5061

1205143558|sip  |0|00|    SIP/2.0 100 Trying

1205143558|sip  |0|00|    Via: SIP/2.0/TLS 10.20.1.223:5061;branch=z9hG4bK8BA0D60F.E19C5E6D7DAA886D;branched=TRUE;ms-internal-info="dgFJqKHJ0AXHZY0sy2IaYnenMPK5auGe9rVA0q4_Ro3qxtXpzhVO8OYQAA"

1205143558|sip  |0|00|    Via: SIP/2.0/TLS 10.22.0.22:42727;branch=z9hG4bK1d5487849DAF07;ms-received-port=42727;ms-received-cid=77D300

1205143558|sip  |0|00|    From: "Terry Adams" <sip:Terry.Adams@sfbdomain.com>;tag=5C30CCBE-7E228105;epid=64167f23b38c

1205143558|sip  |0|00|    To: "Terry Adams" <sip:+61266591007@sfbdomain.com;user=phone>;epid=00907a0f09d8;tag=B640E4F5-9C530CE

1205143558|sip  |0|00|    CSeq: 1 INVITE

1205143558|sip  |0|00|    Call-ID: 0888c5534fafd0fc37bc94f7a523b38c

1205143558|sip  |0|00|    Contact: <sip:Terry.Adams@sfbdomain.com;opaque=user:epid:x5jsdoUew1a9jiDyam965gAA;gruu>

1205143558|sip  |0|00|    Record-Route: <sip:SFB001.sfbdomain.com:5061;transport=tls;opaque=state:T:F:Ci.R781300:Ieh.Lvw6TSBMlsUoTJbFhkstQInxeS6wHgXRHMHiO5ioDln-fhNvUO1qLucUD4gO1tXpzhB_8cegAA;lr;ms-route-sig=hkx-TWiKYeik4c0hYMHXqBiJ4op9gbnpuTatBwsuD7moptXpzhB_8cegAA>;tag=B12630CFFCD2C61F74BDE5EB8A4C611A

1205143558|sip  |0|00|    User-Agent: Spectralink-SL_8440-UA/5.4.4.1156

1205143558|sip  |0|00|    Accept-Language: en

1205143558|sip  |0|00|    P-Preferred-Identity: "Terry Adams" <sip:Terry.Adams@sfbdomain.com>,<tel:+61266591007;ext=1007>

1205143558|sip  |0|00|    Authorization: TLS-DSK qop="auth", realm="SIP Communications Service", opaque="4E02C6ED", crand="D0E0949F", cnum="11", targetname="SFB001.sfbdomain.com", response="bf39acaf3c91ff5826574895f209520c0507fe35"

1205143558|sip  |0|00|    Content-Length: 0

1205143558|sip  |0|00|   

1205143558|sip  |1|00|CTcpSocket::SendData TLS queuedTxData = 0 TotalLen 1313 loop count 1 maxQueueDepth 40000

1205143558|sip  |1|00|CTcpSocket::SendData TLS Sent 1313 loop count 1

1205143558|sip  |3|00|CStkCall::NewCallState 'Unknown'->'Offering' (0x9b9b48)

1205143558|sip  |3|00|GetRemotePartyAddress from 'P-Asserted-Identity'

1205143558|sip  |3|00|CStkCall::NewCallState - BEFORE call to SipOnEvNewCall m_nMediaRTPPort=0

1205143558|ice  |0|00|soIceChannelCreate: channelId=-1 category=1 chanType=0 udpPort=2226 portTurn=2726 tcpPort=-1

1205143558|ice  |0|00|soIceRtpChanList.count=1

1205143558|ice  |2|00|soIceChannelCreate: allocated channelId 5

1205143558|ice  |0|00|soIceChannelCreate: channelId=-1 category=1 chanType=1 udpPort=2227 portTurn=2727 tcpPort=-1

1205143558|ice  |0|00|soIceRtpChanList.count=2

1205143558|ice  |2|00|soIceChannelCreate: allocated channelId 6

1205143558|ice  |0|00|soIceSessionCreate start

1205143558|ice  |0|00|    channelIdAudioRtp=5

1205143558|ice  |0|00|    enableBWManagement=0

1205143558|ice  |0|00|    bwAudioMin=64

1205143558|ice  |0|00|    bwAudioMax=64

1205143558|ice  |0|00|soIceSessionCreate - full functionality so set gatheringStunReqMaxRetransmit = 4

1205143558|ice  |0|00|soIceSessionListRemove: soIceSessionList.count=0

1205143558|ice  |0|00|soIceSessionCreate: sessEntryP (0x426d0f60) - set bInitialInvite true

1205143558|ice  |0|00|soIceSessionList.count=1

1205143558|ice  |1|00|soIceSessionCreate: linked channel (5) with session (3)

1205143558|ice  |1|00|soIceSessionCreate: linked channel (6) with session (3)

1205143558|ice  |2|00|soIceSessionCreate: allocated hSpiSession 0xb1f9d8 sessionId 3 for audio 5 6

1205143558|ice  |0|00|soIceSessionSdpGetCallAnswer

1205143558|ice  |0|00|soIceSessionSdpGetCallAnswer: len=890 pSdpInvite=

1205143558|ice  |0|00|v=0

1205143558|ice  |0|00|o=- 1512444953 1512444953 IN IP4 10.22.0.22

1205143558|ice  |0|00|s=Polycom IP Phone

1205143558|ice  |0|00|c=IN IP4 10.22.0.22

1205143558|ice  |0|00|t=0 0

1205143558|ice  |0|00|a=sendrecv

1205143558|ice  |0|00|m=audio 5350 RTP/SAVP 9 112 8 0 18 101

1205143558|ice  |0|00|a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:/u2wztfJizjOcLml2T1uN/3qiNEZpAiiPqCcyOhn|2^31|1:1

1205143558|ice  |0|00|a=crypto:2 AES_CM_128_HMAC_SHA1_80 inline:gXAfAu4Z6VZ/1ZQYLJe3g4HAGcgwY9w4vDdlLSkJ|2^31

1205143558|ice  |0|00|a=rtpmap:9 G722/8000

1205143558|ice  |0|00|a=rtpmap:112 G7221/16000

1205143558|ice  |0|00|a=fmtp:112 bitrate=24000

1205143558|ice  |0|00|a=rtpmap:8 PCMA/8000

1205143558|ice  |0|00|a=rtpmap:0 PCMU/8000

1205143558|ice  |0|00|a=rtpmap:18 G729/8000

1205143558|ice  |0|00|a=fmtp:18 annexb=no

1205143558|ice  |0|00|a=rtpmap:101 telephone-event/8000

1205143558|ice  |0|00|a=ice-pwd:EK5jmhsUk5xntwoCmEIEAmfT

1205143558|ice  |0|00|a=ice-ufrag:pbdW

1205143558|ice  |0|00|a=rtcp:5351

1205143558|ice  |0|00|a=candidate:1 1 UDP 2130706431 10.22.0.22 5350 typ host

1205143558|ice  |0|00|a=candidate:1 2 UDP 2130706430 10.22.0.22 5351 typ host

1205143558|ice  |0|00|a=candidate:2 1 TCP-ACT 1684733951 10.22.0.22 5350 typ srflx raddr 10.22.0.22 rport 5350

1205143558|ice  |0|00|a=candidate:2 2 TCP-ACT 1684733950 10.22.0.22 5350 typ srflx raddr 10.22.0.22 rport 5350

1205143558|ice  |0|00|soIceSessionSdpGetCallAnswer: RvIceSessionSDPIn returned 0

1205143558|ice  |0|00|*** sdpInpRes.eIceSupport 0

1205143558|ice  |0|00|soIceSessionSdpGetCallAnswer: entryP (0x426d0f60) bInitialInvite=1

1205143558|ice  |0|00|soIceSetLocalHostCandidates - local rtp port 2226

1205143558|ice  |0|00|soIceSetLocalHostCandidates: RvIceSessionSetLocalRtpCandidatesToAllMediaStreams returned 0

1205143558|ice  |0|00|soIceSessionSdpGetCallAnswer: RvIceSessionGatherReflexiveCandidatesEx returned=-1

1205143558|sip  |2|00|SipOnEvNewCall new call appearnce 1 SRTP key È Ô@¨ Ô@

1205143558|sip  |3|00|CStkCall::NewCallState - after call to SipOnEvNewCall m_nMediaRTPPort=2226

1205143558|sip  |2|00|SipOnEvCallNewState 9b9b48,bdb4a0 1,(null)

1205143558|sip  |3|00|CStkCall::NewCallState update held and hold flags, state 1

1205143558|sip  |3|00|CStkCall::NewCallState held 0 hold 0

The Fix!

After realising that this probably had something to do with not having an Edge server deployed on the lab system I was using, I moved the Spectralink over to another system that did have an Edge… and what do you know, the phone started working perfectly. Not having an Edge was in fact the issue. The next question was, is there any configuration that could be done on the Spectralink to make it work without an Edge? (Spoiler: Yes)

As you may or may not know, one of the main configuration items on a Spectralink (or Polycom phones which have basically the same software software forked from the same original base code) is to configure the baseProfile setting to make it run in "Lync" mode. Lync mode makes a whole raft of settings in the background on your behalf, to make your life easier. What are these settings that get made when the base profile is set to Lync mode? Well, to try and make a comparison I used a Polycom VVX running 5.6 firmware (because it was easier to get access to the settings than Spectralink) as a comparison point to see what settings the Lync profile changed in the config:

Lync Base Profile settings (in comparison to Generic mode):

call.defaultTransferType="Blind"

call.enableOnNotRegistered="0"

callLists.collapseDuplicates="0"

callLists.logConsultationCalls="1"

device.baseProfile="Lync"

dialplan.1.applyToCallListDial="0"

dialplan.1.applyToDirectoryDial="1"

dialplan.1.applyToForward="1"

dialplan.1.conflictMatchHandling="1"

dialplan.1.digitmap="x.T"

dialplan.1.digitmap.timeOut="4"

dialplan.1.impossibleMatchHandling="3"

dialplan.1.lyncDigitmap.timeOut="4"

dialplan.applyToDirectoryDial="1"

dialplan.applyToForward="1"

dialplan.applyToPstnDialing="1"

dialplan.applyToRemoteDialing="1"

dialplan.conflictMatchHandling="1"

dialplan.digitmap=""

dialplan.impossibleMatchHandling="3"

dialplan.routing.emergency.1.description=""

dialplan.routing.emergency.1.value=""

dialplan.TranslationInAutoComp="1"

dialplan.userDial.timeOut="4"

feature.btoe.enabled="1"

feature.deviceLock.enable="1"

feature.EWSAutodiscover.enabled="1"

feature.exchangeCalendar.enabled="1"

feature.exchangeCallLog.enabled="1"

feature.exchangeContacts.enabled="1"

feature.exchangeVoiceMail.enabled="1"

feature.logUpload.enabled="1"

feature.lync.abs.enabled="1"

feature.LyncCCCP2010AudioWorkaround.enabled="1"

feature.lyncSafeTransfer.enabled="1"

feature.messaging.enabled="1"

feature.moh.enabled="1"

feature.presence.enabled="1"

locInfo.source="MS_E911_LIS"

reg.1.applyServerDigitMapLocally="1"

reg.1.auth.useLoginCredentials="1"

reg.1.auth.usePinCredentials="1"

reg.1.offerFullCodecListUponResume="0"

reg.1.server.1.registerRetry.baseTimeOut="10"

reg.1.server.1.registerRetry.maxTimeOut="180"

reg.1.server.1.specialInterop="lync2010"

reg.1.server.1.transport="TLS"

reg.1.serverFeatureControl.signalingMethod="serviceMsForwardContact"

reg.1.useTelUriAsLineLabel="0"

roaming_buddies.reg="1"

sec.srtp.holdWithNewKey="0"

sec.srtp.key.lifetime="2^31"

sec.srtp.mki.enabled="1"

sec.srtp.mki.length="1"

sec.srtp.mki.startSessionAtOne="1"

sec.srtp.resumeWithNewKey="0"

sec.TLS.profileSelection.SIP="ApplicationProfile1"

server.log.setting.enabled="1"

softkey.feature.MeetNow="1"

softkey.feature.simplifiedSignIn="1"

tcpIpApp.ice.mode="MSOCS"

tcpIpApp.keepalive.tcp.sip.tls.enable="1"

tcpIpApp.port.rtp.videoPortRange.enable="1"

tcpIpApp.sntp.address="time.windows.com"

tone.dtmf.rfc2833Payload="101"

up.numOfDisplayColumns="2"

up.oneTouchDirectory="1"

up.oneTouchVoiceMail="1"

video.iFrame.delay="2"

video.iFrame.onPacketLoss="1"

voice.codecPref.G7221.24kbps="5"

voice.codecPref.G7221.32kbps="0"

voice.qualityMonitoring.rtcpxr.enable="1"

voIpProt.SIP.allowTransferOnProceeding="0"

voIpProt.SIP.considerTlsDnsEntriesOnly="1"

voIpProt.SIP.failoverOn503Response="0"

voIpProt.SIP.header.diversion.enable="1"

voIpProt.SIP.IM.autoAnswerDelay="40"

voIpProt.SIP.mtls.enable="0"

voIpProt.SIP.serverFeatureControl.cf="1"

voIpProt.SIP.serverFeatureControl.dnd="1"

voIpProt.SIP.serverFeatureControl.localProcessing.cf="0"

voIpProt.SIP.serverFeatureControl.localProcessing.dnd="0"

Based on my earlier assumption that the issue has something to do with the Edge reflexive and relay candidates, the fault was likely caused by something to do with the Microsoft specific ICE interaction. There was one setting this this sizable list that stood out to me - the tcpIpApp.ice.mode="MSOCS" setting. This setting was made specifically to handle the candidate negotiations required by Microsoft Lync/Skype for Business. After disabling this setting, guess what happened? Inbound and outbound calls started working…

TLDR Answer

When you have no Edge server deployed within your Lync/Skype for Business environment and you are deploying Spectralink phones, set the tcpIpApp.ice.mode setting to “disabled”. In a config file this might look something like this:

<!-- If there is an Edge then tcpIpApp.ice.mode="MSOCS" if there is no Edge then tcpIpApp.ice.mode="disabled" -->

<LyncBaseProfile

device.baseProfile.set="1"

device.baseProfile="Lync"

tcpIpApp.ice.mode="disabled"             

> 

</LyncBaseProfile>

The Wrap Up

Well, another day another crazy config setting. There always seems to be an endless supply of gotchas in this business and this is another one to add to your list. Till next time!

Read more →

Skype4B / Lync Certificate Checker Tool

If you’ve ever installed Skype for Business or Lync before, you will know that the system requires PKI Infrastructure and Certificates to function. The reason for this is that all SIP and Web communications within the Skype for Business environment is secure by design and uses certificates for encrypting data. These communications span between servers, clients, phones, PSTN Gateways, Third Party Video equipment and most other integrations you can think of. So without your certificates being deployed properly, you are going to have a lot of trouble getting your environment up and running.

Skype for Business/Lync Edge servers communicate with each other over Mutual Transport Layer Security (MTLS). When using MTLS connections the server originating a message and the server receiving it exchange certificates from mutually trusted Certificate Authorities. The public certificates presented in either direction prove the identity of each server by being signed by a trusted certificate authority. The main thing here to note here is that both servers need to have root certificates installed from each other’s trusted root certificate authority in order for TLS connections to negotiate successfully. This is also the case for federated connections to other organisations via the Skype for Business Edge server. These connections all rely on MTLS for the successful communication between the servers.

Encryption Used in Skype for Business

Traffic type	Protected by
Server-to-server	MTLS
Client-to-server	TLS
Instant messaging and presence	TLS
Audio and video and desktop sharing of media	SRTP (No Certificates Used)
Desktop sharing (signaling)	TLS
Web conferencing	TLS
Meeting content download, address book download, distribution group expansion	HTTPS
Mobile Clients (UCWA)	TLS

In many cases you may not have direct access to the other system you are connecting to in order to check whether the certificate it is using is valid, or has been signed by a trusted root certificate Authority. As a result, you may have issues connecting to the server and need to use complex tools like Wireshark to determine what the certificate being presented by the far end looks like. This can take time and involve installing software on servers, so I wanted to create a simple tool that doesn’t require any installation and can be run straight from a Powershell prompt. After doing some coding, that’s exactly what I created, introducing the Skype for Business Certificate Checker Tool…

Skype4B / Lync Certificate Checker Tool

Features:

• Check the certificate being used by a server using the FQDN/IP and Port number of the server.
• Check the certificate of a Federation SRV record (_sipfederationtls._tcp.domain.com) simply by entering the SIP domain name and ticking the “FED SRV” checkbox.
• Check the SIP SRV record (_sip._tls.domain.com) by simply entering the SIP domain name and ticking the “SIP SRV” checkbox.
• Check the SIP Internal SRV record (_sipinternaltls._tcp.domain.com) by simply entering the SIP domain name and ticking the “SIP INT SRV” checkbox.
• Select the DNS server you would like to use to resolve DNS from by entering a DNS Server IP address in the “DNS Server” field.
• “Show Advanced” checkbox will show all of the information in the certificate.
• The “Show Root Chain” will display the root certificate and all of the intermediate certificates that are applicable in the trust chain for the certificate.
• The “Test DNSLB Pool” checkbox is on by default and will instruct to the tool to test all of the IP Addresses that are resolved for a DNS Name. In the case of Skype for Business, we nearly always have multiple DNS records per A record for the purposes of DNS load balancing.  Rather than having to look all of the servers yourself, the tool will do this for you. Other servers in pool will be displayed in the Information text box in blue colour and will be tested directly via their IP Address rather than the DNS name.
• Import multiple DNS name records from a CSV file. This is useful if you want to check a lot of servers in one sitting. See the “Import File Format” section for more details.
• Save certificate information out to a CSV file. This will save all of the certificate information out in table format that you can open in Excel for record keeping purposes. Note: This export format is different than the one used in conjunction with the “Import” button.
• Comments section – The comments section will have information in it about things that may be wrong with the certificate to help you troubleshoot your issues.

DOWNLOAD HERE

Import File Format

You can import a CSV file containing many domains and servers to test if you choose (for example, this may be useful for checking a large list of federated domains). To do this you will first need to create a CSV file with all of the servers and/or domains that you want to test in it. The format of the CSV for each of the record types will look like:

·        

Header row: Domain,Type,Port

Example Federation Record: "microsoft.com","FED","",

Example SIP Record: "microsoft.com","SIP","",
Example SIP Record: "microsoft.com","SIPINT","",

Example direct Record: "sip.microsoft.com","DIR","5061",

Example file:

Domain,Type,Port

"microsoft.com","FED","",

"microsoft.com","SIP","",
"microsoft.com","SIPINT","",

"sip.microsoft.com","DIR","5061"

The Anatomy of a Certificate

The Certificate Checker Tool can display a few different levels of information about the certificate presented by the server. The default basic view of the certificate will be displayed in the tool as follows:

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Checking FQDN:  sipfed.microsoft.com:5061 Checking IP Address: 167.220.67.163:5061 Certificate Response: Subject: CN=sipfed.microsoft.com, OU=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=WA, C=US Issuer: CN=Microsoft IT SSL SHA2, OU=Microsoft IT, O=Microsoft Corporation, L=Redmond, S=Washington, C=US Not Before: 30/04/2015 2:26:22 PM Not After: 29/04/2017 2:26:22 PM Serial Number: 5A0000F5B0C7CABB89E4624D1E00010000F5B0 Signature Algorithm: sha256RSA Thumbprint: 9E1736ACA8C9731798E7FD3496E7D78454A02E80 Version: 3 HasPrivateKey: False ---------------------------------------------------------------------------------- Comments: - Common Name Match found - FQDN is in SAN list. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

For a Skype for Business or Lync deployment the most important components here are the Subject name, the Not Before and Not After dates. The “Comments” section is provided by the tool to help you troubleshoot issues with the certificate being displayed. This section will automatically check things like the certificate being out of date, the common name/subject alternate names being correct, if there is a Server EKU, and if the certificate has a CLR list included. These comments should help speed up your troubleshooting of certificate issues. Note: the comments will actually be based on all the advanced certificate details, even though the Advanced checkbox is not ticked by default.

Advanced Settings

All certificate details are displayed when the “Advanced” check box is ticked. When you tick this check box and run the tool you will see information as shown below:

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Checking FQDN:  sipfed.microsoft.com:5061 Checking IP Address: 167.220.67.163:5061 Certificate Response: Subject: CN=sipfed.microsoft.com, OU=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=WA, C=US Issuer: CN=Microsoft IT SSL SHA2, OU=Microsoft IT, O=Microsoft Corporation, L=Redmond, S=Washington, C=US Not Before: 30/04/2015 2:26:22 PM Not After: 29/04/2017 2:26:22 PM Serial Number: 5A0000F5B0C7CABB89E4624D1E00010000F5B0 Signature Algorithm: sha256RSA Thumbprint: 9E1736ACA8C9731798E7FD3496E7D78454A02E80 Version: 3 HasPrivateKey: False Archived: False Extension Type: Certificate Policies Oid Value: 2.5.29.32 Data: [1]Certificate Policy:      Policy Identifier=1.3.6.1.4.1.311.42.1      [1,1]Policy Qualifier Info:           Policy Qualifier Id=CPS           Qualifier:                http://www.microsoft.com/pki/mscorp/cps Extension Type: Application Policies Oid Value: 1.3.6.1.4.1.311.21.10 Data: [1]Application Certificate Policy:      Policy Identifier=Server Authentication [2]Application Certificate Policy:      Policy Identifier=Client Authentication Extension Type: Key Usage Oid Value: 2.5.29.15 Data: Digital Signature, Key Encipherment, Data Encipherment (b0) Extension Type: Enhanced Key Usage Oid Value: 2.5.29.37 Data: Server Authentication (1.3.6.1.5.5.7.3.1) Client Authentication (1.3.6.1.5.5.7.3.2) Extension Type: Subject Key Identifier Oid Value: 2.5.29.14 Data: df 62 d3 a8 ef 49 3d 2f ed 10 aa 6a 30 3a 6f f9 54 1b 33 39 Extension Type: Authority Key Identifier Oid Value: 2.5.29.35 Data: KeyID=51 af 24 26 9c f4 68 22 57 80 26 2b 3b 46 62 15 7b 1e cc a5 Extension Type: CRL Distribution Points Oid Value: 2.5.29.31 Data: [1]CRL Distribution Point      Distribution Point Name:           Full Name:                URL=http://mscrl.microsoft.com/pki/mscorp/crl/msitwww2.crl                URL=http://crl.microsoft.com/pki/mscorp/crl/msitwww2.crl Extension Type: Authority Information Access Oid Value: 1.3.6.1.5.5.7.1.1 Data: [1]Authority Info Access      Access Method=Certification Authority Issuer (1.3.6.1.5.5.7.48.2)      Alternative Name:           URL=http://www.microsoft.com/pki/mscorp/msitwww2.crt [2]Authority Info Access      Access Method=On-line Certificate Status Protocol (1.3.6.1.5.5.7.48.1)      Alternative Name:           URL=http://ocsp.msocsp.com Extension Type: Subject Alternative Name Oid Value: 2.5.29.17 Data: DNS Name=sipfed.microsoft.com DNS Name=sipalt.microsoft.com DNS Name=sip.microsoft.com DNS Name=ra30.sbweb.microsoft.com DNS Name=web3.sbweb.microsoft.com DNS Name=web30.sbweb.microsoft.com DNS Name=web31.sbweb.microsoft.com ---------------------------------------------------------------------------------- Comments: - Common Name Match found - FQDN is in SAN list. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

You will have a full view of all attributes contained within the certificate. Using this information you should be able to troubleshoot most certificate related issues. However, there is one more important piece of information that you might need and that is the certificate chain…

The Certificate Chain

The certificate chain is the hierarchy of Certificate Authority servers from the CA server that issued the certificate through the Intermediate Certificate Authorities to the Root Certificate Authority server. The tool will display the certificate chain as follows:

Certificate Chain: Certificate Chain Item 1 Chain Subject Name: CN=sipfed.microsoft.com, OU=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=WA, C=US Chain Issuer name: CN=Microsoft IT SSL SHA2, OU=Microsoft IT, O=Microsoft Corporation, L=Redmond, S=Washington, C=US Chain Not Before: 11/12/2015 05:36:48 Chain Not After: 11/11/2017 05:36:48 Chain Serial Number: 5A000233C22F738FDBCE9CF8B50001000233C2 Chain Signature Algorithm: sha256RSA Chain Thumbprint: A710B806065C2187E387635A9F8D7863A63D702A Chain Version: 3 Chain is valid: True Certificate Chain Item 2 Chain Subject Name: CN=Microsoft IT SSL SHA2, OU=Microsoft IT, O=Microsoft Corporation, L=Redmond, S=Washington, C=US Chain Issuer name: CN=Baltimore CyberTrust Root, OU=CyberTrust, O=Baltimore, C=IE Chain Not Before: 05/08/2014 03:04:09 Chain Not After: 05/08/2018 03:03:30 Chain Serial Number: 0727AA47 Chain Signature Algorithm: sha256RSA Chain Thumbprint: 97EFF3028677894BDD4F9AC53F789BEE5DF4AD86 Chain Version: 3 Chain is valid: True Certificate Chain Item 3 Chain Subject Name: CN=Baltimore CyberTrust Root, OU=CyberTrust, O=Baltimore, C=IE Chain Issuer name: CN=Baltimore CyberTrust Root, OU=CyberTrust, O=Baltimore, C=IE Chain Not Before: 05/13/2000 04:46:00 Chain Not After: 05/13/2025 09:59:00 Chain Serial Number: 020000B9 Chain Signature Algorithm: sha1RSA Chain Thumbprint: D4DE20D05E66FC53FE1A50882C78DB2852CAE474 Chain Version: 3 Chain is valid: True ---------------------------------------------------------------------------------- Root Certificates: - Get Root Certs here: http://cybertrust.omniroot.com/support/sureserver/rootcert_iis.cfm - Download Root Cert: http://cacert.omniroot.com/bc2025.crt ----------------------------------------------------------------------------------

The tool will show you the Subject Name, Issuer Name (which will be the next server in the list/chain), Element Signature Algorithm (can be important, see below), and whether or not the chain is valid.

In addition to displaying the certificate chain the tool will also, where possible, provide a link to a copy of the root certificate for the root CA being used. The tool knows about the major certificate authorities supported by Microsoft for use with Skype for Business and in most cases will give you a direct link to the root certificate for download.

Root Certificates

One very important thing when configuring external federation with partners or public providers is that MTLS is used for these connections. This means that both ends of the connection need to trust the other’s root certificates. You need to ensure that your edge servers have the root certificates of your partners installed. Fortunately, the Cert Checker Tool has you covered here by showing you where you can download the root certificates for common public certificate authorities. This will appear like shown below:

- Get Root Certs here: http://cybertrust.omniroot.com/support/sureserver/rootcert_iis.cfm - Download Root Cert: http://cacert.omniroot.com/bc2025.crt

Before you configure a new partner for federation, you can you use the tool to check what certificate authority they are using for their certificates and as a result which root certificates you need installed on your edge servers.

There is also neat trick you can do to automagically install root certificates on a Windows server or PC (post Windows Vista). Note there is a caveat with this process whereby the third party server must be using a Root Certificate Authority that is trusted by Microsoft as part of their Trusted Root Certificate Program (Microsoft supported root CAs can be confirmed on this list). If this is the case then you just need to browse to a web server that is signed by the root certificate authority of choice and Windows will automatically install the root certificate for you! These root certificates are pushed to Window through Windows Update and will be installed only when you try to connect to a website requiring a particular certificate. So connecting to a federated partner's "dialin.domain.com" web page from all of your Edge servers may be enough to download the root certificates for MTLS trust purposes. There is a lot of documentation about this process on TechNet if you would like to know more. A few Skype for Business community have also written about this phenomenon - Chris and Pat.

The Wrap Up

I hope that this new tool finds you well, and I hope that you have many long years of troubleshooting together. Remember, whilst the flame may flicker from time to time, you must stay strong and think fondly of those times in the early days when you hired the car, threw the work laptop in the boot, and drove to the cabin in the woods; not even one bar of 3G internet access could stop you from fixing that server certificate problem. It’s the memory of those times that will keep you on the straight and narrow when that younger and fancier tool with the sexy universal windows app GUI comes along. Your Powershell Certificate Checker will always be faithful, remember that… now get testing!

Read more →