I had a request to investigate why calls were failing between CUCM Clusters, but only when calls were being transferred/redirected from the Cisco Unity Connection system handler. Internal CUCM extensions could direct call between the two CUCM Clusters, Internal and external call forwarding and transferring work successfully across the clusters. Cisco Unity Connection could also transfer out to external numbers and also internal cluster numbers from the system handlers.
I setup traces in CUCM on Cluster A. I didn’t have visibility of Cluster B configuration. Below is a small snippet of the traces I collected. I’ve mainly just cut the error from the traces. The call was being rejected by cluster B in the H225 setup request. The cluster A was hunting for another Trunk/Gateway to send the call, but couldn’t find an alternate path.
So knowing cluster A is actually sending the initial h225 request was good point to start. I then examined the IP Address from which the H225 setup request was sourced from. A little bit Hex conversion on a bit a paper showed me the IP address. I then queried the Voice Administrator for Cluster B to let me know what IP Addresses had been configured for the ICT Trunk, and bingo the source IP Address was not configured on the Cluster B ICT Trunk. Remember CUCM by default will not accept inbound connections from unknown hosts. There is a service parameter to allow such behaviour though. After resetting the ICT trunk, call were now working from Cluster A to Cluster B.
The reasoning behind why IP Phones could dial across the cluster and the Unity Connection server could not is directly related to what CUCM server the device/CTI Port is registered too, or in the SIP world the primary destination IP Address. The Unity Connection server was directing to CUCM B, while IP Phones that I tested were registered to CUCM A. CUCM A was listed in Cluster B’s ICT Trunk configuration settings however CUCM B was not listed.
48627014.000 |18:53:43.199 |SdlSig |CcSetupReq |null0 |RouteListCdrc(1,100,83,9734) |RouteListCdrc(1,100,83,9734) |1,100,13,40817.2^*^* |[R:N-H:0,N:0,L:0,V:0,Z:0,D:0] CI=28537545 CI.branch=0 sBPL.plid=65 sBPL.l=0 sBPL.pl=5 sBPL.msd=0 FDataType=0opId=0ssType=0 SsKey=0invokeId=0resultExp=Fbpda=F pi.piid=30 pi.l=0 pi2.piid=30 pi2.l=0 pi3.piid=30 pi3.l=0 FQCGPN=pi=0si1 preXCgpn=tn=2npi=1ti=1nd=+642108375375pi=0si3 cgPart= cgPat= cgpn=tn=2npi=1ti=1nd=*64+642108375375pi=1si3 cgpnVM= unXCgpn=tn=2npi=1ti=1nd=+642108375375pi=1si3 cName=locale: 1 Name: UnicodeName: pi: 0 DD=ti=1nd=9014011User=9014011Host=10.129.6.1Port=5060PassWord=Madder=Transport=2mDisplayName=RawUrl=sip:9014011@10.129.6.1:5060pi=0si1 origDD=tn=0npi=1ti=1nd=8310pi=0si1 preXCdpn=tn=0npi=0ti=1nd=4900007pi=0si0 preXTagsList=SUBSCRIBER preXPosMatchList=4900007 cdPart=b165e2c8-d865-4f60-8283-bb946e961281 cdPat=49XXXXX cdpn=tn=0npi=0ti=1nd=4900007pi=0si1 cdpnVMbox= localPatternUsage=5 connectedPatternUsage=2 itrPart= itrPat= LRPart=b165e2c8-d865-4f60-8283-bb946e961281 LRPat=9017006 LR=tn=0npi=0ti=1nd=9017006pi=1si0 LRVM= LRName=locale: 1 Name: UnicodeName: pi: 1 FQOCpdn=ti=1nd=4900007pi=0si1 fFQLRNum=pi=0si1 oPart=b165e2c8-d865-4f60-8283-bb946e961281 oPat=49XXXXX oCpdn=tn=0npi=0ti=1nd=4900007pi=0si1 oCdpnVM= oRFR=130 oName=locale: 1 Name: UnicodeName: pi: 0 ts=SUBSCRIBER posMatches=4900007 withTags= withValues= rdn.l=0IpAddrMode=0 ipAddrType=0 ipv4=10.129.2.1:51025 region=R_Auckland capCount=4 ctiActive=F ctiFarEndDev=2 ctiCCMId=1 cgPtyDev=10.129.6.7 callInst=0 OLF=1Supp DTMF=1DTMF Cfg=1DTMF Payload=0isOffNetDev=T bc.l=3 bc.itr=1 bc.itc=16 bc.trm=0 bc.tm=0 maxForwards=70 cgpnMaskedByRedirect=F callingLRG=b506f727-c1af-2188-5cec-21d15743334d featCallType=0 muteEnabled=0 associatedCallCI=0 featurePriority=1 nonTargetPolicy=0 unconsumedDigits= suppressMOH=F numPlanPkid =58cb38ed-8432-1064-9608-ca1ebc003e4e networkDomain= bitMask=0 SetupReason=0 routeClass=1 sideACmDeviceType=7 protected=1 ControlProcessType=0 tokens=0 isPresent=F transitCount=0 geolocInfo={geolocPkid=, filterPkid=, geolocVal=, devType=3} locPkid=e510c28a-2766-bb3b-21f8-5bbf8781905e locName=L_AUCKLAND deductBW=F fateShareId=QBENZ:28537539 videoTrafficClass=0 oFromAnalogDvc=F bridgeParticipantID= callingUsr= remoteClusterID= isEMCCDevice=F lHPMemCEPN= cHPMemCEPN=TransparentData=null CanSupportSIPTandN=false TransId=0 AllowBitMask=0×0 UserAgentOrServer= OrigDDName=locale: 1 Name: UnicodeName: pi: 0 mCallerId= mCallerName=LatentCaps=null icidVal= icidGenAddr= oioi= tioi= ptParams= receivedPAID= routeHdr= routeCepn= requestURI= PCVFlag=F originallyHadISUP=F isIMSFinalRoute=F IMSMode=0 originalLRG= lastRedirectingLRG= nwLoc=1
48627014.001 |18:53:43.200 |AppInfo |GenAlarm: AlarmName = RouteListExhausted, subFac = CALLMANAGERKeyParam = , severity = 4, AlarmMsg = RouteListName : RL_AUST_ICT, Reason=41, RouteGroups(RG_AUST_ICT)
AppID : Cisco CallManager
ClusterID : Standalone
NodeID : UCMPUB
48627014.002 |18:53:43.200 |AppInfo |GenAlarm: Push_back offset 256 seq 256
48627015.000 |18:53:43.200 |SdlSig |CcRejInd |idle |RouteListControl(1,100,82,35) |RouteListCdrc(1,100,83,9734) |1,100,13,40817.2^*^* |[R:N-H:0,N:2,L:0,V:0,Z:0,D:0] CI=28537545 CI.branch=0 c.l=1 c.cid=8 c.cs=0 c.lc=0 c.r=0 CV=41 rejectType=1 dtmDisconn=F FDataType=0opId=0ssType=0 SsKey=0invokeId=0resultExp=Fbpda=FTransparentData=null CanSupportSIPTandN=false TransId=0 AllowBitMask=0×0 UserAgentOrServer= OrigDDName=locale: 1 Name: UnicodeName: pi: 0 mCallerId= mCallerName= mediaCause=0
48627016.000 |18:53:43.200 |SdlSig |CcRelInd |wait |Cc(1,100,213,1) |RouteListCdrc(1,100,83,9734) |1,100,13,40817.2^*^* |[R:N-H:0,N:1,L:0,V:0,Z:0,D:0] CI=28537545 CI.branch=0 c.l=1 c.cid=8 c.cs=0 c.lc=0 c.r=0 CV=41 isDeskPickup=F dtmDisconn=F FDataType=0opId=0ssType=0 SsKey=0invokeId=0resultExp=Fbpda=FTransparentData=null CanSupportSIPTandN=false TransId=0 AllowBitMask=0×0 UserAgentOrServer= OrigDDName=locale: 1 Name: UnicodeName: pi: 0 mCallerId= mCallerName=
48627017.000 |18:53:43.200 |SdlSig |DStopInd |idle |RouteListControl(1,100,82,35) |RouteListCdrc(1,100,83,9734) |1,100,13,40817.2^*^* |[R:N-H:0,N:1,L:0,V:0,Z:0,D:0]
48627018.000 |18:53:43.200 |SdlSig |CcRelInd |tcc_incoming_call_proceeding4 |Cdcc(1,100,212,81742) |Cc(1,100,213,1) |1,100,13,40817.2^*^* |[R:N-H:0,N:1,L:0,V:0,Z:0,D:0] CI=28537545 CI.branch=0 c.l=1 c.cid=8 c.cs=0 c.lc=0 c.r=0 CV=41 isDeskPickup=F dtmDisconn=F FDataType=0opId=0ssType=0 SsKey=0invokeId=0resultExp=Fbpda=FTransparentData=null CanSupportSIPTandN=false TransId=0 AllowBitMask=0×0 UserAgentOrServer= OrigDDName=locale: 1 Name: UnicodeName: pi: 0 mCallerId= mCallerName=
48627019.000 |18:53:43.200 |SdlSig |DStopConf |routeListExhausted_shutting_down |RouteListCdrc(1,100,83,9734) |RouteListControl(1,100,82,35) |1,100,13,40817.2^*^* |[R:N-H:0,N:2,L:0,V:0,Z:0,D:0] varChannelStatus=0