Many Cisco Agent Desktop (CAD) users at the user's site could not login to Unified Contact Center Express(UCCX)consistently. The users that did get logged in would have intermittent problems transferring calls and often could not hangup a call after completing their conversation.
To temporarily remedy the problem and get the Contact Center Agents logged in a reboot was being done to the Cisco Unified Communication Manager (CUCM), i.e. CallManager Publisher and Subscriber nodes. In addition, they would reboot the Primary and Secondary UCCX servers to resolve the problem. This work around usually had to be initiated every few days as that seemed to be the frequency of the problem.
After a CUCM upgrade to V 188.8.131.52900-4 which the user thought would resolve the problem per the direction of Cisco TAC, the frequency of the reboot workaround became an every morning event.
The user called in one morning and said that now, even after their reboot workaround attempts they had about 5 out of 25 total users on one floor that could not log in period. Then it was determined that other floors were also experiencing the login failures, so the workaround did not work. They even tried the reboot a few more times with no success.
Here is the error that was being seen via a pop up box on the user's PC's when trying to login to UCCX via their Cisco Agent Desktop (CAD) application:
The request to log into the Cisco Unified CCX Application server timed out. Please verify your system is online and try again.
Since the user had two Cisco TAC cases opened up for the ongoing problems, I had their UCCX case requeued and the severity raised to a Priority 1 (P1) as the TAC engineer was not on duty and the user considered the outage as critical to their call center operations. After working all day long with the new Cisco TAC engineer examining UCCX logs, UCCX and CUCM configuration settings the Cisco TAC engineer discovered that we were hitting a software defect within CUCM (Call Manager)184.108.40.206900-4 and that we needed to upgrade to CUCM 7.1.5 to resolve the issue.
Cisco BUG id: CSCth74824 - Slow LDAP authentication communications manager.
Symptom: LDAP authentication delayed by 10 seconds
Condition: CUCM Appliance.
Defect opened to make following optimizations:
From the platform API logs, can see CertifiateManager.getCertificate() method being locked. This method probably does not need a lock. Like the enablePort() method, it should probably use a lock duration of 0.
Also from IMS perspective we can do an optimization. If LDAPS is not configured, don't initialize trust-store.
7.0(0.4) 7.1(3) 7.1(5.10000.12) 8.0(2.10000.12)
8.5(0.98000.8) 8.5(0.98000.16) 7.1(5.98000.120) 7.1(5.31900.2) 7.1(5.31012.2) 8.0(2.42012.1) 7.0(2.24047.1) 8.0(3.11001.3)
Until we could get the CUCM upgrade approved via the user's change management committee and in place we wanted to try to get the users all logged into UCCX via their PC's running CAD.
We ended up going into the UCCX Cisco Unified CM Configuration and in the AXL Service Provider Configuration, we removed the UCCM Publisher from the selected list on the left and put it into the Available AXL Service Provider list on the right hand side. We then turned off the AXL Service on the publisher.
This in affect made the subscriber the only choice of AXL Service providers to use. Since the problem was a sync problem between the publisher and the subscriber CallManager nodes per Cisco TAC.
Again, we had the Agents attempt to login and it still failed.
We then had the agents reboot their client PC's and then go and attempt to login again. It finally worked only after we clicked on Yes to a popup box that said that the user was already logged in and did we want to continue with the login.
We also did a full reboot of the publisher then subscriber, as well as, Primary and Backup UCCX servers. All processes were showing as in-service, so we came up clean. At this point we had up to 24 CAD agents logged in on one floor and had a few other CAD users that were in Sales that gave us some login problems, yet after rebooting their PC's and clicking on Yes to the popup box we were finally able to get them all logged in.
After a few days the change management was approved and the user upgraded their CUCM(CallManager)servers. Version 7.1(5.31012.2)of CUCM was provided from Cisco via a direct FTP download as this was the latest version in this code stream and had the most recent patches included.
Per Cisco the following 7.1.5 versions all contain the fixes for the problem described in this article:
7.1(5.98000.120) 7.1(5.31900.2) 7.1(5.31012.2)
Additional CUCM notes provided from Cisco TAC:
I would request you to upgrade the call manager to 7.1(5b)SU2. At this point in time you are at 220.127.116.11900-4 or 7.1(3b)SU2, so I would recommend you upgrade to 7.1(5b)SU2 and as per the compatibility matrix, there is an upgrade path from 7.1(3b)SU2 to 7.1(5b)SU2. This can be a bit confusing as we have different naming styles for the CUCM versions depending on the reference point it is obtained from. Check out the compatibility matrix below:
Click here to view the Cisco document @ http://www.cisco.com/en/US/docs/voice_ip_comm/cucm/compat/ccmcompmatr.html#wp200646
NOTE: The above-mentioned URL will take you to a non-HP Web site. HP does not control and is not responsible for information outside of the HP Web site.
Also, 7.1(5b)SU2 is compatible with UCCX 7.0(1) SR 5, which is what the user was running on his server.