Shoretel Users Can’t Change Call Handling Mode or Agent Status

TL;WR Probably the SG90 acting up again. Those things are weird. Rebooting the SG90 and the Director server fixed it for me. YMMV.

While moving around some VM’s we had a Shoretel Director server running without a network connection for 4 hours during a maintenance window. Afterwards users couldn’t change their Call Handling Modes or change their agent logged in/out status. It failed from both the phones and from communicator. At first I thought it was a CAS problem; however the phone directory, history, options, and speed dial features were all working correctly.

I popped up the IPDSCASCfgTool (see bottom) to set the log levels for the CAS to include all the DB and CAS flags for a start. After that I used powershell to stream the logs with Get-Content.

Get-Content "C:\Shorline Data\Logs\ipds-190225.000000.Log" -Wait -Tail 0
 17:49:28.837 ( 3264: 3512) >SetUserCHM. User: 123. CHM: 2  
 17:49:28.888 ( 3264: 3512)  
 15:52:01.574 ( 7508: 5168) >CDBWriter::SetUserCHM::CDBUpdateTable::Update() failed. Error: 0xc1200db5.

SetUserCHM was me (unsuccessfully) changing from CHM 1 (Standard) to CHM 2 (In a Meeting) from communicator (testing from a phone will also log here but it’s noisier). That error sent me off looking for database issues, communications problems, etc. No dice. The evt log showed some interesting output though:

15:55:17.013 ( 4080: 4476) [evtl] (**Error**) CEventLibImpl::sendReceiveIPC failed - 0xC126100C  
15:55:17.029 ( 7508: 4036) [evtl] (**Error**) CEventLibImpl::sendReceiveIPC failed - 0xC126100C  
15:55:17.183 ( 4080: 4436) [evtl] (**Error**) CEventLibImpl::sendReceiveIPC failed - 0xC126100C  
15:55:17.183 ( 2992: 6552) [evtl] (**Error**) CEventLibImpl::sendReceiveIPC failed - 0xC126100C  
15:55:17.183 ( 1764: 1856) [evtl] (**Error**) CEventLibImpl::sendReceiveIPC failed - 0xC126100C  
15:55:17.187 ( 4972: 5232) [evtl] (**Error**) CEventLibImpl::sendReceiveIPC failed - 0xC126100C 

After digging around for named pipe issues and doing traces I tried the same on the voice switch and didn’t see any interesting errors. Theoretically the phone’s button and control traffic hits the voice switch and is making it to the director but I couldn’t verify that because I couldn’t get the switch to start a packet cap. Supposedly that’s because of a cipher mismatch: the director server tries to ssh into the voice switch to start the packet cap but it fails to login using it’s certificate.

Anyway I ran out of ideas and just waited until I could restart the switch and it worked. Everything was fine. Once again the SG90 blew up in my face was behaving unexpectedly and sent me one step closer to trying the vswitch.

Another one of those 1/1 google searches:
IPDSCASCfgTool (https://oneview.mitel.com/s/article/How-to-set-the-Log-Levels-for-IPDS-CAS-on-Shoreware-Servers)