Repository navigation
Open
Description
Description
We are using Jigasi with Asterisk (via udp on 127.0.0.1) to allow SIP-based Dialins for our Jitsi Conferences.
This works fine as long as Asterisk is already running. If Asterisk is not reachable for some minutes Jigasi wont try to REGISTER anymore and has to be restarted.
Current behavior
Jun 18 15:46:12 meet-sipgw conmon[2405674]: 2024-06-18 13:46:12.317 INFO: [323] net.java.sip.communicator.impl.protocol.sip.SipLogger.logInfo: Info from the JAIN-SIP stack: Setting SIPMessage peerPacketSource to: /127.0.0.1:5060
Jun 18 15:46:12 meet-sipgw podman[2405647]: 2024-06-18 13:46:12.317 INFO: [323] net.java.sip.communicator.impl.protocol.sip.SipLogger.logInfo: Info from the JAIN-SIP stack: Setting SIPMessage peerPacketSource to: /127.0.0.1:5060
# asterisk stopped...
Jun 18 15:52:22 meet-sipgw conmon[2405674]: 2024-06-18 13:52:22.880 INFO: [34] SipGateway.registrationStateChanged#120: REG STATE CHANGE ProtocolProviderServiceSipImpl(SIP:jigasi@localhost) -> RegistrationStateChangeEvent[ oldState=Registered; newState=RegistrationState=Connection Failed; userRequest=false; reasonCode=-1; reason=A timeout occurred while trying to connect to the server.]
Jun 18 15:52:22 meet-sipgw podman[2405647]: 2024-06-18 13:52:22.880 INFO: [34] SipGateway.registrationStateChanged#120: REG STATE CHANGE ProtocolProviderServiceSipImpl(SIP:jigasi@localhost) -> RegistrationStateChangeEvent[ oldState=Registered; newState=RegistrationState=Connection Failed; userRequest=false; reasonCode=-1; reason=A timeout occurred while trying to connect to the server.]
Jun 18 15:52:22 meet-sipgw conmon[2405674]: 2024-06-18 13:52:22.891 INFO: [34] net.java.sip.communicator.plugin.reconnectplugin.PPReconnectWrapper.scheduleReconnectIfNeeded: Reconnect ProtocolProviderServiceSipImpl(SIP:jigasi@localhost) after 5000 ms.
Jun 18 15:52:22 meet-sipgw podman[2405647]: 2024-06-18 13:52:22.891 INFO: [34] net.java.sip.communicator.plugin.reconnectplugin.PPReconnectWrapper.scheduleReconnectIfNeeded: Reconnect ProtocolProviderServiceSipImpl(SIP:jigasi@localhost) after 5000 ms.
Jun 18 15:52:22 meet-sipgw conmon[2405674]: 2024-06-18 13:52:22.903 INFO: [34] net.java.sip.communicator.impl.protocol.sip.SipLogger.logInfo: Info from the JAIN-SIP stack: the sip stack timer gov.nist.javax.sip.stack.timers.DefaultSipTimer has been stopped
Jun 18 15:52:22 meet-sipgw podman[2405647]: 2024-06-18 13:52:22.903 INFO: [34] net.java.sip.communicator.impl.protocol.sip.SipLogger.logInfo: Info from the JAIN-SIP stack: the sip stack timer gov.nist.javax.sip.stack.timers.DefaultSipTimer has been stopped
Jun 18 15:52:23 meet-sipgw conmon[2405674]: 2024-06-18 13:52:23.905 INFO: [34] net.java.sip.communicator.impl.protocol.sip.SipLogger.logInfo: Info from the JAIN-SIP stack: the sip stack timer gov.nist.javax.sip.stack.timers.DefaultSipTimer has been stopped
Jun 18 15:52:23 meet-sipgw podman[2405647]: 2024-06-18 13:52:23.905 INFO: [34] net.java.sip.communicator.impl.protocol.sip.SipLogger.logInfo: Info from the JAIN-SIP stack: the sip stack timer gov.nist.javax.sip.stack.timers.DefaultSipTimer has been stopped
Jun 18 15:52:27 meet-sipgw conmon[2405674]: 2024-06-18 13:52:27.901 INFO: [37] net.java.sip.communicator.plugin.reconnectplugin.PPReconnectWrapper$ReconnectTask.run: Start reconnecting ProtocolProviderServiceSipImpl(SIP:jigasi@localhost)
Jun 18 15:52:27 meet-sipgw podman[2405647]: 2024-06-18 13:52:27.901 INFO: [37] net.java.sip.communicator.plugin.reconnectplugin.PPReconnectWrapper$ReconnectTask.run: Start reconnecting ProtocolProviderServiceSipImpl(SIP:jigasi@localhost)
Jun 18 15:52:27 meet-sipgw conmon[2405674]: 2024-06-18 13:52:27.910 INFO: [37] net.java.sip.communicator.impl.protocol.sip.SipLogger.logInfo: Info from the JAIN-SIP stack: the sip stack timer gov.nist.javax.sip.stack.timers.DefaultSipTimer has been started
Jun 18 15:52:27 meet-sipgw podman[2405647]: 2024-06-18 13:52:27.910 INFO: [37] net.java.sip.communicator.impl.protocol.sip.SipLogger.logInfo: Info from the JAIN-SIP stack: the sip stack timer gov.nist.javax.sip.stack.timers.DefaultSipTimer has been started
Jun 18 15:52:27 meet-sipgw conmon[2405674]: 2024-06-18 13:52:27.925 INFO: [37] SipGateway.registrationStateChanged#120: REG STATE CHANGE ProtocolProviderServiceSipImpl(SIP:jigasi@localhost) -> RegistrationStateChangeEvent[ oldState=Unregistered; newState=RegistrationState=Registering; userRequest=false; reasonCode=-1; reason=null]
Jun 18 15:52:27 meet-sipgw podman[2405647]: 2024-06-18 13:52:27.925 INFO: [37] SipGateway.registrationStateChanged#120: REG STATE CHANGE ProtocolProviderServiceSipImpl(SIP:jigasi@localhost) -> RegistrationStateChangeEvent[ oldState=Unregistered; newState=RegistrationState=Registering; userRequest=false; reasonCode=-1; reason=null]
Jun 18 15:53:33 meet-sipgw conmon[2405674]: 2024-06-18 13:53:33.901 INFO: [325] SipGateway.registrationStateChanged#120: REG STATE CHANGE ProtocolProviderServiceSipImpl(SIP:jigasi@localhost) -> RegistrationStateChangeEvent[ oldState=Registering; newState=RegistrationState=Connection Failed; userRequest=false; reasonCode=-1; reason=A timeout occurred while trying to connect to the server.]
Jun 18 15:53:33 meet-sipgw podman[2405647]: 2024-06-18 13:53:33.901 INFO: [325] SipGateway.registrationStateChanged#120: REG STATE CHANGE ProtocolProviderServiceSipImpl(SIP:jigasi@localhost) -> RegistrationStateChangeEvent[ oldState=Registering; newState=RegistrationState=Connection Failed; userRequest=false; reasonCode=-1; reason=A timeout occurred while trying to connect to the server.]
Jun 18 15:53:33 meet-sipgw conmon[2405674]: 2024-06-18 13:53:33.911 INFO: [325] net.java.sip.communicator.impl.protocol.sip.SipLogger.logInfo: Info from the JAIN-SIP stack: the sip stack timer gov.nist.javax.sip.stack.timers.DefaultSipTimer has been stopped
Jun 18 15:53:33 meet-sipgw podman[2405647]: 2024-06-18 13:53:33.911 INFO: [325] net.java.sip.communicator.impl.protocol.sip.SipLogger.logInfo: Info from the JAIN-SIP stack: the sip stack timer gov.nist.javax.sip.stack.timers.DefaultSipTimer has been stopped
Jun 18 15:53:34 meet-sipgw conmon[2405674]: 2024-06-18 13:53:34.912 INFO: [325] net.java.sip.communicator.impl.protocol.sip.SipLogger.logInfo: Info from the JAIN-SIP stack: the sip stack timer gov.nist.javax.sip.stack.timers.DefaultSipTimer has been stopped
Jun 18 15:53:34 meet-sipgw podman[2405647]: 2024-06-18 13:53:34.912 INFO: [325] net.java.sip.communicator.impl.protocol.sip.SipLogger.logInfo: Info from the JAIN-SIP stack: the sip stack timer gov.nist.javax.sip.stack.timers.DefaultSipTimer has been stopped
# starting asterisk again
Expected Behavior
It would be desirable to have Jigasi try to re-establish the SIP Registration and not giving up finally.
Possible Solution
Maybe it is possible to configure the SIP-Timers behaviour to not time out?!
Steps to reproduce
- register Jigasi to a SIP-Server
- shut down the SIP-Server
- wait ~ 5 Minutes
- restart SIP-Server
- watch Jigasi not register anymore
Environment details
We are using Jigasi1.1.329-g04ad08fLinux respectively Container Image: jitsi/jigasi:stable-9457-2 with Asterisk 18.10.0~dfsg+~cs6.10.40431411-2 (should not matter).
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Milestone
Relationships
Development
Issue actions
Open in GitHub Copilot app
Activity
damencho commentedon Jun 18, 2024
According to the logs after starting asterisk jigasi tried registering but timed out:
denzs commentedon Jun 19, 2024
Hi @damencho, yeah thats correct. The thing is, this also happens if Jigasi is running fine for weeks and the upstream system then goes unavail for some minutes (due to maintenance from another team)..
I have an idea about SIP-Timers and that this behaviour may be correct regarding RFCs, but from a production point of view the SIP-Bridge to Jitsi just breaks forever (until restarted) if the upstream SIP-Service is maintained for some minutes.
A Setting/Flag to have Jigasi recovery automatically would be very helpful in this case.
damencho commentedon Jun 19, 2024
I don't understand the problem i terms of jigasi. After the service is back jigasi retries again and is not succeeding, not sure how to recover from this.
We are handling this by checking the health of jigasi, when it becomes an healthy a new i stance is scaled up and the old one goes into graceful shutdown.
denzs commentedon Jun 19, 2024
Yeah, that would be my expectation in this case too :)
Example:
Is there a way to check Jigasis Health via http like for the JVBS, or is this only possible via XMPP? :)
damencho commentedon Jun 19, 2024
Yes, there is a health endpoint accessible via http.
denzs commentedon Jun 19, 2024
Do you have any idea, if it is possible to configure Jigasi respecitvely the SIP-Timer Settings in a way that Jigasi does not stop trying to Re-Register? From my point of view it would make sense to be able to configure this behaviour..
And thanks for the hint regarding the http health check, after testing and looking in the source i think i missed configuring the sipChecker to get a good status :)
damencho commentedon Jun 19, 2024
Not sure what you are talking about. There is a reconnect plugin that controls the reconnecting logic for protocol providers and there is no logic of stopping reconnect, if I remember correctly. You can enable debug logging and when you see it stops create a heapdump to try debugging why it may have stopped reconnecting. We haven't seen that in practice, but if there is an incident with the sip provider it is short or we spin up new instances.
https://github.com/jitsi/jitsi/tree/master/modules/plugin/reconnect
denzs commentedon Jun 19, 2024
Thanks for the pointer!
To me it looks as the reconnect plugin doesnt do anything when the problem occurs:
The REGISTER from 13:14:01 is the last try of sending SIP Messages and after several messages like
there is no more SIP from Jigasi..
But anyhow, thanks again for the hint regarding the health endpoint!
I think i can configure podman to use that as an active healthcheck and have the container restarted automatically in case of failures :)
damencho commentedon Jun 19, 2024
You see your provider is in reconnectEnabledProviders, so if connection fails it should be detected by the reconnect plugin and continue attempts later.
denzs commentedon Jun 19, 2024
Right now (14:40) the only (repeating) log messages are:
I still have tcpdump running and the last SIP Message is from 13:14:01 - as you can see in the logs from my last post..
Is there any special switch/configparameter neccessary to have the reconnectplugin to do its magic? :)
As i mentioned, i am using the Jigasi docker image.. maybe the config is missing something?
damencho commentedon Jun 19, 2024
Nope, it should be running always. Maybe there is some bug or something. A heapdump will be helpful, but I'm not sure I will have time to look at it these days.
damencho commentedon Jun 19, 2024
Better use the healthcheck for now
denzs commentedon Jun 20, 2024
I generated a heapdump with
jmap -dump:format=b,file=heap.bin $pid- i am not sure if it helpful if i attach 40MB here.. is there anythin i can provide from the heapdump to provide helpful information?damencho commentedon Jun 20, 2024
Do not attach it, as it holds the jigasi memory and configuration.
Well, it needs somone to check what is going on in the logs. And see in memory whether ReconnectPluginActivator has the sip provider in reconnectEnabledProviders and if it is there, why there is no scheduled task for it - the internal state for its PPReconnectWrapper. https://github.com/jitsi/jitsi/blob/master/modules/plugin/reconnect/src/main/java/net/java/sip/communicator/plugin/reconnectplugin/PPReconnectWrapper.java