Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Connection went offline and Maximum call stack size exceeded when joining >1000 rooms #282

Open
ravikiranj opened this issue Nov 3, 2016 · 5 comments
Open

Connection went offline and Maximum call stack size exceeded when joining >1000 rooms #282

ravikiranj opened this issue Nov 3, 2016 · 5 comments

Comments

@ravikiranj
Copy link

ravikiranj commented Nov 3, 2016 •
edited
Loading

Hi there,
I'm seeing the following errors when hubot tries to join >1000 rooms (@ALL) and I'm not sure how to debug it. I'd appreciate any insight into what's going on. The same setup works if numRooms < 500.

Also, if hubot is the right project for this issue and not hubot-hipchat, I can move it. I created it here since I'm using the hipchat adapter.

[Thu Nov 03 2016 11:10:04 GMT-0700 (PDT)] INFO Joining [email protected]
[Thu Nov 03 2016 11:10:04 GMT-0700 (PDT)] INFO Joining [email protected]


[Thu Nov 03 2016 11:10:10 GMT-0700 (PDT)] INFO Connection went offline
[Thu Nov 03 2016 11:10:10 GMT-0700 (PDT)] INFO Connection went offline
[Thu Nov 03 2016 11:10:10 GMT-0700 (PDT)] INFO Connection went offline
[Thu Nov 03 2016 11:10:10 GMT-0700 (PDT)] INFO Connection went offline
[Thu Nov 03 2016 11:10:10 GMT-0700 (PDT)] INFO Connection went offline
[Thu Nov 03 2016 11:10:10 GMT-0700 (PDT)] INFO Connection went offline
[Thu Nov 03 2016 11:10:10 GMT-0700 (PDT)] INFO Connection went offline
[Thu Nov 03 2016 11:10:10 GMT-0700 (PDT)] INFO Connection went offline
[Thu Nov 03 2016 11:10:10 GMT-0700 (PDT)] INFO Connection went offline
[Thu Nov 03 2016 11:10:10 GMT-0700 (PDT)] INFO Connection went offline
[Thu Nov 03 2016 11:10:10 GMT-0700 (PDT)] INFO Connection went offline
[Thu Nov 03 2016 11:10:10 GMT-0700 (PDT)] INFO Connection went offline
[Thu Nov 03 2016 11:10:10 GMT-0700 (PDT)] INFO Connection went offline
...


[Thu Nov 03 2016 11:10:10 GMT-0700 (PDT)] ERROR RangeError: Maximum call stack size exceeded
  at emitNone (events.js:65:18)
  at Client.emit (events.js:166:7)
  at Client.<anonymous> (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/lib/Client.js:157:12)
  at emitNone (events.js:72:20)
  at Connection.emit (events.js:166:7)
  at emitter.end.emitter.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/node_modules/node-xmpp-core/node_m
odules/reconnect-core/index.js:108:15)
  at Connection.end.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/node_modules/node-xmpp-core/lib/Connect
ion.js:124:5)
  at Client.Session.end (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/lib/session.js:202:21)
  at Connector.module.exports.Connector.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/src/connector.coffee:103:5, <js>:84:19)
  at Connector.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/src/connector.coffee:23:1, <js>:3:61)
  at Connector.onOffline (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/src/connector.coffee:503:3, <js>:455:17)
  at emitNone (events.js:67:13)
  at Client.emit (events.js:166:7)
  at Client.<anonymous> (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/lib/Client.js:157:12)
  at emitNone (events.js:72:20)
  at Connection.emit (events.js:166:7)
  at emitter.end.emitter.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/node_modules/node-xmpp-core/node_m
odules/reconnect-core/index.js:108:15)
  at Connection.end.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/node_modules/node-xmpp-core/lib/Connect
ion.js:124:5)
  at Client.Session.end (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/lib/session.js:202:21)
  at Connector.module.exports.Connector.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/src/connector.coffee:103:5, <js>:84:19)
  at Connector.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/src/connector.coffee:23:1, <js>:3:61)
  at Connector.onOffline (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/src/connector.coffee:503:3, <js>:455:17)
  at emitNone (events.js:67:13)
  at Client.emit (events.js:166:7)
  at Client.<anonymous> (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/lib/Client.js:157:12)
  at emitNone (events.js:72:20)
  at Connection.emit (events.js:166:7)
  at emitter.end.emitter.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/node_modules/node-xmpp-core/node_m
odules/reconnect-core/index.js:108:15)
  at Connection.end.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/node_modules/node-xmpp-core/lib/Connect
ion.js:124:5)

Package versions

"hubot": "^2.19.0"
"hubot-hipchat": "^2.12.0-6"
node: 4.4.7
npm: 2.15.8
@ravikiranj
Copy link
Author

I turned on debug and see below in the log

[Tue Nov 08 2016 11:24:52 GMT-0800 (PST)] INFO Joining [email protected]
[Tue Nov 08 2016 11:24:52 GMT-0800 (PST)] DEBUG  OUT > <presence to="[email protected]/HipChat Bot Plugin Tester"><x xmlns="http://jabber.org/protocol/muc"/><history xmlns="http://jabber.org/protocol/muc" maxstanzas="0"/></presence>
[Tue Nov 08 2016 11:24:52 GMT-0800 (PST)] INFO Joining [email protected]
[Tue Nov 08 2016 11:24:52 GMT-0800 (PST)] DEBUG  OUT > <presence to="[email protected]/HipChat Bot Plugin Tester"><x xmlns="http://jabber.org/protocol/muc"/><history xmlns="http://jabber.org/protocol/muc" maxstanzas="0"/></presence>
[Tue Nov 08 2016 11:24:52 GMT-0800 (PST)] DEBUG No listeners executed; falling back to catch-all
[Tue Nov 08 2016 11:24:52 GMT-0800 (PST)] DEBUG No listeners executed; falling back to catch-all
[Tue Nov 08 2016 11:24:52 GMT-0800 (PST)] DEBUG No listeners executed; falling back to catch-all
[Tue Nov 08 2016 11:24:52 GMT-0800 (PST)] DEBUG No listeners executed; falling back to catch-all
[Tue Nov 08 2016 11:24:52 GMT-0800 (PST)] DEBUG No listeners executed; falling back to catch-all
[Tue Nov 08 2016 11:24:52 GMT-0800 (PST)] DEBUG No listeners executed; falling back to catch-all
[Tue Nov 08 2016 11:24:52 GMT-0800 (PST)] DEBUG No listeners executed; falling back to catch-all
[Tue Nov 08 2016 11:24:52 GMT-0800 (PST)] DEBUG No listeners executed; falling back to catch-all
[Tue Nov 08 2016 11:24:52 GMT-0800 (PST)] DEBUG No listeners executed; falling back to catch-all
[Tue Nov 08 2016 11:24:52 GMT-0800 (PST)] DEBUG No listeners executed; falling back to catch-all

...

[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] DEBUG Disconnecting here
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] INFO Connection went offline
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] DEBUG Disconnecting here
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] INFO Connection went offline
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] DEBUG Disconnecting here
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] INFO Connection went offline
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] DEBUG Disconnecting here
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] INFO Connection went offline
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] DEBUG Disconnecting here
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] INFO Connection went offline
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] DEBUG Disconnecting here
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] INFO Connection went offline
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] DEBUG Disconnecting here
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] INFO Connection went offline
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] DEBUG Disconnecting here
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] INFO Connection went offline
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] DEBUG Disconnecting here
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] INFO Connection went offline
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] DEBUG Disconnecting here
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] INFO Connection went offline
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] DEBUG Disconnecting here
[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] INFO Connection went offline

... 

[Tue Nov 08 2016 11:24:58 GMT-0800 (PST)] ERROR RangeError: Maximum call stack size exceeded
  at Client.emit (events.js:117:44)
  at emitNone (events.js:72:20)
  at Connection.emit (events.js:166:7)
  at emitter.end.emitter.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/node_modules/node-xmpp-core/node_modules/reconnect-core/index.js:108:15)
  at Connection.end.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/node_modules/node-xmpp-core/lib/Connection.js:124:5)
  at Client.Session.end (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/lib/session.js:202:21)
  at Connector.module.exports.Connector.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/src/connector.coffee:103:5, <js>:84:19)
  at Connector.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/src/connector.coffee:23:1, <js>:3:61)
  at Connector.onOffline (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/src/connector.coffee:503:3, <js>:455:17)
  at emitNone (events.js:67:13)
  at Client.emit (events.js:166:7)
  at Client.<anonymous> (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/lib/Client.js:157:12)
  at emitNone (events.js:72:20)
  at Connection.emit (events.js:166:7)
  at emitter.end.emitter.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/node_modules/node-xmpp-core/node_modules/reconnect-core/index.js:108:15)
  at Connection.end.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/node_modules/node-xmpp-core/lib/Connection.js:124:5)
  at Client.Session.end (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/lib/session.js:202:21)
  at Connector.module.exports.Connector.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/src/connector.coffee:103:5, <js>:84:19)
  at Connector.disconnect (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/src/connector.coffee:23:1, <js>:3:61)
  at Connector.onOffline (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/src/connector.coffee:503:3, <js>:455:17)
  at emitNone (events.js:67:13)
  at Client.emit (events.js:166:7)
  at Client.<anonymous> (/home/rjanardhana/ta-hubot/node_modules/hubot-hipchat/node_modules/node-xmpp-client/lib/Client.js:157:12)
  at emitNone (events.js:72:20)
  at Connection.emit (events.js:166:7)

@fehwalker
Copy link

We're seeing the same with ~1100 rooms -- I suspect it may be related to rate limiting (https://developer.atlassian.com/hipchat/guide/hipchat-rest-api/api-rate-limits)

@jbrodley
Copy link

jbrodley commented Dec 31, 2016 •
edited
Loading

I am seeing the same issue on the latest version of hipchat server 2.1.3. Are you on the hosted version of Hipchat or private server? @fehwalker @ravikiranj

@jbrodley
Copy link

jbrodley commented Jan 2, 2017 •
edited
Loading

I ended up opening a ticket with Hipchat support. They were able to look at the logs and realize my issue is that in 2.0.7 and newer server versions XMPP clients are disabled by default. As the Hipchat connector uses the Node JS XMPP Client this is why it started failing after we upgraded. After enabling the XMPP protocol I was able to resolve my issue.

Atlassian has also written up a KB article on this issue. https://confluence.atlassian.com/hipchatkb/hubot-stopped-working-on-hipchat-server-v2-0-7-and-later-867181545.html

@ravikiranj
Copy link
Author

ravikiranj commented Jan 6, 2017 •
edited
Loading

@jbrodley Thank you very much for getting to the bottom of this. Yes, we are on custom hosted hipchat server of 2.0 build 1.4.1. I'll check if that article fixes this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@fehwalker @ravikiranj @jbrodley