Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gearmand 0.33 still required to run mod_gearman? #146

Closed
dirtyren opened this issue Jul 22, 2019 · 7 comments
Closed

gearmand 0.33 still required to run mod_gearman? #146

dirtyren opened this issue Jul 22, 2019 · 7 comments

Comments

@dirtyren
Copy link
Contributor

Hello,

sometime ago the recommended gearmand version was a 0.33 patched that is provided on the console labs repos.
Should we still use that version or could we run mod_gearman with the epel centos7 provided version 1.1.12?
I am running a test machine since Friday and it is running without problems.

Tks.

@sni
Copy link
Owner

sni commented Jul 22, 2019

We are still running 0.33 on all our clients. Haven't found the time to test new releases yet. So please let me know if you find anything.

@dirtyren
Copy link
Contributor Author

Ok @sni , any strange behaviors we find I'll let you know, tks.
So far very stable.

@gzalo
Copy link

gzalo commented Aug 5, 2019

Ok @sni , any strange behaviors we find I'll let you know, tks.
So far very stable.

Are you using Nagios or Naemon? we found that with [lib]gearman 1.12 we were having memory leak problems, but after downgrading to 0.33 they went away.
If you have a small amount of services or restart nagios periodically it does not matter, but you should monitor its memory just in case.

@dirtyren
Copy link
Contributor Author

dirtyren commented Aug 5, 2019

Hey,

Are you using Nagios or Naemon? we found that with [lib]gearman 1.12 we were having memory leak problems, but after downgrading to 0.33 they went away.

naemon, livestatus, mod_gearman and our own broker that uses gearman queues, so far it is working, but the test box has only 86 hosts and 543 services.

If you have a small amount of services or restart nagios periodically it does not matter, but you should monitor its memory just in case.
I am going to monitor memory consumption, tks for the tip and soon I will test it all with 25k services.

@dirtyren
Copy link
Contributor Author

Just to bring this thread up a bit.
We are migrating from CentOS 6 to CentOS 8 and trying to use gearmand 1.1.19.

gearmand after one day is at 97% CPU, using naemon, livestatus and mod_gearman-3.3.,

image

gearman_top wont connect
image

[root@opdev3 ~]# netstat -anp | grep gearmand | grep "ESTABLISHED" | wc -l
986
I am stressing a little bit running checks each 5s to test all the new software on the new OS.

We used to have gearmand 0.33 at 100% CPU when we had a worker in PHP that would loop for some reason connecting and disconnecting to gearmand. But that is not the case here.
Looking on the gearmand github page there are some issues open regarding high CPU usage of gearmand.

Any ideas I could try on this?

[]s.

@sni
Copy link
Owner

sni commented Nov 25, 2020

We upgraded gearman 0.33 to 1.x 12 months ago and only had a single issue so far (ConSol-Monitoring/omd#107). The reason was a gearman worker using timeout. PNP in that case.

@dirtyren
Copy link
Contributor Author

That as caused by a process that was opening connections to gearmand exhausting its available connections. Nothing to do with gearmand, but a behavior with PHP and the gearmand API in objects. It would not disconnect its connection. Related to this comment here
gearman/gearmand#241 (comment)

Tks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants