Server.stop() does not kill the java process in Windows #8

ghost · 2013-02-05T21:22:08Z

When the server is started it does so using subprocess.Popen('browsermob-proxy.bat')

The result process's pid on windows is the cmd.exe process that controls the bat file, but I don't think Windows is linking the java.exe which the bat file spawns.

Calling server.stop() kills the cmd.exe, but a java.exe is left hanging around at about 35MB.

This may be a problem with windows not killing child process correctly and it is apparently not possible to get the java pid in the bat file without some actual windows-style grep for java.exe.

fpoolev · 2014-07-15T20:58:12Z

This problem still exists on Windows 7 as described above.

DarthOpto · 2014-08-29T20:59:41Z

try proxy.close() prior to the server.stop() that helped me, although if the test fails mid way through it still doesn't close the process.

dshuga · 2015-08-10T16:25:52Z

Calling proxy.close() prior to server.stop() didn't resolve this issue for me on Windows 7.

cbdelavenne · 2015-12-16T18:50:27Z

Having the same issue on Mac OS X and Windows Server 2012.

This is my workaround:

import psutil

def terminate_browsermob_processes(self):
    self._browsermob_proxy.close()
    self._browsermob_server.stop()

    # Find BrowserMob-Proxy processes that may still be alive and kill them
    for process in psutil.process_iter():
        try:
            process_info = process.as_dict(attrs=['name', 'cmdline'])
            if process_info.get('name') in ('java', 'java.exe'):
                for cmd_info in process_info.get('cmdline'):
                    if cmd_info == '-Dapp.name=browsermob-proxy':
                        process.kill()
        except psutil.NoSuchProcess:
            pass

ghost · 2017-02-02T19:47:59Z

On Debian 8 (not Windows)

Python: Selenium v3.0.2, python3.4, browsermobproxy v0.5.0
JAVA: BrowserMob Proxy version 2.1.4

When run, it will create these two applicable lines in the ps aux:
/bin/sh /usr/local/browsermobproxy/bin/browsermob-proxy --port=8080
and:
java -Dapp.name=browsermob-proxy -Dbasedir=/usr/local/browsermobproxy -jar /usr/local/browsermobproxy/lib/browsermob-dist-2.1.4.jar --port=8080

But all of the various suggestions (above) to stop the BMP Server always leave this in the ps aux output:

java -Dapp.name=browsermob-proxy -Dbasedir=/usr/local/browsermobproxy -jar /usr/local/browsermobproxy/lib/browsermob-dist-2.1.4.jar --port=8080

I need to get rid of this proc: How? Suggestions?
Something akin to @cbdelavenne 's solution is not possible, it would destroy the ongoing main process.

The shutdown code:

        if(self.route == 'proxify'):
            if(debug):
                logging.info(Fore.YELLOW +'Turning off the browsermob-proxy server')
            #self.proxy.close()
            self.proxy_server.stop()
        if(debug):
            logging.info(Fore.YELLOW +'Turning off the Selenium WebDriver')
        self.driver.quit()
        if(debug):
            logging.info(Fore.YELLOW +'Turning off the X-Window Frameless Visual Buffer (headless browsing)')
        self.display.stop()

Produces this debugging output:

INFO:root:Turning off the browsermob-proxy server
INFO:root:Turning off the Selenium WebDriver
DEBUG:selenium.webdriver.remote.remote_connection:DELETE http://127.0.0.1:43014/session/8e48269a5c3ef89499e896e7bdb59449 {"sessionId": "8e48269a5c3ef89499e896e7bdb59449"}
DEBUG:selenium.webdriver.remote.remote_connection:Finished Request
INFO:root:Turning off the X-Window Frameless Visual Buffer (headless browsing)
DEBUG:pyvirtualdisplay.abstractdisplay:unset DISPLAY
DEBUG:easyprocess:stopping process (pid=29711 cmd="['Xvfb', '-br', '-nolisten', 'tcp', '-screen', '0', '800x600x24', ':1001']")
DEBUG:easyprocess:process is active -> sending SIGTERM
DEBUG:easyprocess:process has ended
DEBUG:easyprocess:return code=0
DEBUG:easyprocess:stdout=
DEBUG:easyprocess:stderr=

server.log:

Running BrowserMob Proxy using LittleProxy implementation. To revert to the legacy implementation, run the proxy with the command-line option '--use-littleproxy false'.
[INFO  2017-02-02T19:10:15,742 net.lightbody.bmp.proxy.Main] (main) Starting BrowserMob Proxy version 2.1.4 
[INFO  2017-02-02T19:10:15,784 org.eclipse.jetty.util.log] (main) jetty-7.x.y-SNAPSHOT 
[INFO  2017-02-02T19:10:15,852 org.eclipse.jetty.util.log] (main) started o.e.j.s.ServletContextHandler{/,null} 
[INFO  2017-02-02T19:10:16,140 org.eclipse.jetty.util.log] (main) Started [email protected]:8080 
[INFO  2017-02-02T19:10:17,643 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Starting proxy at address: 0.0.0.0/0.0.0.0:8081 
[INFO  2017-02-02T19:10:17,685 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Proxy listening with TCP transport 
[INFO  2017-02-02T19:10:17,831 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Proxy started at address: /0:0:0:0:0:0:0:0:8081 
[INFO  2017-02-02T19:10:49,754 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Shutting down proxy server gracefully 
[INFO  2017-02-02T19:10:49,755 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Closing all channels (graceful) 
[INFO  2017-02-02T19:10:49,782 org.littleshoot.proxy.impl.ServerGroup] (qtp359742806-16) Shutting down server group event loops (graceful) 
[INFO  2017-02-02T19:10:51,991 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Done shutting down proxy server

bmp.log:

[INFO  2017-02-02T19:10:15,742 net.lightbody.bmp.proxy.Main] (main) Starting BrowserMob Proxy version 2.1.4 
[INFO  2017-02-02T19:10:15,784 org.eclipse.jetty.util.log] (main) jetty-7.x.y-SNAPSHOT 
[INFO  2017-02-02T19:10:15,852 org.eclipse.jetty.util.log] (main) started o.e.j.s.ServletContextHandler{/,null} 
[INFO  2017-02-02T19:10:16,140 org.eclipse.jetty.util.log] (main) Started [email protected]:8080 
[INFO  2017-02-02T19:10:17,643 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Starting proxy at address: 0.0.0.0/0.0.0.0:8081 
[INFO  2017-02-02T19:10:17,685 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Proxy listening with TCP transport 
[INFO  2017-02-02T19:10:17,831 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Proxy started at address: /0:0:0:0:0:0:0:0:8081 
[INFO  2017-02-02T19:10:49,754 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Shutting down proxy server gracefully 
[INFO  2017-02-02T19:10:49,755 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Closing all channels (graceful) 
[INFO  2017-02-02T19:10:49,782 org.littleshoot.proxy.impl.ServerGroup] (qtp359742806-16) Shutting down server group event loops (graceful) 
[INFO  2017-02-02T19:10:51,991 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp359742806-16) Done shutting down proxy server

andreyrusanov · 2017-06-12T15:28:04Z

Any updates on it? I have the same problem(reproduced on CentOS7 and Ubuntu 16.04)

andreyrusanov · 2017-06-12T18:06:44Z

It seems I found the issue;
It seems that Browsermob process should be started as a group of processes and then has to be terminated as a group. I will try to make a pull request.

andreyrusanov · 2017-06-12T18:36:46Z

PR is made(for Linux only since I don't have Windows machine to test a fix for Windows.

However, I have on assumption on possible fix(see PR description)

michelebenolli · 2019-07-26T14:44:32Z

Why is this marked as closed? The issue is currently present under Windows.

darukavishnikov · 2019-08-16T14:20:06Z

+1
still have the same issue under python

server.stop() ..\..\..\..\..\browsermobproxy\server.py:133: in stop if self.process.poll() is not None: E AttributeError: 'NoneType' object has no attribute 'poll'

Could it be windows permission case?

dasshark · 2020-08-24T14:10:33Z

This can be solved by modifying the stop() method and adding a windows handler like this:

import psutil, signal

parent_process = psutil.Process(self.process.pid)
child_process = parent_process.children(recursive=True)
for child in child_process:
  child.send_signal(signal.SIGTERM)
try:
  parent_process.send_signal(signal.SIGTERM)
except psutil.NoSuchProcess:
  pass

alex-ber · 2021-02-18T19:41:42Z

I've investigate this issue deeply. What happens, than when you start browsermobproxy server (I will reffer to it as bmp_daemon, see https://medium.com/@arkadyt/setting-up-programmatic-aws-access-for-mfa-protected-federated-identities-3be22bcecf4b for details) it uses subprocess.Popen with command that contains bat-file. So, what happen on Windows Python process start's cmd that executes bat-file that starts Java process. When you're calling close() function on bmp_daemon, it successfully kills cmd process. On Windows (and on Mac as per #76) OS doesn't kills grand process automatically (I don't know what's happen on Linux, but I suspect it works their), that is close() function kills cmd process, but on Windows (and Mac) the Java process survives.

Inspired by @dasshark response above, I have witted the following helper function that I'm using calling instead of bmp_daemon.stop().

import psutil, signal
from contextlib import suppress

def closeBmpDaemon(bmp_daemon):
    if bmp_daemon is not None and bmp_daemon.process is not None:
        childs_process = None
        try:
            cmd_process = psutil.Process(bmp_daemon.process.pid)
            childs_process = cmd_process.children(recursive=True)
            childs_process = [*childs_process, cmd_process]

            bmp_daemon.stop()
        finally:
            for child in childs_process:
                # we can't accidentally kill newly created process
                # we can kill only the process we have cached earlier
                # if process was already finished we will get NoSuchProcess
                # that we're just suppressing
                with suppress(psutil.NoSuchProcess):
                    child.send_signal(signal.SIGTERM)

EDIT I have added some comments and make some minor changes.

alex-ber · 2021-04-18T19:29:00Z

I have extracted this function (and more!) above into separate package.
Here https://github.com/alex-ber/selenium-support/blob/main/alexber/seleniumsupport/_impl.py#L46 is source-code.
Here https://pypi.org/project/selenium-support/ description of this package at PyPI
Here https://alex-ber.medium.com/selenium-support-19330843c63a is detail description of this and another useful utility function for Selenium.

KB00100100 · 2021-12-02T06:50:42Z

For Ubuntu 16.04, my personal solution:

add import signal
(line 112) small change

self.process = subprocess.Popen(self.command,
                                        close_fds=True,
                                        preexec_fn=os.setsid,
                                        stdout=self.log_file,
                                        stderr=subprocess.STDOUT)

(line 141) replace self.process.kill() with os.killpg(self.process.pid,signal.SIGUSR1)

alex-ber · 2021-12-02T10:01:16Z

@KB00100100 hopefully my code is cross-platform. In about 2 month I'm going to test my code on Ubuntu. If my code will fail, I will try to use your code snippet. Thanks for sharing.

andreyrusanov mentioned this issue Jun 12, 2017

Run and kill as process group #71

Merged

andreyrusanov mentioned this issue Feb 8, 2018

Java Processes not getting killed in Mac #76

Open

AutomatedTester closed this as completed in #71 May 23, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Server.stop() does not kill the java process in Windows #8

Server.stop() does not kill the java process in Windows #8

ghost commented Feb 5, 2013

fpoolev commented Jul 15, 2014

DarthOpto commented Aug 29, 2014

dshuga commented Aug 10, 2015

cbdelavenne commented Dec 16, 2015

ghost commented Feb 2, 2017 •

edited by ghost

Loading

andreyrusanov commented Jun 12, 2017 •

edited

Loading

andreyrusanov commented Jun 12, 2017

andreyrusanov commented Jun 12, 2017

michelebenolli commented Jul 26, 2019

darukavishnikov commented Aug 16, 2019

dasshark commented Aug 24, 2020 •

edited

Loading

alex-ber commented Feb 18, 2021 •

edited

Loading

alex-ber commented Apr 18, 2021

KB00100100 commented Dec 2, 2021 •

edited

Loading

alex-ber commented Dec 2, 2021

Server.stop() does not kill the java process in Windows #8

Server.stop() does not kill the java process in Windows #8

Comments

ghost commented Feb 5, 2013

fpoolev commented Jul 15, 2014

DarthOpto commented Aug 29, 2014

dshuga commented Aug 10, 2015

cbdelavenne commented Dec 16, 2015

ghost commented Feb 2, 2017 • edited by ghost Loading

On Debian 8 (not Windows)

andreyrusanov commented Jun 12, 2017 • edited Loading

andreyrusanov commented Jun 12, 2017

andreyrusanov commented Jun 12, 2017

michelebenolli commented Jul 26, 2019

darukavishnikov commented Aug 16, 2019

dasshark commented Aug 24, 2020 • edited Loading

alex-ber commented Feb 18, 2021 • edited Loading

alex-ber commented Apr 18, 2021

KB00100100 commented Dec 2, 2021 • edited Loading

alex-ber commented Dec 2, 2021

ghost commented Feb 2, 2017 •

edited by ghost

Loading

andreyrusanov commented Jun 12, 2017 •

edited

Loading

dasshark commented Aug 24, 2020 •

edited

Loading

alex-ber commented Feb 18, 2021 •

edited

Loading

KB00100100 commented Dec 2, 2021 •

edited

Loading