Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to start a worker using process monitor s6-overlay #109

Open
PAXANDDOS opened this issue Feb 14, 2023 · 2 comments
Open

Unable to start a worker using process monitor s6-overlay #109

PAXANDDOS opened this issue Feb 14, 2023 · 2 comments

Comments

@PAXANDDOS
Copy link
Contributor

Hello!
I'm having quite an issue with starting a worker using s6-overlay. It's a process monitor almost like supervisor but created specifically to use inside docker containers.
The worker simply won't start: the queue is not processed and the worker is not getting registered in log files.

/etc/s6-overlay/s6-rc.d/resque/run

#!/command/execlineb -S1

/var/www/app/vendor/bin/resque worker:start --config=/var/www/app/config/queue.yml

Although it does work if ran manually inside the container:

/var/www/app$: vendor/bin/resque worker:start --config=/var/www/app/config/queue.yml
###### or
/var/www/app$: /var/www/app/vendor/bin/resque worker:start --config=/var/www/app/config/queue.yml
###### or
/var/www/app$: /etc/s6-overlay/s6-rc.d/resque/run

I've already made an issue in s6-overlay, please see it for more info: just-containers/s6-overlay#517

@PAXANDDOS
Copy link
Contributor Author

PAXANDDOS commented Feb 16, 2023

The problem was identified by the developer of s6-overlay here: just-containers/s6-overlay#517 (comment)

It appears that worker somehow invokes the stty command, making it terminal-dependant thus blocking execution using process monitors (may be a problem not for s6-overlay only but also for supervisord)

I couldn't find anything stty-related in the code, but there were a bunch of results regarding stty in symfony/console repository, maybe php-resque somehow enables stty through the symfony/console?

@PAXANDDOS
Copy link
Contributor Author

Finally found a workaround!
So since my application also uses Symfony Console I can bring commands to the application level and "wrap" resque commands.
Unfortunately, the library has no constructive Console Application (it's set within the executable file) thus the process is a bit more complicated.

Here's how my "wrapper" command looks like:

#[AsCommand(name: 'queue:worker:start', description: 'Poll for jobs on specified queues and execute job when found')]
final class StartCommand extends Command
{
    protected function configure(): void
    {
        $this->addOption('queue', 'Q', InputOption::VALUE_OPTIONAL, 'The queue(s) to listen on, comma separated.', '*')
            ->addOption('blocking', 'b', InputOption::VALUE_OPTIONAL, 'Use Redis pop blocking or time interval.', true)
            ->addOption('interval', 'i', InputOption::VALUE_OPTIONAL, 'Blocking timeout/interval speed in seconds.', 10)
            ->addOption('timeout', 't', InputOption::VALUE_OPTIONAL, 'Seconds a job may run before timing out.', 60)
            ->addOption('memory', 'm', InputOption::VALUE_OPTIONAL, 'The memory limit in megabytes.', 128)
            ->addOption('pid', 'P', InputOption::VALUE_OPTIONAL, 'Absolute path to PID file, must be writeable by worker.');
    }

    protected function execute(InputInterface $input, OutputInterface $output): int
    {
        $command = new \Resque\Commands\Worker\Start();

        // Clone parameters to new input
        $input = $this->cloneResqueInput($input, $command->getDefinition());

        return $command->run($input, $output);
    }
}

So here configure method is literally copied from the Start command. Then in execute I instantiate the command I want to execute and run cloneResqueInput to make an independent input object which would not be tied to a terminal.

And the cloneResqueInput:

    protected function cloneResqueInput(InputInterface $input, InputDefinition $definition): Input
    {
        $cloned = new ArrayInput([], $definition);
        foreach ($input->getArguments() as $key => $value) {
            if ($key === 'command') {
                continue;
            }
            $cloned->setArgument($key, $value);
        }
        foreach ($input->getOptions() as $key => $value) {
            switch ($key) {
                case 'command':
                    break;
                case 'help':
                    break;
                case 'quiet':
                    break;
                case 'verbose':
                    break;
                case 'version':
                    break;
                case 'ansi':
                    break;
                case 'no-interaction':
                    break;
                default:
                    $cloned->setOption($key, $value);
            }
        }
        $cloned->setOption('config', CONFIG.'queue.yml');
        $cloned->setInteractive(false);

        return $cloned;
    }

Here I create a new Input based on the commands' InputDefinition, and transfer all arguments and options my Input has but without pre-defined Symfony parameters. And since I have the possibility I add my custom config and disable the interactive mode.

So with this "wrapper" I am finally able to start a worker via a process manager. Though this probably removes the ability for workers to listen for signals, but that's alright for me since I don't use them much. Also, I think this entire issue may be because of these signals.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant