Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for running in worker threads #32

Open
OmgImAlexis opened this issue Apr 27, 2020 · 2 comments
Open

Support for running in worker threads #32

OmgImAlexis opened this issue Apr 27, 2020 · 2 comments

Comments

@OmgImAlexis
Copy link

With the default options the following doesn't work.

const python = pythonBridge();

Instead we get the following error. If I set stdio to ['ignore', 'pipe', 'pipe'] it works without an issue. It'd be nice to either detect the worker thread and use different defaults or catch the error and show a nicer message.

TypeError [ERR_INVALID_OPT_VALUE]: The value "WritableWorkerStdio {
  _writableState: WritableState {
    objectMode: false,
    highWaterMark: 16384,
    finalCalled: false,
    needDrain: false,
    ending: false,
    ended: false,
    finished: false,
    destroyed: false,
    decodeStrings: false,
    defaultEncoding: 'utf8',
    length: 97,
    writing: true,
    corked: 0,
    sync: false,
    bufferProcessing: false,
    onwrite: [Function: bound onwrite],
    writecb: [Function],
    writelen: 97,
    bufferedRequest: null,
    lastBufferedRequest: null,
    pendingcb: 1,
    prefinished: false,
    errorEmitted: false,
    emitClose: true,
    autoDestroy: false,
    bufferedRequestCount: 0,
    corkedRequestsFree: {
      next: null,
      entry: null,
      finish: [Function: bound onCorkedFinish]
    }
  },
  writable: true,
  _events: [Object: null prototype] {},
  _eventsCount: 0,
  _maxListeners: undefined,
  [Symbol(kPort)]: MessagePort {
    active: true,
    refed: true,
    _events: [Object: null prototype] {
      newListener: [Function],
      removeListener: [Function],
      message: [Function]
    },
    _eventsCount: 3,
    [Symbol(kWaitingStreams)]: 1
  },
  [Symbol(kName)]: 'stdout',
  [Symbol(kWritableCallbacks)]: [ [Function: bound onwrite] ]
}" is invalid for option "stdio"
    at internal/child_process.js:996:13
    at Array.reduce (<anonymous>)
    at getValidStdio (internal/child_process.js:922:17)
    at ChildProcess.spawn (internal/child_process.js:340:11)
    at Object.spawn (child_process.js:535:9)
    at Function.pythonBridge (/Users/xo/code/scratch/node_modules/python-bridge/index.js:22:28)
    at PostProcessor.<anonymous> (/Users/xo/code/scratch/workers/post-processor.ts:91:24)
    at step (/Users/xo/code/scratch/workers/post-processor.ts:50:23)
    at Object.next (/Users/xo/code/scratch/workers/post-processor.ts:31:53)
    at /Users/xo/code/scratch/workers/post-processor.ts:25:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/xo/code/scratch/workers/post-processor.ts:21:12)
    at PostProcessor.guessit (/Users/xo/code/scratch/workers/post-processor.ts:96:16)
    at /Users/xo/code/scratch/workers/post-processor.ts:78:29
    at Array.map (<anonymous>)
    at PostProcessor.<anonymous> (/Users/xo/code/scratch/workers/post-processor.ts:77:36)
@munro
Copy link
Member

munro commented Aug 26, 2020

wow that's no good, can you share a script I can run to reproduce this? I'd greatly accept a PR to fix this—I haven't used worker threads in Node.js

@munro
Copy link
Member

munro commented Feb 5, 2021

bump @OmgImAlexis do you have a small snippet for me to reproduce this?

I haven't really used worker threads in Node.js, though I guess if I just copy the documentation and call const python = pythonBridge(); it will reproduce—but a snippet to reproduce would be nice!

If I try to think what the issue is, I believe worker threads uses FD 0 on the worker for IPC to the main process—or something like that. I think I'd really have to dig into the how worker threads are working though, and think a bit about a good solution that doesn't end up causing more problems—and then have the automated testing run for single process / main process & worker process. WORK 😫

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants