process, stdout, and such -- using scryer for shell work

[jjtolton]

In Scryer, we jave

?- os:shell("echo $(pwd)", Out).
%@ /home/jay/_project/_project/scryer-modules/cond
%@    Out = 0.

whereas

In Python, you can do the following:

In [1]: import subprocess

In [2]: process = subprocess.Popen(["pwd"], stdout=subprocess.PIPE)

In [3]: output, _error = process.communicate()

In [4]: results = output.decode('utf-8')

In [5]: results
Out[5]: '/tmp'

In Clojure:

(require '[clojure.java.shell :as shell])
(shell/sh "pwd") ;;=> {:exit 0, :out "/tmp\n", :err ""}

Is there a way to easily capture stdout as a stream (perhaps even from write/1 and format/2)?

Or is this something we need to develop?

If this turns out to be something we need to develop I'll convert this to a General instead of a Q&A.

[bakaq]

This is something I have been missing for quite some time also. Something like library(process) would be really helpful.

As a work around (at least in UNIX systems), you probably can use shell/2 to create file pipes (in /tmp for example), run your process connecting stdin, stdout and stderr to them (for example, in bash: program < /tmp/in > /tmp/out 2> /tmp/err (you can also use /dev/null to ignore each one)), and then open/4 them inside Scryer Prolog. I think something like this is basically what subprocess does in Python, and library(process) does in other Prolog implementations. There are some obvious problems with this: Doesn't work on Windows, the shell isn't predictable¹ (I use Nushell for example which is not POSIX), and you may need some way to detach the child process so that shell/2 doesn't block (I think bash's & can do this, but I haven't tested).

Footnotes

1. It seems shell/2 uses the SHELL environment variable to pick the shell, so you can just set it to /bin/sh, which should be available in any UNIX system. ↩

[bakaq]

Here's a minimal example:

:- use_module(library(os)).
:- use_module(library(format)).
:- use_module(library(charsio)).

main :-
    % Fix the shell (this will make this reproducible in any UNIX).
    setenv("SHELL", "/bin/sh"),
    % Ignores the errors if the pipes exist already.
    shell("mkfifo /tmp/in > /dev/null 2> /dev/null",_), 
    shell("mkfifo /tmp/out > /dev/null 2> /dev/null",_),
    shell("cat < /tmp/in > /tmp/out &"),
    open("/tmp/in", write, ProcessIn),
    open("/tmp/out", read, ProcessOut),
    write(ProcessIn, 'asdf'),
    close(ProcessIn),
    get_n_chars(ProcessOut, _, Out),
    close(ProcessOut),
    portray_clause(Out).

$ scryer-prolog subprocess.pl -g main,halt
"asdf".

The order of things here is very particular, because everything blocks everywhere. To make this more usable would need stuff like at_end_of_stream/1 and maybe some kind of way to do a non-blocking open/3.

[jjtolton]

Interesting, so sounds like this is something that will probably have to be implemented at the rust level, otherwise we are going to have to result to hacks like using "inferior" processes and communicating with them via socket to get the result of them communicating with the system, which I suppose it also possible.

some way to detach the child process so that shell/2 doesn't block (I think bash's & can do this, but I haven't tested).

This also means we should probably figure out what the concurrency situation is and get the cont library solid. From my reading of the paper we are going to at least need to implement call_continuation/1 or have some other way of handling callbacks.