File.read/1 missing 1st binary of /dev/stdin which comes from Unix pipe

[pojiro]

Elixir and Erlang/OTP versions

elixir --version
Erlang/OTP 27 [erts-15.0.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Elixir 1.17.2 (compiled with Erlang/OTP 27)

Operating system

Linux Mint 20.3

Current behavior

I prepare two exs files folloings

• name: pipe_test_with_io_read.exs
  • content: IO.read(:stdio, :eof) |> IO.inspect()
• name: pipe_test_with_file_read.exs
  • content: File.read!("/dev/stdin") |> IO.inspect()

The behaviours on bash are followings,

# IO.read works
echo hello | elixir pipe_test_with_io_read.exs 
"hello\n"

# File.read doesn't work
echo hello | elixir pipe_test_with_file_read.exs 
""

---

It seems like File.read cannot get 1st binary which arrived at /dev/stdin. I test my guess by followings,
(ping -c option is Stop after sending count ECHO_REQUEST packets.)

ping -c 1 example.com | elixir pipe_test_with_file_read.exs 
""

ping -c 2 example.com | elixir pipe_test_with_file_read.exs 
"64 bytes from 93.184.215.14 (93.184.215.14): icmp_seq=2 ttl=54 time=127 ms\n\n--- example.com ping statistics ---\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\nrtt min/avg/max/mdev = 126.359/126.589/126.819/0.230 ms\n"

Expected behavior

File.read! can read the anonymous pipe, so it would be nice to behave like following,

echo hello | elixir pipe_test_with_file_read.exs 
"hello\n"

[whatyouhide]

Mh very interesting. However, I'm fairly sure this is happening in Erlang. You can reproduce the same behaviour with:

dbg(:file.read_file(~c"/dev/stdin"))

[josevalim]

Yes, this is an Erlang behaviour. I believe the main reason why it does not work is because Erlang pre-buffers all stdin. For example, if you do yes | erl, Erlang will buffer all input and eventually run out of memory. This means that, trying to use File.read, will always race. There are discussions for changing this, but it depends on other factors, such as the terminal. For now, I recommend using the IO module.

[pojiro]

@whatyouhide @josevalim Thank you.

JFY(and future me)I, I wrote an issue as erlang/otp#8725 . And received an answer for this.
I confirmed that following code works. But we use the -noinput, we can't use IO.read anymore.

echo hello | elixir --erl -noinput pipe_test_with_file_read.exs
"hello\n"

ref. for -noinput, https://www.erlang.org/doc/apps/erts/erl_cmd.html