wasm: add support for GOOS=wasip1 #3861

aykevl · 2023-08-10T20:43:19Z

This is WIP, a preview of what we might want to add (see #3685).

This is bigger than you might expect, for two reasons:

This uses GOOS=wasip1 support in the Go standard library (added in Go 1.21) instead of using our old support for WASM that basically just pretended to be linux/arm as a workaround for lack of native support.
Support is entirely built into the compiler (instead of in a target JSON file), similar to other operating systems (linux, darwin, windows) but unlike our existing js/wasm and wasi/wasm support. I'm not entirely sure this is a good idea, although it does make sense if we expose this feature only using environment variables and not using the -target flag.

It's not fully working, I'll need to continue working on it later. But a basic "hello world" already compiles.

github-actions · 2023-08-10T20:59:04Z

Size difference with the dev branch:

Binary size difference

 before   after   diff
  60748   60748      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/adt7410/main.go
   9768    9768      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/adxl345/main.go
  13372   13372      0   0.00%  tinygo build -size short -o ./build/test.hex -target=pybadge ./examples/amg88xx
   8896    8896      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/apa102/main.go
  11780   11780      0   0.00%  tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/apds9960/proximity/main.go
  10636   10636      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/apa102/itsybitsy-m0/main.go
   8140    8140      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/at24cx/main.go
   8484    8484      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bh1750/main.go
   7708    7708      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/blinkm/main.go
  63192   63192      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmi160/main.go
  28064   28064      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmp180/main.go
  63260   63260      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmp280/main.go
  12432   12432      0   0.00%  tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/bmp388/main.go
   7724    7724      0   0.00%  tinygo build -size short -o ./build/test.hex -target=bluepill ./examples/ds1307/sram/main.go
  22052   22052      0   0.00%  tinygo build -size short -o ./build/test.hex -target=bluepill ./examples/ds1307/time/main.go
  68888   68888      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/ds3231/main.go
   4712    4712      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/easystepper/main.go
  24960   24960      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/espat/espconsole/main.go
  25160   25160      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/espat/esphub/main.go
  24968   24968      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/espat/espstation/main.go
  68640   68640      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/flash/console/spi
  64664   64664      0   0.00%  tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/flash/console/qspi
   7072    7072      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/gc9a01/main.go
  68032   68032      0   0.00%  tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/gps/i2c/main.go
  68444   68444      0   0.00%  tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/gps/uart/main.go
   8576    8576      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/hcsr04/main.go
   5688    5688      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hd44780/customchar/main.go
   5652    5652      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hd44780/text/main.go
  10716   10716      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/hd44780i2c/main.go
  14792   14792      0   0.00%  tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/hts221/main.go
  15896   15896      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hub75/main.go
  10220   10220      0   0.00%  tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/basic
  11024   11024      0   0.00%  tinygo build -size short -o ./build/test.hex -target=xiao ./examples/ili9341/basic
  29212   29212      0   0.00%  tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/pyportal_boing
  10248   10248      0   0.00%  tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/scroll
  11100   11100      0   0.00%  tinygo build -size short -o ./build/test.hex -target=xiao ./examples/ili9341/scroll
 267468  267468      0   0.00%  tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/slideshow
  12124   12124      0   0.00%  tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/lis3dh/main.go
  14148   14148      0   0.00%  tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/lps22hb/main.go
  26156   26156      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/lsm303agr/main.go
  12620   12620      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/lsm6ds3/main.go
  11084   11084      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mag3110/main.go
  10272   10272      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp23017/main.go
  10704   10704      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp23017-multiple/main.go
   9996    9996      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp3008/main.go
  66816   66816      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp2515/main.go
  21968   21968      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/microbitmatrix/main.go
  22024   22024      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit-v2 ./examples/microbitmatrix/main.go
   8636    8636      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mma8653/main.go
   8420    8420      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mpu6050/main.go
  74928   74928      0   0.00%  tinygo build -size short -o ./build/test.hex -target=p1am-100 ./examples/p1am/main.go
  12264   12264      0   0.00%  tinygo build -size short -o ./build/test.hex -target=pico ./examples/pca9685/main.go
   6112    6112      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/pcd8544/setbuffer/main.go
   5124    5124      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/pcd8544/setpixel/main.go
   2889    2889      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino ./examples/servo
   8092    8092      0   0.00%  tinygo build -size short -o ./build/test.hex -target=pybadge ./examples/shifter/main.go
  56340   56340      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/sht3x/main.go
  56312   56312      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/shtc3/main.go
   6452    6452      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ssd1306/i2c_128x32/main.go
   5960    5960      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ssd1306/spi_128x64/main.go
   5728    5728      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ssd1331/main.go
   6420    6420      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/st7735/main.go
   6316    6316      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/st7789/main.go
  17368   17368      0   0.00%  tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/thermistor/main.go
  10284   10284      0   0.00%  tinygo build -size short -o ./build/test.hex -target=circuitplay-bluefruit ./examples/tone
  10952   10952      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/tm1637/main.go
   9536    9536      0   0.00%  tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/touch/resistive/fourwire/main.go
  12532   12532      0   0.00%  tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/touch/resistive/pyportal_touchpaint/main.go
  15852   15852      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/vl53l1x/main.go
  13876   13876      0   0.00%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/vl6180x/main.go
   6484    6484      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd2in13/main.go
   6036    6036      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd2in13x/main.go
   6312    6312      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd4in2/main.go
 137260  137260      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/wifinina/ntpclient/main.go
 137268  137268      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/wifinina/udpstation/main.go
 137528  137528      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/wifinina/tcpclient/main.go
 137660  137660      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/wifinina/webclient/main.go
   7120    7120      0   0.00%  tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/ws2812
   5452    5452      0   0.00%  tinygo build -size short -o ./build/test.bin -target=m5stamp-c3          ./examples/ws2812
  61588   61588      0   0.00%  tinygo build -size short -o ./build/test.hex -target=feather-nrf52840 ./examples/is31fl3731/main.go
   1565    1565      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino   ./examples/ws2812
    896     896      0   0.00%  tinygo build -size short -o ./build/test.hex -target=digispark ./examples/ws2812
  32332   32332      0   0.00%  tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/bme280/main.go
  16816   16816      0   0.00%  tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/microphone/main.go
  11484   11484      0   0.00%  tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/buzzer/main.go
  13076   13076      0   0.00%  tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/veml6070/main.go
   7012    7012      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l293x/simple/main.go
   8968    8968      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l293x/speed/main.go
   6980    6980      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l9110x/simple/main.go
   9524    9524      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l9110x/speed/main.go
   7464    7464      0   0.00%  tinygo build -size short -o ./build/test.hex -target=nucleo-f103rb ./examples/shiftregister/main.go
   8336    8336      0   0.00%  tinygo build -size short -o ./build/test.hex -target=hifive1b ./examples/ssd1351/main.go
  13364   13364      0   0.00%  tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/lis2mdl/main.go
   8516    8516      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/max72xx/main.go
  77260   77260      0   0.00%  tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/dht/main.go
  70732   70732      0   0.00%  tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/alarm/
   7492    7492      0   0.00%  tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/clkout/
  70404   70404      0   0.00%  tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/time/
  70668   70668      0   0.00%  tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/timer/
  12108   12108      0   0.00%  tinygo build -size short -o ./build/test.hex -target=pico ./examples/qmi8658c/main.go
   9024    9024      0   0.00%  tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/ina260/main.go
   9432    9432      0   0.00%  tinygo build -size short -o ./build/test.hex -target=nucleo-l432kc ./examples/aht20/main.go
  72028   72028      0   0.00%  tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/sdcard/console/
  82464   82464      0   0.00%  tinygo build -size short -o ./build/test.hex -target=wioterminal ./examples/rtl8720dn/webclient/
  71808   71808      0   0.00%  tinygo build -size short -o ./build/test.hex -target=wioterminal ./examples/rtl8720dn/webserver/
  98584   98584      0   0.00%  tinygo build -size short -o ./build/test.hex -target=wioterminal ./examples/rtl8720dn/mqttsub/
  60304   60304      0   0.00%  tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/i2csoft/adt7410/
  10400   10400      0   0.00%  tinygo build -size short -o ./build/test.elf -target=wioterminal ./examples/axp192/m5stack-core2-blinky/
   9076    9076      0   0.00%  tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/xpt2046/main.go
  14572   14572      0   0.00%  tinygo build -size short -o ./build/test.hex -target=nucleo-wl55jc ./examples/sx126x/lora_rxtx/
  25884   25884      0   0.00%  tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/ssd1289/main.go
  11348   11348      0   0.00%  tinygo build -size short -o ./build/test.hex -target=pico ./examples/irremote/main.go
  11340   11340      0   0.00%  tinygo build -size short -o ./build/test.hex -target=badger2040 ./examples/uc8151/main.go
  10468   10468      0   0.00%  tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/scd4x/main.go
   8968    8968      0   0.00%  tinygo build -size short -o ./build/test.uf2 -target=circuitplay-express ./examples/makeybutton/main.go
   9732    9732      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ds18b20/main.go
  81336   81336      0   0.00%  tinygo build -size short -o ./build/test.hex -target=nucleo-wl55jc ./examples/lora/lorawan/atcmd/
  15740   15740      0   0.00%  tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/as560x/main.go
   9952    9952      0   0.00%  tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/mpu6886/main.go
   8124    8124      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ttp229/main.go
3468982 3468982      0   0.00%   sum

deadprogram · 2023-08-11T10:46:11Z

This is bigger than you might expect

Perhaps we should move this to milestone for 0.30 release so we do not hold up a Go 1.21 compatible release in the meantime.

@aykevl @codefromthecrypt anyone else what do you think?

aykevl · 2023-08-11T15:27:51Z

Actually I think I can finish this PR quickly. It seems to pass all tests locally at least.

aykevl · 2023-08-11T15:33:20Z

Eventually I'd like to remove our old WASI support entirely (currently usable via -target=wasi), perhaps aliasing -target=wasi to the equivalent of GOOS=wasip1 GOARCH=wasm to not break backwards compatibility too badly. But for now it seems reasonable to keep support for both. It's not too difficult to support both at the same time: it's mainly just slightly more complicated //go:build lines and somewhat slower CI.

deadprogram · 2023-08-11T15:37:27Z

/root/project/src/runtime/os_wasip1.go:8:7: GOOS redeclared in this block
/root/project/src/runtime/os_linux.go:10:7: 	other declaration of GOOS
# runtime
/root/project/src/runtime/os_wasip1.go:8:7: GOOS redeclared in this block
/root/project/src/runtime/os_linux.go:10:7: 	other declaration of GOOS
make: *** [Makefile:449: smoketest] Error 1

Exited with code exit status 2

Actual fail in CircleCI build https://app.circleci.com/pipelines/github/tinygo-org/tinygo/9892/workflows/c400192b-ca42-4e4a-a87c-f7743e786c75/jobs/41137?invite=true#step-118-9

aykevl · 2023-08-11T15:46:53Z

Thanks, fixed now.

This adds true GOOS=wasip1 support in addition to our existing -target=wasi support. The old support for WASI isn't removed, but should be treated as deprecated and will likely be removed eventually to reduce the test burden.

aykevl · 2023-08-11T16:18:50Z

Fixed an issue on tinygo test os (due to the renaming of stat_linux.go).

deadprogram

Looking good from my point of view.

@dgryski @codefromthecrypt @dkegel-fastly what do you think?

deadprogram · 2023-08-14T18:07:27Z

Last call for comments before merging this PR!

cc @dgryski @codefromthecrypt @dkegel-fastly @achille-roussel and anyone else who is interested.

achille-roussel

This looks great 🙌 it's actually a much smaller change than I would have expected, which is great 👍

It might be useful to document when should users select -target=wasi vs GOOS=wasip1 in a README or similar, I believe that the historical implementation still had a few features that were not supported, for example:

I imagine users will probably stubble upon those differences and be confused about why certain functionalities work in one WASI but not the other. I expect we would get questions like "when should we use -target=wasi vs GOOS=wasip1?" and recording those answers will likely reduce the maintenance load.

Thanks for doing this work!

dkegel-fastly · 2023-08-15T16:33:37Z

running my app's tests now...

dkegel-fastly · 2023-08-15T17:13:25Z

In preparation for testing this branch, I did a hopefully clean build of the latest dev branch, but it fails "make test", so maybe hold off until that's settled...? see slack.

deadprogram · 2023-08-17T16:16:48Z

@dkegel-fastly see #3871 to track that issue. Since it is not related to this PR, I am going to merge this one to keep things moving along.

dgryski · 2023-08-18T01:13:06Z

BTW, this seems to pass the test corpus just fine.

aykevl mentioned this pull request Aug 10, 2023

wasm: allow GOOS=wasip1 to substitute for target=wasi #3685

Closed

aykevl force-pushed the wasip1 branch from 1725623 to 7962e89 Compare August 11, 2023 15:27

aykevl marked this pull request as ready for review August 11, 2023 15:27

aykevl force-pushed the wasip1 branch from 7962e89 to ce1cfbf Compare August 11, 2023 15:33

aykevl force-pushed the wasip1 branch from ce1cfbf to 6baddaf Compare August 11, 2023 15:46

deadprogram requested a review from dgryski August 11, 2023 16:02

wasm: add support for GOOS=wasip1

9381463

This adds true GOOS=wasip1 support in addition to our existing -target=wasi support. The old support for WASI isn't removed, but should be treated as deprecated and will likely be removed eventually to reduce the test burden.

aykevl force-pushed the wasip1 branch from 6baddaf to 9381463 Compare August 11, 2023 16:18

aykevl mentioned this pull request Aug 11, 2023

WASI problem #3834

Closed

deadprogram approved these changes Aug 11, 2023

View reviewed changes

achille-roussel approved these changes Aug 15, 2023

View reviewed changes

joeshaw mentioned this pull request Aug 16, 2023

feat(compute/build): support native go fastly/cli#995

Merged

deadprogram merged commit a545f17 into dev Aug 17, 2023
22 checks passed

deadprogram deleted the wasip1 branch August 17, 2023 16:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

wasm: add support for GOOS=wasip1 #3861

wasm: add support for GOOS=wasip1 #3861

aykevl commented Aug 10, 2023 •

edited

Loading

github-actions bot commented Aug 10, 2023 •

edited

Loading

deadprogram commented Aug 11, 2023

aykevl commented Aug 11, 2023

aykevl commented Aug 11, 2023

deadprogram commented Aug 11, 2023

aykevl commented Aug 11, 2023

aykevl commented Aug 11, 2023

deadprogram left a comment

deadprogram commented Aug 14, 2023

achille-roussel left a comment •

edited

Loading

dkegel-fastly commented Aug 15, 2023

dkegel-fastly commented Aug 15, 2023

deadprogram commented Aug 17, 2023

dgryski commented Aug 18, 2023

wasm: add support for GOOS=wasip1 #3861

wasm: add support for GOOS=wasip1 #3861

Conversation

aykevl commented Aug 10, 2023 • edited Loading

github-actions bot commented Aug 10, 2023 • edited Loading

deadprogram commented Aug 11, 2023

aykevl commented Aug 11, 2023

aykevl commented Aug 11, 2023

deadprogram commented Aug 11, 2023

aykevl commented Aug 11, 2023

aykevl commented Aug 11, 2023

deadprogram left a comment

Choose a reason for hiding this comment

deadprogram commented Aug 14, 2023

achille-roussel left a comment • edited Loading

Choose a reason for hiding this comment

dkegel-fastly commented Aug 15, 2023

dkegel-fastly commented Aug 15, 2023

deadprogram commented Aug 17, 2023

dgryski commented Aug 18, 2023

aykevl commented Aug 10, 2023 •

edited

Loading

github-actions bot commented Aug 10, 2023 •

edited

Loading

achille-roussel left a comment •

edited

Loading