Haxe Installation

[rat-moonshine]

With the addition of Haxe project support to Moonshine, we shall be needing installation support of Haxe, Node.js and other related programs through Moonshine SDK Installer.

Here are some initial direction from Josh:

To enable code intelligence, obviously Haxe needs to be installed. Additionally, the language server runs on Node.js. I'm not really familiar with the SDK installer part of Moonshine, so I hope that someone can set that up with these two dependencies. For now, I added fields where these two paths can be configured manually in settings.

If you want to test it out, and you've just installed Haxe for the first time, be sure to open a terminal and manually run "haxelib setup". If you don't, Moonshine will fail to install your project's library dependencies (like OpenFL) and code intelligence won't work. This issue is something that I plan to fix soon. I just discovered it after the merge. It wasn't occurring on my main computer because I was already fully set up for Haxe development there.

We'll probably require to automate the run haxelib setup command once Haxe installation finish. Targeting different compilation platforms (i.e. Windows, macOS, mobile etc.) we might require to look into all the other dependency requirement for individual purpose.

[JoelProminic]

I see that if I install Haxe at the default location, it fails with the App Store build of Moonshine:

: Haxe build directory: /Users/joelanderson/Documents/Flex/workspace/Moonshine-IDE2/ide/MoonshineGUICore
: Command: $HAXE_HOME/haxelib run lime build flash -debug
: /bin/bash: /usr/local/bin/haxelib: Operation not permitted

So, we may need to look into installing this under ~/Downloads/MoonshineSDKs, if possible

[JoelProminic]

@JustinProminic asked about this again recently. We should also install Neko as part of this, and update both paths in Moonshine.

[rat-moonshine]

Instead of using installer to install Haxe, we probably able to download/install using binaries from here https://github.com/HaxeFoundation/haxe/releases.

As an example for macOS, we can download and place the tar.gz files under usr/local/lib/haxe (as standard install location) and need to keep symlinks/executable for Haxe and Haxelib under usr/local/bin (or maybe this part can be handle by haxelib setup). This will help us from getting into installer-GUI to install Haxelib.

We can write an Ant script therefore to automate the other installation process as well.

[rat-moonshine]

I'm having trouble finding a way to pass the default parameter to openfl setup during Ant process:

% haxelib run openfl setup           
lime is up to date
hxcpp is up to date
lime-samples is up to date
openfl-samples is up to date
actuate is up to date
box2d is up to date
layout is up to date
Do you want to install the "openfl" command? [y/n/a] ?

I tried to run the command using haxelib as haxelib run openfl setup --always y - but this immediately completes the execution without giving any details:

% haxelib run openfl setup --always y
%

[JoelProminic]

I see I used the --always option with no y in this issue: Moonshine-IDE/Moonshine-IDE#843 (comment)

You can see my Ant task here

[rat-moonshine]

Problem those I'm facing currently,

1. Running haxelib setup --always /usr/local/lib seems not working correctly. I vaguely remember @JoelProminic suggested another way to execute this is /usr/local/lib | haxelib setup (or likewise command to bypass OpenFL prompts), but all failed with same error while running through an Ant process:

$ haxelib setup --always /usr/local/lib
zsh: command not found: 1
Please enter haxelib repository path with write access
Hit enter for default (/usr/local/lib/haxe/lib)
Path : Error: Eof

2. The second problem that I'm seeing is placing the binaries under /usr/local/lib and /usr/local/bin. This requires user's permission. At this point I'm not clear how we automate such permission during MSKDI installs. I think the binaries can be placed anywhere, perhaps we can place this under /Downloads/MoonshineSDKs, instead of /usr/local/lib - but we needs to place executable under /usr/local/bin too to access globally. At any point, a permission issue comes up during this installation.

[rat-moonshine]

While searching for a way to run shell-script from an app with administrator privilege - so we can copy the files/folders under usr/local/, I found that it maybe possible by running in AppleScript, i.e. do shell script "clear" with administrator privileges. I see running the command in an example script opens me the administrator prompt:

This might require us to break the process in different chunks, i.e. Ant -> AppleScript -> Ant, or something likewise.

[JoelProminic]

These commands worked for me:

# The mkdir command should be replaced by however you are creating the directories normally
mkdir -p /Users/-user-/Downloads/MoonshineSDKs/Haxe/lib
echo '/Users/-user-/Downloads/MoonshineSDKs/Haxe/lib' | haxelib setup

I confirmed I could install a library in this directory.

If we can install Haxe in ~/Downloads/MoonshineSDKs/Haxe, that should resolve the permissions issue as well. We will need to add this directory to PATH when generating the environment, though.

[JoelProminic]

I confirmed that I can download the binaries here, and install them in MoonshineSDKs like this. You'll want to detect the extracted path rather than hard-coding it.

% cd ~/Downloads/MoonshineSDKs/Haxe
% tar xzvf ~/Downloads/haxe-4.2.4-osx.tar.gz
...
% export PATH="/Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe_20211022152000_ab0c054/:$PATH"
% which haxe
/Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe_20211022152000_ab0c054//haxe
# Fix permissions
% xattr -r -d com.apple.quarantine haxe_20211022152000_ab0c054/
% echo '/Users/-user-/Downloads/MoonshineSDKs/Haxe/lib' | haxelib setup
Please enter haxelib repository path with write access
Hit enter for default (/usr/local/lib/haxe/lib)
Path : haxelib repository is now /Users/-user-/Downloads/MoonshineSDKs/Haxe/lib
% haxelib install lime
Downloading lime-7,9,0.zip...
Download complete: 120.62MB in 73.1s (1648.4KB/s)
Installing lime...
  Current version is now 7.9.0
Done

UPDATE: I see that the haxelib path is updated globally, even though I used the haxelib binary from MoonshineSDKs. If this is saved to a directory with restricted permissions, this may cause troubles anyway. I wasn't able to identify the file that saves this information, but let me know what path is reported if you get errors.

[rat-moonshine]

The environment setup command for macOS running from the Ant process - turned out to be unstable between different machine tests.

I moved the process to split in both Ant and AS3 (NativeProcess) to overcome this, as we're doing on Windows. With this split the environment-setup and haxe setup now remains at AS3 side.

I confirm it works accordingly, and I don't see any Exec task failed: /Users/$user/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4/haxelib error either, but this best to be tested by @Aszusz and @JoelProminic , please check.

Some other notes from our earlier discussions:

1. AS3 Ant API has its some logic to measure progress of its ActionScript-based Ant-tag implementations. It helps to follow each of the tag progression. In NativeProcess side we don't have any measurable progress information for its output stream - it just returns a set of bytes and messages along with its stdOut which we're showing here. This is why we don't has any progressing-progress-bar when NativeProcess part is running
2. "Show Log" now populates all the output retuned in split processes
3. A "Installation complete" can be noticed in the middle of the log, and I agree this is confusing. This adds by the SDK Installer library in an automated way when a set of process completes there - process > completion > cleanup. The 'cleanup' section here adds the pre-defined completion message and this triggers during any process ends at library side. This cleanup method used globally, and I think it's better not to modify for this only needs. I adjusted the followed-by message to make it look a continuous process, though. Here is the full log output from the installation:

Installer path: /Users/devsena/Documents/ProminicGITBranch/Moonshine-SDK-Installer/MoonshineSDKInstaller/build/DEPLOY/MoonshineSDKInstaller.app/Contents/Resources
Installer version 2.3.0 (mac)
Using Locale: en_IN
Fetched the SDK download mirror URL from the CGI.
Downloading Haxe SDK v.4.2.4
Starting to untar: /Users/devsena/Library/Application Support/net.prominic.MoonshineSDKInstaller/Local Store/haxe-4.2.4-osx.tar.gz
Completed untar Haxe SDK
Moving files to SDK directory /Users/devsena/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4
Downloading Neko SDK v.2.3.0
Starting to untar: /Users/devsena/Library/Application Support/net.prominic.MoonshineSDKInstaller/Local Store/neko-2.3.0-osx64.tar.gz
Completed untar Haxe SDK
Moving files to SDK directory /Users/devsena/Downloads/MoonshineSDKs/Haxe/neko-2.3.0
Deleted temp files.
Installation complete
Now continues with Haxe/Neko setup
haxelib repository is now /Users/devsena/Downloads/MoonshineSDKs/Haxe/lib
Downloading feathersui-1,0,0-beta,8.zip...
Installing feathersui...
  Current version is now 1.0.0-beta.8
Done
Installing dependency actuate 1.8.9
Downloading actuate-1,8,9.zip...
Installing actuate...
  Current version is now 1.8.9
Done
Installing dependency hxargs 3.0.2
Downloading hxargs-3,0,2.zip...
Installing hxargs...
  Current version is now 3.0.2
Done
Installing dependency openfl 9.1.0
Downloading openfl-9,1,0.zip...
Installing openfl...
  Current version is now 9.1.0
Done
You already have openfl version 9.1.0 installed
You already have actuate version 1.8.9 installed
Downloading lime-7,9,0.zip...
Installing lime...
  Current version is now 7.9.0
Done
Setup completes for Haxe and Neko

For Moonshine > About section we shall discuss at here: Moonshine-IDE/Moonshine-IDE#967.

[Aszusz]

I retested on macOS and Windows and it seems to be working fine in both cases. There weren't any task failed errors.

[JoelProminic]

I can also successfully install on macOS and Windows in today's tests.

The updated logs look good.

I did some quick tests with Haxe Neko projects in Moonshine and Build & Run worked as expected. I'm still having trouble adding a library, but that should be fixed on the Moonshine side (see Moonshine-IDE/Moonshine-IDE#967)

[JoelProminic]

@rat-moonshine and I discussed the lime problem more in today's chat.

This discussion seems very relevant to what we are trying to do. It seems that the problem is that macOS SIP prevents DYLD environment variables from being inherited by child processes. So, haxelib works fine with the local environment, but more complicated commands start to have problems.

Looking at the ideas in this thread, I thought the symlinks were an interesting idea. I did some tests, and this seemed to clear the lime errors. To do this, we need to create symlinks for all .dylib and .ndll files in the Neko installation. For example

ln -s $NEKO_HOME/libneko.dylib $HAXE_HOME/libneko.dylib 
ln -s $NEKO_HOME/libneko.2.dylib $HAXE_HOME/libneko.2.dylib 
ln -s $NEKO_HOME/libneko.2.3.0.dylib $HAXE_HOME/libneko.2.3.0.dylib 
ln -s $NEKO_HOME/std.ndll $HAXE_HOME/std.ndll
ln -s $NEKO_HOME/mod_neko2.ndll $HAXE_HOME/mod_neko2.ndll
ln -s $NEKO_HOME/mysql.ndll $HAXE_HOME/mysql.ndll
ln -s $NEKO_HOME/regexp.ndll $HAXE_HOME/regexp.ndll
ln -s $NEKO_HOME/mysql5.ndll $HAXE_HOME/mysql5.ndll
ln -s $NEKO_HOME/zlib.ndll $HAXE_HOME/zlib.ndll
ln -s $NEKO_HOME/sqlite.ndll $HAXE_HOME/sqlite.ndll
ln -s $NEKO_HOME/mod_tora2.ndll $HAXE_HOME/mod_tora2.ndll
ln -s $NEKO_HOME/ui.ndll $HAXE_HOME/ui.ndll
ln -s $NEKO_HOME/ssl.ndll $HAXE_HOME/ssl.ndll

This needs to be done once as part of installing Haxe and Neko.

UPDATE: the symlinks should only be needed for macOS. @rat-moonshine noticed some Windows problems late in the meeting today, though, so we'll need to investigate that as well.

[JesseProminic]

Project Type	Feature	macOS	Note
Haxe Feathers UI	Project > Build	built and ran successfully
Haxe OpenFL	Project > Build	built and ran successfully
Haxe > AIR	Project > Build	built but failed to run	This command is not supported on platform "AIR"
Haxe > AIR Mobile	Project > built	built but failed to run	This command is not supported on platform "AIR Mobile".
Haxe > C++	Project > Build	built and ran successfully
Haxe > C#	Project > Build	failed	Compilation error. C# compiler not found. Please make sure either Visual Studio or mono is installed or they are reachable by their path.
Haxe > Flash Player	Project >	built but failed to run	stuck at Launching application using adapter: swf
Haxe > HashLink	Project >	built but failed to run	This command is not supported on platform "HashLink"
Haxe > JavaScript	Project >	built but failed to run	This command is not supported on platform "JavaScript"
Haxe > Neko	Project > Build	built and ran successfully
"	Project Settings > Paths > Haxelibs	built and ran successfully HOWEVER	only from the menu Build Project > Build and Run otherwise, /bin/sh: haxelib: command not found
Haxe > PHP	Project > Build	built but failed to run	This command is not supported on platform "PHP"
Haxe > Python	Project > Build	built but failed to run	This command is not supported on platform "Python"

[rat-moonshine]

I moved above test-matrix from @JesseProminic to Moonshine-IDE/Moonshine-IDE#967 (comment) as this testings are Moonshine related. Please, discuss in the linked forum.

[JoelProminic]

I have some reservations about how the symlinks are handled, but the install is working on macOS and Windows for now. We can revisit this if it causes problems later.

[JoelProminic]

We have been seeing these errors lately

: ln: /Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4/libneko.dylib: File exists
: ln: /Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4/libneko.2.dylib: File exists
: ln: /Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4/libneko.2.3.0.dylib: File exists
: ln: /Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4/std.ndll: File exists
: ln: /Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4/mod_neko2.ndll: File exists
: ln: /Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4/mysql.ndll: File exists
: ln: /Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4/regexp.ndll: File exists
: ln: /Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4/mysql5.ndll: File exists
: ln: /Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4/zlib.ndll: File exists
: ln: /Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4/sqlite.ndll: File exists
: ln: /Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4/mod_tora2.ndll: File exists
: ln: /Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4/ui.ndll: File exists
: ln: /Users/-user-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.4/ssl.ndll: File exists

I thought we had written this to hide the warnings if the links already exist.

Tagging @Igazine

[Igazine]

Note:

InstallerSharedCore/src/actionScripts/valueObjects/HelperConstants.as

https://github.com/Moonshine-IDE/Moonshine-IDE/blob/master/ide/MoonshineDESKTOPevolved/src/actionScripts/utils/EnvironmentSetupUtils.as#L366

[Igazine]

Closed this issue while recreated the relevant part as a separate issue in Moonshine-IDE