Blogging about making Booster service cross-platform and making a dotnet tool

.gitignore

@@ -0,0 +1 @@
+*~

user/mgarai/20210915-making-booster-cross-platform.md

@@ -0,0 +1,201 @@
+---
+title: "Making Booster compilation service cross-platform"
+categories: "websharper,fsharp,compilation,cross-platform,linux"
+abstract: "Change a console application used as a service to support cross-platform. RIDs. What they are good for. Executables across platforms. Detached child process. NamedPipe in Linux. IO behavior when using nohup."
+identity: "-1,-1"
+---
+
+## Intro
+
+Make use of experience from making WebSharper Booster console application (used as service) cross-platform. From Windows only to Linux compatible.
+
+## RIDs
+
+RIDs ([Runtime Identifier][runtime-identifier]) are specifiers for compiler to output operating system, version, architecture, "distro kind" specific code.
+
+That is more information that can be inferred from `RuntimeInformation.IsOSPlatform(OSPlatform.Windows)` or `RuntimeInformation.IsOSPlatform(OSPlatform.Linux)`. For further information check [Platform compatibility analyzer][platform-compatibility-analyzer]. For example it's a valid RID to use `linux-musl-x64`, but the `musl` part is not really version specific. It's not easy to infer it from runtime.
+
+### Why specify RID?
+
+Deploying the application itself doesn't need to be RID qualified ([.NET application publishing overview][dotnet-application-publishing-overview]). Simple `dotnet publish` produces the right `.dll` (cross-platform) and an executable, which is current platform specific.
+
+> Publishing an app as framework-dependent produces a cross-platform binary as a `.dll` file, and a platform-specific executable that targets your current platform
+
+When cross-platform executable is needed, than the `dotnet publish` command needs a `-r` flag. Which is the RID. The `<RuntimeIdentifier>` tag in the project file specifies RID.
+
+If the executable expected to be of the right name:
+
+Windows specific
+```shell
+wsfscservice.exe
+```
+
+Linux specific
+```bash
+./wsfscservice
+```
+
+I need to specify RID. This helps finding the running service from the `Task Manager` or `ps`.
+
+### Current platform's RID?
+
+As mentioned before the RID have more information coded in itself than `RuntimeInformation` namespace gives us. If the execution of the application dispatched through `.targets` file definition, we can use the RID that's available there for .NET Core SDK Runtime Identifier. I found that variable while checking through SDK's code reverse engineering how `_UsingDefaultPlatformTarget` is set. For `_UsingDefaultPlatformTarget` = `true` something is used as [default RID][default-runtime-id].
+
+```xml
+<DefaultAppHostRuntimeIdentifier>$(NETCoreSdkRuntimeIdentifier)</DefaultAppHostRuntimeIdentifier>
+```
+
+Some [information][netcoresdkruntimeidentifier-information] about the `MSBuild` variable:
+
+> The NETCoreSdkRuntimeIdentifier MSBuild property determines the bitness of *.comhost.dll
+
+This is a good default value to find out the folder where platform specific `.dll` is located:
+
+```xml
+<!-- where platformspecific wsfscservice/wsfscservice.exe located -->
+<FscToolPath>$(MSBuildThisFileDirectory)/../tools/net5.0/$(NETCoreSdkRuntimeIdentifier)/</FscToolPath>
+```
+
+In the two cases tested `$(NETCoreSdkRuntimeIdentifier)` is substituted for `win-x64` and `linux-x64`.
+
+### Keeping the process's name across different platforms
+
+Alternative would be using the cross-platform `.dll` with `dotnet wsfscservice.dll` but this makes hard to find the running process through all running `dotnet` instances. [Rename the running process is not possible][rename-the-running-process-is-not-possible] or require `start "wsfscservice" dotnet wsfscservice.dll` "hack" which is Windows specific. Also finding the process programmatically is simpler:
+
+```fsharp
+let runningServers =
+    try
+        Process.GetProcessesByName("wsfscservice")
+        |> Array.filter (fun x -> System.String.Equals(x.MainModule.FileName, fileNameOfService, System.StringComparison.OrdinalIgnoreCase))
+    with
+    | e ->
+        // If the processes cannot be queried, because of insufficient rights, the Mutex in service will handle
+        // not running 2 instances
+        nLogger.Error(e, "Could not read running processes of wsfscservice.")
+        [||]
+```
+
+Attach/reattach debugger is [easier][reattach-debugger].
+
+So the choice is [framework-dependent][framework-dependent] but non `--self-contained`. `--self-contained` would make the executable bigger, and makes harder to patch the runtime around the executable.
+
+### `<RuntimeIdentifier>` vs `<RuntimeIdentifiers>`
+
+Expecting that using `<RuntimeIdentifiers>` instead of `<RuntimeIdentifier>` (mind the plural) changes the output structure of published nuget is wrong. When used correctly the RID specific output should be scattered around in `Debug`/`Release` folder (inside for example `win-x64`, `linux-x64`, `linux-musl-x64`). But `dotnet publish` doesn't publish for all different RIDs specified in the `<RuntimeIdentifiers>`. Instead uses directly the `Debug`/`Release` folder. Just like the `dotnet build` command doesn't.
+
+![](/assets/outputs.png)
+
+For that to work we needed RIDs count `publish` command for all executables. Libraries doesn't need extra care. In our build process it looks like this (Fake specific. `DotNet.publish` calls `dotnet publish`)
+
+```fsharp
+let publishExe (mode: BuildMode) fw input output =
+    for rid in [ "win-x64"; "linux-x64"; "linux-musl-x64" ] do
+        let outputPath =
+            __SOURCE_DIRECTORY__ </> "build" </> mode.ToString() </> output </> fw </> rid </> "deploy"
+        DotNet.publish (fun p ->
+            { p with
+                Framework = Some fw
+                OutputPath = Some outputPath
+                NoRestore = true
+                SelfContained = false |> Some // add --self-contained false
+                Runtime = rid |> Some // add -r <RID> from the list above.
+                Configuration = DotNet.BuildConfiguration.fromString (mode.ToString())
+            }) input
+...
+BuildAction.Custom <| fun mode ->
+    publishExe mode "net5.0" "src/compiler/WebSharper.FSharp/WebSharper.FSharp.fsproj" "FSharp"
+    publishExe mode "net5.0" "src/compiler/WebSharper.FSharp.Service/WebSharper.FSharp.Service.fsproj" "FSharp"
+    publishExe mode "net5.0" "src/compiler/WebSharper.CSharp/WebSharper.CSharp.fsproj" "CSharp"
+```
+
+This also can be achieved by simple shell script, which iterates on the RIDs specified in the `<RuntimeIdentifiers>`. If the singular `-r` parameter is not from the listed `<RuntimeIdentifiers>` RIDs from the project file the `dotnet publish` will fail!
+
+Important tags to the project file to be added:
+
+```xml
+<OutputType>Exe</OutputType>
+<TargetFrameworks>net5.0</TargetFrameworks>
+<Name>wsfscservice</Name>
+<AssemblyName>wsfscservice</AssemblyName>
+<RuntimeIdentifiers>win-x64;linux-x64;linux-musl-x64</RuntimeIdentifiers>
+```
+
+### Self contained
+
+It was a surprise that `--self-contained`'s default value is variable. If you specify RID, it's `true` if you just leave `dotnet publish` it's `false`. So to avoid [Publish self-contained][publish-self-contained], `--self-contained false` had to be added.
+
+
+## Running detached child processes
+
+When spinning off an executable the process will become the child process of the main executable. That means however someone starts a process to be background it will be closing with the parent executable. Our Console application can't function as a service.
+
+Starting the executable can be:
+
+```fsharp
+// start a detached wsfscservice.exe. Platform specific.
+let cmdName = if System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(System.Runtime.InteropServices.OSPlatform.Windows) then
+                "wsfscservice.exe" else "wsfscservice"
+let cmdFullPath = (location, cmdName) |> System.IO.Path.Combine
+let startInfo = ProcessStartInfo(cmdFullPath)
+startInfo.CreateNoWindow <- true
+startInfo.UseShellExecute <- false
+startInfo.WindowStyle <- ProcessWindowStyle.Hidden
+proc <- Process.Start(startInfo)
+```
+
+But it will be still child process. The solution is to start the process operation system specifically. Create a `.cmd` for Windows using `start` with `/b` parameter to be [background process][windows-background]. 
+
+```shell
+@echo off
+
+start /d %~dp0 /b wsfscservice.exe
+```
+
+And a `.sh` for Linux. Here [`nohup`][nohup] makes `wsfscservice` immune to hangups and `&` at the end makes it [background process][linux-background].
+
+> If a command is terminated by the control operator &, the shell executes the command in the background in a subshell. The shell does not wait for the command to finish, and the return status is 0
+
+```bash
+#!/usr/bin/env bash
+
+nohup "$(dirname "$BASH_SOURCE")/wsfscservice" >/dev/null 2>&1 &
+```
+
+`Process.Start` can start the platform specific `.cmd`/`.sh`. No matter it returned immediately with `0` code it's sure that the background process is running. `Task Manager` shows `wsfscservice` and `ps -al` show `wsfscservice`. "Start new process, without being a child of the spawning process" question on `Stackoverflow.com` top voted answer also does [something similar][so-use-stub-exe].
+
+```fsharp
+let cmdName = if System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(System.Runtime.InteropServices.OSPlatform.Windows) then
+                "wsfscservice_start.cmd" else "wsfscservice_start.sh"
+```
+
+## Caveat around `nohup`
+
+The service processes commands in an infinite loop. This can be achieved by startup a mailbox processor with `Async.Start`, then on the main thread block the execution. At first blocking the main thread was done by `Console.ReadLine() |> ignore`. Avoiding busy wait. But `nohup`'s IO piping makes this line of code just continue.
+
+Another possible solution for non-busy wait is using:
+
+```fsharp
+use locker = new AutoResetEvent(false)
+locker.WaitOne() |> ignore
+```
+
+That solved that a code working on Windows now keeps the service running on Linux as well.
+
+## NamedPipes on Linux
+
+One thing it's worth mentioning is `NamedPipe`'s `PipeTransmissionMode.Message` is [not supported][not-supported] in the Linux runtime. Had to reimplement the `IsMessageComplete` behavior on `PipeTransmissionMode.Byte` either by pre-sending bytes count (how much byte will a full message be independent of buffer size), or sending separator byte sequence between messages.
+
+[not-supported]: <https://github.com/dotnet/runtime/blob/main/src/libraries/System.IO.Pipes/ref/System.IO.Pipes.cs#L145-L146>
+[so-use-stub-exe]: <https://stackoverflow.com/a/8434682/1859959>
+[linux-background]: <https://linux.die.net/man/1/bash>
+[nohup]: <https://linux.die.net/man/1/nohup>
+[windows-background]: <https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/start>
+[netcoresdkruntimeidentifier-information]: <https://docs.microsoft.com/en-us/dotnet/core/native-interop/expose-components-to-com>
+[default-runtime-id]: <https://github.com/dotnet/sdk/blob/main/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.RuntimeIdentifierInference.targets>
+[publish-self-contained]: <https://docs.microsoft.com/en-us/dotnet/core/deploying/#publish-self-contained>
+[framework-dependent]: <https://docs.microsoft.com/en-us/dotnet/core/deploying/#publish-framework-dependent>
+[reattach-debugger]: <https://github.com/dotnet/runtime/issues/2688#issuecomment-370030584>
+[rename-the-running-process-is-not-possible]: <https://github.com/dotnet/runtime/issues/2688>
+[dotnet-application-publishing-overview]: <https://docs.microsoft.com/en-us/dotnet/core/deploying/>
+[platform-compatibility-analyzer]: <https://docs.microsoft.com/en-us/dotnet/standard/analyzers/platform-compat-analyzer>
+[runtime-identifier]: <https://docs.microsoft.com/en-us/dotnet/core/rid-catalog>

user/mgarai/20210921-writing-a-dotnet-tool.md

@@ -0,0 +1,181 @@
+---
+title: "Writing a dotnet tool"
+categories: "dotnet,tool,cli,fsharp"
+abstract: "Writing a dotnet tool with help of the dotnet CLI"
+identity: "-1,-1"
+---
+
+## Intro
+
+Writing a dotnet tool from scratch have a lot of support from the dotnet CLI. With the following simple steps you can have a tool from the dotnet infrastructure.
+
+## Steps
+
+Create the console application for the tool:
+```ps1
+> dotnet new console -lang F# -n <tool-name>
+> cd <tool-name>
+```
+
+For the `tool-name` check the next section
+
+(Optional) Add a solution file for the project:
+```ps1
+> dotnet new sln
+> dotnet sln add <tool-name>.fsproj
+```
+
+Check build
+```ps1
+> dotnet build
+```
+
+Edit Program.fs. A dotnet tool is basically a console application, so implement what you like.
+
+Add necessary variables for the `.fsproj`. You should see something like this:
+```xml
+<Project Sdk="Microsoft.NET.Sdk">
+
+  <PropertyGroup>
+    <OutputType>Exe</OutputType>
+    <TargetFramework>net5.0</TargetFramework>
+    <RootNamespace><tool_name></RootNamespace>
+    <WarnOn>3390;$(WarnOn)</WarnOn>
+    <!-- these 2 are the added variables -->
+    <PackAsTool>true</PackAsTool>
+    <ToolCommandName><tool-name></ToolCommandName>
+  </PropertyGroup>
+
+  <ItemGroup>
+    <Compile Include="Program.fs" />
+  </ItemGroup>
+
+</Project>
+```
+
+Add `tool-manifest`:
+```ps1
+> dotnet new tool-manifest
+```
+
+Edit `.config\dotnet-tools.json` (the tool manifest). Specify the version and available commands like this:
+```json
+{
+  "version": 1,
+  "isRoot": true,
+  "tools": {
+    "<tool-name>": {
+      "version": "1.0.0",
+      "commands": [ <available-commands> ]
+    }
+  }
+}
+```
+
+Run pack
+```ps1
+> dotnet pack
+> cd bin/Debug
+```
+
+Install the new tool globally. Here the `tool-name` is not the `.nupkg` file name. It's just the `<tool-name>` given at `dotnet new -n <tool-name>`.
+```ps1
+> dotnet tool install -g --add-source . <tool-name>
+```
+
+Verify installation:
+```ps1
+> cd %USERPROFILE%\.dotnet\tools
+or
+> cd $HOME/.dotnet/tools
+---
+> dir
+or 
+> ls
+```
+
+You have a dotnet tool installed.
+
+There is also a [tutorial][tutorial] for that in the `Microsoft` documentation.
+
+## Name of the executable. \<ToolCommandName /\>
+
+Before just creating the console application with the come up name for the tool, consider if `dotnet-` prefix should be added or not.
+
+It's important to understand if you issue command
+```ps1
+> dotnet xy
+```
+in the dotnet ecosystem it's searching for a tool named `xy` prefixed by `dotnet-` and runs it. If anywhere in your `PATH` variable's directories there is a `dotnet-xy` it will run it. It's just a nice to have to install them where `dotnet install` puts `dotnet-xy`.
+
+The installation is versioned. At the installation destination there will be subdirectories for the installed versions like that:
+
+![](/assets/store-folder.png)
+
+The important question is the usage of the tool. For `dotnet xy` go with `dotnet-xy`. For `xy` go with `xy`.
+
+Choosing between the 2 is first step, so the main directory at `dotnet new console -n xy` can be in par with the final executable's name.
+
+For more information go to [this tutorial][custom-location].
+
+### ToolCommandName
+
+The `<AssemblyName>` variable in the `.csproj`/`.fsproj` file doesn't set the executable's name when installed into tools. `<ToolCommandName>` does. `dotnet pack` will use `<ToolCommandName>` which goes into 
+`%USERPROFILE%\.dotnet\tools` or `$HOME/.dotnet/tools` or where `--tool-path` points to in `dotnet tool install --tool-path ...`. All should be in the `PATH` environment variable (be careful, `--tool-path` is not by default).
+
+## FAQ
+**Question:** I changed the code, but no change is visible in the installed tool. Why?
+
+**Answer:** Have you repacked the changes?
+```ps1
+> dotnet pack
+> cd bin/Debug
+> dotnet tool install -g --add-source . <tool-name>
+```
+
+Have you changed version, and forgot to bump it accidentally?
+```ps1
+> dotnet tool install -g --add-source . --version 1.0.0 <tool-name>
+instead of
+> dotnet tool install -g --add-source . --version 1.1.0 <tool-name>
+```
+
+###
+**Q:** I have the following error:
+```
+error NU1101: Unable to find package <tool-name>. No packages exist with this id in source(s): dotnet-websharper-GitHub, fsc feed, Microsoft Visual Studio Offline Packages, nuget.org
+The tool package could not be restored.
+Tool '<tool-name>' failed to install. This failure may have been caused by:
+
+* You are attempting to install a preview release and did not use the --version option to specify the version.
+* A package by this name was found, but it was not a .NET tool.
+* The required NuGet feed cannot be accessed, perhaps because of an Internet connection problem.
+* You mistyped the name of the tool.
+
+For more reasons, including package naming enforcement, visit https://aka.ms/failure-installing-tool
+```
+How can I solve it?
+
+**A:** This can mean anything. Usually the problem is missing `--add-source <folder-to-.nupkg>`, or using the file name instead of `[PACKAGE_NAME]`.
+```ps1
+> dotnet tool install -g --version 1.0.0 <tool-name>.1.0.0.nupkg
+instead of
+> dotnet tool install -g --add-source . --version 1.0.0 <tool-name>
+```
+###
+**Q:** Can't find the executable in `%USERPROFILE%\.dotnet\tools`. Why?
+
+**A:** Have you provided `-g` flag?
+```ps1
+> dotnet tool install -g ...
+```
+###
+**Q:** `dotnet tool list` doesn't list what I have provided in the tools manifest. Why?
+
+**A:** `dotnet tool list` also differentiate between global and local tools. Have you added `-g` flag for your installed global tool?
+```ps1
+> dotnet tool list -g
+```
+
+[tutorial]: <https://docs.microsoft.com/en-us/dotnet/core/tools/global-tools-how-to-create>
+[custom-location]: <https://docs.microsoft.com/en-us/dotnet/core/tools/global-tools-how-to-use#use-the-tool-as-a-global-tool-installed-in-a-custom-location>