Trouble building on macOS (my workarounds and suggestions)

[AlienKevin]

This library looks awesome but I had trouble building fstalign on macOS sonoma (the issues likely apply to M1 machines running ventura as well). I will outline the steps I followed to build this library, the issues encountered along the way, and temporary workarounds I developed.

1. git clone https://github.com/revdotcom/fstalign
2. mkdir build && cd build
3. cmake .. -DOPENFST_ROOT="/opt/homebrew/Cellar/openfst/1.8.2" -DDYNAMIC_OPENFST=ON
   Issue 1: The following ICU libraries were not found: uc (required)
   Reason: The path to icu4c in CMakeList.txt is not for a brew installation:
   list(APPEND CMAKE_PREFIX_PATH "/usr/local/opt/icu4c") # for Mac users
   Workaround: Run export CMAKE_PREFIX_PATH="/opt/homebrew/opt/icu4c" and export CPATH="/opt/homebrew/opt/icu4c/include" before cmake
4. make
   Issue 2: OpenFST dependency fails to build due to usage of C++17 features
   Reason: Like many other macOS users, I manage my dependencies through brew. Unfortunately, brew usually provides only the latest version of a package and the latest OpenFST version is 1.8.2, which uses C++17 features. This library uses C++14 and does not build with C++17. I can't find a straight-forward way to update the CMakeLists.txt to specify a separate C++ version for OpenFST only.
   Workaround: Download a old version of brew's openfst formula, uninstall OpenFST 1.8.2 and install the older 1.7.9 version which doesn't require C++17. This requires tweaking the old 1.7.9 ruby formula so it took me a while to figure out. See appendix for detailed steps.
   Issue 3: With issue 2 fixed, there's still a remaining issue with the Catch2 dependency.
   Reason: Older versions of Catch2 uses some x86 assembly code internally that is not compatabile with ARM. Newer versions updated to support both architectures but the version of Catch2 referenced in this library is too old and does not support ARM.
   Workaround: Remove the existing Catch2 submodule and download a newer version. I chose v2.13.8 because it supports ARM and shouldn't cause compatability issues with this library:
   cd third-party
rm -rf Catch2
git clone --branch v2.13.8 https://github.com/catchorg/Catch2
   Rerun make and it should build successfully now.

After all workarounds are applied, this library finally builds on macOS! To save future users the same trouble as I went through, I have a few suggestions:

1. Add the brew path to OpenFST in CMakeList.txt
2. Either update this library to support C++17 or update the CMakeList.txt to support C++17 for OpenFST 1.8.2
3. Accept libfst.dylib in place of libfst.so on macOS. (Not sure if this is still an issue on OpenFST 1.8.2)
4. Update Catch2 to at least v2.13.8

Appendix

Workaround to install an older version of OpenFST

Create a file named openfst.rb in the current working directory. Below is the openfst.rb script I adapted from the original formula for 1.7.9. The bottle section has to be updated to use the new syntax.

class Openfst < Formula
  desc "Library for weighted finite-state transducers"
  homepage "http://www.openfst.org/twiki/bin/view/FST/WebHome"
  url "http://openfst.org/twiki/pub/FST/FstDownload/openfst-1.7.9.tar.gz"
  sha256 "9319aeb31d1e2950ae25449884e255cc2bc9dfaf987f601590763e61a10fbdde"
  license "Apache-2.0"

  livecheck do
    url "http://www.openfst.org/twiki/bin/view/FST/FstDownload"
    regex(/href=.*?openfst[._-]v?(\d+(?:\.\d+)+)\.t/i)
  end

  bottle do
    rebuild 1
    sha256 cellar: :any, big_sur: "a0289323819255885b7d45a89e2ebd88512f8153c1c956017258a61b07c29506"
    sha256 cellar: :any, catalina: "b32fb6cb0eb43a7d8775d8bfc760c49471586eeb33797f3d44a8b53cd45dc792"
    sha256 cellar: :any, mojave: "7e5a450f383ddfeddcb7ee8d240e7db576fcc32a25c199d6a35eba40fea920d9"
    sha256 cellar: :any, high_sierra: "0635e790f390be0a97c78a434e723280339fe0f0d86ee55c4a34339840f160a7"
  end

  def install
    system "./configure", "--disable-dependency-tracking",
                          "--disable-silent-rules",
                          "--prefix=#{prefix}"
    system "make"
    system "make", "install"
  end

  test do
    (testpath/"text.fst").write <<~EOS
      0 1 a x .5
      0 1 b y 1.5
      1 2 c z 2.5
      2 3.5
    EOS

    (testpath/"isyms.txt").write <<~EOS
      <eps> 0
      a 1
      b 2
      c 3
    EOS

    (testpath/"osyms.txt").write <<~EOS
      <eps> 0
      x 1
      y 2
      z 3
    EOS

    system bin/"fstcompile", "--isymbols=isyms.txt", "--osymbols=osyms.txt", "text.fst", "binary.fst"
    assert_predicate testpath/"binary.fst", :exist?
  end
end

To uninstall 1.8.2, do:

brew remove openfst

To install the older 1.7.9 from the formula above, do:

brew install openfst.rb

Ignore the Error: Failed to load cask: openfst.rb message.

Brew will create a libfst.dylib file but this library requires a libfst.so file. You need to create a symbolic link named libfst.so to the dylib file for the compiler to find the binary.

ln -s /opt/homebrew/Cellar/openfst/1.7.9/lib/libfst.dylib /opt/homebrew/Cellar/openfst/1.7.9/lib/libfst.so

After the older version is installed, clean the CMakeCache.txt and rerun cmake using the older version:

rm CMakeCache.txt
cmake .. -DOPENFST_ROOT="/opt/homebrew/Cellar/openfst/1.7.9" -DDYNAMIC_OPENFST=ON

Reference: https://nelson.cloud/how-to-install-older-versions-of-homebrew-packages/

[qmac]

@AlienKevin Thank you so much for this thorough exploration and write up! We will start looking into your suggestions. I know that C++17 is top of mind for us and you've already given us a great blueprint for debugging going forward!

[Rehanchy]

Thanks for sharing the steps, I have tried the method to build fstalign on my macOS ventura(M2Max), the cmakelist definately is outdated for adding a wrong cmake prefix path for mac users. I can confirm that installing openfst 1.7.9 solved my problem and I succesfully built the fstalign. The other method of changing Catch2 to newer version doesn't work for me, the openfst1.8.2 got lots of error related to std:: namespace problem, which I believe is still an issue of C++ version.
So, if anyone using macOS is trying to build fstalign for now, I'd recommend you follow @AlienKevin 's step and choose the method of installing the older version of openfst.

[icodeformybhasa]

@Rehanchy did you update the bottle section for ventura? If yes what values did you use, I can't seem to find it.

[migueljette]

I confirm these steps worked for me on my most recent laptop (Apple M3 with OS X 14.5)! Nice!