-
Notifications
You must be signed in to change notification settings - Fork 273
Experimental OMake backend
OMake is one of the compilation backends for OASIS. It is extremely fast and on my machine with 10 cores builds BAP 10 times faster (in two minutes from scratch on mine 2Gh machine). It is also extremely fast on rebuilding and can redeploy BAP after a deep change in the matter of milliseconds. It also has a lot of interesting features, like incremental building, when an omake server is polling for changes in the file system and triggers rebuild as soon as you hit save. OMake also watches for the changes in dependencies outside of the source tree, so if you have any external library updated it will notice it and correctly rebuild the dependencies (no more "inconsistent assumptions").
Go to the root directory of the BAP repository and perform the following steps:
- (optional) install BAP dependencies
opam pin add . -n
opam install bap --deps-only
opam pin remove bap
# install ghidra debian package, if you haven't yet
sudo add-apt-repository ppa:ivg/ghidra -y
sudo apt install libghidra-dev -y
sudo apt install libghidra-data
- install the fixed version of oasis
opam pin add oasis https://github.com/BinaryAnalysisPlatform/oasis.git
- install omake
opam install omake
- run the specially prepared configure script
./configure-omake --with-llvm-version=10 --with-llvm-config=llvm-config-10
- everything else is as usual
make && make reinstall
There are a couple of caveats though, that is why it is not our default build system. First of all, for omake backend to work, we had to submit a few patches to OASIS. While they are pending, we need to use our own fork:
opam pin add oasis [email protected]:BinaryAnalysisPlatform/oasis.git
Another issue is that OMake backend doesn't support OASIS variables and conditional compilation. This immediately bars omake as a default backend, since we rely on the configuration variables a lot. But for the development mode, when we build all targets in the repository, it is not a big deal. As a workaround, we turn all conditional compilation into unconditional and hardcode variables for LLVM and C++ (and any other that we might need in the future) in corresponding OMakefiles.
For the same reason, we also hardcode opam as the default OCaml package manager, so this backend doesn't support opamless environments.
Another important limitation is that it is in general unsafe to use the omake backend with compilers that are earlier than 4.08. While I didn't have any problems, I can't guarantee that they do not exist :) In any case, we will soon drop 4.07 and everyone is encouraged to use 4.08 or later, especially if they are developing BAP and using omake for that.
There are also a couple of non-essential issues with omake, which we will collect here (and occasionally fix in upstream)
Yes, omake indeed builds in the same tree as the source code, polluting it with the build artifacts. Potentially, it could build in a separate folder using the virtual mount feature of OMake. But this feature is occluded by the files, generated by OASIS, which are not seen by virtual mounts. We can, theoretically, fix it on the OASIS file, but so far it is not as annoying as I would expect, especially if you're using Emacs with ido mode. Not sure why, but by default ido ignores all built artifact, you have to add only .om and .omc to the list of ignored files, e.g.,
(setq oasis-garbage-extensions '("omc" "om"))
(dolist (ext oasis-garbage-extensions)
(add-to-list 'ido-ignore-files (concat "\." ext)))
We also extended the .gitingnore file so that git didn't get overwhelmed by all the garbage. If you need to remove this garbage, then just do git clean -idx.
It could be an indentation error or an unbound variable. Just delete this file and it will be recreated (hopefully) without errors. This issue is still under investigation, looks like a race condition. Fortunately, it occurs rather rarely.
To enable directory tracking we need to pass the -w option to omake, e.g., using the following compilation command,
omake -w && make reinstall
Since OASIS doesn't support C++ sources we are specifying only C sources in the oasis specification and C++ targets are not registered in the omake backend. As a result a change to one of the C++ components remained unchanged and the libraries are not rebuilt. Mitigation, (re)compile the project after a changes in CXX_FOLDER with the following command:
(cd _build/built_plugins && rm -rf CXX_DEPS); (cd CXX_FOLDER && omake clean); omake -w && make reinstall
Where CXX_DEPS is the list of plugins that depend on the library in CXX_FOLDER, e.g.,
(cd _build/built_plugins && rm -rf llvm x86); (cd lib/bap_llvm && omake clean); omake -w && make reinstall