Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve ccache instructions Fixes #4215 #4218

Merged
merged 1 commit into from
Sep 17, 2024
Merged

Improve ccache instructions Fixes #4215 #4218

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 12 additions & 72 deletions docs/build-speed.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,47 +72,8 @@ We suggest to use [**ccache**](https://ccache.dev/) to cache the compilation of
Ccache works by wrapping the C++ compilers, storing the compilation results, and skipping the compilation
if an intermediate compilation result was originally stored.

To install it, you can follow the [official installation instructions](https://github.com/ccache/ccache/blob/master/doc/INSTALL.md).

On macOS, we can install ccache with `brew install ccache`.
Once installed you can configure it as follows to cache NDK compile results:

```
ln -s $(which ccache) /usr/local/bin/gcc
ln -s $(which ccache) /usr/local/bin/g++
ln -s $(which ccache) /usr/local/bin/cc
ln -s $(which ccache) /usr/local/bin/c++
ln -s $(which ccache) /usr/local/bin/clang
ln -s $(which ccache) /usr/local/bin/clang++
```
Comment on lines -80 to -87
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those instructions are needed for Android, why are you removing them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, I was focussed on the iOS use cache and given the new script calls ccache directly I assumed they weren't needed.

I can create another PR to put it back. However is it definitely needed. I noticed this in the cmake files
https://github.com/facebook/react-native/blob/main/packages/react-native/ReactAndroid/src/main/jni/CMakeLists.txt#L16-L21
https://github.com/facebook/react-native/blob/main/packages/react-native/ReactAndroid/src/main/jni/CMakeLists.txt#L16-L21

I can do some testing?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we would have to verify if it's still needed or not

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm so I can't even get ccache to work on android.
From my testing the ndk compiler gets used
/home/johnf/opt/Android/SDK/ndk/26.1.10909125/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++


This will create symbolic links to `ccache` inside the `/usr/local/bin/` which are called `gcc`, `g++`, and so on.

This works as long as `/usr/local/bin/` comes first than `/usr/bin/` inside your `$PATH` variable, which is the default.

You can verify that it works using the `which` command:

```
$ which gcc
/usr/local/bin/gcc
```

If the results is `/usr/local/bin/gcc`, then you're effectively calling `ccache` which will wrap the `gcc` calls.

:::caution
Please note that this setup of `ccache` will affect all the compilations that you're running on your machine, not only those related to React Native. Use it at your own risk. If you're failing to install/compile other software, this might be the reason. If that is the case, you can remove the symlink you created with:

```
unlink /usr/local/bin/gcc
unlink /usr/local/bin/g++
unlink /usr/local/bin/cc
unlink /usr/local/bin/c++
unlink /usr/local/bin/clang
unlink /usr/local/bin/clang++
```

to revert your machine to the original status and use the default compilers.
:::
Ccache is available in the package manager for most operating systems. On macOS, we can install ccache with `brew install ccache`.
Or you can follow the [official installation instructions](https://github.com/ccache/ccache/blob/master/doc/INSTALL.md) to install from source.

You can then do two clean builds (e.g. on Android you can first run `yarn react-native run-android`, delete the `android/app/build` folder and run the first command once more). You will notice that the second build was way faster than the first one (it should take seconds rather than minutes).
While building, you can verify that `ccache` works correctly and check the cache hits/miss rate `ccache -s`
Expand All @@ -139,44 +100,23 @@ Should you need to wipe your cache, you can do so with `ccache --clear`

#### XCode Specific Setup

To make sure `ccache` works correctly with iOS and XCode, you need to follow a couple of extra steps:

1. You must alter the way Xcode and `xcodebuild` call for the compiler command. By default they use _fully specified paths_ to the compiler binaries, so the symbolic links installed in `/usr/local/bin` will not be used. You may configure Xcode to use _relative_ names for the compilers using either of these two options:
To make sure `ccache` works correctly with iOS and XCode, you need to enable React Native support for ccache in `ios/Podfile`.

- environment variables prefixed on the command line if you use a direct command line: `CLANG=clang CLANGPLUSPLUS=clang++ LD=clang LDPLUSPLUS=clang++ xcodebuild <rest of xcodebuild command line>`
- A `post_install` section in your `ios/Podfile` that alters the compiler in your Xcode workspace during the `pod install` step:
Open `ios/Podfile` in your editor and uncomment the `ccache_enabled` line.

```ruby
post_install do |installer|
react_native_post_install(installer)

# ...possibly other post_install items here

installer.pods_project.targets.each do |target|
target.build_configurations.each do |config|
# Using the un-qualified names means you can swap in different implementations, for example ccache
config.build_settings["CC"] = "clang"
config.build_settings["LD"] = "clang"
config.build_settings["CXX"] = "clang++"
config.build_settings["LDPLUSPLUS"] = "clang++"
end
end

__apply_Xcode_12_5_M1_post_install_workaround(installer)
# https://github.com/facebook/react-native/blob/main/packages/react-native/scripts/react_native_pods.rb#L197-L202
react_native_post_install(
installer,
config[:reactNativePath],
:mac_catalyst_enabled => false,
# TODO: Uncomment the line below
:ccache_enabled => true
)
end
```

2. You need a ccache configuration that allows for a certain level of sloppiness and cache behavior such that ccache registers cache hits during Xcode compiles. The ccache configuration variables that are different from standard are as follows if configured by environment variable:

```bash
export CCACHE_SLOPPINESS=clang_index_store,file_stat_matches,include_file_ctime,include_file_mtime,ivfsoverlay,pch_defines,modules,system_headers,time_macros
export CCACHE_FILECLONE=true
export CCACHE_DEPEND=true
export CCACHE_INODECACHE=true
```

The same may be configured in a `ccache.conf` file or any other mechanism ccache provides. More on this can be found in the [official ccache manual](https://ccache.dev/manual/4.3.html).

#### Using this approach on a CI

Ccache uses the `/Users/$USER/Library/Caches/ccache` folder on macOS to store the cache.
Expand Down
84 changes: 12 additions & 72 deletions website/versioned_docs/version-0.74/build-speed.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,47 +72,8 @@ We suggest to use [**ccache**](https://ccache.dev/) to cache the compilation of
Ccache works by wrapping the C++ compilers, storing the compilation results, and skipping the compilation
if an intermediate compilation result was originally stored.

To install it, you can follow the [official installation instructions](https://github.com/ccache/ccache/blob/master/doc/INSTALL.md).

On macOS, we can install ccache with `brew install ccache`.
Once installed you can configure it as follows to cache NDK compile results:

```
ln -s $(which ccache) /usr/local/bin/gcc
ln -s $(which ccache) /usr/local/bin/g++
ln -s $(which ccache) /usr/local/bin/cc
ln -s $(which ccache) /usr/local/bin/c++
ln -s $(which ccache) /usr/local/bin/clang
ln -s $(which ccache) /usr/local/bin/clang++
```

This will create symbolic links to `ccache` inside the `/usr/local/bin/` which are called `gcc`, `g++`, and so on.

This works as long as `/usr/local/bin/` comes first than `/usr/bin/` inside your `$PATH` variable, which is the default.

You can verify that it works using the `which` command:

```
$ which gcc
/usr/local/bin/gcc
```

If the results is `/usr/local/bin/gcc`, then you're effectively calling `ccache` which will wrap the `gcc` calls.

:::caution
Please note that this setup of `ccache` will affect all the compilations that you're running on your machine, not only those related to React Native. Use it at your own risk. If you're failing to install/compile other software, this might be the reason. If that is the case, you can remove the symlink you created with:

```
unlink /usr/local/bin/gcc
unlink /usr/local/bin/g++
unlink /usr/local/bin/cc
unlink /usr/local/bin/c++
unlink /usr/local/bin/clang
unlink /usr/local/bin/clang++
```

to revert your machine to the original status and use the default compilers.
:::
Ccache is available in the package manager for most operating systems. On macOS, we can install ccache with `brew install ccache`.
Or you can follow the [official installation instructions](https://github.com/ccache/ccache/blob/master/doc/INSTALL.md) to install from source.

You can then do two clean builds (e.g. on Android you can first run `yarn react-native run-android`, delete the `android/app/build` folder and run the first command once more). You will notice that the second build was way faster than the first one (it should take seconds rather than minutes).
While building, you can verify that `ccache` works correctly and check the cache hits/miss rate `ccache -s`
Expand All @@ -139,44 +100,23 @@ Should you need to wipe your cache, you can do so with `ccache --clear`

#### XCode Specific Setup

To make sure `ccache` works correctly with iOS and XCode, you need to follow a couple of extra steps:

1. You must alter the way Xcode and `xcodebuild` call for the compiler command. By default they use _fully specified paths_ to the compiler binaries, so the symbolic links installed in `/usr/local/bin` will not be used. You may configure Xcode to use _relative_ names for the compilers using either of these two options:
To make sure `ccache` works correctly with iOS and XCode, you need to enable React Native support for ccache in `ios/Podfile`.

- environment variables prefixed on the command line if you use a direct command line: `CLANG=clang CLANGPLUSPLUS=clang++ LD=clang LDPLUSPLUS=clang++ xcodebuild <rest of xcodebuild command line>`
- A `post_install` section in your `ios/Podfile` that alters the compiler in your Xcode workspace during the `pod install` step:
Open `ios/Podfile` in your editor and uncomment the `ccache_enabled` line.

```ruby
post_install do |installer|
react_native_post_install(installer)

# ...possibly other post_install items here

installer.pods_project.targets.each do |target|
target.build_configurations.each do |config|
# Using the un-qualified names means you can swap in different implementations, for example ccache
config.build_settings["CC"] = "clang"
config.build_settings["LD"] = "clang"
config.build_settings["CXX"] = "clang++"
config.build_settings["LDPLUSPLUS"] = "clang++"
end
end

__apply_Xcode_12_5_M1_post_install_workaround(installer)
# https://github.com/facebook/react-native/blob/main/packages/react-native/scripts/react_native_pods.rb#L197-L202
react_native_post_install(
installer,
config[:reactNativePath],
:mac_catalyst_enabled => false,
# TODO: Uncomment the line below
:ccache_enabled => true
)
end
```

2. You need a ccache configuration that allows for a certain level of sloppiness and cache behavior such that ccache registers cache hits during Xcode compiles. The ccache configuration variables that are different from standard are as follows if configured by environment variable:

```bash
export CCACHE_SLOPPINESS=clang_index_store,file_stat_matches,include_file_ctime,include_file_mtime,ivfsoverlay,pch_defines,modules,system_headers,time_macros
export CCACHE_FILECLONE=true
export CCACHE_DEPEND=true
export CCACHE_INODECACHE=true
```

The same may be configured in a `ccache.conf` file or any other mechanism ccache provides. More on this can be found in the [official ccache manual](https://ccache.dev/manual/4.3.html).

#### Using this approach on a CI

Ccache uses the `/Users/$USER/Library/Caches/ccache` folder on macOS to store the cache.
Expand Down
84 changes: 12 additions & 72 deletions website/versioned_docs/version-0.75/build-speed.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,47 +72,8 @@ We suggest to use [**ccache**](https://ccache.dev/) to cache the compilation of
Ccache works by wrapping the C++ compilers, storing the compilation results, and skipping the compilation
if an intermediate compilation result was originally stored.

To install it, you can follow the [official installation instructions](https://github.com/ccache/ccache/blob/master/doc/INSTALL.md).

On macOS, we can install ccache with `brew install ccache`.
Once installed you can configure it as follows to cache NDK compile results:

```
ln -s $(which ccache) /usr/local/bin/gcc
ln -s $(which ccache) /usr/local/bin/g++
ln -s $(which ccache) /usr/local/bin/cc
ln -s $(which ccache) /usr/local/bin/c++
ln -s $(which ccache) /usr/local/bin/clang
ln -s $(which ccache) /usr/local/bin/clang++
```

This will create symbolic links to `ccache` inside the `/usr/local/bin/` which are called `gcc`, `g++`, and so on.

This works as long as `/usr/local/bin/` comes first than `/usr/bin/` inside your `$PATH` variable, which is the default.

You can verify that it works using the `which` command:

```
$ which gcc
/usr/local/bin/gcc
```

If the results is `/usr/local/bin/gcc`, then you're effectively calling `ccache` which will wrap the `gcc` calls.

:::caution
Please note that this setup of `ccache` will affect all the compilations that you're running on your machine, not only those related to React Native. Use it at your own risk. If you're failing to install/compile other software, this might be the reason. If that is the case, you can remove the symlink you created with:

```
unlink /usr/local/bin/gcc
unlink /usr/local/bin/g++
unlink /usr/local/bin/cc
unlink /usr/local/bin/c++
unlink /usr/local/bin/clang
unlink /usr/local/bin/clang++
```

to revert your machine to the original status and use the default compilers.
:::
Ccache is available in the package manager for most operating systems. On macOS, we can install ccache with `brew install ccache`.
Or you can follow the [official installation instructions](https://github.com/ccache/ccache/blob/master/doc/INSTALL.md) to install from source.

You can then do two clean builds (e.g. on Android you can first run `yarn react-native run-android`, delete the `android/app/build` folder and run the first command once more). You will notice that the second build was way faster than the first one (it should take seconds rather than minutes).
While building, you can verify that `ccache` works correctly and check the cache hits/miss rate `ccache -s`
Expand All @@ -139,44 +100,23 @@ Should you need to wipe your cache, you can do so with `ccache --clear`

#### XCode Specific Setup

To make sure `ccache` works correctly with iOS and XCode, you need to follow a couple of extra steps:

1. You must alter the way Xcode and `xcodebuild` call for the compiler command. By default they use _fully specified paths_ to the compiler binaries, so the symbolic links installed in `/usr/local/bin` will not be used. You may configure Xcode to use _relative_ names for the compilers using either of these two options:
To make sure `ccache` works correctly with iOS and XCode, you need to enable React Native support for ccache in `ios/Podfile`.

- environment variables prefixed on the command line if you use a direct command line: `CLANG=clang CLANGPLUSPLUS=clang++ LD=clang LDPLUSPLUS=clang++ xcodebuild <rest of xcodebuild command line>`
- A `post_install` section in your `ios/Podfile` that alters the compiler in your Xcode workspace during the `pod install` step:
Open `ios/Podfile` in your editor and uncomment the `ccache_enabled` line.

```ruby
post_install do |installer|
react_native_post_install(installer)

# ...possibly other post_install items here

installer.pods_project.targets.each do |target|
target.build_configurations.each do |config|
# Using the un-qualified names means you can swap in different implementations, for example ccache
config.build_settings["CC"] = "clang"
config.build_settings["LD"] = "clang"
config.build_settings["CXX"] = "clang++"
config.build_settings["LDPLUSPLUS"] = "clang++"
end
end

__apply_Xcode_12_5_M1_post_install_workaround(installer)
# https://github.com/facebook/react-native/blob/main/packages/react-native/scripts/react_native_pods.rb#L197-L202
react_native_post_install(
installer,
config[:reactNativePath],
:mac_catalyst_enabled => false,
# TODO: Uncomment the line below
:ccache_enabled => true
)
end
```

2. You need a ccache configuration that allows for a certain level of sloppiness and cache behavior such that ccache registers cache hits during Xcode compiles. The ccache configuration variables that are different from standard are as follows if configured by environment variable:

```bash
export CCACHE_SLOPPINESS=clang_index_store,file_stat_matches,include_file_ctime,include_file_mtime,ivfsoverlay,pch_defines,modules,system_headers,time_macros
export CCACHE_FILECLONE=true
export CCACHE_DEPEND=true
export CCACHE_INODECACHE=true
```

The same may be configured in a `ccache.conf` file or any other mechanism ccache provides. More on this can be found in the [official ccache manual](https://ccache.dev/manual/4.3.html).

#### Using this approach on a CI

Ccache uses the `/Users/$USER/Library/Caches/ccache` folder on macOS to store the cache.
Expand Down