Skip to content

Commit

Permalink
Bugfix rust module incremental build support (#649)
Browse files Browse the repository at this point in the history
## Description

Previously, the incremental build support for rust modules was broken
because the previous attempt of updating the last modified time using
utime() was not permanent (for an unknown reason). This commit does two
things:

1. Updates the logic to use pathlib.Path's `touch()` function, which has
proven to reliably update the last modified time of the file.

2. Always touch the toml file instead of checking to see if any of the
rust files has been touched since the last build. The previous logic was
unnecessary complicated because the cargo build system will decide if a
re-build is necessary, and make will only copy the file if the efi has
changed.

- [ ] Impacts functionality?
- **Functionality** - Does the change ultimately impact how firmware
functions?
- Examples: Add a new library, publish a new PPI, update an algorithm,
...
- [ ] Impacts security?
- **Security** - Does the change have a direct security impact on an
application,
    flow, or firmware?
  - Examples: Crypto algorithm change, buffer overflow fix, parameter
    validation improvement, ...
- [ ] Breaking change?
- **Breaking change** - Will anyone consuming this change experience a
break
    in build or boot behavior?
- Examples: Add a new library class, move a module to a different repo,
call
    a function in a new library class in a pre-existing module, ...
- [ ] Includes tests?
  - **Tests** - Does the change include any explicit test code?
  - Examples: Unit tests, integration tests, robot tests, ...
- [ ] Includes documentation?
- **Documentation** - Does the change contain explicit documentation
additions
    outside direct code modifications (and comments)?
- Examples: Update readme file, add feature readme file, link to
documentation
    on an a separate Web page, ...

## How This Was Tested

1. Verified changes to a rust module propagated into the final firmware.
2. Verified changes to a rust library propagated to a rust module that
then also propagated into the final firmware.
3. Verified that making no changes to a rust module or library resulted
in cargo build skipping the re-build and the final efi remained the same
exact efi of the previous build (efi was not modified or re-copied)

This was verified on Windows and Ubuntu 22.04

## Integration Instructions

Clear your Conf folder
  • Loading branch information
Javagedes authored Dec 15, 2023
1 parent 78632a1 commit 76281f0
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 37 deletions.
5 changes: 2 additions & 3 deletions BaseTools/Conf/build_rule.template
Original file line number Diff line number Diff line change
Expand Up @@ -119,8 +119,9 @@
# 2.25 - Split OemData rule into intermediate binary and final binary rules.
# 2.26 - Rename OemData to ProductData.
# 2.27 - Add Rust build support.
# 2.28 - Remove Rust Incremental Build Support Hack

#!VERSION=2.27
#!VERSION=2.28

[C-Code-File]
<InputFile>
Expand Down Expand Up @@ -293,8 +294,6 @@
$(OUTPUT_DIR)(+)$(MODULE_NAME).efi

<Command>
# Temporary_Rust_Todo - Remove .efi files to better support Rust incremental build for now.
$(RM) $(OUTPUT_DIR)(+)*.efi
$(CARGO) make $(CARGOMAKE_FLAGS) -e RUSTFLAGS="-C link-arg=/MAP:$(DEBUG_DIR)(+)$(MODULE_NAME).map $(RUST_FLAGS)" build $(MODULE_NAME)
"$(GENFW)" -e $(MODULE_TYPE) -o $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(DEBUG_DIR)(+)$(TARGET_TRIPLE)(+)$(RUST_TARGET)(+)$(MODULE_NAME).efi $(GENFW_FLAGS)
$(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).map $(OUTPUT_DIR)(+)$(MODULE_NAME).map
Expand Down
27 changes: 0 additions & 27 deletions BaseTools/Source/Python/Common/Misc.py
Original file line number Diff line number Diff line change
Expand Up @@ -1931,30 +1931,3 @@ def CopyDict(ori_dict):
#
def RemoveCComments(ctext):
return re.sub('//.*?\n|/\*.*?\*/', '\n', ctext, flags=re.S)


# MU_CHANGE [BEGIN] - Add Rust build support

# Update all .toml file's mtime
# If toml related source file changed. update .toml file's mtime
# all toml file saved in file TomlFileListFileName
#
def UpdateTomlFileMTime(TomlFileListFileName):
try:
with open(TomlFileListFileName, 'r') as file:
lines = file.readlines()
for toml in lines:
TomlFile = toml[0:-1]
TomlSourceDir = os.path.join(PathClass(TomlFile).Dir, "src")
TomlSourceFiles = glob.glob(TomlSourceDir + "/*.rs")
LatestFile = max(TomlSourceFiles, key=os.path.getmtime)
LatestFileMTime = os.path.getmtime(LatestFile)
CurrentMTime = os.path.getmtime(str(TomlFile))
if CurrentMTime < LatestFileMTime:
os.utime(str(TomlFile), (os.path.getatime(LatestFile), LatestFileMTime))
except FileNotFoundError:
pass
except:
pass

# MU_CHANGE [END] - Add Rust build support
15 changes: 8 additions & 7 deletions BaseTools/Source/Python/build/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
from linecache import getlines
from subprocess import Popen,PIPE, STDOUT
from collections import OrderedDict, defaultdict
import pathlib

from AutoGen.PlatformAutoGen import PlatformAutoGen
from AutoGen.ModuleAutoGen import ModuleAutoGen
Expand Down Expand Up @@ -66,9 +67,6 @@
from AutoGen.IncludesAutoGen import IncludesAutoGen
from GenFds.GenFds import resetFdsGlobalVariable
from AutoGen.AutoGen import CalculatePriorityValue
# MU_CHANGE [BEGIN] - Add Rust build support
from Common.Misc import UpdateTomlFileMTime
# MU_CHANGE [END] - Add Rust build support

## standard targets of build command
gSupportedTarget = ['all', 'genc', 'genmake', 'modules', 'libraries', 'fds', 'clean', 'cleanall', 'cleanlib', 'run']
Expand Down Expand Up @@ -2345,10 +2343,13 @@ def _MultiThreadBuildPlatform(self):
EdkLogger.quiet("[cache Summary]: Makecache miss num: %s " % len(self.MakeCacheMiss))

# MU_CHANGE [BEGIN] - Add Rust build support

# if .tmol releated rust src changed, update .toml file's modified time
UpdateTomlFileMTime(os.path.join(Wa.BuildDir, 'RustFileWatch.lst'))

# Touch all toml files (to update the last modified time) so
# that make will actually see that a source file is newer than
# the .efi file and copy it to the expected directory(s).
for m in self.AllModules:
for src in m.SourceFileList:
if ".toml" in src.Path.lower():
pathlib.Path(src.Path).touch(exist_ok=True)
# MU_CHANGE [END] - Add Rust build support

for Arch in Wa.ArchList:
Expand Down

0 comments on commit 76281f0

Please sign in to comment.