[23.2] Don't attempt to commit in dry_run mode

[23.2] Don't attempt to commit in dry_run mode #30706