Skip to content

Commit

Permalink
Use raw CLI inputs for random seed instead of derived inputs and use …
Browse files Browse the repository at this point in the history
…proper hashing algorithm

Signed-off-by: Alex Saveau <[email protected]>
  • Loading branch information
SUPERCILEX committed Dec 23, 2023
1 parent 18dacba commit 106d7db
Show file tree
Hide file tree
Showing 191 changed files with 653,965 additions and 752,033 deletions.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Generate a reproducibly random tree in the current directory with *approximately
```console
$ ftzz ./simple -n 1M
About 1,000,000 files will be generated in approximately 1,000 directories distributed across a tree of maximum depth 5 where each directory contains approximately 4 other directories.
Created 1,002,953 files across 1,193 directories.
Created 1,003,229 files across 1,259 directories.

```

Expand All @@ -33,7 +33,7 @@ Generate *exactly* 1 million files:
```console
$ ftzz ./exact -en 1M
Exactly 1,000,000 files will be generated in approximately 1,000 directories distributed across a tree of maximum depth 5 where each directory contains approximately 4 other directories.
Created 1,000,000 files across 1,193 directories.
Created 1,000,000 files across 1,259 directories.

```

Expand All @@ -42,7 +42,7 @@ Generate ~10_000 files with ~1 MB of random data spread across them:
```console
$ ftzz ./with_data -n 10K -b 1M
About 10,000 files will be generated in approximately 1,000 directories distributed across a tree of maximum depth 5 where each directory contains approximately 4 other directories. Each file will contain approximately 100 bytes of random data.
Created 9,652 files (960.1 KB) across 930 directories.
Created 9,312 files (924.6 KB) across 1,570 directories.

```

Expand All @@ -53,11 +53,11 @@ change the starting seed:
```console
$ ftzz ./unseeded -n 100
About 100 files will be generated in approximately 100 directories distributed across a tree of maximum depth 5 where each directory contains approximately 3 other directories.
Created 32 files across 214 directories.
Created 45 files across 198 directories.

$ ftzz ./seeded -n 100 --seed 42 # Or $RANDOM
About 100 files will be generated in approximately 100 directories distributed across a tree of maximum depth 5 where each directory contains approximately 3 other directories.
Created 78 files across 59 directories.
Created 83 files across 110 directories.

```

Expand Down
14 changes: 14 additions & 0 deletions api.golden
Original file line number Diff line number Diff line change
Expand Up @@ -166,8 +166,18 @@ pub struct ftzz::NumFilesWithRatio
impl ftzz::NumFilesWithRatio
pub fn ftzz::NumFilesWithRatio::from_num_files(num_files: core::num::nonzero::NonZeroU64) -> Self
pub fn ftzz::NumFilesWithRatio::new(num_files: core::num::nonzero::NonZeroU64, file_to_dir_ratio: core::num::nonzero::NonZeroU64) -> core::result::Result<Self, ftzz::NumFilesWithRatioError>
impl core::clone::Clone for ftzz::NumFilesWithRatio
pub fn ftzz::NumFilesWithRatio::clone(&self) -> ftzz::NumFilesWithRatio
impl core::cmp::Eq for ftzz::NumFilesWithRatio
impl core::cmp::PartialEq for ftzz::NumFilesWithRatio
pub fn ftzz::NumFilesWithRatio::eq(&self, other: &ftzz::NumFilesWithRatio) -> bool
impl core::fmt::Debug for ftzz::NumFilesWithRatio
pub fn ftzz::NumFilesWithRatio::fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result
impl core::hash::Hash for ftzz::NumFilesWithRatio
pub fn ftzz::NumFilesWithRatio::hash<__H: core::hash::Hasher>(&self, state: &mut __H)
impl core::marker::Copy for ftzz::NumFilesWithRatio
impl core::marker::StructuralEq for ftzz::NumFilesWithRatio
impl core::marker::StructuralPartialEq for ftzz::NumFilesWithRatio
impl core::marker::Send for ftzz::NumFilesWithRatio
impl core::marker::Sync for ftzz::NumFilesWithRatio
impl core::marker::Unpin for ftzz::NumFilesWithRatio
Expand All @@ -181,6 +191,10 @@ pub fn ftzz::NumFilesWithRatio::try_from(value: U) -> core::result::Result<T, <T
impl<T, U> core::convert::TryInto<U> for ftzz::NumFilesWithRatio where U: core::convert::TryFrom<T>
pub type ftzz::NumFilesWithRatio::Error = <U as core::convert::TryFrom<T>>::Error
pub fn ftzz::NumFilesWithRatio::try_into(self) -> core::result::Result<U, <U as core::convert::TryFrom<T>>::Error>
impl<T> alloc::borrow::ToOwned for ftzz::NumFilesWithRatio where T: core::clone::Clone
pub type ftzz::NumFilesWithRatio::Owned = T
pub fn ftzz::NumFilesWithRatio::clone_into(&self, target: &mut T)
pub fn ftzz::NumFilesWithRatio::to_owned(&self) -> T
impl<T> core::any::Any for ftzz::NumFilesWithRatio where T: 'static + core::marker::Sized
pub fn ftzz::NumFilesWithRatio::type_id(&self) -> core::any::TypeId
impl<T> core::borrow::Borrow<T> for ftzz::NumFilesWithRatio where T: core::marker::Sized
Expand Down
16 changes: 8 additions & 8 deletions src/generator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ use std::{
cmp::max,
fmt::Write,
fs::create_dir_all,
hash::{DefaultHasher, Hash, Hasher},
num::{NonZeroU64, NonZeroUsize},
path::PathBuf,
process::ExitCode,
Expand Down Expand Up @@ -38,7 +39,7 @@ pub enum Error {
RuntimeCreation,
}

#[derive(Debug)]
#[derive(Copy, Clone, Eq, PartialEq, Hash, Debug)]
pub struct NumFilesWithRatio {
num_files: NonZeroU64,
file_to_dir_ratio: NonZeroU64,
Expand Down Expand Up @@ -246,7 +247,11 @@ fn validated_options(
bytes_per_file,
dirs_per_dir,
max_depth,
seed,
seed: {
let mut hasher = DefaultHasher::new();
(num_files_with_ratio, max_depth, seed).hash(&mut hasher);
hasher.finish()
},
human_info: HumanInfo {
dirs_per_dir: dirs_per_dir.round() as usize,
total_dirs: num_dirs.round() as usize,
Expand Down Expand Up @@ -405,12 +410,7 @@ async fn run_generator_async(
let bytes = NonZeroU64::new(bytes);
let dynamic = DynamicGenerator {
num_dirs_distr: truncatable_normal(dirs_per_dir),
random: {
let seed = ((files.get().wrapping_add(max_depth.into()) as f64 * dirs_per_dir) as u64)
.wrapping_add(seed);
log!(Level::Debug, "Starting seed: {seed}");
Xoshiro256PlusPlus::seed_from_u64(seed)
},
random: Xoshiro256PlusPlus::seed_from_u64(seed),

bytes: bytes.map(|_| GeneratorBytes {
num_bytes_distr: truncatable_normal(bytes_per_file),
Expand Down
6 changes: 3 additions & 3 deletions testdata/cmds/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
```console
$ ftzz -n 1K -b 100 --files-exact --bytes-exact all-exact-explicit
Exactly 1,000 files will be generated in approximately 1,000 directories distributed across a tree of maximum depth 5 where each directory contains approximately 4 other directories. Each file will contain approximately 0 bytes of random data totaling exactly 100 bytes.
Created 1,000 files (100 B) across 506 directories.
Created 1,000 files (100 B) across 983 directories.

```

Expand All @@ -21,7 +21,7 @@ Info output:
```console
$ ftzz -vv -n 1K verbose
About 1,000 files will be generated in approximately 1,000 directories distributed across a tree of maximum depth 5 where each directory contains approximately 4 other directories.
INFO [ftzz::generator] Starting config: Configuration { root_dir: "verbose", files: 1000, bytes: 0, files_exact: false, bytes_exact: false, fill_byte: None, dirs_per_dir: 3.9810717055349727, bytes_per_file: 0.0, max_depth: 5, seed: 0, human_info: HumanInfo { dirs_per_dir: 4, total_dirs: 1000, bytes_per_files: 0 } }
Created 835 files across 476 directories.
INFO [ftzz::generator] Starting config: Configuration { root_dir: "verbose", files: 1000, bytes: 0, files_exact: false, bytes_exact: false, fill_byte: None, dirs_per_dir: 3.9810717055349727, bytes_per_file: 0.0, max_depth: 5, seed: 2268264718836998093, human_info: HumanInfo { dirs_per_dir: 4, total_dirs: 1000, bytes_per_files: 0 } }
Created 578 files across 1,033 directories.

```
8 changes: 4 additions & 4 deletions testdata/cmds/numbers.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Plain nums:
```console
$ ftzz --files 1000 --max-depth 10 --ftd-ratio 100 plain
About 1,000 files will be generated in approximately 10 directories distributed across a tree of maximum depth 10 where each directory contains approximately 1 other directory.
Created 994 files across 92 directories.
Created 996 files across 42 directories.

```

Expand All @@ -12,7 +12,7 @@ SI numbers:
```console
$ ftzz --files 1K --max-depth 0.01K --ftd-ratio 0.0001M si
About 1,000 files will be generated in approximately 10 directories distributed across a tree of maximum depth 10 where each directory contains approximately 1 other directory.
Created 994 files across 92 directories.
Created 996 files across 42 directories.

```

Expand All @@ -21,7 +21,7 @@ Commas:
```console
$ ftzz --files 1,000 --max-depth 0,010 --ftd-ratio 1,0,0 commas
About 1,000 files will be generated in approximately 10 directories distributed across a tree of maximum depth 10 where each directory contains approximately 1 other directory.
Created 994 files across 92 directories.
Created 996 files across 42 directories.

```

Expand All @@ -30,6 +30,6 @@ Underscores:
```console
$ ftzz --files 1_000 --max-depth 1_0 --ftd-ratio 1_0_0 underscores
About 1,000 files will be generated in approximately 10 directories distributed across a tree of maximum depth 10 where each directory contains approximately 1 other directory.
Created 994 files across 92 directories.
Created 996 files across 42 directories.

```
Loading

0 comments on commit 106d7db

Please sign in to comment.