Skip to content

ka-de/dataset-tools

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

dataset-tools

Features


check

This versatile Rust program provides various checking and analysis functionalities for codebases and datasets to help maintain quality and consistency. It offers the following key features:

Attribute Scanning

Scans Rust files for built-in attributes, highlights them and gives you the whole list.

Multiline Detection

Find multiple lines in text files, this can cause issues with certain training tasks where multiple lines means that the captions are picked in a random order.

Optimization Verification

Analyzes Cargo.toml files for correct optimization settings.

Pedantic Warning Check

Ensure Rust files have the attribute set for pedantic warnings in clippy.

With more things to come, eventually!

Release Build


cargo build --workspace -Z build-std --target x86_64-pc-windows-msvc --release

Run clippy to Fix Warnings


cargo clippy --workspace --all-targets

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages