Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding plugin DupFileManager, and updating RenameFile and FileMonitor. #422

Merged
merged 39 commits into from
Aug 28, 2024

Conversation

David-Maisonave
Copy link
Contributor

@David-Maisonave David-Maisonave commented Aug 28, 2024

  • Adding new plugin DupFileManager

    • Has the following Task options:
      • Tag Duplicates - Set tag DuplicateMarkForDeletion to the duplicates with lower resolution, duration, file name length, or black list path.
      • Delete Tagged Duplicates - Delete scenes having DuplicateMarkForDeletion tag.
      • Delete Duplicates - Delete duplicate scenes. Performs deletion without first tagging.
    • Features:
      • Optionally merge metadata (tags, performers, studios, etc...) from duplicate to the primary (file-to-keep), before deleting duplicate files.
      • Has a 3 tier path selection to determine which duplicates to keep, and which should be candidates for deletions.
        • Whitelist - List of paths NOT to be deleted.
          • E.g. C:\Favorite,E:\MustKeep\
        • Gray-List - List of preferential paths to determine which duplicate should be the primary.
          • E.g. C:\2nd_Favorite,H:\ShouldKeep\
        • Blacklist - List of LEAST preferential paths to determine primary candidates for deletion.
          • E.g. C:\Downloads,F:\DeleteMeFirst\
      • Max Dup Process - This option allows users to run the tasks with a limited number of files. This allows user to test out the tag/deletion process at a limited scope before running it against the full Stash library.
  • Updated FileMonitor

    • Added logic to stop running multiple scan jobs.
    • Added API Key option. Needed for user using usernames to access Stash.
    • Added validation check for scheduled plugins
    • Separated unit testing into two parts.
    • Increase max log file size.
    • Set status logging to debug level.
    • Added UI option to enable DupFileManager delete duplicate task in the scheduler.
    • Added "every" option for every day of the week in the scheduler.
    • Added scheduler option to start plugin without the Task Scheduler.
    • Changed the format for plugins in the scheduler.
  • RenameFile Updates

    • Fixed bug that was adding duplicate resolution
    • Fixed issue with Studio triggering folder being created on rename.
    • Replace path separators with OS independent {os.sep}.
    • Integrated changes to use StashPluginHelper class, and stream line the code.
    • Put rename back as default method over move.
    • Removed functions which are already included in stashapi and StashPluginHelper.

Where's was an (if not) condition which should have been an (if) condition. This caused issues when renaming a file without populating the title field and with an associated performer.
Added new plugin ChangeFileMonitor.
Made following changes to RenameFile plugin.
	Added fields galleries, resolution, and width.
	Fixed bug associated with studio.
	Added logic to limit the log file size.
	Added logic to only get [Change success] logging when no errors occurs.
	Change default fields to include studio.
	Added postfix styles, which was mainly needed to properly format resolution field, but can be used for the other fields.
	Consolidated [Include Existing Key Field] options into one option.
	Cleaned up code and updated version
Added new plugin ChangeFileMonitor.
Made following changes to RenameFile plugin.
	Added fields galleries, resolution, and width.
	Fixed bug associated with studio.
	Added logic to limit the log file size.
	Added logic to only get [Change success] logging when no errors occurs.
	Change default fields to include studio.
	Added postfix styles, which was mainly needed to properly format resolution field, but can be used for the other fields.
	Consolidated [Include Existing Key Field] options into one option.
	Cleaned up code and updated version
Removed unused UI variable endpoint.
Added logic that allows ChangeFileMonitor to run as a script.
Updated README.md file to explain how to run it as a script.
Can start FileMonitor from Stash UI as a service.
Stop FileMonitor from the UI.
Scheduler service for multiple UI task.
Scheduler service for plugins.
UI option to trim backup DB files.
Config file option to exclude paths.
Config file option to only include paths.
Config file option to only include specified file types.

Very minor changes to RenameFile plugin.
1. Fixed bug that added duplicate resolution, width, height , scene_date, video_codec, and frame_rate when title is not populated.
2. Added excludeTags field, and pre-populated it with DuplicateMarkForDeletion and DuplicateWhitelistFile to avoid conflicts with up coming plugin DupFileManager.
Added API Key logic to allow FileMonitor to work in service mode when user has configured a Stash username and password.
To slim down the main config, moved self_unit_test and task_examples to their own configuration files.
Added more self_unit_test, and a selfUnitTest activation field that is normally false.
When field is true, it turns on the task for unit testing.
Change default rename using Move, in order to avoid access issues when scene is being played.
100's of file changes at the same time caused FileMonitor to run many dozens of scan jobs.
Added logic to have FileMonitor delay new scan jobs while last scan job is still running.
Separated unit testing into two parts.
Increase max log file size.
Set status logging to debug level.
Added UI option to enable DupFileManager delete duplicate task  in the scheduler.
Added "every" option for every day of the week in the scheduler.
Added scheduler option to start plugin without the Task Scheduler.
Changed the format for plugins in the scheduler.
Fixed issue with Studio triggering folder being created on rename.
Integrated changes to use StashPluginHelper class, and stream line the code.
Put rename back as default method over move.
Removed functions which are already included in stashapi and StashPluginHelper.
Added option to avoid "Delete Tagged Duplicates" without turnOnSchedulerDeleteDup enabled.
@DogmaDragon DogmaDragon merged commit 97999c1 into stashapp:main Aug 28, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants