Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to notify when synced folder overcomes set size limit #5868

Merged
merged 57 commits into from
Aug 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
75f0e53
Const autofy variables in ProcessDirectoryJob::processFileAnalyzeRemo…
claucambra Jun 27, 2023
17de46c
Clean up declaration of DiscoveryPhase::checkSelectiveSyncNewFolder b…
claucambra Jun 27, 2023
e5f399f
Move active folder size limit check to independent convenience method
claucambra Jun 27, 2023
e3dc6be
Extract processServerNew into a proper method instead of large lambda
claucambra Jun 27, 2023
7145d73
Add DiscoveryPhase::checkSelectiveSyncExistingFolder
claucambra Jun 28, 2023
5844b27
Check selective sync state directories that have been updated and now…
claucambra Jun 28, 2023
ac218b0
Use single convenience method to handle trailing slash in paths for F…
claucambra Jun 28, 2023
10dd03f
Add slot for existing folders becoming big in Folder
claucambra Jun 28, 2023
6f2e63d
Create and connect specific signal for existing folder now being disc…
claucambra Jun 28, 2023
5acacdd
Make checkFolderSizeLimit more task agnostic
claucambra Jul 3, 2023
aa20c10
Properly check folder size on server
claucambra Jul 3, 2023
e8b8211
Add a checkbox in UI to obey folder size limit for existing folders
claucambra Jul 4, 2023
b06cbf4
Add config settings for notifying existing folders going over limit
claucambra Jul 4, 2023
e6f5fb6
Hook up existing folder notification size checkbox in general settings
claucambra Jul 4, 2023
6105733
Do not notify if setting is not enabled for folders going over limit
claucambra Jul 4, 2023
83b6903
Ensure notify big folders checkbox state is changed when parent check…
claucambra Jul 4, 2023
2a4ba39
Do not notify about folders growing beyond threshold by default
claucambra Jul 4, 2023
a97bdb9
Clean up AccountSettings::refreshSelectiveSyncStatus
claucambra Jul 4, 2023
017dca2
Move findPathInList from discoveryphase to common syncjournaldb
claucambra Jul 4, 2023
e05cf76
Add specific message for existing folders exceeding warning limit in …
claucambra Jul 4, 2023
993ee6c
Remove redundant "optional" gui log functions which relly just call g…
claucambra Jul 4, 2023
2bcefd6
Add methods to Folder so that other classes can easily add folder pat…
claucambra Jul 5, 2023
7e17e92
Add convenience methods to folderman so that classes can add and remo…
claucambra Jul 5, 2023
29cc9c0
Handle WHITELIST_FOLDER and BLACKLIST_FOLDER verbs in activitylistmodel
claucambra Jul 5, 2023
10989fa
Add method to get user pointer in UserModel by AccountState pointer
claucambra Jul 5, 2023
e985367
Add method to add notification activity in activity list in User
claucambra Jul 5, 2023
fe59d4e
Ensure whitelist/blacklist procedure removes activity
claucambra Jul 5, 2023
1293106
Create more than one converted activity link, therby respecting maxac…
claucambra Jul 5, 2023
c975d40
Do not create dismiss links for notifications as these are not used a…
claucambra Jul 5, 2023
7d1fa16
Add activity for existing folder going over set limit
claucambra Jul 5, 2023
5f250a5
Ensure newly whitelisted or blacklisted paths are removed from undeci…
claucambra Jul 5, 2023
e2535d6
Prevent double notification of folder now big in activities
claucambra Jul 5, 2023
f9bde98
Prevent possible accountsettings crash by sorting blacklist before ch…
claucambra Jul 5, 2023
6682b3a
Ensure folder whitelist and blacklist helper methods remove path from…
claucambra Jul 5, 2023
a93279b
Default to adding existing now big folders to whitelist when undecided
claucambra Jul 5, 2023
74c4ec8
Const autofy variables in ProcessDirectoryJob::processFile
claucambra Jul 10, 2023
1b73011
Constify SyncJournalDb::findPathInSelectiveSyncList
claucambra Jul 10, 2023
82f6fd5
Remove use of old foreach in SyncJournalDb::setSelectiveSyncList
claucambra Jul 10, 2023
9fa5ae3
Use simpler sort methods for white and black lists in discoveryphase
claucambra Jul 10, 2023
ac8e9c4
Fix trailing slash procedure in DiscoveryPhase::checkSelectiveSyncNew…
claucambra Jul 10, 2023
3b208bf
Move trailingSlashPath convenience function into Utility class
claucambra Jul 10, 2023
21656cc
Replace all manual isntancesof adding trailing slash with use of util…
claucambra Jul 10, 2023
4cdf525
Report size for directories in syncenginetestutils propfind response
claucambra Jul 18, 2023
0b13be6
Add test for correct reporting of existing folder overcoming size limit
claucambra Jul 18, 2023
1342f81
Add config option to stop synchronising existing folders when they gr…
claucambra Jul 24, 2023
bd7ccf5
Add UI element in general settings to stop syncing existing folders t…
claucambra Jul 24, 2023
2324a60
Stop current sync when existing folder becomes big and we have enable…
claucambra Jul 24, 2023
449bac8
Do not always re-notify user on existing folder now big if this folde…
claucambra Jul 24, 2023
af45017
Do not clear the whitelist on sync success
claucambra Jul 24, 2023
9b5391b
Ensure generated activity buttons are within count set by maxActionBu…
claucambra Aug 3, 2023
e987fe8
Fix generation of context menu activity links
claucambra Aug 3, 2023
5db7926
Fix breakages in testactivitylistmodel caused by recent activitylistm…
claucambra Aug 3, 2023
e1f342c
Declare whitelist and blacklist activity verbs as static constexprs i…
claucambra Aug 3, 2023
cbad434
Extract notifying of existing folder now big to separate static method
claucambra Aug 3, 2023
a7fcb84
Extract folder now big activity generation into separate method
claucambra Aug 3, 2023
6d56621
Deduplicate whitelistFolderPath and blacklistFolderPath
claucambra Aug 3, 2023
a5d7c75
Add additional logging for folder size check
claucambra Aug 3, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 29 additions & 5 deletions src/common/syncjournaldb.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,33 @@ bool SyncJournalDb::maybeMigrateDb(const QString &localPath, const QString &abso
return true;
}

bool SyncJournalDb::findPathInSelectiveSyncList(const QStringList &list, const QString &path)
{
Q_ASSERT(std::is_sorted(list.cbegin(), list.cend()));

if (list.size() == 1 && list.first() == QStringLiteral("/")) {
// Special case for the case "/" is there, it matches everything
return true;
}

const QString pathSlash = path + QLatin1Char('/');

// Since the list is sorted, we can do a binary search.
// If the path is a prefix of another item or right after in the lexical order.
auto it = std::lower_bound(list.cbegin(), list.cend(), pathSlash);

if (it != list.cend() && *it == pathSlash) {
return true;
}

if (it == list.cbegin()) {
return false;
}
--it;
Q_ASSERT(it->endsWith(QLatin1Char('/'))); // Folder::setSelectiveSyncBlackList makes sure of that
return pathSlash.startsWith(*it);
}

bool SyncJournalDb::exists()
{
QMutexLocker locker(&_mutex);
Expand Down Expand Up @@ -1958,10 +1985,7 @@ QStringList SyncJournalDb::getSelectiveSyncList(SyncJournalDb::SelectiveSyncList
if (!next.hasData)
break;

auto entry = query->stringValue(0);
if (!entry.endsWith(QLatin1Char('/'))) {
entry.append(QLatin1Char('/'));
}
const auto entry = Utility::trailingSlashPath(query->stringValue(0));
result.append(entry);
}
*ok = true;
Expand All @@ -1986,7 +2010,7 @@ void SyncJournalDb::setSelectiveSyncList(SyncJournalDb::SelectiveSyncListType ty
}

SqlQuery insQuery("INSERT INTO selectivesync VALUES (?1, ?2)", _db);
foreach (const auto &path, list) {
for (const auto &path : list) {
insQuery.reset_and_clear_bindings();
insQuery.bindValue(1, path);
insQuery.bindValue(2, int(type));
Expand Down
3 changes: 3 additions & 0 deletions src/common/syncjournaldb.h
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,9 @@ class OCSYNC_EXPORT SyncJournalDb : public QObject
/// Migrate a csync_journal to the new path, if necessary. Returns false on error
static bool maybeMigrateDb(const QString &localPath, const QString &absoluteJournalPath);

/// Given a sorted list of paths ending with '/', return whether or not the given path is within one of the paths of the list
static bool findPathInSelectiveSyncList(const QStringList &list, const QString &path);

// To verify that the record could be found check with SyncJournalFileRecord::isValid()
[[nodiscard]] bool getFileRecord(const QString &filename, SyncJournalFileRecord *rec) { return getFileRecord(filename.toUtf8(), rec); }
[[nodiscard]] bool getFileRecord(const QByteArray &filename, SyncJournalFileRecord *rec);
Expand Down
6 changes: 6 additions & 0 deletions src/common/utility.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -723,4 +723,10 @@ bool Utility::isCaseClashConflictFile(const QString &name)
return bname.contains(QStringLiteral("(case clash from"));
}

QString Utility::trailingSlashPath(const QString &path)
{
static const auto slash = QLatin1Char('/');
return path.endsWith(slash) ? path : QString(path + slash);
}

} // namespace OCC
2 changes: 2 additions & 0 deletions src/common/utility.h
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,8 @@ namespace Utility {
*/
OCSYNC_EXPORT void registerUriHandlerForLocalEditing();

OCSYNC_EXPORT QString trailingSlashPath(const QString &path);

#ifdef Q_OS_WIN
OCSYNC_EXPORT bool registryKeyExists(HKEY hRootKey, const QString &subKey);
OCSYNC_EXPORT QVariant registryGetKeyValue(HKEY hRootKey, const QString &subKey, const QString &valueName);
Expand Down
80 changes: 57 additions & 23 deletions src/gui/accountsettings.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@


#include "accountsettings.h"
#include "common/syncjournaldb.h"
#include "common/syncjournalfilerecord.h"
#include "qmessagebox.h"
#include "ui_accountsettings.h"
Expand Down Expand Up @@ -1480,48 +1481,81 @@ void AccountSettings::folderTerminateSyncAndUpdateBlackList(const QStringList &b

void AccountSettings::refreshSelectiveSyncStatus()
{
QString msg;
auto cnt = 0;
QString unsyncedFoldersString;
QString becameBigFoldersString;

const auto folders = FolderMan::instance()->map().values();

static const auto folderSeparatorString = QStringLiteral(", ");
static const auto folderLinkString = [](const QString &slashlessFolderPath, const QString &folderName) {
return QStringLiteral("<a href=\"%1?folder=%2\">%1</a>").arg(slashlessFolderPath, folderName);
};
static const auto appendFolderDisplayString = [](QString &foldersString, const QString &folderDisplayString) {
if (!foldersString.isEmpty()) {
foldersString += folderSeparatorString;
}
foldersString += folderDisplayString;
};

_ui->bigFolderUi->setVisible(false);

for (const auto folder : folders) {
if (folder->accountState() != _accountState) {
continue;
}

auto ok = false;
auto blacklistOk = false;
const auto undecidedList = folder->journalDb()->getSelectiveSyncList(SyncJournalDb::SelectiveSyncUndecidedList, &ok);
auto blacklist = folder->journalDb()->getSelectiveSyncList(SyncJournalDb::SelectiveSyncBlackList, &blacklistOk);
blacklist.sort();

for (const auto &it : undecidedList) {
// FIXME: add the folder alias in a hoover hint.
// folder->alias() + QLatin1String("/")
if (cnt++) {
msg += QLatin1String(", ");
}
auto myFolder = (it);
if (myFolder.endsWith('/')) {
myFolder.chop(1);
}
const auto theIndx = _model->indexForPath(folder, myFolder);
if (theIndx.isValid()) {
msg += QString::fromLatin1("<a href=\"%1?folder=%2\">%1</a>")
.arg(Utility::escape(myFolder), Utility::escape(folder->alias()));

const auto folderTrailingSlash = Utility::trailingSlashPath(it);
const auto folderWithoutTrailingSlash = it.endsWith('/') ? it.left(it.length() - 1) : it;
const auto escapedFolderString = Utility::escape(folderWithoutTrailingSlash);
const auto escapedFolderName = Utility::escape(folder->alias());
const auto folderIdx = _model->indexForPath(folder, folderWithoutTrailingSlash);

// If we do not know the index yet then do not provide a link string
const auto folderDisplayString = folderIdx.isValid() ? folderLinkString(escapedFolderString, escapedFolderName) : folderWithoutTrailingSlash;

// The new big folder procedure automatically places these new big folders in the blacklist.
// This is not the case for existing folders discovered to have gone beyond the limit.
// So we need to check if the folder is in the blacklist or not and tweak the message accordingly.
if (SyncJournalDb::findPathInSelectiveSyncList(blacklist, folderTrailingSlash)) {
appendFolderDisplayString(unsyncedFoldersString, folderDisplayString);
} else {
msg += myFolder; // no link because we do not know the index yet.
appendFolderDisplayString(becameBigFoldersString, folderDisplayString);
}
}
}

if (!msg.isEmpty()) {
ConfigFile cfg;
const auto info = !cfg.confirmExternalStorage() ?
tr("There are folders that were not synchronized because they are too big: ") :
!cfg.newBigFolderSizeLimit().first ?
tr("There are folders that were not synchronized because they are external storages: ") :
tr("There are folders that were not synchronized because they are too big or external storages: ");
ConfigFile cfg;
QString infoString;

_ui->selectiveSyncNotification->setText(info + msg);
_ui->bigFolderUi->setVisible(true);
if (!unsyncedFoldersString.isEmpty()) {
infoString += !cfg.confirmExternalStorage() ? tr("There are folders that were not synchronized because they are too big: ")
: !cfg.newBigFolderSizeLimit().first ? tr("There are folders that were not synchronized because they are external storages: ")
: tr("There are folders that were not synchronized because they are too big or external storages: ");

infoString += unsyncedFoldersString;
}

if (!becameBigFoldersString.isEmpty()) {
if (!infoString.isEmpty()) {
infoString += QStringLiteral("\n");
}

const auto folderSizeLimitString = QString::number(cfg.newBigFolderSizeLimit().second);
infoString += tr("There are folders that have grown in size beyond %1MB: %2").arg(folderSizeLimitString, becameBigFoldersString);
}

_ui->selectiveSyncNotification->setText(infoString);
_ui->bigFolderUi->setVisible(!infoString.isEmpty());
}

bool AccountSettings::event(QEvent *e)
Expand Down
Loading
Loading