Merge pull request #949 from sabeechen/dev

Release v0.112.1

.devcontainer/devcontainer.json

@@ -20,7 +20,6 @@
         "mikoz.black-py"]
       },
     "runArgs": ["--init", "--privileged"],
-    "extensions": ["ms-python.python", "wholroyd.jinja","ms-python.vscode-pylance"],
     "forwardPorts": [3000],
     "mounts": [ "source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind" ]
   }

.github/workflows/prod_push.yaml

@@ -17,15 +17,17 @@ jobs:
         path: dev
         persist-credentials: false
 
-    - name: Login to DockerHub
-      uses: docker/login-action@v3.0.0
+    - name: Log in to the Container registry
+      uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
       with:
-        username: ${{ secrets.DOCKERHUB_USERNAME }}
-        password: ${{ secrets.DOCKERHUB_ACCESS_TOKEN }}
+        registry: ghcr.io
+        username: ${{ github.actor }}
+        password: ${{ secrets.GITHUB_TOKEN }}
 
     - name: Publish Addon Image
       uses: home-assistant/builder@master
       with:
         args: |
           --all \
-          --target dev/hassio-google-drive-backup
+          --target dev/hassio-google-drive-backup \
+          --docker-hub "ghcr.io/sabeechen"

.github/workflows/test_staging_push.yml

@@ -32,18 +32,20 @@ jobs:
       run: |
           python3 staging/update.py dev staging
 
-    - name: Login to DockerHub
-      uses: docker/login-action@v3.0.0
+    - name: Log in to the Container registry
+      uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
       with:
-        username: ${{ secrets.DOCKERHUB_USERNAME }}
-        password: ${{ secrets.DOCKERHUB_ACCESS_TOKEN }}
+        registry: ghcr.io
+        username: ${{ github.actor }}
+        password: ${{ secrets.GITHUB_TOKEN }}
 
     - name: Publish Staging Addon image
       uses: home-assistant/builder@master
       with:
         args: |
           --all \
-          --target dev/hassio-google-drive-backup
+          --target dev/hassio-google-drive-backup \
+          --docker-hub "ghcr.io/sabeechen"
 
     - name: Publish Staging Addon Version
       run: |

hassio-google-drive-backup/CHANGELOG.md

@@ -1,21 +1,38 @@
+## v0.112.1 [2023-11-03]
+
+- Added warnings about using the "Stop Addons" feature.  I plan on removing this in the near future.  If you'd like to keep the feature around, please give your feedback in [this GitHub issue](https://github.com/sabeechen/hassio-google-drive-backup/issues/940).
+- When backups are stuck in the "pending" state, the addon now provides you with the Supervisor logs to help figure out whats wrong.
+- Added support for the "exclude Home Assistant database" options for automatic backups
+- Added configuration options to limit the speed of uploads to Google Drive
+- When Google Drive doesn't have enough space, the addon now explains how much space you're using and how much is left.  This was a source of confusion for users.
+- When the addon halts because it needs to delete more than one backup, it now tells you which backups will be deleted.
+- Fixed a bug when using "stop addons" that prevented it from recognizing addons in the "starting" state.
+- The addon's containers are now donwloaded from Github (previously was DockerHub)
+- Added another redundant token provider, hosted on heroku, that the addon uses for its cloud-required component when you aren't using your own google app credentials.
+
 ## v0.111.1 [2023-06-19]
+
 - Support for the new network storage features in Home Assistant.  The addon will now create backups in what Home Assistant has configured as its default backup location.  This can be overridden in the addon's settings.
 - Raised the addon's required permissions to "Admin" in order to access the supervisor's mount API.
 - Fixed a CSS error causing toast messages to render partially off screen on small displays.
 - Fixed misreporting of some error codes from Google Drive when a partial upload can't be resumed.
 
 ## v0.110.4 [2023-04-28]
+
 - Fix a whitespace error causing authorization to fail.
 
 ## v0.110.3 [2023-03-24]
+
 - Fix an error causing "Days Between Backups" to be ignored when "Time of Day" for a backup is set.
 - Fix a bug causing some timezones to make the addon to fail to start.
 
 ## v0.110.2 [2023-03-24]
+
 - Fix a potential cause of SSL errors when communicating with Google Drive
 - Fix a bug causing backups to be requested indefinitely if scheduled during DST transitions.
 
 ## v0.110.1 [2023-01-09]
+
 - Adds some additional options for donating
 - Mitgigates SD card corruption by redundantly storing config files needed for addon startup.
 - Avoid global throttling of Google Drive API calls by:

hassio-google-drive-backup/DOCS.md

@@ -178,7 +178,7 @@ When true, requires your home assistant username and password to access the Web
 
 #### Option: `use_ssl` (default: False)
 
-When true, requires your home assistant username and password to access the Web UI.
+When true, the Web UI exposed by `expose_extra_server` will be served over SSL (HTTPS).
 
 #### Option: `certfile` (default: `/ssl/certfile.pem`)
 

hassio-google-drive-backup/backup/config/settings.py

@@ -52,6 +52,7 @@ class Setting(Enum):
     # Partial backups
     EXCLUDE_FOLDERS = "exclude_folders"
     EXCLUDE_ADDONS = "exclude_addons"
+    EXCLUDE_HA_DATABASE = "exclude_ha_database"
 
     STOP_ADDONS = "stop_addons"
     DISABLE_WATCHDOG_WHEN_STOPPING = "disable_watchdog_when_stopping"
@@ -214,6 +215,8 @@ def key(self):
     Setting.EXCLUDE_FOLDERS: "",
     Setting.EXCLUDE_ADDONS: "",
 
+    Setting.EXCLUDE_HA_DATABASE: False,
+
     Setting.STOP_ADDONS: "",
     Setting.DISABLE_WATCHDOG_WHEN_STOPPING: False,
 
@@ -355,6 +358,7 @@ def key(self):
     # Partial backup settings
     Setting.EXCLUDE_FOLDERS: "str?",
     Setting.EXCLUDE_ADDONS: "str?",
+    Setting.EXCLUDE_HA_DATABASE: "bool?",
 
     Setting.STOP_ADDONS: "str?",
     Setting.DISABLE_WATCHDOG_WHEN_STOPPING: "bool?",

hassio-google-drive-backup/backup/ha/hasource.py

@@ -51,6 +51,7 @@ def __init__(self, backupType, protected, options: CreateOptions, request_info,
         self._time = time
         self._pending_subverted = False
         self._start_time = time.now()
+        self._logs = None
 
     def considerForPurge(self) -> bool:
         return False
@@ -111,6 +112,15 @@ def isStale(self):
             Setting.FAILED_BACKUP_TIMEOUT_SECONDS))
         staleTime = self.getFailureTime() + delta
         return self._time.now() >= staleTime
+
+    def attach_logs(self, logs):
+        self._logs = logs
+
+    def error_logs(self):
+        if self._logs:
+            return self._logs
+        else:
+            return None
 
     def madeByTheAddon(self):
         return True
@@ -512,6 +522,11 @@ async def _requestAsync(self, pending: PendingBackup, start=[]) -> None:
                 logger.error("Backup failed:")
                 logger.printException(e)
                 pending.failed(e, self.time.now())
+            try:
+                pending.attach_logs(await self.harequests.getSuperLogs())
+            except Exception as e:
+                logger.error("Failed to get sueprvisor logs after failed backup request")
+                logger.printException(e)
         finally:
             await self.stopper.startAddons()
             self.trigger()
@@ -522,8 +537,10 @@ def _buildBackupInfo(self, options: CreateOptions):
             addons.append(addon['slug'])
         request_info: Dict[str, Any] = {
             'addons': [],
-            'folders': []
+            'folders': [],
         }
+        if self.config.get(Setting.EXCLUDE_HA_DATABASE):
+            request_info['homeassistant_exclude_database'] = True
         folders = list(map(lambda f: f['slug'], FOLDERS))
         type_name = "Full"
         for folder in folders:

hassio-google-drive-backup/backup/static/js/scripts.js

@@ -109,6 +109,12 @@ function errorReports(send) {
   var jqxhr = $.get("errorreports?send=" + send)
   $('#error_reports_card').fadeOut(500)
 }
+
+function dismissRemoveStopAddons() {
+  var jqxhr = $.get("dismiss_remove_stop_addons");
+  $('#stop_addon_removal_survey').fadeOut(500);
+}
+
 hideIngress = false;
 function exposeServer(expose) {
   var url = "exposeserver?expose=" + expose;
@@ -606,7 +612,12 @@ function populateBackupDiv(backup_div, backups, icon) {
 
       if (backup.isPending) {
         $("#loading" + backup.slug).show();
-        $("#backup_card" + backup.slug).css("cursor", "auto");
+        if (backup.super_logs) {
+          $("#backup_card" + backup.slug).css("cursor", "pointer");
+          $("#has-logs", template).show();
+        } else {
+          $("#backup_card" + backup.slug).css("cursor", "auto");
+        }
       } else {
         $("#loading" + backup.slug).hide();
         $("#backup_card" + backup.slug).css("cursor", "pointer");
@@ -744,6 +755,8 @@ function processStatusUpdate(data) {
     question_card = "warn_upgrade_backups_card";
   } else if (data.warn_oob_oauth) {
     question_card = "warn_creds_deprecated_card";
+  } else if(data.warn_stop_addons) {
+    question_card = "stop_addon_removal_survey";
   }
 
   $('.question-card').each(function (i) {

hassio-google-drive-backup/backup/static/layouts/partials/modals/about.jinja2

@@ -6,7 +6,7 @@
       I'm an individual who works on projects like this in my spare time.
       If you like it and would like to support me, you can star my projects on
       <a href="https://github.com/sabeechen" target="_blank" rel="noreferrer">Github</a>
-      or contribute financially usign any of the methods below, they all go into the same bucket used to support the add-on:
+      or contribute financially using any of the methods below, they all go into the same bucket used to support the add-on:
       <div class="flex-wrap">
         <a target="_blank" rel="noreferrer" href="https://www.buymeacoffee.com/sabeechen" class="donate-button">
           <img src="static/{{ version }}/images/bmc-button.svg" alt="Support me">

hassio-google-drive-backup/backup/static/layouts/partials/modals/backup/details.jinja2

@@ -362,9 +362,30 @@
           </div>`;
     }
 
+    function colorizeLog(log) {
+      if (log.includes("WARN")) return `<span class="super-log-warn">${log}</span>`;
+      if (log.includes("INFO")) return `<span class="super-log-info">${log}</span>`;
+      if (log.includes("ERROR")) return `<span class="super-log-error">${log}</span>`;
+      if (log.includes("DEBUG")) return `<span class="super-log-debug">${log}</span>`;
+      return `<span class="super-log-info">${log}</span>`;
+    }
+
     function showDetails(target) {
       var backup = $(target).data('backup');
       if (backup.isPending) {
+        if (backup.super_logs) {
+          let modal = $('#pending_backup_modal');
+          modal.data('slug', backup.slug);
+          modal.data('backup', backup);
+          const logs = backup.super_logs.split('\n');
+          const container = document.getElementById('super_logs_pending_request');
+          logs.forEach(log => {
+            const colorizedLog = colorizeLog(log);
+            container.innerHTML += `${colorizedLog}\n`;
+          });
+          M.Modal.getInstance(document.getElementById('pending_backup_modal')).open();
+          container.scrollTop = container.scrollHeight;
+        }
         return;
       }
 

hassio-google-drive-backup/backup/static/layouts/partials/modals/pending_backup_info.jinja2

@@ -0,0 +1,65 @@
+<div id="pending_backup_modal" class="modal">
+  <style>
+    .tall {
+      display: flex; /* Use Flexbox */
+      flex-direction: column; /* Stack children vertically */
+      height: 100%; /* Set a specific height for the parent container */
+      box-sizing: border-box;
+    }
+    .tall div {
+      flex-grow: 1; /* Grow to take the remaining space */
+      resize: none; /* Optional: prevent manual resizing */
+    }
+
+    .super-log-container {
+      background-color: black;
+      color: white;
+      width: 95%;
+      height: 400px;
+      margin: 20px auto;
+      overflow-y: scroll;
+      font-family: monospace;
+      white-space: pre-line; /* preserves your newline characters */
+      padding: 10px;
+      border-radius: 4px;
+    }
+  
+    /* Log level color coding */
+    .super-log-warn {
+      color: yellow;
+    }
+  
+    .super-log-error {
+      color: red;
+    }
+  
+    .super-log-debug {
+      color: lightgreen;
+    }
+    .super-log-info {
+      color: white;
+    }
+  </style>
+  <div class="modal-content">
+    <h4>Pending Backup Info</h4>
+    <p>This addon thinks a backup is in progress, but that just an assumption.  In reality Home Assistant doesn't tell this addon the reason the 
+      request to create a backup was rejected, a backup already being in progress is just the most common reason.  A lot of less common reasons can cause it to be rejected like
+
+      <ul class="browser-default">
+        <li>You're using a network share to store backups, and its unavailable</li>
+        <li>Something is wrong with your hard drive and Home Assistant can't read/write to it</li>
+        <li>Home Assistant has frozen itself because it thinks something is wrong</li>
+      </ul>
+      The only place these less common errors show up is in the Supervisor's logs.  The Supervisor is a part of Home Assistant that manages it in the background.  Its logs are 
+      kind of hard to get to, so for conveinence a snapshot of the logs right after the backup was most recently requested is shown below.  If you're seeing a backup thats stuck 
+      in the "pending" state, these logs might provide a clue why.  If you're able to identify and resolve that problem, restart this addon to have it try creating a backup again.</p>
+
+      <h5>Supervisor Logs</h5>
+    <div class="tall">
+      <div class="super-log-container" id="super_logs_pending_request"></div>
+    </div>
+  </div>
+  <div class="modal-footer">
+    <a href="#!" class="modal-close btn-flat">Close</a>
+  </div>
+</div>

hassio-google-drive-backup/backup/static/layouts/partials/modals/settings.jinja2

@@ -436,6 +436,16 @@
             </div>
           </div>
         </div>
+        <div class="row">
+          <div class="col s12">
+            <label>
+              <input type="checkbox" name="exclude_ha_database" id="exclude_ha_database"class="filled-in checkbox-ha" />
+              <span>Exclude Home Asisstant Database from Backups</span>
+              <br />
+              <span class="helper-text">Don't include the Home Assistant database in backups.  Useful if you need to save space and don't care about history.  Applies to both full and partial backups.</span>
+            </label>
+          </div>
+        </div>
         <div class="row">
           <div class="col s12">
             <label>
@@ -477,9 +487,8 @@
               <span>Stop addons</span>
               <br />
               <span class="helper-text">
-                Addons selected below will be stopped before backup and started after.  This is generally only
-                recommended as a last resort if an addon can't be backed up while running, which is a rare.  In
-                some error situations an addon can't be restarted after backup and you'll have to do it manually.
+                This feature is going to be removed in a future release.  If you'd like to keep it around, give your feedback <a href="https://github.com/sabeechen/hassio-google-drive-backup/issues/940" target="_blank">here</a>.
+                Addons selected below will be stopped before backup and started after.
               </span>
             </label>
           </div>

hassio-google-drive-backup/backup/static/layouts/partials/navbar.jinja2

@@ -13,6 +13,7 @@
 {% include 'layouts/partials/modals/about.jinja2' %}
 {% include 'layouts/partials/modals/about.jinja2' %}
 {% include 'layouts/partials/modals/crypto_donate.jinja2' %}
+{% include 'layouts/partials/modals/pending_backup_info.jinja2' %}
 
 {% if coordEnabled %}
 <script type="text/javascript">
@@ -37,6 +38,7 @@
     $('.collapsible').collapsible();
     $('.sidenav').sidenav();
     $('.fixed-action-btn').floatingActionButton();
+    $('#pending_backup_modal').modal();
 
     const dropdownTriggers = document.querySelectorAll('.dropdown-trigger');
     const instances = M.Dropdown.init(dropdownTriggers, { 'constrainWidth': false });