Improve sidebar performance

[imfing]

Discussed in #116

^{Originally posted by thangisme October 2, 2023}
With around 4K page, it takes a while to build the site. Which is not the case with other themes.

Template Metrics:

     cumulative       average       maximum      cache  percent  cached  total  
       duration      duration      duration  potential   cached   count  count  template
     ----------      --------      --------  ---------  -------  ------  -----  --------
  1h11m15.223847094s  1.700566367s  3.723108386s          0        0       0   2514  docs/single.html
  29m25.170236303s  566.304214ms  2.228949419s         29        0       0   3117  partials/sidebar.html
  17m4.13874766s  1.712606601s   3.15029002s          0        0       0    598  docs/list.html
  9m44.776173335s  187.488353ms  1.667255081s        100        0       0   3119  partials/theme-toggle.html
  8m26.370114506s  162.715332ms   2.49520532s         52        0       0   3112  partials/breadcrumb.html
  29.106906383s    9.338115ms  2.961663767s         84        0       0   3117  partials/scripts.html
  18.830250172s    6.050851ms   1.24994449s         12        0       0   3112  partials/components/pager.html
  17.525731147s    5.622627ms  1.394370744s         16        0       0   3117  partials/head.html
   8.323090391s    2.670224ms  935.323309ms          3        0       0   3117  partials/footer.html
   4.253779071s  1.417926357s  1.718321097s          0        0       0      3  _default/list.html
   3.254211742s  3.254211742s  3.254211742s          0        0       0      1  _default/single.html
   3.125610084s    1.002762ms  302.079788ms         94        0       0   3117  partials/navbar.html
   2.958057535s  2.958057535s  2.958057535s          0        0       0      1  js/flexsearch.js
   2.772303655s  2.772303655s  2.772303655s          0        0       0      1  json/search-data.json
   2.324423135s      84.138µs   10.123734ms         32        0       0  27626  partials/utils/icon.html
   1.646970444s  1.646970444s  1.646970444s          0        0       0      1  hextra-home.html
   1.108133334s   16.539303ms  743.714675ms          0        0       0     67  shortcodes/card.html
   473.390865ms     188.002µs    1.148266ms          0        0       0   2518  partials/utils/fragments.html
   408.630441ms     679.917µs   12.980529ms          0        0       0    601  _default/list.rss.xml
   328.305514ms     105.361µs      829.74µs        100        0       0   3116  partials/toc.html
   325.333061ms   40.666632ms  325.054054ms          0        0       0      8  shortcodes/hextra/feature-card.html
   173.680592ms      55.756µs    1.771161ms        100        0       0   3115  partials/components/last-updated.html
   171.013371ms      54.864µs    2.740091ms          7        0       0   3117  partials/opengraph.html
   140.474137ms      22.533µs    1.019029ms        100        0       0   6234  partials/search.html
   140.084677ms      44.942µs    1.298499ms         19        0       0   3117  partials/utils/page-description.html
   129.101685ms      41.431µs    3.434975ms        100        0       0   3116  partials/components/comments.html
     83.38736ms      26.752µs    1.462905ms        100        0       0   3117  partials/utils/page-width.html
    74.589641ms      23.929µs   10.828863ms        100      100    3116   3117  partials/head-css.html
    25.315915ms       8.121µs    9.142074ms        100      100    3116   3117  partials/favicons.html
    24.946966ms   24.946966ms   24.946966ms          0        0       0      1  _internal/_default/sitemap.xml
    21.683486ms       8.625µs      56.337µs          0        0       0   2514  shortcodes/rawhtml.html
     4.409197ms     881.839µs    1.228166ms          0        0       0      5  shortcodes/tab.html
     1.837825ms         589ns      46.988µs        100        0       0   3117  partials/custom/head-end.html
      624.158µs     104.026µs     586.997µs          0        0       0      6  shortcodes/cards.html
      385.126µs     192.563µs     364.407µs        100        0       0      2  partials/language-switch.html
      251.657µs     251.657µs     251.657µs          0        0       0      1  shortcodes/hextra/feature-grid.html
      220.678µs     220.678µs     220.678µs          0        0       0      1  shortcodes/icon.html
      162.938µs      81.469µs      143.15µs          0        0       0      2  shortcodes/tabs.html
      160.524µs     160.524µs     160.524µs          0        0       0      1  shortcodes/hextra/hero-badge.html
      118.484µs     118.484µs     118.484µs          0        0       0      1  shortcodes/hextra/hero-headline.html
      113.495µs     113.495µs     113.495µs          0        0       0      1  404.html


                   |  EN   
-------------------+-------
  Pages            | 3719  
  Paginator pages  |    0  
  Non-page files   |    0  
  Static files     | 6726  
  Processed images |    0  
  Aliases          |    0  
  Sitemaps         |    1  
  Cleaned          |    0  

Total in 348461 ms

Someone at Hugo forum pointed out :

https://github.com/imfing/hextra/blob/main/layouts/partials/sidebar.html#L66-L95 4

The number of iterations increases quadratically in relation to the number of content pages (i.e., 3000 pages = 90,000,000 iterations).

Could you take a look?

[imfing]

For site with under 1000 pages, the build speed is tolerable. With ~1000 pages, the nested sidebar traversal and mobile navigation bar would consume tremendous time during build process.

Here's the overview of the plan:

• Cache the entire sidebar and navigation when site is large, and dynamically expand the sidebar for the active page from client side
• Set a configurable limit (e.g. 1000 pages)
  • when # of pages is smaller than it, dynamically build the page sidebars
  • when # of pages is larger than it, cache the sidebar, and add the client side script

[divinepaul]

bump. same issue.
also affects SEO with a lot of internal links spamming each page. i did notice the sidebar-data-source and #376 any chance it will be merged soon ?
loving the theme so far.

[imfing]

bump. same issue.

also affects SEO with a lot of internal links spamming each page. i did notice the sidebar-data-source and #376 any chance it will be merged soon ?

loving the theme so far.

@divinepaul don't have any eta yet, this is much more work than i expected