KeyError: 'grouped'

[m-i-l]

Just spotted the following error in the logs. Not sure what triggered it - will need to investigate further.

[Sat Nov 02 12:59:57.012363 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018] [2024-11-02 12:59:57,010] ERROR in app: Exception on / [GET]
[Sat Nov 02 12:59:57.012461 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018] Traceback (most recent call last):
[Sat Nov 02 12:59:57.012479 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1473, in wsgi_app
[Sat Nov 02 12:59:57.012496 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]     response = self.full_dispatch_request()
[Sat Nov 02 12:59:57.012515 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Sat Nov 02 12:59:57.012538 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 882, in full_dispatch_request
[Sat Nov 02 12:59:57.012555 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]     rv = self.handle_user_exception(e)
[Sat Nov 02 12:59:57.012572 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Sat Nov 02 12:59:57.012588 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 880, in full_dispatch_request
[Sat Nov 02 12:59:57.012684 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]     rv = self.dispatch_request()
[Sat Nov 02 12:59:57.012742 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]          ^^^^^^^^^^^^^^^^^^^^^^^
[Sat Nov 02 12:59:57.012788 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 865, in dispatch_request
[Sat Nov 02 12:59:57.012843 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
[Sat Nov 02 12:59:57.012888 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Sat Nov 02 12:59:57.012943 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]   File "/usr/local/apache2/htdocs/dynamic/searchmysite/search/search.py", line 31, in search
[Sat Nov 02 12:59:57.013006 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]     (total_results, total_domains) = get_no_of_results(search_results, groupbydomain)
[Sat Nov 02 12:59:57.013059 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Sat Nov 02 12:59:57.013115 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]   File "/usr/local/apache2/htdocs/dynamic/searchmysite/searchutils.py", line 211, in get_no_of_results
[Sat Nov 02 12:59:57.013167 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]     total_results = search_results['grouped']['domain']['matches']
[Sat Nov 02 12:59:57.013223 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018]                     ~~~~~~~~~~~~~~^^^^^^^^^^^
[Sat Nov 02 12:59:57.013274 2024] [wsgi:error] [pid 10:tid 281472973074720] [client 172.18.0.1:52018] KeyError: 'grouped'
172.18.0.1 - - [02/Nov/2024:12:59:55 +0000] "GET /search/?q=argos+garden+furniture+habitat&page=17601 HTTP/1.1" 500 265
[Sat Nov 02 12:59:57.017562 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014] [2024-11-02 12:59:57,011] ERROR in app: Exception on / [GET]
[Sat Nov 02 12:59:57.017770 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014] Traceback (most recent call last):
[Sat Nov 02 12:59:57.017834 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1473, in wsgi_app
[Sat Nov 02 12:59:57.017892 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]     response = self.full_dispatch_request()
[Sat Nov 02 12:59:57.017945 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Sat Nov 02 12:59:57.018000 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 882, in full_dispatch_request
[Sat Nov 02 12:59:57.018054 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]     rv = self.handle_user_exception(e)
[Sat Nov 02 12:59:57.018107 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Sat Nov 02 12:59:57.018159 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 880, in full_dispatch_request
[Sat Nov 02 12:59:57.018216 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]     rv = self.dispatch_request()
[Sat Nov 02 12:59:57.018267 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]          ^^^^^^^^^^^^^^^^^^^^^^^
[Sat Nov 02 12:59:57.018322 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 865, in dispatch_request
[Sat Nov 02 12:59:57.018375 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
[Sat Nov 02 12:59:57.018430 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Sat Nov 02 12:59:57.018479 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]   File "/usr/local/apache2/htdocs/dynamic/searchmysite/search/search.py", line 31, in search
[Sat Nov 02 12:59:57.018533 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]     (total_results, total_domains) = get_no_of_results(search_results, groupbydomain)
[Sat Nov 02 12:59:57.018584 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Sat Nov 02 12:59:57.018657 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]   File "/usr/local/apache2/htdocs/dynamic/searchmysite/searchutils.py", line 211, in get_no_of_results
[Sat Nov 02 12:59:57.018705 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]     total_results = search_results['grouped']['domain']['matches']
[Sat Nov 02 12:59:57.018759 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014]                     ~~~~~~~~~~~~~~^^^^^^^^^^^
[Sat Nov 02 12:59:57.018802 2024] [wsgi:error] [pid 10:tid 281472981528864] [client 172.18.0.1:52014] KeyError: 'grouped'

There are an awful lot of bots active now, including one which has spent hour hammering the server with a search for "argos garden furniture habitat" and is now on page 17672 of the results.