Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Specific job setting does result in 503 Service Unavailable error ("Storage is temporarily not available") #187

Open
rugk opened this issue Sep 6, 2023 · 0 comments
Open

Specific job setting does result in 503 Service Unavailable error ("Storage is temporarily not available") #187

rugk opened this issue Sep 6, 2023 · 0 comments

Comments

@rugk
Copy link

rugk commented Sep 6, 2023 •
edited
Loading

Describe the bug
I have run some jobs successfully, but a specific job (with maybe a lot of files), cannot be started?

To Reproduce
I have no idea. I can reproduce it when I select two specific directories. Other settings do not seem to matter.

  • They are quite big and contain quite many photos. (114,6 GB + 2,1 GB)
  • I never have storage problems or such errors when using Nextcloud otherwise.
  • I do not use any external storage (in terms of what Nextcloud means like a remote storage), it is just a hard disk. (Maybe some remote shares may be included in subfolders I am not sure, but well)
  • Two other jobs were successful and I can start other jobs. I do not see pattern (except of "this big directory") yet.

And the server response I can see in the devtools comes very fast, it does not seem as if it is indexing something or so. It took 853 ms.

The following request is executed (copied as fetch):

await fetch("https://*****/apps/mediadc/api/v1/tasks/run", {
    "credentials": "include",
    "headers": {
        "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/117.0",
        "Accept": "application/json, text/plain, */*",
        "Accept-Language": "de,en-US;q=0.7,en;q=0.3",
        "Content-Type": "application/json",
        "requesttoken": "***************",
        "Sec-Fetch-Dest": "empty",
        "Sec-Fetch-Mode": "cors",
        "Sec-Fetch-Site": "same-origin",
        "Sec-GPC": "1",
        "Alt-Used": "***************",
    },
    "body": "{\"targetDirectoryIds\":\"[\\\"132366\\\",\\\"1329883\\\"]\",\"excludeList\":{\"user\":{\"mask\":[],\"fileid\":[]},\"admin\":{\"mask\":[],\"fileid\":[]}},\"collectorSettings\":{\"hashing_algorithm\":\"dhash\",\"similarity_threshold\":\"99\",\"hash_size\":\"16\",\"target_mtype\":0,\"finish_notification\":true}}",
    "method": "POST",
    "mode": "cors"
});

POST data JSON:

{"targetDirectoryIds":"[\"132366\",\"1329883\"]","excludeList":{"user":{"mask":[],"fileid":[]},"admin":{"mask":[],"fileid":[]}},"collectorSettings":{"hashing_algorithm":"dhash","similarity_threshold":"99","hash_size":"16","target_mtype":0,"finish_notification":true}}

The server reply is this:

503 Service Unavailable
HTTP/3 503 Service Unavailable

Response headers 
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
date: Wed, 06 Sep 2023 19:38:35 GMT
referrer-policy: no-referrer
x-robots-tag: noindex, nofollow
status: 503 Service Unavailable
content-type: text/html; charset=UTF-8
strict-transport-security: max-age=63072000; includeSubDomains; preload
x-permitted-cross-domain-policies: none
content-encoding: gzip
server: Caddy, Caddy
pragma: no-cache
x-powered-by: PHP/8.2.10
cache-control: no-store, no-cache, must-revalidate
expires: Thu, 19 Nov 1981 08:52:00 GMT
x-xss-protection: 1; mode=block
vary: Accept-Encoding
content-security-policy: ****

grafik

Storage is temporarily not available

Full HTML below:

Full HTML 
<!DOCTYPE html>
<html class="ng-csp" data-placeholder-focus="false" lang="de" data-locale="de_DE" translate="no" >
	<head
 data-requesttoken="">
		<meta charset="utf-8">
		<title>
			Nextcloud		</title>
		<meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0">
				<meta name="apple-itunes-app" content="app-id=1125420102">
				<meta name="theme-color" content="#316b26">
		<link rel="icon" href="/apps/theming/favicon?v=9a88a012">
		<link rel="apple-touch-icon" href="/apps/theming/icon?v=9a88a012">
		<link rel="mask-icon" sizes="any" href="/core/img/favicon-mask.svg" color="#316b26">
		<link rel="manifest" href="/apps/theming/manifest?v=9a88a012">
		<link rel="stylesheet" href="/core/css/server.css?v=0dbbf340-37">
<link rel="stylesheet" href="/custom_apps/cospend/css/cospend-search.css?v=58ce695c-37">
<link rel="stylesheet" href="/apps/theming/css/default.css?v=34e90de1-37">
<link rel="stylesheet" href="/core/css/guest.css?v=0dbbf340-37">
		<script nonce="Rk1CNmtlbXpORUQ2VitsR3dvSlpDWVhzRjFBYVNYYVVaQ2FPVHVJYjZ2MD06UXZjMTlJT0hSeTY0SG9NMDl1MW9hcmFHZWhkVWZnL3lMV1BqQTVCUW9idz0=" defer src="/dist/core-common.js?v=0dbbf340-37"></script>
<script nonce="Rk1CNmtlbXpORUQ2VitsR3dvSlpDWVhzRjFBYVNYYVVaQ2FPVHVJYjZ2MD06UXZjMTlJT0hSeTY0SG9NMDl1MW9hcmFHZWhkVWZnL3lMV1BqQTVCUW9idz0=" defer src="/dist/core-main.js?v=0dbbf340-37"></script>
<script nonce="Rk1CNmtlbXpORUQ2VitsR3dvSlpDWVhzRjFBYVNYYVVaQ2FPVHVJYjZ2MD06UXZjMTlJT0hSeTY0SG9NMDl1MW9hcmFHZWhkVWZnL3lMV1BqQTVCUW9idz0=" defer src="/core/l10n/de.js?v=0dbbf340-37"></script>
<script nonce="Rk1CNmtlbXpORUQ2VitsR3dvSlpDWVhzRjFBYVNYYVVaQ2FPVHVJYjZ2MD06UXZjMTlJT0hSeTY0SG9NMDl1MW9hcmFHZWhkVWZnL3lMV1BqQTVCUW9idz0=" defer src="/dist/core-files_fileinfo.js?v=0dbbf340-37"></script>
<script nonce="Rk1CNmtlbXpORUQ2VitsR3dvSlpDWVhzRjFBYVNYYVVaQ2FPVHVJYjZ2MD06UXZjMTlJT0hSeTY0SG9NMDl1MW9hcmFHZWhkVWZnL3lMV1BqQTVCUW9idz0=" defer src="/dist/core-files_client.js?v=0dbbf340-37"></script>
<script nonce="Rk1CNmtlbXpORUQ2VitsR3dvSlpDWVhzRjFBYVNYYVVaQ2FPVHVJYjZ2MD06UXZjMTlJT0hSeTY0SG9NMDl1MW9hcmFHZWhkVWZnL3lMV1BqQTVCUW9idz0=" defer src="/js/core/merged-template-prepend.js?v=0dbbf340-37"></script>
<script nonce="Rk1CNmtlbXpORUQ2VitsR3dvSlpDWVhzRjFBYVNYYVVaQ2FPVHVJYjZ2MD06UXZjMTlJT0hSeTY0SG9NMDl1MW9hcmFHZWhkVWZnL3lMV1BqQTVCUW9idz0=" defer src="/apps/files_sharing/l10n/de.js?v=0dbbf340-37"></script>
<script nonce="Rk1CNmtlbXpORUQ2VitsR3dvSlpDWVhzRjFBYVNYYVVaQ2FPVHVJYjZ2MD06UXZjMTlJT0hSeTY0SG9NMDl1MW9hcmFHZWhkVWZnL3lMV1BqQTVCUW9idz0=" defer src="/dist/files_sharing-main.js?v=0dbbf340-37"></script>
			</head>
	<body id="body-login">
		<noscript>
	<div id="nojavascript">
		<div>
			Diese Anwendung benötigt JavaScript zum ordnungsgemäßen Betrieb. Bitte <a href="https://www.enable-javascript.com/" target="_blank" rel="noreferrer noopener">aktiviere JavaScript</a> und lade die Seite neu.		</div>
	</div>
</noscript>
					<input type="hidden" id="initial-state-core-versionHash" value="IjBkYmJmMzQwIg==">
					<input type="hidden" id="initial-state-comments-maxAutoCompleteResults" value="MTA=">
				<div class="wrapper">
			<div class="v-align">
									<header role="banner">
						<div id="header">
							<div class="logo"></div>
						</div>
					</header>
								<main>
					<h1 class="hidden-visually">
						Nextcloud					</h1>
					<div class="guest-box">
	<h2>Fehler</h2>
	<ul>
			<li>
			<p></p>
							<p class='hint'>Storage is temporarily not available</p>
					</li>
		</ul>
</div>
				</main>
			</div>
		</div>
		<footer role="contentinfo">
			<p class="info">
				<a href="****" target="_blank" rel="noreferrer noopener" class="entity-name">Nextcloud</a> – ein sicherer Ort für all deine Daten			</p>
		</footer>
	</body>
</html>

Also in the GUI that button keeps on spinning endlessly (nothing else happens):
grafik

Nothing can be found in the Nextcloud log about this. I even set the log level to debug in that Python API thing (default seems to be Warn), but it did not log more:
grafik

Expected behavior
Create a task or at least show (or log) a useful error.

Desktop (please complete the following information):

  • OS: Fedora CoreOS, but Nextcloud is running in the official Docker container.
  • CPU: x86-64
  • Nextcloud version [e.g. 25] 27
  • Database configuration mariadb 10
  • Python version using binaries
  • MediaDC version 0.3.6
  • cloud_py_api version 0.1.8

More system reports (copied):

System report from admin settings 
{
  "nextcloud-version": "27.0.2.1",
  "app-versions": {
    "cloud_py_api-version": "0.1.8",
    "mediadc-version": "0.3.6"
  },
  "is-videos-supported": false,
  "is-snap": false,
  "arch": "amd64",
  "webserver": "Caddy/v2.7.4",
  "database": {
    "type": "mysql",
    "version": "10.6.15",
    "size": "642736128"
  },
  "php-version": "8.2.10",
  "php-interpreter": "php",
  "python-interpreter-setting": "/usr/bin/python3",
  "os": "Linux",
  "os-release": "6.4.7-200.fc38.x86_64",
  "machine-type": "x86_64"
}

Python API:

{
  "nextcloud-version": "27.0.2.1",
  "app-versions": {
    "cloud_py_api-version": "0.1.8"
  },
  "is-videos-supported": false,
  "is-snap": false,
  "arch": "amd64",
  "webserver": "Caddy/v2.7.4",
  "database": {
    "type": "mysql",
    "version": "10.6.15",
    "size": "642736128"
  },
  "php-version": "8.2.10",
  "php-interpreter": "php",
  "python-interpreter-setting": "/usr/bin/python3",
  "os": "Linux",
  "os-release": "6.4.7-200.fc38.x86_64",
  "machine-type": "x86_64"
}

Additional context
Nextcloud issue report data: https://gist.github.com/rugk/93b16842749408755ac64bdf7331f462

Unrelated(?), because in my Nextcloud log it does not happen at the same time when I trigger the request this error appears:

{
    "reqId": "ksGAxoR4j99BDwyu6uGP",
    "level": 3,
    "time": "2023-09-06T20:06:27+02:00",
    "remoteAddr": "****",
    "user": "rugk",
    "app": "mediadc",
    "method": "DELETE",
    "url": "/apps/mediadc/api/v1/tasks/3",
    "message": "Can't terminate CollectorTask background process.\n{\"id\":3,\"type\":\"manual\",\"owner\":\"rugk\",\"target_directory_ids\":\"[\\\"485\\\"]\",\"exclude_list\":\"{\\\"user\\\":{\\\"mask\\\":[],\\\"fileid\\\":[]},\\\"admin\\\":{\\\"mask\\\":[],\\\"fileid\\\":[]}}\",\"collector_settings\":\"{\\\"hashing_algorithm\\\":\\\"dhash\\\",\\\"similarity_threshold\\\":100,\\\"hash_size\\\":16,\\\"target_mtype\\\":0,\\\"finish_notification\\\":true,\\\"duplicated\\\":false}\",\"files_scanned\":38,\"files_total\":5722,\"files_total_size\":12232414183,\"deleted_files_count\":0,\"deleted_files_size\":0,\"created_time\":1694023458,\"finished_time\":0,\"updated_time\":1694023515,\"py_pid\":253,\"errors\":\"\"}",
    "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/117.0",
    "version": "27.0.2.1",
    "data": {
        "app": "mediadc"
    },
    "id": "64f8dc0a6c349"
}

[mediadc] Fehler: Can't terminate CollectorTask background process.
{"id":3,"type":"manual","owner":"rugk","target_directory_ids":"["485"]","exclude_list":"{"user":{"mask":[],"fileid":[]},"admin":{"mask":[],"fileid":[]}}","collector_settings":"{"hashing_algorithm":"dhash","similarity_threshold":100,"hash_size":16,"target_mtype":0,"finish_notification":true,"duplicated":false}","files_scanned":38,"files_total":5722,"files_total_size":12232414183,"deleted_files_count":0,"deleted_files_size":0,"created_time":1694023458,"finished_time":0,"updated_time":1694023515,"py_pid":253,"errors":""}

DELETE /apps/mediadc/api/v1/tasks/3
from ***** by rugk at 2023-09-06T20:06:27+02:00

But that was just that I had to delete one (stopped, and before of that stop the) task to execute the new one (also BTW don't get why already finished tasks count against the limit of running at most 3 tasks?Edit: Oh I misinterpreted that, it is 1 for running. forget that.). And I pressed the delete or stop button multiple times so…
@rugk rugk changed the title Specific job setting does result in 503 Service Unavailable error Specific job setting does result in 503 Service Unavailable error ("Storage is temporarily not available") Sep 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@rugk