Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IntegrityError at /api/v2/search when searching remote handles #642

Open
Ultimator14 opened this issue Sep 3, 2023 · 16 comments
Open

IntegrityError at /api/v2/search when searching remote handles #642

Ultimator14 opened this issue Sep 3, 2023 · 16 comments
Labels
area/api The OAuth/client app API bug Something isn't working pri/medium Medium Priority

Comments

@Ultimator14
Copy link

I'm using Tusky as client app for my own takahe server. In the takahe configuration I enabled ERROR_EMAILS.
Whenever I search for a (remote) handle the first time e.g. @[email protected] using the Tusky search, I get two mails informing me about a server error.

Internal Server Error: /api/v2/search

IntegrityError at /api/v2/search
doppelter Schlüsselwert verletzt Unique-Constraint »users_identity_actor_uri_key«
DETAIL:  Schlüssel »(actor_uri)=(https://jointakahe.takahe.social/@[email protected]/)« existiert bereits.

(full message see below)

This happens independently of the remote server software. I noticed this for mastodon, takahe, kbin and also some others. This only happens if the user is unknown to the server, so the very first time I do the search. Afterwards, the error does no longer occur but only the pinned posts are shown in the Tusky app and follows are also not displayed correctly (see screenshot).

Log in error mail (The log leaks some usernames and passwords, I redacted them manually):

Internal Server Error: /api/v2/search

IntegrityError at /api/v2/search
doppelter Schlüsselwert verletzt Unique-Constraint »users_identity_actor_uri_key«
DETAIL:  Schlüssel »(actor_uri)=(https://jointakahe.takahe.social/@[email protected]/)« existiert bereits.

Request Method: GET
Request URL: https://social.pygos.space/api/v2/search?q=%40takahe%40jointakahe.org&type=statuses&resolve=true&limit=20&offset=0&following=false
Django Version: 4.2.4
Python Executable: /usr/local/bin/python3
Python Version: 3.11.5
Python Path: ['/takahe', '/usr/local/bin', '/usr/local/lib/python311.zip', '/usr/local/lib/python3.11', '/usr/local/lib/python3.11/lib-dynload', '/usr/local/lib/python3.11/site-packages']
Server time: Sun, 03 Sep 2023 20:34:01 +0000
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.postgres',
'corsheaders',
'django_htmx',
'hatchway',
'core',
'activities',
'api',
'mediaproxy',
'stator',
'users']
Installed Middleware:
['core.middleware.SentryTaggingMiddleware',
'django.middleware.security.SecurityMiddleware',
'corsheaders.middleware.CorsMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django_htmx.middleware.HtmxMiddleware',
'core.middleware.HeadersMiddleware',
'core.middleware.ConfigLoadingMiddleware',
'api.middleware.ApiTokenMiddleware',
'users.middleware.DomainMiddleware']


Traceback (most recent call last):
 File "/takahe/activities/services/search.py", line 37, in search_identities_handle
   raise Identity.DoesNotExist()
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

During handling of the above exception (), another exception occurred:
 File "/takahe/users/models/identity.py", line 405, in by_username_and_domain
   return cls.objects.get(
          
 File "/usr/local/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
   return getattr(self.get_queryset(), name)(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 637, in get
   raise self.model.DoesNotExist(
   ^

During handling of the above exception (Identity matching query does not exist.), another exception occurred:
 File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
   return self.cursor.execute(sql, params)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/psycopg/cursor.py", line 737, in execute
   raise ex.with_traceback(None)
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The above exception (doppelter Schlüsselwert verletzt Unique-Constraint »users_identity_actor_uri_key«
DETAIL:  Schlüssel »(actor_uri)=(https://jointakahe.takahe.social/@[email protected]/)« existiert bereits.) was the direct cause of the following exception:
 File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
   response = get_response(request)
              ^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
   response = wrapped_callback(request, *callback_args, **callback_kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/takahe/api/decorators.py", line 46, in inner
   return function(request, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/hatchway/view.py", line 302, in __call__
   response = self.view(request, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/takahe/api/views/search.py", line 35, in search
   search_result = searcher.search_all()
                   ^^^^^^^^^^^^^^^^^^^^^
 File "/takahe/activities/services/search.py", line 138, in search_all
   "identities": self.search_identities_handle(),
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/takahe/activities/services/search.py", line 47, in search_identities_handle
   identity = Identity.by_username_and_domain(
              
 File "/takahe/users/models/identity.py", line 423, in by_username_and_domain
   return cls.objects.create(
          
 File "/usr/local/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
   return getattr(self.get_queryset(), name)(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 658, in create
   obj.save(force_insert=True, using=self.db)
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 814, in save
   self.save_base(
   ^
 File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 877, in save_base
   updated = self._save_table(
             
 File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1020, in _save_table
   results = self._do_insert(
             
 File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1061, in _do_insert
   return manager._insert(
          
 File "/usr/local/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
   return getattr(self.get_queryset(), name)(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1805, in _insert
   return query.get_compiler(using=using).execute_sql(returning_fields)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1822, in execute_sql
   cursor.execute(sql, params)
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
   return self._execute_with_wrappers(
          
 File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
   return executor(sql, params, many, context)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
   with self.db.wrap_database_errors:
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
   raise dj_exc_value.with_traceback(traceback) from exc_value
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
   return self.cursor.execute(sql, params)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/psycopg/cursor.py", line 737, in execute
   raise ex.with_traceback(None)
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Exception Type: IntegrityError at /api/v2/search
Exception Value: doppelter Schlüsselwert verletzt Unique-Constraint »users_identity_actor_uri_key«
DETAIL:  Schlüssel »(actor_uri)=(https://jointakahe.takahe.social/@[email protected]/)« existiert bereits.
Raised during: hatchway.view.inner
Request information:
USER: [[email protected]](mailto:[email protected])

GET:
q = '@[email protected]'
type = 'statuses'
resolve = 'true'
limit = '20'
offset = '0'
following = 'false'

POST: No POST data

FILES: No FILES data

COOKIES: No cookie data

META:
HTTP_ACCEPT_ENCODING = 'gzip'
HTTP_AUTHORIZATION = 'Bearer <REDACTED>'
HTTP_CONNECTION = 'close'
HTTP_FORWARDED = 'for=93.205.182.221;proto=https;host="social.pygos.space"'
HTTP_HOST = 'social.pygos.space'
HTTP_USER_AGENT = 'Tusky/23.0 Android/13 OkHttp/4.11.0'
HTTP_X_FORWARDED_FOR = '93.205.182.221, 192.168.157.12'
HTTP_X_FORWARDED_HOST = 'social.pygos.space'
HTTP_X_FORWARDED_PROTO = 'https'
HTTP_X_HOST = 'social.pygos.space'
PATH_INFO = '/api/v2/search'
QUERY_STRING = 'q=%40takahe%40jointakahe.org&type=statuses&resolve=true&limit=20&offset=0&following=false'
RAW_URI = '/api/v2/search?q=%40takahe%40jointakahe.org&type=statuses&resolve=true&limit=20&offset=0&following=false'
REMOTE_ADDR = '127.0.0.1'
REMOTE_PORT = '52502'
REQUEST_METHOD = 'GET'
SCRIPT_NAME = ''
SERVER_NAME = '0.0.0.0'
SERVER_PORT = '8001'
SERVER_PROTOCOL = 'HTTP/1.1'
SERVER_SOFTWARE = 'gunicorn/20.1.0'
gunicorn.socket = <socket.socket fd=9, family=2, type=1, proto=0, laddr=('127.0.0.1', 8001), raddr=('127.0.0.1', 52502)>
wsgi.errors = <gunicorn.http.wsgi.WSGIErrorsWrapper object at 0x7f7b18e590>
wsgi.file_wrapper = <class 'gunicorn.http.wsgi.FileWrapper'>
wsgi.input = <gunicorn.http.body.Body object at 0x7f7b1d3190>
wsgi.input_terminated = True
wsgi.multiprocess = True
wsgi.multithread = False
wsgi.run_once = False
wsgi.url_scheme = 'https'
wsgi.version = '(1, 0)'

Settings:
Using settings module takahe.settings
ABSOLUTE_URL_OVERRIDES = {}
ADMINS = [('Admin', '[<REDACTED>@pygos.space](mailto:<REDACTED>@pygos.space)')]
ALLOWED_HOSTS = ['*']
APPEND_SLASH = True
AUTHENTICATION_BACKENDS = ['django.contrib.auth.backends.ModelBackend']
AUTH_PASSWORD_VALIDATORS = '********************'
AUTH_USER_MODEL = 'users.User'
AUTO_ADMIN_EMAIL = None
BASE_DIR = PosixPath('/takahe')
CACHES = {'default': {'BACKEND': 'django.core.cache.backends.dummy.DummyCache', 'LOCATION': ''}}
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_KEY_PREFIX = '********************'
CACHE_MIDDLEWARE_SECONDS = 600
CORS_ALLOW_CREDENTIALS = True
CORS_ALLOW_HEADERS = "('accept', 'accept-encoding', 'authorization', 'content-type', 'dnt', 'origin', 'user-agent', 'x-csrftoken', 'x-requested-with', 'Idempotency-Key')"
CORS_EXPOSE_HEADERS = "('link',)"
CORS_ORIGIN_ALLOW_ALL = True
CORS_ORIGIN_WHITELIST = []
CORS_PREFLIGHT_MAX_AGE = 604800
CSRF_COOKIE_AGE = 31449600
CSRF_COOKIE_DOMAIN = None
CSRF_COOKIE_HTTPONLY = False
CSRF_COOKIE_MASKED = False
CSRF_COOKIE_NAME = 'csrftoken'
CSRF_COOKIE_PATH = '/'
CSRF_COOKIE_SAMESITE = 'Lax'
CSRF_COOKIE_SECURE = False
CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure'
CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN'
CSRF_TRUSTED_ORIGINS = ['https://social.pygos.space/']
CSRF_USE_SESSIONS = False
DATABASES = {'default': {'NAME': 'takahe', 'USER': '<REDACTED>', 'PASSWORD': '********************', 'HOST': 'localhost', 'PORT': '', 'CONN_MAX_AGE': 600, 'OPTIONS': {'sslmode': 'disable'}, 'ENGINE': 'django.db.backends.postgresql', 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_HEALTH_CHECKS': False, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'MIGRATE': True, 'MIRROR': None, 'NAME': None}}}
DATABASE_ROUTERS = []
DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440
DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000
DATA_UPLOAD_MAX_NUMBER_FILES = 100
DATETIME_FORMAT = 'N j, Y, P'
DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M']
DATE_FORMAT = 'N j, Y'
DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y']
DEBUG = False
DEBUG_PROPAGATE_EXCEPTIONS = False
DECIMAL_SEPARATOR = '.'
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'
DEFAULT_CHARSET = 'utf-8'
DEFAULT_EXCEPTION_REPORTER = 'django.views.debug.ExceptionReporter'
DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter'
DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'
DEFAULT_FROM_EMAIL = 'webmaster@localhost'
DEFAULT_INDEX_TABLESPACE = ''
DEFAULT_TABLESPACE = ''
DISALLOWED_USER_AGENTS = []
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'pygos.space'
EMAIL_HOST_PASSWORD = '********************'
EMAIL_HOST_USER = '<REDACTED>'
EMAIL_PORT = 587
EMAIL_SSL_CERTFILE = None
EMAIL_SSL_KEYFILE = '********************'
EMAIL_SUBJECT_PREFIX = '[Django] '
EMAIL_TIMEOUT = None
EMAIL_USE_LOCALTIME = False
EMAIL_USE_SSL = False
EMAIL_USE_TLS = True
FILE_UPLOAD_DIRECTORY_PERMISSIONS = None
FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler']
FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440
FILE_UPLOAD_PERMISSIONS = 420
FILE_UPLOAD_TEMP_DIR = None
FIRST_DAY_OF_WEEK = 0
FIXTURE_DIRS = []
FORCE_SCRIPT_NAME = None
FORMAT_MODULE_PATH = None
FORM_RENDERER = 'django.forms.renderers.DjangoTemplates'
IGNORABLE_404_URLS = []
INSTALLED_APPS = ['django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.postgres', 'corsheaders', 'django_htmx', 'hatchway', 'core', 'activities', 'api', 'mediaproxy', 'stator', 'users']
INTERNAL_IPS = []
JSONLD_MAX_SIZE = 51200
LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ar-dz', 'Algerian Arabic'), ('ast', 'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('ckb', 'Central Kurdish (Sorani)'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en-gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', 'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('hy', 'Armenian'), ('ia', 'Interlingua'), ('id', 'Indonesian'), ('ig', 'Igbo'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'Japanese'), ('ka', 'Georgian'), ('kab', 'Kabyle'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('ky', 'Kyrgyz'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('ms', 'Malay'), ('my', 'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Telugu'), ('tg', 'Tajik'), ('th', 'Thai'), ('tk', 'Turkmen'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('uz', 'Uzbek'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'), ('zh-hant', 'Traditional Chinese')]
LANGUAGES_BIDI = ['he', 'ar', 'ar-dz', 'ckb', 'fa', 'ur']
LANGUAGE_CODE = 'en-us'
LANGUAGE_COOKIE_AGE = None
LANGUAGE_COOKIE_DOMAIN = None
LANGUAGE_COOKIE_HTTPONLY = False
LANGUAGE_COOKIE_NAME = 'django_language'
LANGUAGE_COOKIE_PATH = '/'
LANGUAGE_COOKIE_SAMESITE = None
LANGUAGE_COOKIE_SECURE = False
LOCALE_PATHS = []
LOGGING = {}
LOGGING_CONFIG = 'logging.config.dictConfig'
LOGIN_REDIRECT_URL = '/'
LOGIN_URL = '/auth/login/'
LOGOUT_REDIRECT_URL = '/'
LOGOUT_URL = '/auth/logout/'
MAIN_DOMAIN = 'social.pygos.space'
MANAGERS = []
MEDIA_ROOT = '/takahe/media/'
MEDIA_URL = 'https://social.pygos.space/media/'
MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage'
MIDDLEWARE = ['core.middleware.SentryTaggingMiddleware', 'django.middleware.security.SecurityMiddleware', 'corsheaders.middleware.CorsMiddleware', 'whitenoise.middleware.WhiteNoiseMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django_htmx.middleware.HtmxMiddleware', 'core.middleware.HeadersMiddleware', 'core.middleware.ConfigLoadingMiddleware', 'api.middleware.ApiTokenMiddleware', 'users.middleware.DomainMiddleware']
MIGRATION_MODULES = {}
MONTH_DAY_FORMAT = 'F j'
NUMBER_GROUPING = 0
PASSWORD_HASHERS = '********************'
PASSWORD_RESET_TIMEOUT = '********************'
PREPEND_WWW = False
ROBOTS_TXT_DISALLOWED_USER_AGENTS = []
ROOT_URLCONF = 'takahe.urls'
SECRET_KEY = '********************'
SECRET_KEY_FALLBACKS = '********************'
SECURE_CONTENT_TYPE_NOSNIFF = True
SECURE_CROSS_ORIGIN_OPENER_POLICY = 'same-origin'
SECURE_HSTS_INCLUDE_SUBDOMAINS = False
SECURE_HSTS_PRELOAD = False
SECURE_HSTS_SECONDS = 0
SECURE_PROXY_SSL_HEADER = "('HTTP_X_FORWARDED_PROTO', 'https')"
SECURE_REDIRECT_EXEMPT = []
SECURE_REFERRER_POLICY = 'same-origin'
SECURE_SSL_HOST = None
SECURE_SSL_REDIRECT = False
SERVER_EMAIL = '[[email protected]](mailto:[email protected])'
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 1209600
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_NAME = 'sessionid'
SESSION_COOKIE_PATH = '/'
SESSION_COOKIE_SAMESITE = 'Lax'
SESSION_COOKIE_SECURE = False
SESSION_ENGINE = 'django.contrib.sessions.backends.signed_cookies'
SESSION_EXPIRE_AT_BROWSER_CLOSE = False
SESSION_FILE_PATH = None
SESSION_SAVE_EVERY_REQUEST = False
SESSION_SERIALIZER = 'django.contrib.sessions.serializers.JSONSerializer'
SETTINGS_MODULE = 'takahe.settings'
SETUP = Settings(DATABASE_SERVER=ImplicitHostname('postgres://<REDACTED>@localhost/takahe?sslmode=disable', scheme='postgres', user='<REDACTED>', password='<REDACTED>', host='localhost', host_type='int_domain', path='/takahe', query='sslmode=disable'), ENVIRONMENT='development', DEBUG=False, DEBUG_TOOLBAR=False, LOCAL_SETTINGS=False, SECRET_KEY='<REDACTED>', STATOR_TOKEN='68f9cb85d28c1dfd842128bba90f4082e680cf2fbc400c743729fb1b5c52880462ca68cd412b071ff7662d37609547095504ddbf19156cdb9415409fe042b57fd48a0edc658ec58e73b57ecedaa79bf9671176a73ab85008eb037398b9423f0a4ae680688d7183c257d07a703eb5733667681f127b8cd4b9ca9c499cc616cea0', ALLOWED_HOSTS=['*'], CORS_HOSTS=[], CSRF_HOSTS=['https://social.pygos.space'], USE_PROXY_HEADERS=True, SENTRY_DSN=None, SENTRY_SAMPLE_RATE=1.0, SENTRY_TRACES_SAMPLE_RATE=0.01, SENTRY_CAPTURE_MESSAGES=False, SENTRY_EXPERIMENTAL_PROFILES_TRACES_SAMPLE_RATE=0.0, MAIN_DOMAIN='social.pygos.space', EMAIL_SERVER=AnyUrl('smtp://<REDACTED>@pygos.space:587?tls=true', scheme='smtp', user='<REDACTED>', password='<REDACTED>', host='pygos.space', tld='space', host_type='domain', port='587', query='tls=true'), EMAIL_FROM='[email protected]', AUTO_ADMIN_EMAIL=None, ERROR_EMAILS=['<REDACTED>@pygos.space'], ROBOTS_TXT_DISALLOWED_USER_AGENTS=[], MEDIA_URL='https://social.pygos.space/media/', MEDIA_ROOT='/takahe/media/', MEDIA_BACKEND=MediaBackendUrl('local://', scheme='local'), MEDIA_BACKEND_S3_ACL='public-read', MEDIA_MAX_IMAGE_FILESIZE_MB=10, AVATAR_MAX_IMAGE_FILESIZE_KB=1000, EMOJI_MAX_IMAGE_FILESIZE_KB=200, REMOTE_TIMEOUT=5.0, SEARCH=True, CACHES_DEFAULT=None, STATOR_CONCURRENCY=50, STATOR_CONCURRENCY_PER_MODEL=15, VAPID_PUBLIC_KEY=None, VAPID_PRIVATE_KEY=None, PGHOST=None, PGPORT=5432, PGNAME='takahe', PGUSER='postgres', PGPASSWORD=None)
SHORT_DATETIME_FORMAT = 'm/d/Y P'
SHORT_DATE_FORMAT = 'm/d/Y'
SIGNING_BACKEND = 'django.core.signing.TimestampSigner'
SILENCED_SYSTEM_CHECKS = []
STATICFILES_DIRS = [PosixPath('/takahe/static')]
STATICFILES_FINDERS = ['django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder']
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
STATIC_ROOT = PosixPath('/takahe/static-collected')
STATIC_URL = '/static/'
STATOR_CONCURRENCY = 50
STATOR_CONCURRENCY_PER_MODEL = 15
STATOR_TOKEN = '********************'
STORAGES = {'staticfiles': {'BACKEND': 'django.contrib.staticfiles.storage.ManifestStaticFilesStorage'}, 'default': {'BACKEND': 'django.core.files.storage.FileSystemStorage'}}
TAKAHE_ENV_FILE = '.env'
TAKAHE_USER_AGENT = 'python-httpx/0.24.1 (Takahe/0.10.0-dev; +https://social.pygos.space/)'
TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [PosixPath('/takahe/templates')], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', 'core.context.config_context', 'users.context.user_context']}}]
TEST_NON_SERIALIZED_APPS = []
TEST_RUNNER = 'django.test.runner.DiscoverRunner'
THOUSAND_SEPARATOR = ','
TIME_FORMAT = 'P'
TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
TIME_ZONE = 'UTC'
USE_DEPRECATED_PYTZ = False
USE_I18N = True
USE_L10N = True
USE_THOUSAND_SEPARATOR = False
USE_TZ = True
USE_X_FORWARDED_HOST = False
USE_X_FORWARDED_PORT = False
WHITENOISE_MAX_AGE = 3600
WSGI_APPLICATION = 'takahe.wsgi.application'
X_FRAME_OPTIONS = 'DENY'
YEAR_MONTH_FORMAT = 'F Y'

Screenshot from the Tusky app:
Screenshot_20230903-224602_Tusky

@andrewgodwin
Copy link
Member

What version are you running? I suspect this was already fixed as part of #634, it's just not in a release yet.

@Ultimator14
Copy link
Author

I'm on main. The build is from yesterday

@Ultimator14
Copy link
Author

Ultimator14 commented Sep 3, 2023

The error does not occur when I reduce gunicorn workers to 1 (GUNICORN_EXTRA_CMD_ARGS: "--reload -w 1").
But posts and follows are still not displayed. That is probably another issue.

@andrewgodwin
Copy link
Member

andrewgodwin commented Sep 4, 2023

OK - I'll see if I can investigate it soon then. It might be a few weeks. Looks like some sort of error in the new fetching code, maybe - have you tried the current stable release to see if it still breaks there?

@andrewgodwin andrewgodwin added bug Something isn't working area/api The OAuth/client app API pri/medium Medium Priority labels Sep 4, 2023
@Ultimator14
Copy link
Author

No I haven't and unfortunately I can't downgrade due to database issues. I'll try to check this with a test instance.

@andrewgodwin
Copy link
Member

Ah right, we've had migrations on main since the release (which is incidentally why I have not done a patch release yet). Let me know what you find - if this is caused by our fix for the other error breaking search, it would be good to know.

@Ultimator14
Copy link
Author

But posts and follows are still not displayed. That is probably another issue.

I did some research and at least for mastodon, only follows of the local instance are shown in the mobile app. And for posts, only future posts are downloaded to the local instance and counted.
That means if I search for a new user once, the user is known to my server from that point in time. If I check the users profile via the mobile app, it shows me 0 follows and 0 posts (except there are pinned posts, which will be backfilled). If the user posts something from now on, it appears on the profile and the post numbers goes up by one. If I follow, the follow number goes up by one. To get the actual number of posts and followers, I have to check the web page of the remote profile.

It looks like this is desired behavior.

if this is caused by our fix for the other error breaking search, it would be good to know.

The same error also occurs on 0.9.0. Error log for version 0.9.0 below.

takahe-web      | Internal Server Error: /api/v2/search
takahe-web      | Traceback (most recent call last):
takahe-web      |   File "/takahe/activities/services/search.py", line 38, in search_identities_handle
takahe-web      |     raise Identity.DoesNotExist()
takahe-web      | users.models.identity.Identity.DoesNotExist
takahe-web      | 
takahe-web      | During handling of the above exception, another exception occurred:
takahe-web      | 
takahe-web      | Traceback (most recent call last):
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/asgiref/sync.py", line 349, in main_wrap                                                                                                         
takahe-web      |     raise exc_info[1]
takahe-web      |   File "/takahe/users/models/identity.py", line 362, in by_username_and_domain
takahe-web      |     return cls.objects.get(
takahe-web      |            ^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
takahe-web      |     return getattr(self.get_queryset(), name)(*args, **kwargs)
takahe-web      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 637, in get                                                                                                     
takahe-web      |     raise self.model.DoesNotExist(
takahe-web      | users.models.identity.Identity.DoesNotExist: Identity matching query does not exist.
takahe-web      | 
takahe-web      | During handling of the above exception, another exception occurred:
takahe-web      | 
takahe-web      | Traceback (most recent call last):
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
takahe-web      |     return self.cursor.execute(sql, params)
takahe-web      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/psycopg/cursor.py", line 737, in execute                                                                                                         
takahe-web      |     raise ex.with_traceback(None)
takahe-web      | psycopg.errors.UniqueViolation: duplicate key value violates unique constraint "users_identity_actor_uri_key"
takahe-web      | DETAIL:  Key (actor_uri)=(https://jointakahe.takahe.social/@[email protected]/) already exists.
takahe-web      | The above exception was the direct cause of the following exception:
takahe-web      | 
takahe-web      | Traceback (most recent call last):
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
takahe-web      |     response = get_response(request)
takahe-web      |                ^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
takahe-web      |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
takahe-web      |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/takahe/api/decorators.py", line 46, in inner
takahe-web      |     return function(request, *args, **kwargs)
takahe-web      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/hatchway/view.py", line 302, in __call__                                                                                                         
takahe-web      |     response = self.view(request, **kwargs)
takahe-web      |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/takahe/api/views/search.py", line 35, in search
takahe-web      |     search_result = searcher.search_all()
takahe-web      |                     ^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/takahe/activities/services/search.py", line 139, in search_all
takahe-web      |     "identities": self.search_identities_handle(),
takahe-web      |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/takahe/activities/services/search.py", line 48, in search_identities_handle
takahe-web      |     identity = Identity.by_username_and_domain(
takahe-web      |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/takahe/users/models/identity.py", line 382, in by_username_and_domain
takahe-web      |     return cls.objects.create(
takahe-web      |            ^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
takahe-web      |     return getattr(self.get_queryset(), name)(*args, **kwargs)
takahe-web      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 658, in create
takahe-web      |     obj.save(force_insert=True, using=self.db)
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 814, in save                                                                                                     
takahe-web      |     self.save_base(
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 877, in save_base
takahe-web      |     updated = self._save_table(
takahe-web      |               ^^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1020, in _save_table
takahe-web      |     results = self._do_insert(
takahe-web      |               ^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1061, in _do_insert
takahe-web      |     return manager._insert(
takahe-web      |            ^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
takahe-web      |     return getattr(self.get_queryset(), name)(*args, **kwargs)
takahe-web      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1805, in _insert
takahe-web      |     return query.get_compiler(using=using).execute_sql(returning_fields)
takahe-web      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1822, in execute_sql
takahe-web      |     cursor.execute(sql, params)
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 102, in execute
takahe-web      |     return super().execute(sql, params)
takahe-web      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
takahe-web      |     return self._execute_with_wrappers(
takahe-web      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
takahe-web      |     return executor(sql, params, many, context)
takahe-web      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
takahe-web      |     with self.db.wrap_database_errors:
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__                                                                                                        
takahe-web      |     raise dj_exc_value.with_traceback(traceback) from exc_value
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
takahe-web      |     return self.cursor.execute(sql, params)
takahe-web      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
takahe-web      |   File "/usr/local/lib/python3.11/site-packages/psycopg/cursor.py", line 737, in execute                                                                                                         
takahe-web      |     raise ex.with_traceback(None)
takahe-web      | django.db.utils.IntegrityError: duplicate key value violates unique constraint "users_identity_actor_uri_key"
takahe-web      | DETAIL:  Key (actor_uri)=(https://jointakahe.takahe.social/@[email protected]/) already exists.

@andrewgodwin
Copy link
Member

Thanks, appreciate you checking! I can't easily replicate it on my own server, but the traceback you provided should hopefully be enough to put a potential fix in place anyway.

@andrewgodwin
Copy link
Member

Hmm, taking at a look at the source code, we already have something in place to stop this happening - it's meant to check if the user exists right before it makes it.

Are you able to go into the Django Admin for me and get a screenshot of what the https://jointakahe.takahe.social/@[email protected]/ / @[email protected] identity looks like, if it even exists? I suspect something about the record is corrupted but I don't know how.

@Ultimator14
Copy link
Author

Are you able to go into the Django Admin for me and get a screenshot of what the https://jointakahe.takahe.social/@[email protected]/ / @[email protected] identity looks like, if it even exists? I suspect something about the record is corrupted but I don't know how.

Like this?

Bildschirmfoto vom 2023-09-15 18-12-41

Without looking at the code I assume this is a concurrency issue because it only occurs the first time a search for a user. I guess there are two processes that check for the existence of the db entry, both get the result that it doesn't exist and both try to create it. The creation fails the second time because the first one already created it.
The issue also doesn't occur with only one worker.

@andrewgodwin
Copy link
Member

Ah, I missed that it only happens on the first search for a user, sorry - you're probably right. The search endpoint should only be called once, but my guess is that the client you're using calls it twice for some reason.

@andrewgodwin
Copy link
Member

OK, the commit I just landed above should fix it - I made that whole process happen in a transaction. Let me know if it doesn't and we can reopen.

@Ultimator14
Copy link
Author

Ah nvm the error still occurs. Same as before :/

@Ultimator14
Copy link
Author

I did some testing and added a few print statements to see whats wrong.

Diff
diff --git a/users/models/identity.py b/users/models/identity.py                                                                                                                                                   
index bebaa31..46f5b14 100644                                                                                                                                                                                      
--- a/users/models/identity.py                                                                                                                                                                                     
+++ b/users/models/identity.py                                                                                                                                                                                     
@@ -2,6 +2,9 @@ import ssl                                                                                                                                                                                         
 from functools import cached_property, partial                                                                                                                                                                    
 from typing import Literal, Optional                                                                                                                                                                              
 from urllib.parse import urlparse                                                                                                                                                                                 
+import traceback                                                                                                                                                                                                  
+from datetime import datetime                                                                                                                                                                                     
+from random import randint                                                                                                                                                                                        
                                                                                                                                                                                                                   
 import httpx
 import urlman

@@ -396,14 +399,20 @@ class Identity(StatorModel):
             domain = domain.lower()
 
         with transaction.atomic():
+            debug_nonce = randint(0, 2**16 - 1)
+            tprint = lambda x: print(str(debug_nonce) + " " + str(datetime.now().strftime("%d.%m.%Y %H:%M:%S.%f")) + " " + x)
+            #traceback.print_stack()
+            tprint("Entering transaction atomic")
             try:
                 if local:
+                    tprint("local-before-return")
                     return cls.objects.get(
                         username__iexact=username,
                         domain_id=domain,
                         local=True,
                     )
                 else:
+                    tprint("else-before-return")
                     return cls.objects.get(
                         username__iexact=username,
                         domain_id=domain,
@@ -412,9 +421,11 @@ class Identity(StatorModel):
                 if fetch and not local:
                     actor_uri, handle = cls.fetch_webfinger(f"{username}@{domain}")
                     if handle is None:
+                        tprint("handle-before-return")
                         return None
                     # See if this actually does match an existing actor
                     try:
+                        tprint("try-before-return")
                         return cls.objects.get(actor_uri=actor_uri)
                     except cls.DoesNotExist:
                         pass
@@ -422,13 +433,16 @@ class Identity(StatorModel):
                     username, domain = handle.split("@")
                     if not domain_instance:
                         domain_instance = Domain.get_remote_domain(domain)
+                    tprint("domain-before-return")
                     return cls.objects.create(
                         actor_uri=actor_uri,
                         username=username,
                         domain_id=domain_instance,
                         local=False,
                     )
+                tprint("final-before-return")
                 return None
+            tprint("no-return hit")
 
     @classmethod
     def by_actor_uri(cls, uri, create=False, transient=False) -> "Identity":

And I got the following output (random number per process, time, message), sorted by time:

244 16.09.2023 09:38:36.219160 Entering transaction atomic
244 16.09.2023 09:38:36.219348 else-before-return

61503 16.09.2023 09:38:36.227405 Entering transaction atomic
61503 16.09.2023 09:38:36.227565 else-before-return
61503 16.09.2023 09:38:36.682489 try-before-return
61503 16.09.2023 09:38:36.687559 domain-before-return

244 16.09.2023 09:38:36.704224 try-before-return

So there are two processes that run concurrently. But even with atomic transaction, the second process starts executing the code before the first one is finished.

Maybe also relevant: I see no error in the docker log, it's only sent via mail due to the TAKAHE_ERROR_EMAILS setting.

@andrewgodwin
Copy link
Member

Atomic transactions don't guarantee that the code isn't run concurrently, just that the view of the database should be linear - though that's obviously not quite what's happening here.

I have some other ideas about how to fix this, so I'll try one of those in the next commit.

@AstraLuma
Copy link
Contributor

Did that next commit happen?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/api The OAuth/client app API bug Something isn't working pri/medium Medium Priority
Projects
None yet
Development

No branches or pull requests

3 participants