Unable to upload large number of subjects because of token expiration

[himeshr]

env: prod
username: sachink@cini_uat

Issue:

• On csv uploading of large number of subjects, we are facing repeated 401 errors during upload of Subjects.
• Once these fail, with TokenExpiredException for a row, all consecutive rows also repeatedly fail with the same error, as token is not refreshed intermediately.
• Uploads need to be done in 1000s and hence smaller size uploads is not easier to do

Technical details:

In rules-server, we make use of "upload-user" username and its password in getUploadUserToken(), to fetch token, that is to be used to invoke Avni APIs like "/upload" or "/upload/media", etc.. looks like some issue with token refreshing.

Refer attached file for error stack trace: ErrorStackTrace.txt
Link of the error file

[1t5j0y]

Issue is not with the token for upload-user on rules-server. That is getting set for each call.
Issue is with the token expiring for the user that triggered the upload job which is set in the request to rule server.

To unblock implementation team for now, have advised them to logout/login to webapp just before upload and chunk the upload into batches of 1000 subjects. The upload roughly taking 25 mins for 500 subjects.

Will need to discuss long term solution.

[1t5j0y]

Issue was that the token for the user who triggered the upload was being used for calls from rules-server back to avni-server to fetch rules (/rule/search) and ruledependencies (/ruleDependency/search) which would expire during upload.
Fixed to use token for upload-user for these calls as well.