Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add retry handler for matrix rate limits #121

Open
wants to merge 35 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
7fda8f9
feat: add retry handler to await ratelimiting when creating new rooms
Sep 27, 2023
5b524c2
refactor: add `dev.medienhaus.meta` to initial state instead of separ…
Sep 27, 2023
720c47c
refactor: better logging, add fallback in case no ms are returned fro…
Sep 27, 2023
fd12fb3
feat: add user feadback while syncing and translations
Sep 27, 2023
5fa9bba
refactor: move retry function into Matrix.js for global use and renam…
Oct 4, 2023
5f07e0b
refactor: use retry function when creating applications folder and se…
Oct 4, 2023
59790a9
chore: improve grammar
Oct 4, 2023
0b24c5d
refactor: add retry handling when encountering rate limits
Oct 4, 2023
6b0e9c0
refactor: improve logic and fix messages being sent twice
Oct 4, 2023
aefd37d
chore: remove debug log
Oct 4, 2023
5273d24
refactor: show user feedback when syncing pads from server
Oct 4, 2023
21567e6
merge main into spaces-ratelimiting
Oct 5, 2023
b66e40e
fix: wrap error message in translation.
Oct 5, 2023
60a09fa
refactor: remove retries parameter since it is not being used after r…
Oct 5, 2023
5de86f8
Merge branch 'main' into spaces-ratelimiting
Oct 10, 2023
03478f2
merge main into spaces-ratelimiting
Nov 1, 2023
bc046fc
chore: remove multiple empty lines
Nov 1, 2023
b2c2bb3
chore: remove unused import
Nov 1, 2023
a7d1f8c
Merge branch 'main' into spaces-ratelimiting
Nov 15, 2023
925d736
chore: improve translation
Nov 15, 2023
283f680
chore: make linter happy
Nov 15, 2023
f8e60f4
chore: make linter happy
Nov 15, 2023
716c3b8
Merge branch 'main' into spaces-ratelimiting
fnwbr Nov 15, 2023
5981d4b
refactor: move retry logic into Matrix.js
Nov 16, 2023
7244efa
refactor: remove retry logic
Nov 16, 2023
7bf72c8
Merge branch 'main' into spaces-ratelimiting
fnwbr Nov 21, 2023
2cf7045
Improve UI around synchronizing Spacedeck entries
fnwbr Nov 21, 2023
6bb5679
Decrease <LoadingSpinnerInline> size
fnwbr Nov 21, 2023
5cc5585
merge main into matrix-ratelimiting
aofn Jan 9, 2024
944e41b
fix: manipulate state correctly
aofn Jan 9, 2024
0af0eac
fix: only parse link to function which creates the necessary object
aofn Jan 9, 2024
51d9879
fix: use loading spinner inline style from main
aofn Jan 9, 2024
4f517dc
Merge branch 'main' into matrix-ratelimiting
aofn Jan 30, 2024
2abeac0
merge main into matrix-ratelimiting
aofn Feb 28, 2024
37b9a04
merge main into matrix-ratelimiting
aofn Feb 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 90 additions & 13 deletions lib/Matrix.js
Original file line number Diff line number Diff line change
Expand Up @@ -343,19 +343,27 @@ function useMatrixProvider(auth) {

const createRoom = useCallback(
async (name, isSpace, topic, joinRule, type, template, parentId) => {
const room = await authenticationProvider.createRoom(name, isSpace, topic, joinRule, type, template, parentId);
const room = await authenticationProvider.createRoom(name, isSpace, topic, joinRule, type, template, parentId)
.catch((error) => {
return handleRateLimit(error, () => createRoom(name, isSpace, topic, joinRule, type, template, parentId))
.catch(error => {
return error.message;
});
});
// in case of recursion room will already return the desired roomId
const roomId = room.room_id || room;

if (isSpace) {
if (!spaces.has(room.room_id)) {
setSpaces(setRoom(room.room_id));
if (!spaces.has(roomId)) {
setSpaces(setRoom(roomId));
}
} else {
if (!rooms.has(room.room_id)) {
setRooms(setRoom(room.room_id));
if (!rooms.has(roomId)) {
setRooms(setRoom(roomId));
}
}

return room.room_id;
return roomId;
},
[authenticationProvider, rooms, setRoom, setRooms, setSpaces, spaces],
);
Expand All @@ -375,14 +383,16 @@ function useMatrixProvider(auth) {
setApplicationsFolder(existingApplicationsSpace.roomId);
} else {
logger.debug('Creating new root "Applications" space...');
const newApplicationsSpace = await createRoom(
const createNewApplicationsSpace = async () => await createRoom(
'Applications',
true,
'This is your private applications space. You can find all your application data in here.',
'invite',
'context',
'applications',
);
)
.catch(error => logger.debug(error));
const newApplicationsSpace = await createNewApplicationsSpace();
logger.debug('Created new "Applications" space', { newApplicationsSpace });
setApplicationsFolder(newApplicationsSpace);
}
Expand Down Expand Up @@ -420,23 +430,32 @@ function useMatrixProvider(auth) {
});
} else {
logger.debug('Creating new service space...', { serviceType: element });
const roomId = await createRoom(
const createNewServiceRoom = async () => await createRoom(
element,
true,
`This is your private space for the application ${element}. You can find all your ${element} data in here.`,
'invite',
'context',
'application',
applicationsFolder,
);
logger.debug('Created new service space', { serviceType: element, roomId });
await authenticationProvider.addSpaceChild(applicationsFolder, roomId);
).catch(error => logger.debug(error));

const roomId = await createNewServiceRoom();
logger.debug('Created new service space...', { serviceType: element, roomId });

const addServiceRoomToParent = async () => await authenticationProvider.addSpaceChild(applicationsFolder, roomId)
.catch(error => {
return handleRateLimit(error, () => addServiceRoomToParent())
.catch(error => logger.debug(error.message));
});

await addServiceRoomToParent();
setServiceSpaces((object) => {
object[element] = roomId;
});
}
}
}, 500);
}, 1000);

applicationsFolder && _.isEmpty(serviceSpaces) && lookForServiceSpaces();

Expand Down Expand Up @@ -492,6 +511,61 @@ function useMatrixProvider(auth) {
return metaEvent;
};

/**
* Handles retrying a function when encountering a rate limiting error (HTTP 429) from matrix.
*
* @param {Object} error - The error object, typically containing an HTTP status code and additional data.
* @param {Function} retryFunction - The function to retry after a delay in case of a rate limiting error.
* @returns {Promise<*>} - A Promise that resolves with the result of the retryFunction.
*
* @throws {Error} - Throws an error if `error` is falsy or does not have an `httpStatus` property.
*/
const handleRateLimit = async (error, retryFunction) => {
// Handle other errors
if (error.httpStatus !== 429) throw new Error(error.data.error || 'Something went wrong. Please try again.');
// Handle rate limiting with retry_after_ms
const retryAfterMs = error.data['retry_after_ms'] || 5000;
logger.debug('Retry after (ms):', retryAfterMs);

// Retry the function after the specified delay, defaults to 5000ms
await new Promise((resolve) => setTimeout(resolve, retryAfterMs));

return retryFunction();
};

/**
* Adds a child to a parent space using the authentication provider.
*
* @param {string} parent - The ID or name of the parent space.
* @param {string} child - The ID or name of the child space or room to be added.
* @param {boolean} suggested - Indicates whether the addition is suggested (i.e. in element).
* @returns {Promise<string>} A promise that resolves with a success message or rejects with an error message.
* @throws {Error} If the authentication provider encounters an error other than 429 during the operation.
*/
const addSpaceChild = async (parent, child, suggested) => {
await authenticationProvider.addSpaceChild(parent, child, suggested)
.catch((error) => {
return handleRateLimit(error, () => addSpaceChild(parent, child, suggested));
});
};

/**
* Sends a text message to a specified matrix room using the Matrix client.
*
* @param {string} roomId - The ID of the room where the message will be sent.
* @param {string} message - The text message to be sent.
* @returns {Promise<string>} A promise that resolves with a success message
* @throws {Error} If there is an issue other than 429 sending the message or if a rate limit is encountered.
*/
const sendMessage = async (roomId, message) => {
await matrixClient.sendMessage(roomId, {
msgtype: 'm.text',
body: message,
}).catch((error) => {
return handleRateLimit(error, () => sendMessage(roomId, message));
});
};

return {
rooms,
spaces,
Expand All @@ -507,6 +581,9 @@ function useMatrixProvider(auth) {
hydrateRoomContent,
getMetaEvent,
isConnectedToServer,
handleRateLimit,
addSpaceChild,
sendMessage,
};
}

Expand Down
15 changes: 8 additions & 7 deletions lib/auth/MatrixAuthProvider.js
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,14 @@ class MatrixAuthProvider {
type: 'm.room.join_rules',
content: { join_rule: joinRule }, // can be set to either public, invite or knock
},
],
{
type: 'dev.medienhaus.meta',
content: {
type: type,
template: template,
version: '0.4',
},
}],
power_level_content_override: {
// we only want users with moderation rights to be able to do any actions, people joining the room will have a default level of 0.
ban: 50,
Expand Down Expand Up @@ -171,12 +178,6 @@ class MatrixAuthProvider {
}

const room = await this.matrixClient.createRoom(opts);
const medienhausMetaEvent = {
type: type,
template: template,
version: '0.4',
};
await this.matrixClient.sendStateEvent(room.room_id, 'dev.medienhaus.meta', medienhausMetaEvent);

return room;
}
Expand Down
27 changes: 18 additions & 9 deletions pages/etherpad/[[...roomId]].js
Original file line number Diff line number Diff line change
Expand Up @@ -64,11 +64,11 @@
const auth = useAuth();
const matrix = useMatrix();

const matrixClient = auth.getAuthenticationProvider('matrix').getMatrixClient();
const etherpad = auth.getAuthenticationProvider('etherpad');

const [serverPads, setServerPads] = useState(null);
const [isDeletingPad, setIsDeletingPad] = useState(false);
const [errorMessage, setErrorMessage] = useState('');

/**
* A roomId is set when the route is /etherpad/<roomId>, otherwise it's undefined
Expand Down Expand Up @@ -137,17 +137,24 @@
if (!link || !name) return;

logger.debug('Creating new Matrix room for pad', { link, name });
const room = await matrix
.createRoom(name, false, '', 'invite', 'content', 'etherpad', matrix.serviceSpaces.etherpad)
.catch((error) => setErrorMessage(error.message));

const room = await matrix.createRoom(name, false, '', 'invite', 'content', 'etherpad', matrix.serviceSpaces.etherpad);
await auth.getAuthenticationProvider('matrix').addSpaceChild(matrix.serviceSpaces.etherpad, room);
await matrixClient.sendMessage(room, {
msgtype: 'm.text',
body: link,
});
await matrix.addSpaceChild(matrix.serviceSpaces.etherpad, room).catch((error) => setErrorMessage(error.message));

await matrix.sendMessage(room, link).catch((error) => setErrorMessage(error.message));

if (getConfig().publicRuntimeConfig.authProviders.etherpad.myPads?.api) {
await etherpad.syncAllPads();
setServerPads(etherpad.getAllPads());
}

setErrorMessage('');

return room;
},
[auth, matrix, matrixClient],
[auth, matrix, etherpad],

Check failure

Code scanning / ESLint

verifies the list of dependencies for Hooks like useEffect and similar Error

React Hook useCallback has an unnecessary dependency: 'auth'. Either exclude it or remove the dependency array.
);

/**
Expand Down Expand Up @@ -270,6 +277,7 @@

const listEntries = useMemo(() => {
return matrix.spaces.get(matrix.serviceSpaces.etherpad)?.children?.map((writeRoomId) => {
if (!matrix.roomContents?.get(writeRoomId)) return;
const name = _.get(matrix.rooms.get(writeRoomId), 'name');
const etherpadId = matrix.roomContents
.get(writeRoomId)
Expand All @@ -295,7 +303,7 @@
}, [matrix.roomContents, matrix.rooms, matrix.serviceSpaces.etherpad, matrix.spaces, roomId, serverPads]);

// Add the following parameters to the iframe URL:
// - user's Matrix displayname as parameter so that it shows up in Etherpad as username
// - user's Matrix display name as parameter so that it shows up in Etherpad as username
// - user's MyPads auth token so that we skip having to enter a password for password protected pads owned by user
let iframeUrl;

Expand Down Expand Up @@ -335,6 +343,7 @@
<ServiceTable>
<ServiceTable.Body>{listEntries}</ServiceTable.Body>
</ServiceTable>
{errorMessage && <ErrorMessage>{t(errorMessage)}</ErrorMessage>}
</>
)}
</DefaultLayout.Sidebar>
Expand Down
38 changes: 18 additions & 20 deletions pages/spacedeck/[[...roomId]].js
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ export default function Spacedeck() {
const spacedeck = auth.getAuthenticationProvider('spacedeck');

// Whenever the roomId changes (e.g. after a new sketch was created), automatically focus that element.
// This makes the sidebar scroll to the element if it is outside of the current viewport.
// This makes the sidebar scroll to the element if it is outside the current viewport.
const selectedSketchRef = useRef(null);
useEffect(() => {
selectedSketchRef.current?.focus();
Expand Down Expand Up @@ -150,25 +150,23 @@ export default function Spacedeck() {
}, [spacedeck]);

async function createSketchRoom(link, name, parent = serviceSpaceId) {
// eslint-disable-next-line no-undef
logger.debug('creating room for ' + name);
const room = await matrix.createRoom(name, false, '', 'invite', 'content', 'spacedeck', parent).catch(() => {
setErrorMessage(t('Something went wrong when trying to create a new room'));
});
await auth
.getAuthenticationProvider('matrix')
.addSpaceChild(parent, room)
.catch(() => {
setErrorMessage(t("Couldn't add the new room to your sketch folder"));
});
await matrixClient
.sendMessage(room, {
msgtype: 'm.text',
body: link,
})
.catch(() => {
setErrorMessage(t('Something went wrong when trying to save the new sketch link'));
});
// Create the room with retry handling
const room = await matrix.createRoom(name, false, '', 'invite', 'content', 'spacedeck', parent)
.catch(error => setErrorMessage(error));

// Log debug information about current progress
logger.debug(`Created room for ${name} with id ${room}`);

// Add the room as a child to the parent space
await matrix.addSpaceChild(parent, room)
.catch(error => setErrorMessage(error));

// Log debug information about current progress
logger.debug('Added %s to parent %s', name, parent);

// Send the message to the room with retry handling
await matrix.sendMessage(room, link)
.catch(error => setErrorMessage(error));

return room;
}
Expand Down
48 changes: 25 additions & 23 deletions public/locales/de/_defaults.json
Original file line number Diff line number Diff line change
@@ -1,25 +1,27 @@
{
"Cancel": "Abbrechen",
"Confirm": "Bestätigen",
"Copy link to clipboard": "Link kopieren",
"Connection lost": "Verbindung unterbrochen",
"Delete": "Löschen",
"Make sure your link includes \"{{url}}\"": "Dein Link muss \"{{url}}\" beinhalten",
"Username": "Benutzername",
"Name": "Bezeichnung",
"Password": "Passwort",
"password protected": "Passwortgeschützt",
"Upload": "Hochladen",
"Save": "Speichern",
"Select action": "Aktion wählen",
"What would you like to do?": "Was möchtest du tun?",
"{{error}}, please wait {{time}} seconds.": "{{error}}, bitte warte {{time}} Sekunden.",
"Invitations": "Einladungen",
"From": "Von",
"Accept": "Annehmen",
"Decline": "Ablehnen",
"The session for {{service}} has expired.": "Die Sitzung für {{service}} ist abgelaufen.",
"Please enter your account password to continue using {{service}}:": "Bitte gib dein Passwort ein, um {{service}} wieder nutzen zu können:",
"Something went wrong! Please try again.": "Da ist etwas schief gelaufen! Bitte versuche es erneut.",
"loading...": "lädt..."
"Accept": "Annehmen",
"Cancel": "Abbrechen",
"Confirm": "Bestätigen",
"Copy link to clipboard": "Link kopieren",
"Connection lost": "Verbindung unterbrochen",
"Decline": "Ablehnen",
"Delete": "Löschen",
"From": "Von",
"Invitations": "Einladungen",
"loading...": "lädt...",
"Make sure your link includes \"{{url}}\"": "Dein Link muss \"{{url}}\" beinhalten",
"Name": "Bezeichnung",
"Password": "Passwort",
"password protected": "Passwortgeschützt",
"Please enter your account password to continue using {{service}}:": "Bitte gib dein Passwort ein, um {{service}} wieder nutzen zu können:",
"Save": "Speichern",
"Select action": "Aktion wählen",
"Something went wrong! Please try again.": "Da ist etwas schief gelaufen! Bitte versuche es erneut.",
"Syncing more entries": "Weitere Einträge werden geladen",
"The session for {{service}} has expired.": "Die Sitzung für {{service}} ist abgelaufen.",
"Upload": "Hochladen",
"Username": "Benutzername",
"What would you like to do?": "Was möchtest Du tun?",
"{{error}}, please wait {{time}} seconds.": "{{error}}, bitte warte {{time}} Sekunden."

}
Loading