Wallet logger interface, more (responsive) logs #1516

ekzyis · 2024-10-24T23:19:35Z

Description

Close #1380

based on #1510, #1511, #1515

TODO:

wallet logger interface: replace addWalletLog with logger interface
responsive receive logs: don't refetch receive logs after save, poll during save
more (receive) logs
incoming payments; amount of payments; especially info around errors
test various errors on unhappy paths

Video

Current status (a610b93):

2024-11-06.03-10-33.mp4

Additional Context

we currently only log wallet attached / wallet details updated and wallet enabled / wallet disabled for receives so if you only attach spending credentials, there's no such log. ideally, we would only log these once, no matter if send and/or recv was attached. but how without creating a mess? previously, we logged them separately but that was noisy.
I believe there is a bug where we don't store the invoice description of unwrapped invoices since it's empty in the database even though the stored bolt11 does contain a description (see missing invoice description in last receiver log in video).

Checklist

Are your changes backwards compatible? Please answer below:

On a scale of 1-10 how well and how have you QA'd this change and any features it might affect? Please answer below:

For frontend changes: Tested on mobile? Please answer below:

Did you introduce any new environment variables? If so, call them out explicitly here:

ekzyis · 2024-10-31T12:18:03Z

Current status: 2cb3342

on successful autowithdrawal:

1m	[lnbits]	OK	↙ payment received: 1,000,000 msats
1m	[lnbits]	INFO	created invoice for 1,000,000 msats
1m	[lnbits]	INFO	↙ incoming payment: 1,000,000 msats

if wallet failed to create invoice:

6s	[lnbits]	ERROR	failed to create invoice: test error
6s	[lnbits]	INFO	↙ incoming payment: 1,000,000 msats

if wallet does not support msats:

4m	[lnbits]	OK	↙ payment received: 9,000 msats
4m	[lnbits]	WARN	wallet does not support msats
4m	[lnbits]	INFO	created invoice for 9,000 msats
4m	[lnbits]	INFO	↙ incoming payment: 9,900 msats

if withdrawal itself failed:

1m	[lnbits]	ERROR	withdrawal failed: no route found
1m	[lnbits]	INFO	created invoice for 1,000,000 msats
1m	[lnbits]	INFO	↙ incoming payment: 1,000,000 msats

TODO:

p2p zap logs on error and success
error handling (HTTP vs JSON)
rich log: show details of logs like payment hash, link to invoice, info how to fix etc. when clicked/uncollapsed (?)
log wallet enabled / wallet details updated only once but also show it if only send is configured ... how ideally?

ekzyis · 2024-11-05T03:52:32Z

How "rich log" looks so far:

2024-11-05.04-39-23.mp4

(not really rich yet since it's just text, no links to anywhere)

I am not super happy with + and - as indicators that the log can be expanded / collapsed but the SVGs that we use in ContextAwareToggle don't align well in the table. Same for these unicode arrows: 🢓, 🢒 (as you can see the arrow-down is not vertically aligned)

* remove unused logNav class * rename classes

* the code assignment was broken anyway * we already log withdrawal errors using .catch on payViaPaymentRequest

ekzyis · 2024-11-06T02:13:37Z

Current status (a610b93):

2024-11-06.03-10-33.mp4

I believe there is a bug where we don't store the invoice description since it's empty in the database even though the stored bolt11 does contain a description (see missing invoice description in last receiver log).

ekzyis · 2024-11-06T02:41:47Z

api/resolvers/wallet.js

+  // server implementation of wallet logger interface on client
+  const log = (level) => async (message, context = {}) => {
+    try {
+      await models.walletLog.create({


no console.log here since I think wallets logs inside the server logs would be more noise than signal

ekzyis · 2024-11-06T02:47:11Z

api/resolvers/wallet.js

        testCreateInvoice:
          walletDef.testCreateInvoice && validateLightning && canReceive({ def: walletDef, config: data })
-            ? (data) => walletDef.testCreateInvoice(data, { me, models })
+            ? (data) => walletDef.testCreateInvoice(data, { logger, me, models })


we pass logger to testCreateInvoice of the wallets but we only use the logger for NWC. So far, I saw no need to add wallet-specific logging to the other wallets.

ekzyis · 2024-11-06T02:51:42Z

lib/url.js

+export function assertContentTypeJson (res) {
+  const contentType = res.headers.get('content-type')
+  if (!contentType || !contentType.includes('application/json')) {
+    throw new Error(`POST ${res.url}: ${res.status} ${res.statusText}`)


no more gigantic error messages about how we couldn't parse the HTML body as JSON when a reverse proxy throws a gateway timeout

ekzyis · 2024-11-06T02:53:24Z

wallets/lnc/client.js

@@ -26,7 +26,7 @@ async function disconnect (lnc, logger) {
      }, 50)
      logger.info('disconnected')
    } catch (err) {
-      logger.error('failed to disconnect from lnc', err)
+      logger.error('failed to disconnect from lnc: ' + err)


logger no longer supports a variable number of arguments because second argument is now context

ekzyis · 2024-11-06T02:56:43Z

worker/wallet.js

@@ -294,7 +292,7 @@ export async function checkWithdrawal ({ data: { hash, withdrawal, invoice }, bo

    const fee = Number(wdrwl.payment.fee_mtokens)
    const paid = Number(wdrwl.payment.mtokens) - fee
-    const [{ confirm_withdrawl: code }] = await serialize([
+    const [[{ confirm_withdrawl: code }]] = await serialize([


for some reason the return type changed at some point

ekzyis · 2024-11-06T02:57:32Z

worker/wallet.js

-        const message = `autowithdrawal of ${
-          numWithUnits(msatsToSats(paid), { abbreviate: false })} with ${
-          numWithUnits(msatsToSats(fee), { abbreviate: false })} as fee`
-        await addWalletLog({ wallet: dbWdrwl.wallet, level: 'SUCCESS', message }, { models })


we don't need this because we log success/errors using .then and .catch on payViaPaymentRequest

ekzyis added enhancement improvements to existing features wallets labels Oct 24, 2024

ekzyis marked this pull request as draft October 24, 2024 23:19

ekzyis changed the title ~~Wallet logger interface, more and more responsive receive logs,~~ Wallet logger interface, more (responsive) logs Oct 24, 2024

ekzyis force-pushed the wallet-logs branch 3 times, most recently from 6b2a906 to 6161c48 Compare October 31, 2024 12:03

ekzyis force-pushed the wallet-logs branch 2 times, most recently from 2cb3342 to 7a24ffb Compare November 5, 2024 00:28

ekzyis force-pushed the wallet-logs branch from 4b6a5c3 to f444308 Compare November 5, 2024 03:54

ekzyis added 16 commits November 5, 2024 16:31

Inject wallet logger interface

922d321

Include method in NWC logs

f7ccd63

Fix wrong page total

dfd0b3b

Poll for new logs every second

8320ca6

Fix overlapping pagination

3c98e9e

Remove unused total

90adf62

Better logs for incoming payments

0e190ed

Use _setLogs instead of wrapper

5f18765

Remove inconsistent receive log

1e146d8

Remove console.log from wallet logger on server

c7a8aae

Fix missing 'wallet detached' log

3263c2d

Fix confirm_withdrawl code

93c9708

Remove duplicate autowithdrawal log

bca632a

Add context to log

68a4ed5

Add more context

8d26348

Better table styling

e8629fb

ekzyis force-pushed the wallet-logs branch from f444308 to e8629fb Compare November 5, 2024 17:13

Move CSS for wallet logs into one file

ed4108e

* remove unused logNav class * rename classes

ekzyis added 4 commits November 5, 2024 22:57

Align key with second column

a5b8d65

Fix TypeError if context empty

c63c9cd

Check content-type header before calling res.json()

2e41724

Fix duplicate 'failed to create invoice'

6257e08

ekzyis force-pushed the wallet-logs branch from 40617b3 to 6257e08 Compare November 5, 2024 22:38

ekzyis added 4 commits November 6, 2024 00:02

Parse details from LND error

4d69d39

Fix invalid DOM property 'colspan'

9eea356

P2P zap logs with context

2a5e4bd

Remove unnecessary withdrawal error log

a610b93

* the code assignment was broken anyway * we already log withdrawal errors using .catch on payViaPaymentRequest

Don't show outgoing fee to receiver to avoid confusion

99fd631

ekzyis commented Nov 6, 2024

View reviewed changes

Fix typo in comment

90b95b3

ekzyis commented Nov 6, 2024

View reviewed changes

Log if invoice was canceled by payer

be9b940

ekzyis force-pushed the wallet-logs branch from 2a84110 to be9b940 Compare November 6, 2024 20:26

ekzyis added 2 commits November 6, 2024 21:41

Automatically populate context from bolt11

f53e29e

Fix missing context

f8b040b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wallet logger interface, more (responsive) logs #1516

Wallet logger interface, more (responsive) logs #1516

ekzyis commented Oct 24, 2024 •

edited

Loading

ekzyis commented Oct 31, 2024 •

edited

Loading

ekzyis commented Nov 5, 2024

ekzyis commented Nov 6, 2024

ekzyis Nov 6, 2024

ekzyis Nov 6, 2024

ekzyis Nov 6, 2024

ekzyis Nov 6, 2024

ekzyis Nov 6, 2024

ekzyis Nov 6, 2024

Wallet logger interface, more (responsive) logs #1516

Are you sure you want to change the base?

Wallet logger interface, more (responsive) logs #1516

Conversation

ekzyis commented Oct 24, 2024 • edited Loading

Description

Video

Additional Context

Checklist

ekzyis commented Oct 31, 2024 • edited Loading

ekzyis commented Nov 5, 2024

ekzyis commented Nov 6, 2024

ekzyis Nov 6, 2024

Choose a reason for hiding this comment

ekzyis Nov 6, 2024

Choose a reason for hiding this comment

ekzyis Nov 6, 2024

Choose a reason for hiding this comment

ekzyis Nov 6, 2024

Choose a reason for hiding this comment

ekzyis Nov 6, 2024

Choose a reason for hiding this comment

ekzyis Nov 6, 2024

Choose a reason for hiding this comment

ekzyis commented Oct 24, 2024 •

edited

Loading

ekzyis commented Oct 31, 2024 •

edited

Loading