receiver and sender fallback and p2p lnurlp #1523

riccardobl · 2024-10-27T13:49:12Z

The PR implements fallbacks on receiver, lnurlp for attached wallets and does some refactoring to the wallet interface.
Credit fee priority is toggleable for easier testing.

Closes #1496
Closes #1494
as side effect it should fix this too #1500

Changelog:

made pessimistic actions retryable
Solved some circular dependencies [1]
- wallet/server doesn't self-import to get walletDefs anymore
- moved addWalletLog to wallet/server
wallet/server exports functions to create wrapped and un-wrapped hodl and regular invoices
- createHodlInvoice and createInvoice : create an invoice for an user id (or for stacker news [2]), they will first try to return a wrapped p2p invoice, if it is not possible they will fallback to the SN custodial wallet. If the wrapping fails, the next wallet is used.
- createFeeCreditInvoice : create a "virtual" invoice to represent CC transfers,
Moved some constants to lib/constants and helpers under the appropriate lib/ module
~~Added some types with jsdoc~~ removed because they made the diff hard to follow, will add them back later
Strict bigint <-> number casts: bigint are kept bigint unless casting them to number is absolutely needed and only through the helper that checks for range.
implemented lnurlp/@stacker.news ln-address as pessimistic paid action TRANSFER : so it can use the p2p flow
make sybil fee configurable: actions can implement getSybilFeePercent to change the default sybil fee
implemented receiver fallback
implemented sender fallback : attachments -> fee credit -> qr

_{[1] The current implementation functions correctly, but it relies on a specific export/import order, which is not best practice. This approach can introduce unintended issues as the dependency tree grows more complex, potentially leading to circular dependencies that change the order in unexpected ways even if everything appears correct.}
_{[2] the constant `SN_WALLET` can be passed as userId to target stacker news}

Tests

setup

stacker1:
- fee credits: 1000
- sending attachments [good, bad]
- receiving attachments []
stacker2:
- fee credits: 0
- sending attachments: [bad, good]
- receiving attachments: []
stacker3:
- fee credits: 1000
- sending attachments: [bad,bad]
- receiving attachments []
stacker4:
- fee credits: 0
- sending attachments: []
- receiving attachments: [good, bad]
stacker5:
- fee credits: 0
- sending attachments: []
- receiving attachments: [bad, good]
stacker6:
- fee credits: 0
- sending attachments: []
- receiving attachments: [bad, bad]
stacker7:
- fee credits: 0
- sending attachments: []
- receiving attachments: []

Zap

send receive	stacker1	stacker2	stacker3	stacker4
stacker4	[x] p2p	[x] p2p	[x] cc	-
sacker5	[x] p2p	[x] p2p	[x] cc	[x] qr on retry
stacker6	[x] cc	[x ] cc	[x] cc	[x] qr on retry
stacker7	[x] cc	[x] cc	[x] cc	[x] qr on retry

post result

stacker1 posts spending from attached wallet
stacker2 posts spending from attached wallet
stacker3 posts spending from attached wallet (bug: item still says retry until the page is refreshed)
stacker4 fails, shows qr on manual retry
stacker5 fails, shows qr on manual retry
stacker6 fails, shows qr on manual retry

LNURLP

stacker3: cc
stacker4: p2p
stacker5: p2p
stacker6: cc

riccardobl · 2024-11-02T16:46:54Z

on hold until #1507 is merged, i will rebase any change on top of the new code

riccardobl · 2024-11-04T13:53:18Z

api/paidAction/index.js

-      throw new Error('forceFeeCredits is set, but user does not have enough fee credits')
+    // if forceFeeCredit or the action is free or prioritizeFeeCredits perform a fee credit action
+    if (me && !disableFeeCredit && ((forceFeeCredits || cost === 0n) || (prioritizeFeeCredits && (me?.msats ?? 0n) >= cost))) {
+      const invoiceData = await createFeeCreditInvoice(receiverUserId, { msats: cost, description })


Paid actions had to check if they were paid with fee credit, or with an invoice, until now this wasn't a problem because actions never changed from one method to the other, but now with a post nov5-update fallback mechanism (ie. p2p first, cc last, both on send and receive) , we can have a situations where an action changes payment method.

Using a virtual invoice for fee credit payments is a simple way to treat every action as if it had an invoice.

riccardobl · 2024-11-04T13:53:53Z

api/paidAction/index.js

-  const action = paidActions[actionType]
+async function performAction (invoiceEntry, paidAction, args, context) {
+  const { retryForInvoice } = context
+  if (retryForInvoice && paidAction.retry) {


if the action supports retrying, we use it. Otherwise we just rerun the action

riccardobl · 2024-11-04T13:54:45Z

api/paidAction/index.js

+    context.me = context.me ? await context.models.user.findUnique({ where: { id: context.me.id } }) : undefined
+    context.cost = context.cost ?? await paidAction.getCost(args, context)
+    if (context.cost < 0n) throw new Error('Cost cannot be negative')
+    context.sybilFeePercent = context.sybilFeePercent ?? paidAction.getSybilFeePercent ? await paidAction.getSybilFeePercent(args, context) : undefined


paid actions can set their sybil fee

riccardobl · 2024-11-04T13:55:07Z

api/paidAction/index.js

+    if (context.sybilFeePercent !== undefined && context.sybilFeePercent < 0n) throw new Error('Sybil fee percent cannot be negative')
+
+    context.description = context.me?.hideInvoiceDesc ? undefined : (context.description ?? await paidAction.describe(args, context))
+    context.descriptionHash = context.descriptionHash ?? (paidAction.describeHash ? await paidAction.describeHash(args, context) : undefined)


paid actions can return a custom description hash

riccardobl · 2024-11-04T13:55:31Z

api/paidAction/index.js

+
+    context.description = context.me?.hideInvoiceDesc ? undefined : (context.description ?? await paidAction.describe(args, context))
+    context.descriptionHash = context.descriptionHash ?? (paidAction.describeHash ? await paidAction.describeHash(args, context) : undefined)
+    context.fallbackToSN = context.fallbackToSN ?? true


we can disable fallback to the centralized SN wallet

riccardobl · 2024-11-04T14:01:07Z

lib/constants.js

+
+// the fee for the zap sybil service
+export const ZAP_SYBIL_FEE_PERCENT = 30n
+export const MAX_FEE_ESTIMATE_PERMILE = 25n // the maximum fee relative to outgoing we'll allow for the fee estimate


reduce bigint <-> number conversions

riccardobl · 2024-11-04T14:01:41Z

lib/validate.js

+export const toPositive = (x) => {
+  if (typeof x === 'bigint') return toPositiveBigInt(x)
+  return toPositiveNumber(x)
+}


Just helpers to strictly handle bigints

riccardobl · 2024-11-04T14:02:03Z

pages/api/lnurlp/[username]/pay.js

        descriptionHash = createHash('sha256').update(noteStr).digest('hex')
      } else {
        res.status(400).json({ status: 'ERROR', reason: 'invalid NIP-57 note' })
        return
      }
    } else {
-      description = user.hideInvoiceDesc ? undefined : `Funding @${username} on stacker.news`
+      description = `Funding @${username} on stacker.news`
+      description += comment ? `: ${comment}` : '.'


we don't need to handle invoice privacy here, it is done by the paid action

riccardobl · 2024-11-04T14:03:39Z

wallets/wrap.js

-      wrapped.description = undefined
-      wrapped.description_hash = undefined
-    }
-


this is handled upstream by the create*Invoice code in wallets/server

riccardobl · 2024-11-04T14:04:11Z

worker/autowithdraw.js

+      description: 'SN: autowithdrawal',
+      expiry: 360,
+      fallbackToSN: false, // if the user doesn't have an attached wallet, there is nothing to withdraw to
+      direct: true // no need to wrap this one


this is to disable wrapping, that would otherwise happen automatically because the destination is an user wallet.
It is a bit ugly to have a random flag like that, but supposedly this is one of the few situations where we need this behavior (?)

riccardobl · 2024-11-04T14:08:07Z

wallets/server.js

+  if (receiverUser.hideInvoiceDesc) {
+    description = undefined
+    descriptionHash = undefined
+  }


receiver privacy

riccardobl · 2024-11-05T20:10:07Z

components/use-paid-mutation.js

+      }
+
+      try {
+        await invoiceHelper.cancel(invoice)


What if somehow the wallet failed but is still trying to pay? (maybe an improper timeout?). Safer to wait for the invoice to be cancelled, so we don't double invoice.

riccardobl · 2024-11-05T20:10:46Z

components/use-paid-mutation.js

    try {
-      return await waitForWalletPayment(invoice, waitFor)
+      console.log('could not pay with any wallet, will try with fee credits...')
+      const retry = await retryPaidAction({ variables: { invoiceId: parseInt(invoice.id), forceFeeCredits: true } })


we use fee credits only if the p2p payment is not possible

riccardobl · 2024-11-06T19:54:36Z

components/use-paid-mutation.js

+      }
+      await waitForQrPayment(invoice, null, { persistOnNavigate, waitFor })
+      return { invoice, response }
+    }


anons gets the qr code right away

riccardobl · 2024-11-06T19:57:12Z

api/paidAction/index.js

+  } else {
+    if (failedInvoice.retriable) retriable = true
+    context.retriedWithFeeCredits = !failedInvoice.retriableWithFeeCredits
+    context.retriedManually = !failedInvoice.retriableManually


should this be a single state with an enum? Maybe not, the 3 flags are not necessarily dependent to each other and we don't need to follow a specific flow.

riccardobl · 2024-11-06T19:59:36Z

api/paidAction/index.js

-  if (!action.supportsOptimism) {
-    throw new Error(`retryPaidAction - action does not support optimism ${actionType}`)
+  let retriable = false
+  if (forceRetry) {


the code limits retries so that after a while we know that everything we tried failed, and we consider the action dead (both client and serverside).
However only the user knows if maybe it is a temporary issue with its node or something like that, so there is this forceRetry flag that the retry button can use in the ui to restart the cycle over.

riccardobl · 2024-11-06T20:00:11Z

api/paidAction/index.js

-  const invoiceArgs = await createSNInvoice(actionType, args, context)
+  context.retryForInvoice = failedInvoice
+  context.forceFeeCredits = forceFeeCredits
+  context.actionAttempt = failedInvoice.actionAttempt + 1 // next attempt


this is used to cycle through the wallets when they look fine to us, but the user client code cannot pay them, so we ensures every retry is for another wallet, eventually we'll find a wallet the user can pay, maybe.

riccardobl marked this pull request as draft October 27, 2024 22:54

riccardobl changed the title ~~minor wallet refactoring (server only)~~ fallback + lnurlp and minor refactoring with types (server only) Oct 28, 2024

riccardobl mentioned this pull request Oct 28, 2024

Improve wallet fallbacks #1489

Closed

riccardobl force-pushed the walletref branch 2 times, most recently from 8b22e03 to e647d2e Compare October 30, 2024 00:19

riccardobl added the blocked label Nov 2, 2024

riccardobl added 8 commits November 3, 2024 18:38

move hmac functions under lib/crypto

2830bac

solve circular dependency between wallet resolver and wallets/server

eaf4390

strict bigints

9d8c48b

wallet refactor: improve fallbacks and invoicing apis

a5126b2

'transfer' paidAction

46a8e4c

pass comment as invoice description

3c36e86

add notifications for p2p withdrawals (p2p transfer)

811f770

remove types

36c8cfb

riccardobl force-pushed the walletref branch from 49e4fb9 to 36c8cfb Compare November 4, 2024 13:39

riccardobl changed the title ~~fallback + lnurlp and minor refactoring with types (server only)~~ receiver fallback and p2p lnurlp Nov 4, 2024

Merge branch 'master' into walletref

f1bc81f

riccardobl commented Nov 4, 2024

View reviewed changes

riccardobl removed the blocked label Nov 4, 2024

riccardobl marked this pull request as ready for review November 4, 2024 14:05

riccardobl commented Nov 4, 2024

View reviewed changes

riccardobl marked this pull request as draft November 4, 2024 21:36

riccardobl added 2 commits November 4, 2024 22:37

cycle wallets on retry

e2f1912

handle sender fallback

ddf9f2b

riccardobl changed the title ~~receiver fallback and p2p lnurlp~~ receiver and sender fallback and p2p lnurlp Nov 4, 2024

riccardobl added 4 commits November 5, 2024 18:32

Merge branch 'master' into walletref

9b9627d

wait for invoice cancellation

7838c33

fix: await retry mutation

1a14f8d

fix wallet usage and add logs

947cc1a

riccardobl commented Nov 5, 2024

View reviewed changes

riccardobl added 2 commits November 6, 2024 01:29

give two more chances to retry an invoice with fee credit and manually

e4fa21f

Merge branch 'master' into walletref

7a3a6d6

riccardobl force-pushed the walletref branch from 0e0d7d3 to c36b04e Compare November 6, 2024 13:08

make invoices retriable two times more with feecredits and qr

860fab0

riccardobl force-pushed the walletref branch from c36b04e to 860fab0 Compare November 6, 2024 13:08

riccardobl added 4 commits November 6, 2024 18:41

fix retry logic

a569b59

fix retry logic

7d1a745

fix retry and qr logic

185570c

show lnurlp notifications

730e005

riccardobl commented Nov 6, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

receiver and sender fallback and p2p lnurlp #1523

receiver and sender fallback and p2p lnurlp #1523

riccardobl commented Oct 27, 2024 •

edited

Loading

riccardobl commented Nov 2, 2024

riccardobl Nov 4, 2024

riccardobl Nov 4, 2024

riccardobl Nov 4, 2024

riccardobl Nov 4, 2024

riccardobl Nov 4, 2024

riccardobl Nov 4, 2024

riccardobl Nov 4, 2024

riccardobl Nov 4, 2024

riccardobl Nov 4, 2024

riccardobl Nov 4, 2024 •

edited

Loading

riccardobl Nov 4, 2024

riccardobl Nov 5, 2024

riccardobl Nov 5, 2024

riccardobl Nov 6, 2024

riccardobl Nov 6, 2024

riccardobl Nov 6, 2024

riccardobl Nov 6, 2024 •

edited

Loading

receiver and sender fallback and p2p lnurlp #1523

Are you sure you want to change the base?

receiver and sender fallback and p2p lnurlp #1523

Conversation

riccardobl commented Oct 27, 2024 • edited Loading

Tests

setup

Zap

post result

LNURLP

riccardobl commented Nov 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

riccardobl Nov 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

riccardobl Nov 6, 2024 • edited Loading

Choose a reason for hiding this comment

riccardobl commented Oct 27, 2024 •

edited

Loading

riccardobl Nov 4, 2024 •

edited

Loading

riccardobl Nov 6, 2024 •

edited

Loading