source-stripe-native: PaymentMethod missing data, long backfills

[jonwihl]

A user has reported that they're only seeing 25%-33% of the expected number of PaymentMethods records in their Postgres materialization.

Also, the user said that the PaymentMethod stream takes a really long time to backfill (multiple days).

[Alex-Bair]

Regarding the really long time to backfill, that's unfortunately the nature of the stream. PaymentMethods is a child stream of the Customers stream. Meaning, we have to get a batch of customer IDs, then ask the API what payment method each customer has. There's no bulk GET API where we can retrieve all payment methods, we have to ask for payment methods associated with a specific customer. Although there are two documented endpoints for retrieving payment methods (/payment_methods and /customer/:id/payment_methods) , they both require a customer ID as either a query param or within the path. I poured over the docs and tested various endpoints, but it looks like this is the only way to backfill PaymentMethods.

For users that have a significant number of customers in Stripe, this backfill process takes a really long time.