Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show token prices #1382

Merged
merged 34 commits into from
Mar 7, 2024
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
0b03759
Implement fetching coin prices from CoinGecko
mudrila Feb 27, 2024
2942280
Leverage useFormatCoins for showing treasury total on DAO dashboard page
mudrila Feb 28, 2024
22b4d13
empty commit
adamgall Feb 28, 2024
324edbe
Merge branch 'develop' into fix/show-token-prices
adamgall Feb 28, 2024
e1f77e1
Add Netlify function tokenPrices, route request to CoinGecko, cache t…
mudrila Mar 4, 2024
2d6de29
Refactor and cleanup of storing/getting prices
mudrila Mar 4, 2024
ea7aa8f
Fix returningprice for ETH
mudrila Mar 4, 2024
c5b477a
Try fixing token fiat balance overflow
mudrila Mar 4, 2024
88f356a
Use ethers.constants.MaxUint256 instead of Number max value
mudrila Mar 4, 2024
3e5f170
Use smaller multiplicator
mudrila Mar 4, 2024
59dd067
Add math.round
mudrila Mar 4, 2024
e5afb36
Merge branch 'develop' into fix/show-token-prices
mudrila Mar 4, 2024
6781722
Rename netlify function, fix returning response error, better typing …
mudrila Mar 6, 2024
8633535
Handle better CoinGecko requesting error and return cached prices
mudrila Mar 6, 2024
a3783d4
Show cached prices on FE if request to CoinGecko failed
mudrila Mar 6, 2024
97930a9
Added some more explicit code and comments
adamgall Mar 6, 2024
9252d6e
Do not execute request to netlify function if tokensAddresses is empt…
mudrila Mar 6, 2024
50e148d
Merge pull request #1415 from decent-dao/fix/show-token-prices-2
mudrila Mar 6, 2024
88899cf
Merge branch 'develop' into fix/show-token-prices
mudrila Mar 6, 2024
2c92f7b
Trying changing possibleCachedTokenPrices filtering null values
mudrila Mar 6, 2024
873df06
Fix filtering logic of uncached token prices
mudrila Mar 6, 2024
15e1564
Use rawTokenAddresses for allUncachedTokenAddresses instead of splitt…
mudrila Mar 6, 2024
7c03888
Handle a lot of cases:
adamgall Mar 7, 2024
b207d66
Refactor to compartmentalize code
adamgall Mar 7, 2024
e70037d
Split stores per network
mudrila Mar 7, 2024
91c0326
Add network param to the tokenPrices function
mudrila Mar 7, 2024
2248280
Fix token address chain namespacing
mudrila Mar 7, 2024
621771c
Swap expiration with time of fetching
mudrila Mar 7, 2024
96420f8
Cleanup comments
mudrila Mar 7, 2024
dfaa285
Reverse adding cacheTime
mudrila Mar 7, 2024
b5a93f5
Fix cached token addresses filtering
mudrila Mar 7, 2024
2f85687
Convert minutes to seconds, specify in variable names that we're deal…
adamgall Mar 7, 2024
7720d94
Merge commit '11303a767df67e67e8007ca20e07894b268e3269' into fix/show…
adamgall Mar 7, 2024
daf34df
Prettier updates
adamgall Mar 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .env
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,9 @@ NEXT_PUBLIC_INFURA_IPFS_API_SECRET=""
NEXT_PUBLIC_SITE_URL="https://app.dev.fractalframework.xyz/"
# WalletConnect Cloud Project ID
NEXT_PUBLIC_WALLET_CONNECT_PROJECT_ID=""
# CoinGecko API key
COINGECKO_API_KEY=""
mudrila marked this conversation as resolved.
Show resolved Hide resolved
# Minutes to cache prices for token addresses
TOKEN_PRICE_CACHE_INTERVAL_MINUTES=""
# Shutter Public Key
NEXT_PUBLIC_SHUTTER_EON_PUBKEY=0x0e6493bbb4ee8b19aa9b70367685049ff01dc9382c46aed83f8bc07d2a5ba3e6030bd83b942c1fd3dff5b79bef3b40bf6b666e51e7f0be14ed62daaffad47435265f5c9403b1a801921981f7d8659a9bd91fe92fb1cf9afdb16178a532adfaf51a237103874bb03afafe9cab2118dae1be5f08a0a28bf488c1581e9db4bc23ca
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,6 @@ yarn-error.log*

# GraphQL generated files
.graphclient/

# Local Netlify folder
.netlify
2 changes: 1 addition & 1 deletion .nvmrc
Original file line number Diff line number Diff line change
@@ -1 +1 @@
18.12.1
18.14.0
268 changes: 268 additions & 0 deletions netlify/functions/tokenPrices.mts
Original file line number Diff line number Diff line change
@@ -0,0 +1,268 @@
// eslint-disable-next-line import/named
import { Store, getStore } from '@netlify/blobs';
import { ethers } from 'ethers';

const PUBLIC_DEMO_API_BASE_URL = 'https://api.coingecko.com/api/v3/';
const AUTH_QUERY_PARAM = `?x_cg_demo_api_key=${process.env.COINGECKO_API_KEY}`;

type TokenPriceWithMetadata = {
data: {
tokenAddress: string;
price: number;
};
metadata: {
fetched: number;
};
};

type Config = {
store: Store;
now: number;
cacheTime: number;
};

const SUPPORTED_NETWORKS = ['ethereum'] as const;
type SupportedNetworks = typeof SUPPORTED_NETWORKS[number];

function sanitizeUserInput(tokensString: string, network: SupportedNetworks) {
const rawTokenAddresses = tokensString.split(',');
const needNativeAsset = rawTokenAddresses.map(address => address.toLowerCase()).includes(network);
const validTokenAddresses = rawTokenAddresses.filter(address => ethers.utils.isAddress(address));
const lowerCaseTokenAddresses = validTokenAddresses.map(address => address.toLowerCase());
const tokens = [...new Set(lowerCaseTokenAddresses)];
if (needNativeAsset) tokens.push(network);
return { tokens, needNativeAsset };
}

async function splitData(
config: Config,
tokens: string[],
responseBodyCallback: (address: string, price: number) => void,
network: SupportedNetworks
) {
// Try to get all of the tokens from our store.
// Any token address that we don't have a record for will
// come back as null.
const possibleCachedTokenPrices = await Promise.all(
tokens.map(
tokenAddress =>
config.store.getWithMetadata(`${network}/${tokenAddress}`, {
type: 'json',
}) as Promise<TokenPriceWithMetadata> | null
)
);

// Filter out the null values, leaving us with an array of
// TokenPricesWithMetadata. All of these TokenPrices will be either
// expired or unexpired.
const cachedTokenPrices = possibleCachedTokenPrices.filter(
(possible): possible is TokenPriceWithMetadata => possible !== null
);

// Let's pull out all of the expired addresses from our cache.
const expiredCachedTokenAddresses = cachedTokenPrices
.filter(tokenPrice => tokenPrice.metadata.fetched + config.cacheTime < config.now)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should be converting this cacheTime from "minutes" to "seconds" heh

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should just specify the env var value in miliseconds? config.now gives milliseconds and I feel like it's more appropriate to not convert minutes to milliseconds here. WDYT?

Copy link
Member

@adamgall adamgall Mar 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config.now gives seconds (https://github.com/decent-dao/fractal-interface/pull/1382/files#diff-284b3930e569c4259e85b685d36c139226ef2a210eaeb25f614b752fd1b2e910R197)

We did name the env var TOKEN_PRICE_CACHE_INTERVAL_MINUTES, which specifies that the value should be MINUTES. Minutes are a lot easier to reason about than milliseconds, when setting configs in an env var.

I think the cleaner solution is to convert the value to seconds when setting the config property here: https://github.com/decent-dao/fractal-interface/pull/1382/files#diff-284b3930e569c4259e85b685d36c139226ef2a210eaeb25f614b752fd1b2e910R198

const cacheTime = parseInt(process.env.TOKEN_PRICE_CACHE_INTERVAL_MINUTES) * 60

My TL;DR is that:

  • I'd like the user-facing number to be denominated in "minutes" (because that seems like an appropriate resolution for setting things like a cache expiration).
  • I'd like the code and data to be denominated in seconds (because we're using Unix timestamps here and those are in seconds).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mudrila i just pushed up the fix, no worries

.map(tokenPrice => tokenPrice.data.tokenAddress);

// Finally let's get a list of all of the token addresses that
// we don't have any record of in our cache.
const uncachedTokenAddresses = tokens.filter(
address =>
!cachedTokenPrices.find(cachedTokenPrice => cachedTokenPrice.data.tokenAddress === address)
);

// We'll update our response with those cached expired and
// unexpired prices.
cachedTokenPrices.forEach(tokenPrice => {
responseBodyCallback(tokenPrice.data.tokenAddress, tokenPrice.data.price);
});

return { expiredCachedTokenAddresses, uncachedTokenAddresses };
}

function getTokenPricesUrl(tokens: string[], network: SupportedNetworks) {
const tokenPricesUrl = `${PUBLIC_DEMO_API_BASE_URL}simple/token_price/${network}/${AUTH_QUERY_PARAM}&vs_currencies=usd&contract_addresses=${tokens.join(
','
)}`;
return tokenPricesUrl;
}

function getNativeAssetPriceUrl(network: SupportedNetworks) {
const nativeAssetPriceUrl = `${PUBLIC_DEMO_API_BASE_URL}simple/price${AUTH_QUERY_PARAM}&ids=${network}&vs_currencies=usd`;
return nativeAssetPriceUrl;
}

async function storeTokenPrice(
config: Config,
tokenAddress: string,
price: number,
network: SupportedNetworks
) {
await config.store.setJSON(
`${network}/${tokenAddress}`,
{ tokenAddress, price },
{ metadata: { fetched: config.now } }
);
}

async function processTokenPricesResponse(
config: Config,
tokenPricesResponseJson: Record<string, { usd?: number }>,
responseBodyCallback: (address: string, price: number) => void,
network: SupportedNetworks
) {
const coinGeckoResponseAddresses = Object.keys(tokenPricesResponseJson);
for await (const tokenAddress of coinGeckoResponseAddresses) {
// Sometimes no USD price is returned. If this happens,
// we should consider it as though CoinGecko doesn't support
// this address and fallback to 0.
const price = tokenPricesResponseJson[tokenAddress].usd || 0;
const sanitizedAddress = tokenAddress.toLowerCase();

responseBodyCallback(sanitizedAddress, price);
await storeTokenPrice(config, sanitizedAddress, price, network);
}

return coinGeckoResponseAddresses;
}

async function processUnknownAddresses(
config: Config,
needPricesTokenAddresses: string[],
responseAddresses: string[],
network: SupportedNetworks
) {
const unknownAddresses = needPricesTokenAddresses
.filter(x => !responseAddresses.includes(x))
.map(address => address.toLowerCase());
for await (const tokenAddress of unknownAddresses) {
await storeTokenPrice(config, tokenAddress, 0, network);
}
}

async function coinGeckoRequestAndResponse(
config: Config,
url: string,
responseBodyCallback: (address: string, price: number) => void,
network: SupportedNetworks
) {
// Make the request to CoinGecko.
// Response is of shape:
// {
// [tokenAddress]: { usd: 1234 },
// }
let ethPriceResponseJson: Record<string, { usd?: number }>;
try {
const ethPriceResponse = await fetch(url);
ethPriceResponseJson = await ethPriceResponse.json();
} catch (e) {
throw e;
}

// Update the cache with the new price and update
// the response object.
const responseAddresses = processTokenPricesResponse(
config,
ethPriceResponseJson,
responseBodyCallback,
network
);

return responseAddresses;
}

export default async function getTokenPrices(request: Request) {
if (!process.env.COINGECKO_API_KEY) {
console.error('CoinGecko API key is missing');
return Response.json({ error: 'Error while fetching prices' }, { status: 503 });
}

if (!process.env.TOKEN_PRICE_CACHE_INTERVAL_MINUTES) {
console.error('TOKEN_PRICE_CACHE_INTERVAL_MINUTES is not set');
return Response.json({ error: 'Error while fetching prices' }, { status: 503 });
}

const requestSearchParams = new URL(request.url).searchParams;
const tokensStringParam = requestSearchParams.get('tokens');
if (!tokensStringParam) {
return Response.json({ error: 'Tokens missing from request' }, { status: 400 });
}

const networkParam = requestSearchParams.get('network') as SupportedNetworks;
if (!networkParam || !SUPPORTED_NETWORKS.includes(networkParam)) {
return Response.json({ error: 'Requested network is not supported' }, { status: 400 });
}

const store = getStore('token-prices');
const now = Math.floor(Date.now() / 1000);
const cacheTime = parseInt(process.env.TOKEN_PRICE_CACHE_INTERVAL_MINUTES);
const config = { store, now, cacheTime };

const { tokens, needNativeAsset } = sanitizeUserInput(tokensStringParam, networkParam);

// Let's immediately build up our repsonse object, containing each
// token address and an value of 0. We'll modify this along the way
// populating it with more relevant prices.
const responseBody: Record<string, number> = tokens.reduce((p, c) => ({ ...p, [c]: 0 }), {});

const { expiredCachedTokenAddresses, uncachedTokenAddresses } = await splitData(
config,
tokens,
(address, price) => {
responseBody[address] = price;
},
networkParam
);

// If there are no expired token prices, and no token addresses that we
// don't have a cached value for at all, we can early return!
if (expiredCachedTokenAddresses.length === 0 && uncachedTokenAddresses.length === 0) {
return Response.json({ data: responseBody });
}

// If we got here, then we have either some expired prices for given tokens,
// or no prices at all for given tokens.

// First, let's build up our list of token addresses to query CoinGecko with,
// which is all uncached tokens and tokens that have expired.
// Remove native asset name if it's in this list.
const needPricesTokenAddresses = [
...uncachedTokenAddresses,
...expiredCachedTokenAddresses,
].filter(address => address !== networkParam);

let responseAddresses: string[];
try {
responseAddresses = await coinGeckoRequestAndResponse(
config,
getTokenPricesUrl(needPricesTokenAddresses, networkParam),
(address, price) => {
responseBody[address] = price;
},
networkParam
);
} catch (e) {
console.error('Error while querying CoinGecko', e);
return Response.json({ error: 'Error while fetching prices', data: responseBody });
}
await processUnknownAddresses(config, needPricesTokenAddresses, responseAddresses, networkParam);

// Do we need to get the price of our chain's gas token?
if (needNativeAsset) {
try {
await coinGeckoRequestAndResponse(
config,
getNativeAssetPriceUrl(networkParam),
(address, price) => {
responseBody[address] = price;
},
networkParam
);
} catch (e) {
console.error('Error while querying CoinGecko', e);
return Response.json({ error: 'Error while fetching prices', data: responseBody });
}
}

return Response.json({ data: responseBody });
}
Loading