Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert getCommunities to TRPC and add additional flags. #8777

Merged
merged 47 commits into from
Aug 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
95c8e00
Basic outline of TRPC getCommunities call.
jnaviask Aug 6, 2024
c0558eb
Add base filter and TRPC export.
jnaviask Aug 6, 2024
03c3ca0
Add CommunityTag schema and fix pagination.
jnaviask Aug 6, 2024
0837946
Complete code for query.
jnaviask Aug 6, 2024
a26d28f
Fix type error in deprecated get_communities route.
jnaviask Aug 6, 2024
3f3632f
Update group lifecycle test to use expected correct but error values.
jnaviask Aug 6, 2024
3f52906
Add TODO notes and fix framework input types.
jnaviask Aug 6, 2024
c0258da
Fix parallelism in tag schema, add unit test, TODO to rewrite.
jnaviask Aug 6, 2024
fe56f81
Query tentatively complete, needs testing.
jnaviask Aug 7, 2024
4b565ad
Revert database.ts change.
jnaviask Aug 7, 2024
423c389
Fix group lifecycle test.
jnaviask Aug 7, 2024
df5f0dc
Add GetCommunities query tests to stake lifecycle.
jnaviask Aug 7, 2024
6201573
Rewrite refresh-all-memberships to use new TRPC query.
jnaviask Aug 7, 2024
7ccd527
Fix some code quality errors.
jnaviask Aug 7, 2024
53a6397
Temp patch to disable ts and allow testing to proceed.
jnaviask Aug 7, 2024
f5751f6
Fix tag naming in server controllers.
jnaviask Aug 7, 2024
8afecd2
Make TRPC queries POST rather than GET requests.
jnaviask Aug 7, 2024
df8e0b5
Remove ts-ignores by asserting default zod values.
jnaviask Aug 7, 2024
ea4f7be
Enum type for order_by rather than parsing in query logic.
jnaviask Aug 7, 2024
7dd9196
Remove TODO comments.
jnaviask Aug 7, 2024
a69aa60
Switch TRPC back to GET and use preprocess for tag_ids.
jnaviask Aug 7, 2024
7935af2
Always return stake and tag data.
jnaviask Aug 7, 2024
d02b724
Add bulkOffchain data and stake and tags to GetCommunity() query.
jnaviask Aug 7, 2024
fd53e69
Revert z.infer to z.input change.
jnaviask Aug 7, 2024
5b7d342
Revert unnecessary changes.
jnaviask Aug 7, 2024
cb3a59b
Use separate queries for bulkOffchain data.
jnaviask Aug 7, 2024
d34715d
Remove unnecessary cast.
jnaviask Aug 8, 2024
479ecfe
Skip duplicate test.
jnaviask Aug 8, 2024
bd5f887
Fix SQL infer by changing optional to nullish.
jnaviask Aug 8, 2024
d296b46
Fix nesting in sql output and convert all optional to nullish.
jnaviask Aug 8, 2024
6e4662f
Clean up community schema and seed unwrap nulls.
jnaviask Aug 8, 2024
6126320
Clean up Community schema.
jnaviask Aug 8, 2024
c3665a8
Use CTEs with json_agg to fix GetCommunities query.
jnaviask Aug 8, 2024
4109739
Fix CTE-based query.
jnaviask Aug 8, 2024
db90a84
Final fixes to query.
jnaviask Aug 8, 2024
b492cd2
Fix test linting.
jnaviask Aug 8, 2024
c3fdde7
Revert capitalization changes on groups and topics.
jnaviask Aug 8, 2024
cf10fdc
Make created_at and updated_at dates instead of any.
jnaviask Aug 8, 2024
2bdd0d5
builds and checks types with model-zod conventions
Aug 9, 2024
d071935
fix tests and linting errors
Aug 9, 2024
baeeb11
Fix broken linking test.
jnaviask Aug 9, 2024
0f38d0f
fix test
Aug 9, 2024
a1d6b8f
Remove CommunityBanner from include as it is queried separately.
jnaviask Aug 9, 2024
104be38
Merge branch 'master' into 8756.jake.get-communities-route-extension
jnaviask Aug 9, 2024
6659d15
Change dates to coerce + make Community associations nullish.
jnaviask Aug 9, 2024
4644df6
Revert testing changes to communities page.
jnaviask Aug 9, 2024
fbf5dc0
Fix build errors.
jnaviask Aug 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions libs/core/src/integration/outbox.schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ import * as events from './events.schemas';
const BaseOutboxProperties = z.object({
event_id: PG_INT.optional(),
relayed: z.boolean().optional(),
created_at: z.date().optional(),
updated_at: z.date().optional(),
created_at: z.coerce.date().optional(),
updated_at: z.coerce.date().optional(),
});

export const Outbox = z.union([
Expand Down
2 changes: 1 addition & 1 deletion libs/core/src/integration/util.schemas.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { z } from 'zod';

// All events should carry this common metadata
export const EventMetadata = z.object({
created_at: z.date().nullish().describe('When the event was emitted'),
created_at: z.coerce.date().nullish().describe('When the event was emitted'),
// TODO: TBD
// aggregateType: z.enum(Aggregates).describe("Event emitter aggregate type")
// aggregateId: z.string().describe("Event emitter aggregate id")
Expand Down
246 changes: 246 additions & 0 deletions libs/model/src/community/GetCommunities.query.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,246 @@
import { type Query } from '@hicommonwealth/core';
import * as schemas from '@hicommonwealth/schemas';
import { QueryTypes } from 'sequelize';
import { z } from 'zod';
import { models } from '../database';

export function GetCommunities(): Query<typeof schemas.GetCommunities> {
return {
...schemas.GetCommunities,
auth: [],
secure: false,
body: async ({ payload }) => {
const {
base,
include_node_info,
stake_enabled,
has_groups,
tag_ids,
cursor,
limit,
order_by,
order_direction,
} = payload;

// pagination configuration
const direction = order_direction || 'DESC';
const order_col = order_by || 'profile_count';
const offset = limit! * (cursor! - 1);

// note that tags are queried based on the INTERSECTION of provided tags
const filtering_tags = tag_ids && tag_ids.length > 0;
const replacements = filtering_tags ? { tag_ids } : {};

const communityCTE = `
WITH "community_CTE" AS (
SELECT "Community"."id",
"Community"."chain_node_id",
"Community"."name",
"Community"."discord_config_id",
"Community"."description",
"Community"."token_name",
"Community"."social_links",
"Community"."default_symbol",
"Community"."network",
"Community"."base",
"Community"."ss58_prefix",
"Community"."icon_url",
"Community"."active",
"Community"."stages_enabled",
"Community"."custom_stages",
"Community"."custom_domain",
"Community"."block_explorer_ids",
"Community"."collapsed_on_homepage",
"Community"."type",
"Community"."substrate_spec",
"Community"."has_chain_events_listener",
"Community"."default_summary_view",
"Community"."default_page",
"Community"."has_homepage",
"Community"."hide_projects",
"Community"."terms",
"Community"."bech32_prefix",
"Community"."admin_only_polling",
"Community"."discord_bot_webhooks_enabled",
"Community"."directory_page_enabled",
"Community"."directory_page_chain_node_id",
"Community"."thread_count",
"Community"."profile_count",
"Community"."namespace",
"Community"."namespace_address",
"Community"."created_at",
"Community"."updated_at",
"Community"."redirect",
"Community"."snapshot_spaces",
"Community"."include_in_digest_email"
FROM "Communities" AS "Community"
WHERE "Community"."active" = true
${
base
? `
AND "Community"."base" = '${base}'
`
: ''
}
${
has_groups
? `
AND (
SELECT "community_id"
FROM "Groups" AS "Groups"
WHERE ( "Groups"."community_id" = "Community"."id" )
LIMIT 1
) IS NOT NULL
`
: ''
}
${
stake_enabled
? `
AND (
SELECT "community_id"
FROM "CommunityStakes" AS "CommunityStakes"
WHERE (
"CommunityStakes"."stake_enabled" = true AND
"CommunityStakes"."community_id" = "Community"."id"
)
LIMIT 1
) IS NOT NULL
`
: ''
}
${
filtering_tags
? `
AND (
SELECT COUNT ( DISTINCT "CommunityTags"."tag_id" )
FROM "CommunityTags" AS "CommunityTags"
WHERE (
"CommunityTags"."community_id" = "Community"."id" AND
"CommunityTags"."tag_id" IN (:tag_ids)
)
) = ${tag_ids.length}
`
: ''
}
)
`,
communityTagsCTE = `
, "CommunityTags_CTE" AS (
SELECT "community_id",
json_agg(json_build_object(
'community_id', "community_id",
'tag_id', "tag_id",
'Tag', json_build_object('id', "CommunityTags->Tag"."id", 'name', "CommunityTags->Tag"."name")
)) as "CommunityTags"
FROM "CommunityTags"
JOIN "Tags" AS "CommunityTags->Tag" ON "CommunityTags"."tag_id" = "CommunityTags->Tag"."id"
WHERE "CommunityTags"."community_id" IN (SELECT "id" FROM "community_CTE")
GROUP BY "community_id"
)
`,
communityStakesCTE = `
, "CommunityStakes_CTE" AS (
SELECT "community_id",
json_agg(json_build_object(
'community_id', "community_id",
'stake_id', "stake_id",
'stake_token', "stake_token",
'vote_weight', "vote_weight",
'stake_enabled', "stake_enabled",
'created_at', "created_at",
'updated_at', "updated_at"
)) as "CommunityStakes"
FROM "CommunityStakes"
WHERE "CommunityStakes"."community_id" IN (SELECT "id" FROM "community_CTE")
GROUP BY "community_id"
)
`,
groupsCTE = has_groups
? `
, "Groups_CTE" AS (
SELECT "community_id",
json_agg(json_build_object(
'id', "id",
'community_id', "community_id",
'metadata', "metadata",
'requirements', "requirements",
'is_system_managed', "is_system_managed",
'created_at', "created_at",
'updated_at', "updated_at"
)) as "Groups"
FROM "Groups"
WHERE "Groups"."community_id" IN (SELECT "id" FROM "community_CTE")
GROUP BY "community_id"
)
`
: '';

const sql = `
${communityCTE}
${communityTagsCTE}
${communityStakesCTE}
${groupsCTE}
SELECT
"community_CTE".*,
count(*) OVER() AS total,
"CommunityTags_CTE"."CommunityTags" as "CommunityTags",
"CommunityStakes_CTE"."CommunityStakes" as "CommunityStakes"
${has_groups ? `, "Groups_CTE"."Groups" as "groups"` : ''}
${
include_node_info
? `
, "ChainNode"."id" AS "ChainNode.id",
"ChainNode"."url" AS "ChainNode.url",
"ChainNode"."eth_chain_id" AS "ChainNode.eth_chain_id",
"ChainNode"."cosmos_chain_id" AS "ChainNode.cosmos_chain_id",
"ChainNode"."alt_wallet_url" AS "ChainNode.alt_wallet_url",
"ChainNode"."balance_type" AS "ChainNode.balance_type",
"ChainNode"."name" AS "ChainNode.name",
"ChainNode"."description" AS "ChainNode.description",
"ChainNode"."health" AS "ChainNode.health",
"ChainNode"."ss58" AS "ChainNode.ss58",
"ChainNode"."bech32" AS "ChainNode.bech32",
"ChainNode"."cosmos_gov_version" AS "ChainNode.cosmos_gov_version",
"ChainNode"."block_explorer" AS "ChainNode.block_explorer",
"ChainNode"."slip44" AS "ChainNode.slip44",
"ChainNode"."created_at" AS "ChainNode.created_at",
"ChainNode"."updated_at" AS "ChainNode.updated_at"
`
: ''
}
FROM "community_CTE"
LEFT OUTER JOIN "CommunityTags_CTE" ON "community_CTE"."id" = "CommunityTags_CTE"."community_id"
LEFT OUTER JOIN "CommunityStakes_CTE" ON "community_CTE"."id" = "CommunityStakes_CTE"."community_id"
${
has_groups
? 'LEFT OUTER JOIN "Groups_CTE" ON "community_CTE"."id" = "Groups_CTE"."community_id"'
: ''
}
${
include_node_info
? 'LEFT OUTER JOIN "ChainNodes" AS "ChainNode" ON "community_CTE"."chain_node_id" = "ChainNode"."id"'
: ''
}
ORDER BY "community_CTE"."${order_col}" ${direction} LIMIT ${limit} OFFSET ${offset};
`;

const communities = await models.sequelize.query<
z.infer<typeof schemas.Community> & { total?: number }
>(sql, {
replacements,
type: QueryTypes.SELECT,
nest: true,
});

return schemas.buildPaginatedResponse(
communities,
+(communities.at(0)?.total ?? 0),
{
limit,
offset,
},
);
},
};
}
88 changes: 80 additions & 8 deletions libs/model/src/community/GetCommunity.query.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
import { type Query } from '@hicommonwealth/core';
import * as schemas from '@hicommonwealth/schemas';
import { Includeable } from 'sequelize';
import { Includeable, Op } from 'sequelize';
import { models } from '../database';
import { CommunityAttributes } from '../models';

export function GetCommunity(): Query<typeof schemas.GetCommunity> {
return {
Expand All @@ -10,20 +11,91 @@ export function GetCommunity(): Query<typeof schemas.GetCommunity> {
secure: false,
body: async ({ payload }) => {
const where = { id: payload.id };
const include: Includeable[] = [];
const include: Includeable[] = [
{
model: models.CommunityStake,
},
{
model: models.CommunityTags,
include: [
{
model: models.Tags,
},
],
},
];
if (payload.include_node_info) {
include.push({
model: models.ChainNode,
required: true,
});
}

return (
await models.Community.findOne({
where,
include,
})
)?.toJSON();
const result = await models.Community.findOne({
where,
include,
});

if (!result) {
return;
}

const [
adminsAndMods,
numVotingThreads,
numTotalThreads,
communityBanner,
] = await (<
Promise<
[
Array<{ address: string; role: 'admin' | 'moderator' }>,
number,
number,
{ banner_text: string } | undefined,
]
>
>Promise.all([
models.Address.findAll({
where: {
community_id: payload.id,
[Op.or]: [{ role: 'admin' }, { role: 'moderator' }],
},
attributes: ['address', 'role'],
}),
models.Thread.count({
where: {
community_id: payload.id,
stage: 'voting',
},
}),
models.Thread.count({
where: {
community_id: payload.id,
marked_as_spam_at: null,
},
}),
models.CommunityBanner.findOne({
where: {
community_id: payload.id,
},
}),
]));

return {
...result.toJSON(),
adminsAndMods,
numVotingThreads,
numTotalThreads,
communityBanner: communityBanner?.banner_text,
} as CommunityAttributes & {
numVotingThreads: number;
numTotalThreads: number;
adminsAndMods: Array<{
jnaviask marked this conversation as resolved.
Show resolved Hide resolved
address: string;
role: 'admin' | 'moderator';
}>;
communityBanner: string | undefined;
};
},
};
}
1 change: 1 addition & 0 deletions libs/model/src/community/index.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
export * from './CreateGroup.command';
export * from './CreateStakeTransaction.command';
export * from './GenerateStakeholderGroups.command';
export * from './GetCommunities.query';
export * from './GetCommunity.query';
export * from './GetCommunityStake.query';
export * from './GetMembers.query';
Expand Down
2 changes: 2 additions & 0 deletions libs/model/src/models/community.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import type { AddressInstance } from './address';
import type { ChainNodeAttributes, ChainNodeInstance } from './chain_node';
import type { CommentAttributes } from './comment';
import type { CommunityAlertAttributes } from './community_alerts';
import type { CommunityTagsAttributes } from './community_tags';
import type { ContractInstance } from './contract';
import type { StarredCommunityAttributes } from './starred_community';
import type { ThreadAttributes } from './thread';
Expand All @@ -15,6 +16,7 @@ import type { UserAttributes } from './user';
export type CommunityAttributes = z.infer<typeof Community> & {
// associations
ChainNode?: ChainNodeAttributes;
CommunityTags?: CommunityTagsAttributes[];
StarredCommunities?: StarredCommunityAttributes[];
Threads?: ThreadAttributes[] | ThreadAttributes['id'][];
Comments?: CommentAttributes[] | CommentAttributes['id'][];
Expand Down
Loading
Loading