data sources app v2

[josh-chamberlain]

Context

We released this app at the end of 2023.

We're iterating and expanding on our existing prototypes, linking them together into something more cohesive and engaging. The goal is an app where people can find data and collaborators to help them understand police systems.

We're doing the work on the data-sources-app-v2 fork.

Rollout

• v2 alpha: start using v2 search/results #540
• v2 beta: semi-closed launch #543
  • contains V1 to V2 Database Migration Plan #456
• v2 launch: open the floodgates #544

Feature	Now	Alpha	Beta	Launch
Search, results, details	v1 app	v2 app	v2 app	v2 app
New source	v1*	v1*	v2	v2
New request	v1*	v1*	v2 (visible signed in)	v2 (visible to all)
Source & request approval	v1*	v1*	retool	retool
Edit data sources	v1*	v1*	retool	retool
Sign up	-	-	v2 (hidden, invite only)	v2 (open)
Log in	-	-	v2	v2
Follow	-	-	v2	v2
Notify	-	-	v2	v2

* = in Airtable

Front end views

• All these views are public, but some actions are only available to logged-in users.
• Since we tend to use one branch for the back end tweaks needed for any front end feature, they're one issue.

In rough order of priority, but see the kanban:

• v2: main page/search #249
• v2: search results #250
• v2: new data source #252
• v2: new request #251
• v2: data source details #253
• v2: user profile #274
• v2: request details #254

Front end docs

• v2: Update client documentation during v2 updates #379

Front end testing (can be a fast follow)

• Remove page-level client unit testing and implement e2e (integration) tests using Playwright pdap.io#203

Back end changes

In addition to minor back end changes referenced in the issues above.

• Replace Airtable #32
• v2 auth & permissions refactor #318
• v2 Testing Overhaul #271
• v2: Logging for endpoint calls #334
• v2: requests support for back end #256
• v2: follow search notifications #26
  • Create user email verification system #463
  • Create automation job for sending notifications to users about followed locations #493

Style

• Most of these views have a ~75% pane on the left for information and form actions, and a 25% panel on the right for details and CTAs.
• We should see how a pretty vanilla application of our design system looks, then add anticipated new components from the wireframe

Design system:

• Update release workflow to include a beta release strategy design-system#60
• Update styling to match search v2 updates design-system#61
• Update checkbox input to be viable design-system#62
• Create loading spinner / shimmer design-system#63
• Create dropdown component design-system#64
• Add accessibility linting design-system#89

Docs

Throughout: READMEs for devs; docs pages for people looking for data; GitHub and Discord wayfinding for volunteers.

[joshuagraber]

Hey, @josh-chamberlain, I've had a look through the issues here. Everything looks pretty straightforward, I just have a few comments and questions:

General:

• Colors, etc, are changing, yes? If so, design-system will need a new major version to reflect that. Would it also be possible to add some component style definitions in Figma for this purpose?
  Note: (I'm going to create a beta release flow for design-system so that we can release these updates easily without going to production. It's a bit overdue anyway. Symlinking for testing is not ideal.)
• We will also need some new components in design-system for this.
  • Checkbox input
    • Style 1: expressed in "record types" section
    • Style 2: expressed in "location" section
  • Dropdown component (for filtering section)
  • Loading spinner and/or loading shimmer. (Maybe we want to think about this at the design level? Given that a lot of these pages and their sub-components will require data fetching, I think we need better loading states either way.)
• Will different users have different permissions/roles to gate the features they are permitted to interact with?

For #249

What does "coarse selection" mean?

For #250

users who are logged in can click a "Follow search" button to be notified about new sources in the search

Once we have "saved searches," is the plan to build some kind of event emitter to send these notifications? Do we mean to use email, sms, push?

For #254

links to a github issue for the request, where collaborators can comment

Are we auto-creating this issue when a user makes a request? Will the backend handle this or should we expect to do it client-side?

Nice to have client enhancements that I'd love to get done here as well since we're re-doing most of the client:

• Lazy loading of route components (have been meaning to add this anyway for performance reasons)
• Lazy loading of any sub-components that require expensive network calls
• Subtle transition animations for routes and elements entering / exiting the DOM

Thank you. I've added some issues to design-system to start on some of this (60, 61, 62, 63, 64).

[josh-chamberlain]

@joshuagraber thanks for taking a look!

• colors, etc should remain the same. The wireframe is intended to be a layout onto which our existing colors, fonts, etc will fit.
• new design system components will be needed, but some combination of you, Vikram and I will have at least one session where we do some detail work on those components/interactions. loading states, YES.
• "coarse selection" refers to record_type grouping into broader categories. "Field Contacts" and "Calls for Service" are both under the "Police & Public Interactions" category. For users without much context or terminology at their fingertips, the hope is that these coarse groups will help them get what they want. Full list here.
• for "follow" type updates, v2: follow search notifications #26 represents the backend work for that
• the back end will handle requests via v2: requests support for back end #256
• I added your issues above

[joshuagraber]

Nice @josh-chamberlain thank you for the context!