Analytics Page Updates

[aj-codecov]

Context:

The page is composed of two pieces, the chart at the top and then the table of repos and their coverage at the bottom. The chart is configurable by date and repo (multi select). It shows the maximum coverage for a commit in one of the selected repos as the data point in the chart (See https://codecovio.atlassian.net/browse/CODE-2441). Each data point is a one week bucket. This means that if you select a couple of repos, the data point on the chart will be the highest coverage number from any commit within those repos within the week.

Problem(s) to solve:

1. There is no way for a user to know we use “max”

2. There is no way for a user to see more granularly than a weeklong bucket, leading to odd behavior if you select a time period less than one week.

   • message shows saying “not enough data to render graph”, which is true, but hover shows a single data point exists

   • If time period is a Sunday to Sunday (the date we list for each “bucket” is always a Sunday), it only shows the first Sunday’s data, not the second Sunday’s based on the hover

3. Their is no indication of what commit data might be coming from within the chart

4. Commit data can come from a commit where CI failed and the chart doesn’t check for this at any point

5. When we select multiple repos, what do we surface? It’s impossible to know.

   • Do we show the data of a single commit from any selected repo of the weeklong bucket that happened to be the high water mark for coverage that week?
   • Do we show an average of the highest coverage commits from all the repos?

6. The table below the chart shows repos, when they were last updated, tracked lines, and what current coverage is

   • none of this info is relevant to the chart above beyond the repo name

   • “last updated” is especially confusing as we don’t use the most up to date data in the chart

   • tracked lines gives an indication or repo size, but this doesn’t matter as it doesn’t inform the chart data in any way - maybe if the chart showed average this would matter more?

   • Test coverage is the worst offender here as it indicates a coverage percentage directly in conflict with what we display in the chart

7. The table results don’t change at all when the parameters of the chart are changed. If your coverage is 100% based on your most recent commit and you choose a time frame from three months ago when your coverage was worse, it’ll still be saying 100% in the table, the chart will show the old coverage though.

Proposed Solution(s):

Table layout

• Mimic the layout of the file tree within a repo

  • Repo | Tracked lines (total) | Covered (total) | Partial (total) | Missed (total) | Coverage % (total hits / (sum of total hit + total partial + total miss))

Context Selection

• Allow a user to select Min/Max/Average

• Allow a user to select their time bucket (days, weeks, months) a la flags tab

• These changes should change the data in the table below

  *Last updated could be relevant commit and link to a commit

  *Tracked lines/covered/partial/missed/coverage % would all be relevant to the parameters of the graph

Additional information

• Inform a user data is being pulled from their default branch per repo

• Inform a user what commit is providing the info in the graph

• Inform a user the line of the chart with multiple repos selected is an average

  *Alternatively, if we don’t want to make it the average, explain what it is

[aj-codecov]

Initial Scope:

• Allow a user to select Min/Max/Average

• Allow a user to select their time bucket (days, weeks, months) a la flags tab

Stretch Scope:

• These changes should change the data in the table below

  • Last updated could be relevant commit and link to a commit

  • Tracked lines/covered/partial/missed/coverage % would all be relevant to the parameters of the graph

[aj-codecov]

Related issue: codecov/gazebo#2315

[codecovdesign]

related: https://sentry.zendesk.com/agent/tickets/112977