-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Add repository metrics #177
Open
steffen
wants to merge
20
commits into
Autodesk:master
Choose a base branch
from
steffen:master
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
6cebdbb
First version of integrated repository metrics
steffen d5320c7
Add passing around nwo in query string in navigation links
steffen 69b3577
Add loading repository specific data
steffen 2480fd4
Add back to instance statistics link to repo view
steffen 4495986
Add overview table to overview page
steffen 5b33604
Add first version of creating a report for monitored repositories
steffen b68d0a3
Add report for GitHub api request types by count
steffen 0d2b923
Add abstract ReportRepository class
steffen 93c8f0f
Add monitored repositories
steffen 06384ab
Add showing repository name in page header
steffen 13ee2ec
Add writing repository name directly into the header to avoid flickering
steffen d9495fd
Add git-sizer metrics
steffen 44b62be
Make "passing" repository to scripts work
steffen 221fafc
Exclude internal api requests from api request types data
steffen a75e34a
Add repository metrics support for non-remote runs
steffen 654b75b
Add repository Git traffic metrics
steffen 7008ec9
Consolidate git-sizer report and include all quantity values
steffen 189d46c
Tweaked charts to focus on actionable metrics
steffen c1b1a8a
Make updater scripts executable
steffen eff5fca
Fix issue with relative dataURLs by only supporting absolute dataURLs
steffen File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Add report for GitHub api request types by count
- Loading branch information
commit b68d0a3d1367583324d7e4afb232739badde7187
There are no files selected for viewing
21 changes: 21 additions & 0 deletions
21
updater/reports/repository/ReportGitHubApiRequestTypesByCount.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
from ..Report import * | ||
|
||
# Lists the number of GitHub api request types | ||
# Ordered by count in descending order | ||
class ReportGitHubApiRequestTypesByCount(Report): | ||
def name(self): | ||
return "github-api-request-types-by-count" | ||
|
||
def metaName(self): | ||
return self.repository + "/" + self.name() | ||
|
||
def fileName(self): | ||
return os.path.join(self.dataDirectory, "repository", self.repositoryOwner, self.repositoryName, self.name() + ".tsv") | ||
|
||
# The data is overwritten every day, so skip reading the old data | ||
def readData(self): | ||
pass | ||
|
||
def updateData(self): | ||
self.header, self.data = self.parseData( | ||
self.executeScript(self.scriptPath("repository/github-api-request-types-by-count.sh"))) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
19 changes: 19 additions & 0 deletions
19
updater/scripts/repository/github-api-request-types-by-count.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
#!/bin/bash | ||
|
||
# | ||
# Count number of API calls per route | ||
# | ||
|
||
echo -e "type\tcount" | ||
|
||
zcat -f /var/log/github/unicorn.log.1* | | ||
grep -F 'request_category=api' | | ||
grep -Fv 'remote_address=127.0.0.1' | | ||
grep "$1" | | ||
grep -oP 'route=\K\S+' | | ||
grep -Fvx 'nil' | | ||
sort | | ||
uniq -ic | | ||
sort -rn | | ||
head -20 | | ||
awk '{ printf("%s\t%s\n", $2, $1) }' | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That looks pretty similar to https://github.com/Autodesk/hubble/blob/master/updater/scripts/api-requests-by-user.sh ... should we add a repo option to the original script to avoid code duplication?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point! There are other reports I wanted to add for repo metrics that are pretty much the same as well. I'll explore adding a repo option to the original scripts.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@larsxschneider I added a repo option for
git-download.sh
. Let me know if you think that's a good solution and if it's ok to use for other reports/scripts as well.