Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: initial commit #1

Merged
merged 3 commits into from
Jul 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@ A clear and concise description of what you expected to happen.

If applicable, add screenshots to help explain your problem.

**Kitex version:**
**Goref version:**

Please provide the version of Kitex you are using.
Please provide the version of goref you are using.

**Environment:**

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ jobs:
unit-benchmark-test:
strategy:
matrix:
go: [ "1.18", "1.19", "1.20", "1.21", "1.22" ]
go: [ "1.21", "1.22" ]
os: [ X64 ]
runs-on: ${{ matrix.os }}
steps:
Expand Down
1 change: 0 additions & 1 deletion .golangci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ run:
# include `vendor` `third_party` `testdata` `examples` `Godeps` `builtin`
skip-dirs-use-default: true
skip-dirs:
- kitex_gen
skip-files:
- ".*\\.mock\\.go$"
# output configuration options
Expand Down
8 changes: 8 additions & 0 deletions .licenserc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,12 @@ header:
- '**/*.go'
- '**/*.s'

paths-ignore:
- pkg/proc/eval.go
- pkg/proc/mem.go
- pkg/proc/protobuf.go
- pkg/proc/region.go
- pkg/proc/variables.go
- testdata

comment: on-failure
10 changes: 5 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ We use [git-flow](https://nvie.com/posts/a-successful-git-branching-model/) as o

## Bugs
### 1. How to Find Known Issues
We are using [Github Issues](https://github.com/cloudwego/kitex/issues) for our public bugs. We keep a close eye on this and try to make it clear when we have an internal fix in progress. Before filing a new task, try to make sure your problem doesn’t already exist.
We are using [Github Issues](https://github.com/cloudwego/goref/issues) for our public bugs. We keep a close eye on this and try to make it clear when we have an internal fix in progress. Before filing a new task, try to make sure your problem doesn’t already exist.

### 2. Reporting New Issues
Providing a reduced test code is a recommended way for reporting issues. Then can placed in:
Expand All @@ -23,12 +23,12 @@ Please do not report the safe disclosure of bugs to public issues. Contact us by

## Submit a Pull Request
Before you submit your Pull Request (PR) consider the following guidelines:
1. Search [GitHub](https://github.com/cloudwego/kitex/pulls) for an open or closed PR that relates to your submission. You don't want to duplicate existing efforts.
1. Search [GitHub](https://github.com/cloudwego/goref/pulls) for an open or closed PR that relates to your submission. You don't want to duplicate existing efforts.
2. Be sure that an issue describes the problem you're fixing, or documents the design for the feature you'd like to add. Discussing the design upfront helps to ensure that we're ready to accept your work.
3. [Fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) the cloudwego/kitex repo.
3. [Fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) the cloudwego/goref repo.
4. In your forked repository, make your changes in a new git branch:
```
git checkout -b my-fix-branch develop
git checkout -b my-fix-branch main
```
5. Create your patch, including appropriate test cases.
6. Follow our [Style Guides](#code-style-guides).
Expand All @@ -38,7 +38,7 @@ Before you submit your Pull Request (PR) consider the following guidelines:
```
git push origin my-fix-branch
```
9. In GitHub, send a pull request to `kitex:develop`
9. In GitHub, send a pull request to `goref:main`

## Contribution Prerequisites
- Our development environment keeps up with [Go Official](https://golang.org/project/).
Expand Down
45 changes: 44 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,44 @@
# .github
# Goref

[![WebSite](https://img.shields.io/website?up_message=cloudwego&url=https%3A%2F%2Fwww.cloudwego.io%2F)](https://www.cloudwego.io/)
[![License](https://img.shields.io/github/license/cloudwego/goref)](https://github.com/cloudwego/goref/blob/main/LICENSE-APACHE)

Goref is a Go heap object reference analysis tool based on delve.
It can display the space and object count distribution of Go memory references, which is helpful for efficiently locating memory leak issues or viewing persistent heap objects to optimize GC overhead.

## Installation

Clone the git repository and build:

```
$ git clone https://github.com/cloudwego/goref
$ cd goref
$ go install github.com/cloudwego/goref/cmd/grf
```

> Supported go version to compile the command tool: go1.21 ~ go1.22.

## Usage

Attach to a running process with its PID, and then use go pprof tool to open the output file.

```
$ grf attach ${PID}
successfully output to `grf.out`
$ go tool pprof -http=:5079 ./grf.out
```

The opened HTML page displays the reference distribution of the heap memory. You can choose to view the "inuse space" or "inuse objects".

It also supports analyzing core files, e.g.

```
$ grf core ${execfile} ${corefile}
successfully output to `grf.out`
```

> Supported go version for executable file: go1.17 ~ go1.22.

## Credit

Thanks to [Delve](https://github.com/go-delve/delve) for providing powerful golang debugger.
144 changes: 144 additions & 0 deletions cmd/grf/cmds/commands.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
// Copyright 2024 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package cmds

import (
"errors"
"fmt"
"os"
"strconv"

"github.com/go-delve/delve/pkg/config"
"github.com/go-delve/delve/pkg/logflags"
"github.com/go-delve/delve/service/debugger"
"github.com/spf13/cobra"

myproc "github.com/cloudwego/goref/pkg/proc"
)

var (
// rootCommand is the root of the command tree.
rootCommand *cobra.Command

conf *config.Config
loadConfErr error
outFile string
)

// New returns an initialized command tree.
func New(docCall bool) *cobra.Command {
// Config setup and load.
conf, loadConfErr = config.LoadConfig()

// Main dlv root command.
rootCommand = &cobra.Command{
Use: "grf",
Short: "Goref is a Go heap object reference analysis tool based on delve.",
Long: "Goref is a Go heap object reference analysis tool based on delve.",
}
rootCommand.CompletionOptions.DisableDefaultCmd = true

// 'attach' subcommand.
attachCommand := &cobra.Command{
Use: "attach pid [executable]",
Short: "Attach to running process and begin scanning.",
Long: `Attach to an already running process and begin scanning its memory.

This command will cause Goref to take control of an already running process and begin scanning object references.
You'll have to wait for goref until it outputs 'successfully output to ...', or kill it to terminate scanning.
`,
PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
if len(args) == 0 {
return errors.New("you must provide a PID")
}
return nil
},
Run: attachCmd,
}
attachCommand.Flags().StringVarP(&outFile, "out", "o", "grf.out", "output file name")
rootCommand.AddCommand(attachCommand)

coreCommand := &cobra.Command{
Use: "core <executable> <core>",
Short: "Scan a core dump.",
Long: `Scan a core dump (only supports linux and windows core dumps).

The core command will open the specified core file and the associated executable and begin scanning object references.
You'll have to wait for goref until it outputs 'successfully output to ...', or kill it to terminate scanning.`,
PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
if len(args) < 2 {
return errors.New("you must provide a core file and an executable")
}
return nil
},
Run: coreCmd,
}
coreCommand.Flags().StringVarP(&outFile, "out", "o", "grf.out", "output file name")
rootCommand.AddCommand(coreCommand)

return rootCommand
}

func attachCmd(_ *cobra.Command, args []string) {
var pid int
var exeFile string
if len(args) > 0 {
var err error
pid, err = strconv.Atoi(args[0])
if err != nil {
fmt.Fprintf(os.Stderr, "Invalid pid: %s\n", args[0])
os.Exit(1)
}
}
if len(args) > 1 {
exeFile = args[1]
}
os.Exit(execute(pid, exeFile, "", outFile, conf))
}

func coreCmd(_ *cobra.Command, args []string) {
os.Exit(execute(0, args[0], args[1], outFile, conf))
}

func execute(attachPid int, exeFile, coreFile, outFile string, conf *config.Config) int {
if loadConfErr != nil {
logflags.DebuggerLogger().Errorf("%v", loadConfErr)
}

dConf := debugger.Config{
AttachPid: attachPid,
Backend: "default",
CoreFile: coreFile,
DebugInfoDirectories: conf.DebugInfoDirectories,
AttachWaitFor: "",
AttachWaitForInterval: 1,
AttachWaitForDuration: 0,
}
var args []string
if exeFile != "" {
args = []string{exeFile}
}
dbg, err := debugger.New(&dConf, args)
if err != nil {
fmt.Fprintln(os.Stderr, err.Error())
return 1
}
t := dbg.Target()
if err = myproc.ObjectReference(t, outFile); err != nil {
fmt.Fprintln(os.Stderr, err.Error())
return 1
}
return 0
}
23 changes: 23 additions & 0 deletions cmd/grf/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
// Copyright 2024 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package main

import (
"github.com/cloudwego/goref/cmd/grf/cmds"
)

func main() {
cmds.New(false).Execute()
}
24 changes: 24 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
module github.com/cloudwego/goref

go 1.21

toolchain go1.22.2

require (
github.com/go-delve/delve v1.22.2-0.20240701043435-faac701e9f79
github.com/modern-go/reflect2 v1.0.2
github.com/spf13/cobra v1.8.0
jayantxie marked this conversation as resolved.
Show resolved Hide resolved
)

require (
github.com/cilium/ebpf v0.11.0 // indirect
github.com/hashicorp/golang-lru v1.0.2 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/spf13/pflag v1.0.5 // indirect
golang.org/x/arch v0.6.0 // indirect
golang.org/x/exp v0.0.0-20230224173230-c95f2b4c22f2 // indirect
golang.org/x/sys v0.17.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
)
56 changes: 56 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
github.com/cilium/ebpf v0.11.0 h1:V8gS/bTCCjX9uUnkUFUpPsksM8n1lXBAvHcpiFk1X2Y=
github.com/cilium/ebpf v0.11.0/go.mod h1:WE7CZAnqOL2RouJ4f1uyNhqr2P4CCvXFIqdRDUgWsVs=
github.com/cpuguy83/go-md2man/v2 v2.0.3/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/creack/pty v1.1.20 h1:VIPb/a2s17qNeQgDnkfZC35RScx+blkKF8GV68n80J4=
github.com/creack/pty v1.1.20/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/frankban/quicktest v1.14.5 h1:dfYrrRyLtiqT9GyKXgdh+k4inNeTvmGbuSgZ3lx3GhA=
github.com/frankban/quicktest v1.14.5/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/go-delve/delve v1.22.2-0.20240701043435-faac701e9f79 h1:wDUy5rTtpM5oCscyPYQCDiTjWNNdN2j+vNpEHdKoUI0=
github.com/go-delve/delve v1.22.2-0.20240701043435-faac701e9f79/go.mod h1:iS7XgxZVcrCf9piPKK1RT21pEbjtgzrYlPDl2uja6gQ=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/hashicorp/golang-lru v1.0.2 h1:dV3g9Z/unq5DpblPpw+Oqcv4dU/1omnb4Ok8iPY6p1c=
github.com/hashicorp/golang-lru v1.0.2/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/spf13/cobra v1.8.0 h1:7aJaZx1B85qltLMc546zn58BxxfZdR/W22ej9CFoEf0=
github.com/spf13/cobra v1.8.0/go.mod h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyhcho=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
golang.org/x/arch v0.6.0 h1:S0JTfE48HbRj80+4tbvZDYsJ3tGv6BUU3XxyZ7CirAc=
golang.org/x/arch v0.6.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys=
golang.org/x/exp v0.0.0-20230224173230-c95f2b4c22f2 h1:Jvc7gsqn21cJHCmAWx0LiimpP18LZmUxkT5Mp7EZ1mI=
golang.org/x/exp v0.0.0-20230224173230-c95f2b4c22f2/go.mod h1:CxIveKay+FTh1D0yPZemJVgC/95VzuuOLq5Qi4xnoYc=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0 h1:25cE3gD+tdBA7lp7QfhuV+rJiE9YXTcS3VG1SqssI/Y=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
Loading
Loading