Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Xray Source Control Service #811

Closed
wants to merge 78 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
a664ea4
Promote version to 1.28.1 (#733)
omerzi Apr 5, 2023
0b22cb3
Added the Frogbot badge to the README (#740)
eyalbe4 Apr 13, 2023
93d1ee0
Merge remote-tracking branch 'upstream/dev' into masterupstream
sverdlov93 Apr 19, 2023
60012f0
Merge remote-tracking branch 'origin/dev'
omerzi May 2, 2023
72340a7
Merge branch 'dev'
eyalbe4 May 16, 2023
3c0e410
Merge remote-tracking branch 'origin/dev'
talarian1 May 17, 2023
7dd161b
Merge remote-tracking branch 'origin/dev'
omerzi May 21, 2023
88f4c5c
Merge remote-tracking branch 'origin/dev'
talarian1 May 23, 2023
21b99ad
Merge branch 'master' into dev
yahavi May 23, 2023
b71c280
Promoted version to 1.29.1
yahavi May 23, 2023
8f7755c
Merge branch 'dev'
eyalbe4 Jun 10, 2023
e6ba583
Merge branch 'dev'
eyalbe4 Jun 25, 2023
b624428
Merge branch 'dev'
RobiNino Jun 29, 2023
da0c018
Merge branch 'master' into dev
omerzi Jul 12, 2023
f750025
Promoted version to 1.31.1
omerzi Jul 12, 2023
e1d57bd
Merge remote-tracking branch 'origin/dev'
omerzi Jul 18, 2023
886ff5f
Merge branch 'dev'
eyalbe4 Jul 31, 2023
b1ae836
Promoted version to 1.31.3
eyalbe4 Jul 31, 2023
df38e23
Merge branch 'dev'
eyalbe4 Jul 31, 2023
46b7fb9
Merge branch 'dev'
eyalbe4 Aug 3, 2023
5f58723
Resolve conflicts
eyalbe4 Aug 3, 2023
0ca9b47
Refactor to interface
EyalDelarea Aug 15, 2023
a911794
Split scan service
EyalDelarea Aug 15, 2023
5386aaa
Working XSC call
EyalDelarea Aug 16, 2023
a80d5b9
optional multi-scan-id
EyalDelarea Aug 16, 2023
c660dc9
refactor
EyalDelarea Aug 16, 2023
c57e00d
Check XSC
EyalDelarea Aug 16, 2023
6b23b57
fix multi-scan-id
EyalDelarea Aug 16, 2023
a72281b
Fix code analysis errors
EyalDelarea Aug 16, 2023
b0358b2
fix always using xsc
EyalDelarea Aug 17, 2023
18c6c93
fix static check
EyalDelarea Aug 17, 2023
b882364
Merge branch 'dev' of https://github.com/jfrog/jfrog-client-go into xsc
EyalDelarea Aug 17, 2023
ebc2382
Fix choose XSC server when enabled
EyalDelarea Aug 17, 2023
692ffe5
Handle XSC/XRAY url provided from flags and config.
EyalDelarea Aug 17, 2023
29442bd
fix static check
EyalDelarea Aug 17, 2023
a9a7d92
Refactor
EyalDelarea Aug 17, 2023
a469339
Comment
EyalDelarea Aug 20, 2023
58b7b7f
Fix date & multi scan on get
EyalDelarea Aug 20, 2023
94e1af2
Merge branch 'dev' of https://github.com/jfrog/jfrog-client-go into xsc
EyalDelarea Aug 20, 2023
b6162da
Update fields
EyalDelarea Aug 21, 2023
a30da83
Fix wrong url
EyalDelarea Aug 21, 2023
c4a3271
remove fake param
EyalDelarea Aug 22, 2023
c822cb0
add xsc tests
EyalDelarea Aug 23, 2023
cd9b278
trigger tests workflow
EyalDelarea Aug 23, 2023
bb862af
trigger tests workflow new secrets
EyalDelarea Aug 23, 2023
e2a654e
refactor and add version test for xsc
EyalDelarea Aug 23, 2023
e47d7f8
test xsc without context
EyalDelarea Aug 23, 2023
3647f3b
Update README new test flag
EyalDelarea Aug 23, 2023
fd2543a
Add default to mock server
EyalDelarea Aug 24, 2023
75aaa3d
pull dev
EyalDelarea Aug 24, 2023
3812c20
Merge branch 'dev' of https://github.com/jfrog/jfrog-client-go into xsc
EyalDelarea Aug 28, 2023
56c6cac
fix import
EyalDelarea Aug 28, 2023
97b4701
Update test and README.md
EyalDelarea Aug 29, 2023
a43644e
Add XSC const
EyalDelarea Aug 29, 2023
d99342c
fix endpoint
EyalDelarea Aug 30, 2023
349551c
Merge branch 'dev' of https://github.com/jfrog/jfrog-client-go into xsc
EyalDelarea Aug 31, 2023
6822658
dont overwrite msi for testings purposes
EyalDelarea Aug 31, 2023
59e8b9e
Allow pass msi as env
EyalDelarea Aug 31, 2023
2a27e77
Merge branch 'dev' of https://github.com/jfrog/jfrog-client-go into xsc
EyalDelarea Sep 3, 2023
5e95ddc
fix pull dev
EyalDelarea Sep 3, 2023
e15b99a
pull dev
EyalDelarea Sep 4, 2023
1e6715d
fix merge
EyalDelarea Sep 4, 2023
d354c90
Merge branch 'dev' of https://github.com/jfrog/jfrog-client-go into xsc
EyalDelarea Sep 6, 2023
16e4917
Small refactor
EyalDelarea Sep 7, 2023
35ab59b
refactor
EyalDelarea Sep 7, 2023
09e885b
edit xsc test
EyalDelarea Sep 7, 2023
fe5814b
fix format
EyalDelarea Sep 7, 2023
db77870
break import cycles
EyalDelarea Sep 8, 2023
d79c668
cr
EyalDelarea Sep 8, 2023
5119d42
return only version and not bool
EyalDelarea Sep 8, 2023
1f8eabc
Add XSC version log
EyalDelarea Sep 8, 2023
d74eec0
fix test
EyalDelarea Sep 8, 2023
ba49d30
CR
EyalDelarea Sep 8, 2023
cd281a3
CR
EyalDelarea Sep 10, 2023
412c1e6
Move XSC tests to unit tests and remove from suits
EyalDelarea Sep 10, 2023
da2082c
remove unused fields
EyalDelarea Sep 10, 2023
584f264
fix static check
EyalDelarea Sep 10, 2023
7bf2c8a
remove format diffs
EyalDelarea Sep 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions access/auth/accessdetails.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,18 @@ type accessDetails struct {
auth.CommonConfigFields
}

func (rt *accessDetails) GetXscUrl() string {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every Xray manager have this function as this is how we check if you have XSC.
It's build on top of xray.
XrayManager.GetXscUrl -> send get request and then get yes or not.

panic("Failed: Method is not implemented")
}

func (rt *accessDetails) SetXscUrl(url string) {
panic("Failed: Method is not implemented")
}

func (rt *accessDetails) GetVersion() (string, error) {
panic("Failed: Method is not implemented")
}

func (rt *accessDetails) GetXscVersion() string {
panic("Failed: Method is not implemented")
}
12 changes: 12 additions & 0 deletions artifactory/auth/rtdetails.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,18 @@ type artifactoryDetails struct {
auth.CommonConfigFields
}

func (rt *artifactoryDetails) SetXscUrl(url string) {
panic("Failed: Method is not implemented")
}

func (rt *artifactoryDetails) GetXscUrl() string {
panic("Failed: Method is not implemented")
}

func (rt *artifactoryDetails) GetXscVersion() string {
panic("Failed: Method is not implemented")
}

func (rt *artifactoryDetails) GetVersion() (string, error) {
var err error
if rt.Version == "" {
Expand Down
9 changes: 9 additions & 0 deletions artifactory/services/utils/tests/xray/consts.go
Original file line number Diff line number Diff line change
Expand Up @@ -1433,3 +1433,12 @@ var MapResponse = map[string]map[string]string {
"ReportDetails": LicensesReportDetailsResponse,
},
}

const gitInfoSentResponse = `{"multi_scan_id": "f2a8d4fe-40e6-11ee-84e4-02ee10c7f40e"}`

const scanGraphResponse = `{"scan_id": "9c9dbd61-f544-4e33-4613-34727043d71f"}`

// Empty body as tests currently do not check response body.
const getScanResultsResponse = `{}`

const xscVersionResponse = `{"xsc_version": "0.0.0"}`
26 changes: 24 additions & 2 deletions artifactory/services/utils/tests/xray/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,6 @@ func reportHandler(w http.ResponseWriter, r *http.Request) {
log.Error(err)
http.Error(w, err.Error(), http.StatusInternalServerError)
}

return
}
case http.MethodPost:
Expand Down Expand Up @@ -114,7 +113,7 @@ func reportHandler(w http.ResponseWriter, r *http.Request) {
}
case http.MethodDelete:
if numSegments == 0 {
_, err := fmt.Fprint(w, XrayReportDeleteResponse)
_, err := fmt.Fprint(w, XrayReportDeleteResponse)
if err != nil {
log.Error(err)
http.Error(w, err.Error(), http.StatusInternalServerError)
Expand All @@ -140,6 +139,27 @@ func entitlementsHandler(w http.ResponseWriter, r *http.Request) {
}
}

func securityHandler(w http.ResponseWriter, r *http.Request) {
var err error
endpoint := r.URL.Path[strings.LastIndex(r.URL.Path, "/")+1:]
switch endpoint {
case "gitinfo":
_, err = fmt.Fprint(w, gitInfoSentResponse)
case "graph":
_, err = fmt.Fprint(w, scanGraphResponse)
case "9c9dbd61-f544-4e33-4613-34727043d71f":
_, err = fmt.Fprint(w, getScanResultsResponse)
case "version":
_, err = fmt.Fprint(w, xscVersionResponse)
default:
panic("Unexpected request URI:" + r.RequestURI)
}
if err != nil {
log.Error(err)
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}

func buildScanHandler(w http.ResponseWriter, r *http.Request) {
argsSegment := strings.Split(r.URL.Path, services.BuildScanAPI)[1]
switch r.Method {
Expand Down Expand Up @@ -188,6 +208,8 @@ func StartXrayMockServer() int {
handlers["/api/xray/scanBuild"] = scanBuildHandler
handlers["/api/v2/summary/artifact"] = artifactSummaryHandler
handlers["/api/v1/entitlements/feature/"] = entitlementsHandler
handlers["/xsc/"] = securityHandler
handlers["/xray/"] = securityHandler
handlers[fmt.Sprintf("/%s/", services.ReportsAPI)] = reportHandler
handlers[fmt.Sprintf("/%s/", services.BuildScanAPI)] = buildScanHandler
handlers["/"] = http.NotFound
Expand Down
5 changes: 4 additions & 1 deletion auth/servicedetails.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,11 @@ type ServiceDetails interface {
GetSshAuthHeaders() map[string]string
GetClient() *jfroghttpclient.JfrogHttpClient
GetVersion() (string, error)
GetXscUrl() string
GetXscVersion() string

SetUrl(url string)
SetXscUrl(url string)
SetUser(user string)
SetPassword(password string)
SetApiKey(apiKey string)
Expand All @@ -51,12 +54,12 @@ type ServiceDetails interface {
AuthenticateSsh(sshKey, sshPassphrase string) error
InitSsh() error
RunPreRequestFunctions(httpClientDetails *httputils.HttpClientDetails) error

CreateHttpClientDetails() httputils.HttpClientDetails
}

type CommonConfigFields struct {
Url string `json:"-"`
XscUrl string `json:"-"`
User string `json:"-"`
Password string `json:"-"`
ApiKey string `json:"-"`
Expand Down
12 changes: 12 additions & 0 deletions distribution/auth/dsdetails.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,18 @@ type distributionDetails struct {
auth.CommonConfigFields
}

func (ds *distributionDetails) GetXscUrl() string {
panic("Failed: Method is not implemented")
}

func (ds *distributionDetails) SetXscUrl(url string) {
panic("Failed: Method is not implemented")
}

func (ds *distributionDetails) GetXscVersion() string {
panic("Failed: Method is not implemented")
}

func (ds *distributionDetails) GetVersion() (string, error) {
var err error
if ds.Version == "" {
Expand Down
12 changes: 12 additions & 0 deletions lifecycle/auth/lifecycledetails.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,18 @@ type lifecycleDetails struct {
auth.CommonConfigFields
}

func (rt *lifecycleDetails) GetXscUrl() string {
panic("Failed: Method is not implemented")
}

func (rt *lifecycleDetails) SetXscUrl(url string) {
panic("Failed: Method is not implemented")
}

func (rt *lifecycleDetails) GetVersion() (string, error) {
panic("Failed: Method is not implemented")
}

func (rt *lifecycleDetails) GetXscVersion() string {
panic("Failed: Method is not implemented")
}
12 changes: 12 additions & 0 deletions pipelines/auth/pipelinesdetails.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,18 @@ type pipelinesDetails struct {
auth.CommonConfigFields
}

func (pd *pipelinesDetails) GetXscUrl() string {
panic("Failed: Method is not implemented")
}

func (pd *pipelinesDetails) SetXscUrl(url string) {
panic("Failed: Method is not implemented")
}

func (pd *pipelinesDetails) GetXscVersion() string {
panic("Failed: Method is not implemented")
}

func (pd *pipelinesDetails) GetVersion() (string, error) {
var err error
if pd.Version == "" {
Expand Down
1 change: 1 addition & 0 deletions tests/jfrogclient_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ func setupIntegrationTests() {
createXrayWatchManager()
createXrayPolicyManager()
createXrayBinMgrManager()
createSecurityServiceManager()
}
if *TestPipelines {
createPipelinesIntegrationsManager()
Expand Down
44 changes: 39 additions & 5 deletions tests/utils_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import (
"errors"
"flag"
"fmt"
"github.com/jfrog/jfrog-client-go/config"
"net/http"
"os"
"path/filepath"
Expand Down Expand Up @@ -110,6 +111,7 @@ var (
testsXrayWatchService *xrayServices.WatchService
testsXrayPolicyService *xrayServices.PolicyService
testXrayBinMgrService *xrayServices.BinMgrService
securityServiceManager xrayServices.SecurityServiceManager

// Pipelines Services
testsPipelinesIntegrationsService *pipelinesServices.IntegrationsService
Expand Down Expand Up @@ -140,10 +142,10 @@ const (
func init() {
ciRunId = flag.String("ci.runId", "", "A unique identifier used as a suffix to create repositories in the tests")
TestArtifactory = flag.Bool("test.artifactory", false, "Test Artifactory")
TestDistribution = flag.Bool("test.distribution", false, "Test distribution")
TestXray = flag.Bool("test.xray", false, "Test xray")
TestPipelines = flag.Bool("test.pipelines", false, "Test pipelines")
TestAccess = flag.Bool("test.access", false, "Test access")
TestDistribution = flag.Bool("test.distribution", false, "Test Distribution")
TestXray = flag.Bool("test.xray", false, "Test Xray")
TestPipelines = flag.Bool("test.pipelines", false, "Test Pipelines")
TestAccess = flag.Bool("test.access", false, "Test Access")
TestRepositories = flag.Bool("test.repositories", false, "Test repositories in Artifactory")
RtUrl = flag.String("rt.url", "http://localhost:8081/artifactory", "Artifactory url")
DistUrl = flag.String("ds.url", "", "Distribution url")
Expand Down Expand Up @@ -542,7 +544,11 @@ func GetDistDetails() auth.ServiceDetails {

func GetXrayDetails() auth.ServiceDetails {
xrayDetails := xrayAuth.NewXrayDetails()
xrayDetails.SetUrl(clientutils.AddTrailingSlashIfNeeded(*XrayUrl))
xrayUrl := clientutils.AddTrailingSlashIfNeeded(*XrayUrl)
xrayDetails.SetUrl(xrayUrl)
xrayDetails.SetXscUrl(strings.Replace(xrayUrl, "xray", "xsc", 1))
xrayDetails.SetXscVersion("0.0.0")

setAuthenticationDetail(xrayDetails)
return xrayDetails
}
Expand Down Expand Up @@ -1115,6 +1121,34 @@ func createAccessPingManager() {
testsAccessPingService.ServiceDetails = accessDetails
}

func createSecurityServiceManager() {
xrayDetails := xrayAuth.NewXrayDetails()
xrayUrl := clientutils.AddTrailingSlashIfNeeded(*XrayUrl)
xrayDetails.SetUrl(xrayUrl)
xrayDetails.SetXscUrl(strings.Replace(xrayUrl, "/xray/", "/xsc/", 1))
// XSC version is used to determine if XSC in enabled or not.
xrayDetails.SetXscVersion("0.0.0")

setAuthenticationDetail(xrayDetails)
initSecurityManagerByServerDetails(xrayDetails)
}

// Init securityServiceManager as XSC or Xray manager depends on the server details.
func initSecurityManagerByServerDetails(xrayDetails *xrayAuth.XrayDetails) {
xsc := auth.ServiceDetails(xrayDetails)
serviceConfig, err := config.NewConfigBuilder().
SetServiceDetails(xsc).
SetCertificatesPath(xsc.GetClientCertPath()).
Build()
failOnHttpClientCreation(err)
securityService, err := xrayServices.New(serviceConfig)
if err != nil {
log.Error("failed to initialize securityServiceManager")
os.Exit(1)
}
securityServiceManager = securityService
}

func getUniqueField(prefix string) string {
return strings.Join([]string{prefix, strconv.FormatInt(time.Now().Unix(), 10), runtime.GOOS}, "-")
}
3 changes: 2 additions & 1 deletion tests/xray_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -105,11 +105,12 @@ func initXrayTest(t *testing.T) {
if !*TestXray {
t.Skip("Skipping xray test. To run xray test add the '-test.xray=true' option.")
}
createSecurityServiceManager()
}

type testXrayDetails struct {
auth.ServiceDetails
version string
version string
}

func newTestXrayDetails(serviceDetails auth.ServiceDetails) testXrayDetails {
Expand Down
26 changes: 26 additions & 0 deletions tests/xrayscan_test.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
package tests

import (
"github.com/jfrog/jfrog-client-go/auth"
"github.com/jfrog/jfrog-client-go/config"
services2 "github.com/jfrog/jfrog-client-go/xray/services"
"github.com/stretchr/testify/assert"
"strconv"
"strings"
"testing"
Expand Down Expand Up @@ -46,6 +50,28 @@ func TestNewXrayScanService(t *testing.T) {
}
}

func TestXrayScanGraph(t *testing.T) {
initXrayTest(t)
mockScanId := "9c9dbd61-f544-4e33-4613-34727043d71f"
xrayServerPort := xray.StartXrayMockServer()
xrayDetails := newTestXrayDetails(GetXrayDetails())
xrayDetails.SetUrl("http://localhost:" + strconv.Itoa(xrayServerPort) + "/xray/")

cfp := auth.ServiceDetails(xrayDetails)
serviceConfig, err := config.NewConfigBuilder().
SetServiceDetails(cfp).
Build()
assert.NoError(t, err)
securityServiceManager, err = services2.New(serviceConfig)
assert.NoError(t, err)

scanId, err := securityServiceManager.ScanGraph(&services2.XrayGraphScanParams{})
assert.NoError(t, err)
assert.Equal(t, mockScanId, scanId)
_, err = securityServiceManager.GetScanGraphResults(scanId, false, false)
assert.NoError(t, err)
}

func scanBuild(t *testing.T, buildName, buildNumber, expected string) {
params := services.NewXrayScanParams()
params.BuildName = buildName
Expand Down
Loading
Loading