Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(everything): merge main into v2 :salute_1: #2930

Draft
wants to merge 47 commits into
base: v2-dev
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
4b164f8
all: upgrade minimum supported Go release to 1.22 (#2850)
nsrip-dd Sep 9, 2024
4574919
appsec: missing SQLi and LFI RC capabilities (#2851)
eliottness Sep 10, 2024
0ffa615
profiler: record Orchestrion, activation information (#2814)
nsrip-dd Sep 10, 2024
d1b31e8
internal/version: bump to v1.69.0-dev (#2854)
nsrip-dd Sep 11, 2024
e1ca75f
internal/civisibility/integrations/gotesting: fixes for orchestrion a…
tonyredondo Sep 12, 2024
afcff94
contrib/aws/aws-sdk-go-v2/aws: Fix streamName nil pointer panic (#2846)
webdestroya Sep 12, 2024
e081e4a
contrib/cloud.google.com/go/pubsub.v1: split tracing code (#2852)
rarguelloF Sep 12, 2024
07c4b72
internal/civisibility/integrations/gotesting: add the `DD_CIVISIBILIT…
tonyredondo Sep 13, 2024
660a6a2
internal/stacktrace: upgrade msgp and add stacktrace.GetSpanValue (#2…
eliottness Sep 13, 2024
63f207d
internal/civisibility/integrations/gotesting: improve the isCiVisibil…
tonyredondo Sep 13, 2024
d9505b3
Regenerate stats payload to add is_trace_root (#2856)
ajgajg1134 Sep 16, 2024
6d0e1b8
add aws credentials for system tests (#2870)
wconti27 Sep 16, 2024
1f0966d
internal/apps: move setup-smoke-test (#2871)
felixge Sep 17, 2024
3646321
internal/apps: Add two new apps (#2868)
felixge Sep 18, 2024
b94c2c3
appsec: upgrade go-libddwaf v3.3.0 -> v3.4.0 (#2877)
eliottness Sep 18, 2024
054d19d
internal/civisibility/utils/net: a new http client for rapid endpoint…
tonyredondo Sep 19, 2024
8860b39
internal/civisibility: adds the new vcpu_count metric for tslv events…
tonyredondo Sep 20, 2024
ba18110
appsec: Suspicious Attacker Blocking (#2878)
eliottness Sep 23, 2024
7699f9e
internal/appsec: refactor listeners (#2862)
eliottness Sep 24, 2024
101d4da
profiler: avoid metrics profile log noise when stopping profiling (#2…
nsrip-dd Sep 24, 2024
eef52d3
ddtrace/tracer: update log msg to accurately count dropped traces (#2…
hannahkm Sep 24, 2024
ac73f9b
profiler: suppress errors if the profiler is stopped (#2886)
nsrip-dd Sep 25, 2024
bed7121
internal/datastreams: fix Processor goroutine leaks (#2880)
ggambetti Sep 26, 2024
043dcd1
feat(.github/workflows): stricter GitHub token default permission com…
darccio Sep 26, 2024
82f0b8e
internal/civisibility: change the default value for DD_CIVISIBILITY_E…
tonyredondo Sep 27, 2024
b0bb4c7
contrib/99designs/gqlgen/tracer.go: nil check response (#2792)
dienvoandpadcojp Sep 27, 2024
daa6823
Fix typo in README.md (#2898)
fkmy Sep 30, 2024
e5a17f2
appsec: Attacker Fingerprinting (#2899)
eliottness Oct 1, 2024
0f6ba13
contrib/grpc: attempt to fix flaky tests (#2872)
RomainMuller Oct 1, 2024
438d47d
chore: prefix system tests env var (#2902)
wconti27 Oct 2, 2024
ef90025
internal/appsec: fix derivatives serdes on simple types (#2905)
eliottness Oct 2, 2024
3b1e8a9
internal/civisibility: test session logical names (#2904)
tonyredondo Oct 3, 2024
e90c07d
Fix and Feature: standardize config behavior across tracing libs (#2873)
mtoffl01 Oct 3, 2024
d3f686e
internal/civisibility: automatic test retries (#2892)
tonyredondo Oct 3, 2024
d50070a
internal/civisibility: fixes the test parent status when the auto-ret…
tonyredondo Oct 4, 2024
966abf2
internal/civisibility: add support for the test.source.end tag (#2911)
tonyredondo Oct 4, 2024
2a34b15
internal/civisibility: add and improve debug logs (#2912)
tonyredondo Oct 4, 2024
7704733
Implement DD_TRACE_LOG_DIRECTORY (#2901)
mtoffl01 Oct 8, 2024
fc58656
internal/telemetry: skip TestProductChange/profiler_start,_tracer_sta…
nsrip-dd Oct 10, 2024
2f2854f
internal/civisibility: add extra tags to github action pull_requests …
tonyredondo Oct 10, 2024
5b9a8af
internal/civisibility: add early flake detection feature (#2916)
tonyredondo Oct 10, 2024
93311db
internal/version: bump to v1.70.0 (#2919)
eliottness Oct 14, 2024
4f34a1c
initial passthrough
hannahkm Oct 15, 2024
2eeaab3
fix remaining v1 import links
hannahkm Oct 15, 2024
1de5adb
fix dyngo imports
hannahkm Oct 15, 2024
ea693ee
fix(all) v1 import urls and go mod
hannahkm Oct 15, 2024
fdd983b
go mod tidy grpc
hannahkm Oct 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/appsec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,9 @@ concurrency:
# Automatically cancel previous runs if a new one is triggered to conserve resources.
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.ref }}

permissions:
contents: read

jobs:
# Prepare the cache of Go modules to share it will the other jobs.
# This maximizes cache hits and minimizes the time spent downloading Go modules.
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/datadog-static-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@ on: [push]

name: Datadog Static Analysis

permissions:
contents: read
pull-requests: write

jobs:
static-analysis:
runs-on: ubuntu-latest
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ on:
- reopened
- opened
- edited
permissions:
contents: read
issues: write
jobs:
label_issues:
if: contains(github.event.issue.title, 'contrib')
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/ecosystems-label-pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ on:
- opened
- reopened
- edited
permissions:
contents: read
pull-requests: write
jobs:
label_issues:
runs-on: ubuntu-latest
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/govulncheck.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ on:
- cron: '00 00 * * *'
workflow_dispatch:

permissions:
contents: read

jobs:
govulncheck-tests:
runs-on: ubuntu-latest
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/multios-unit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,9 @@ on:
env:
DD_APPSEC_WAF_TIMEOUT: 1m # Increase time WAF time budget to reduce CI flakiness

permissions:
contents: read

jobs:
test-multi-os:
runs-on: "${{ inputs.runs-on }}"
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/parametric-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ on:
schedule:
- cron: '00 04 * * 2-6'

permissions:
contents: read

jobs:
parametric-tests:
if: github.event_name != 'pull_request' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == 'DataDog/dd-trace-go')
Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/smoke-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ on:
env:
TEST_RESULTS: /tmp/test-results # path to where test results will be saved

permissions:
contents: read

jobs:
go-get-u:
# Run go get -u to upgrade dd-trace-go dependencies to their
Expand Down Expand Up @@ -90,7 +93,7 @@ jobs:
ref: ${{ inputs.ref || github.ref }}
- uses: actions/setup-go@v3
with:
go-version: "1.21"
go-version: "1.22"
cache: true
- name: go mod tidy
run: |-
Expand Down Expand Up @@ -185,7 +188,7 @@ jobs:
uses: docker/build-push-action@v5
with:
context: .
file: ./internal/apps/setup-smoke-test/Dockerfile
file: ./internal/setup-smoke-test/Dockerfile
push: false
load: true
tags: smoke-test
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/stale.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ on:
schedule:
- cron: '30 1 * * *'

permissions:
contents: read
issues: write

jobs:
stale:
runs-on: ubuntu-latest
Expand Down
7 changes: 7 additions & 0 deletions .github/workflows/system-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ on:
schedule:
- cron: '00 04 * * 2-6'

permissions:
contents: read

jobs:
system-tests:
if: github.event_name != 'pull_request' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == 'DataDog/dd-trace-go')
Expand All @@ -43,6 +46,8 @@ jobs:
- uds-echo
scenario:
- DEFAULT
- INTEGRATIONS
- CROSSED_TRACING_LIBRARIES
- APPSEC_DISABLED
- APPSEC_BLOCKING
- APPSEC_BLOCKING_FULL_DENYLIST
Expand Down Expand Up @@ -103,6 +108,8 @@ jobs:
DD_API_KEY: ${{ secrets.DD_API_KEY }}
SYSTEM_TESTS_E2E_DD_API_KEY: ${{ secrets.SYSTEM_TESTS_E2E_DD_API_KEY }}
SYSTEM_TESTS_E2E_DD_APP_KEY: ${{ secrets.SYSTEM_TESTS_E2E_DD_APP_KEY }}
SYSTEM_TESTS_AWS_ACCESS_KEY_ID: ${{ secrets.SYSTEM_TESTS_IDM_AWS_ACCESS_KEY_ID }}
SYSTEM_TESTS_AWS_SECRET_ACCESS_KEY: ${{ secrets.SYSTEM_TESTS_IDM_AWS_SECRET_ACCESS_KEY }}
name: Test (${{ matrix.weblog-variant }}, ${{ matrix.scenario }})
steps:
- name: Checkout system tests
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/test-apps.cue
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,10 @@ env: {
DD_TAGS: "github_run_id:${{ github.run_id }} github_run_number:${{ github.run_number }} ${{ inputs['arg: tags'] }}",
}

permissions: {
contents: "read",
}

jobs: {
for i, scenario in #scenarios {
for j, env in #envs {
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/test-apps.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ name: Test Apps
env:
DD_ENV: github
DD_TAGS: 'github_run_id:${{ github.run_id }} github_run_number:${{ github.run_number }} ${{ inputs[''arg: tags''] }}'
permissions:
contents: read
jobs:
job-0-0:
name: unit-of-work/v1 (prod)
Expand Down
9 changes: 8 additions & 1 deletion .github/workflows/unit-integration-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ env:
# without having to download a newer one.
GOTOOLCHAIN: local

permissions:
contents: read

jobs:
copyright:
runs-on: ubuntu-latest
Expand All @@ -28,10 +31,14 @@ jobs:
uses: actions/checkout@v3
with:
ref: ${{ inputs.ref || github.ref }}
- name: Setup Go
- name: Setup Go
uses: ./.github/actions/setup-go
with:
go-version: ${{ inputs.go-version }}
- name: Setup go
uses: actions/setup-go@v5
with:
go-version: stable
- name: Copyright
run: |
go run checkcopyright.go
Expand Down
51 changes: 30 additions & 21 deletions appsec/appsec.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ var appsecDisabledLog sync.Once
// Note that passing the raw bytes of the HTTP request body is not expected and would
// result in inaccurate attack detection.
// This function always returns nil when appsec is disabled.
func MonitorParsedHTTPBody(ctx context.Context, body interface{}) error {
func MonitorParsedHTTPBody(ctx context.Context, body any) error {
if !appsec.Enabled() {
appsecDisabledLog.Do(func() { log.Warn("appsec: not enabled. Body blocking checks won't be performed.") })
return nil
Expand All @@ -60,7 +60,15 @@ func SetUser(ctx context.Context, id string, opts ...tracer.UserMonitoringOption
appsecDisabledLog.Do(func() { log.Warn("appsec: not enabled. User blocking checks won't be performed.") })
return nil
}
return sharedsec.MonitorUser(ctx, id)

op, errPtr := usersec.StartUserLoginOperation(ctx, usersec.UserLoginOperationArgs{})
op.Finish(usersec.UserLoginOperationRes{
UserID: id,
SessionID: getSessionID(opts...),
Success: true,
})

return *errPtr
}

// TrackUserLoginSuccessEvent sets a successful user login event, with the given
Expand All @@ -76,17 +84,7 @@ func SetUser(ctx context.Context, id string, opts ...tracer.UserMonitoringOption
// Take-Over (ATO) monitoring, ultimately blocking the IP address and/or user id
// associated to them.
func TrackUserLoginSuccessEvent(ctx context.Context, uid string, md map[string]string, opts ...tracer.UserMonitoringOption) error {
span := getRootSpan(ctx)
if span == nil {
return nil
}

const tagPrefix = "appsec.events.users.login.success."
span.SetTag(tagPrefix+"track", true)
for k, v := range md {
span.SetTag(tagPrefix+k, v)
}
span.SetTag(ext.ManualKeep, true)
TrackCustomEvent(ctx, "users.login.success", md)
return SetUser(ctx, uid, opts...)
}

Expand All @@ -106,14 +104,15 @@ func TrackUserLoginFailureEvent(ctx context.Context, uid string, exists bool, md
return
}

const tagPrefix = "appsec.events.users.login.failure."
span.SetTag(tagPrefix+"track", true)
span.SetTag(tagPrefix+"usr.id", uid)
span.SetTag(tagPrefix+"usr.exists", exists)
for k, v := range md {
span.SetTag(tagPrefix+k, v)
}
span.SetTag(ext.ManualKeep, true)
// We need to do the first call to SetTag ourselves because the map taken by TrackCustomEvent is map[string]string
// and not map [string]any, so the `exists` boolean variable does not fit int
span.SetTag("appsec.events.users.login.failure.usr.exists", exists)
span.SetTag("appsec.events.users.login.failure.usr.id", uid)

TrackCustomEvent(ctx, "users.login.failure", md)

op, _ := usersec.StartUserLoginOperation(ctx, usersec.UserLoginOperationArgs{})
op.Finish(usersec.UserLoginOperationRes{UserID: uid, Success: false})
}

// TrackCustomEvent sets a custom event as service entry span tags. This span is
Expand Down Expand Up @@ -145,3 +144,13 @@ func getRootSpan(ctx context.Context) *tracer.Span {
}
return span.Root()
}

func getSessionID(opts ...tracer.UserMonitoringOption) string {
cfg := &tracer.UserMonitoringConfig{
Metadata: make(map[string]string),
}
for _, opt := range opts {
opt(cfg)
}
return cfg.SessionID
}
Loading
Loading