[Reachability] Scan summary and better docs (#1379)

Changelog.md

@@ -1,5 +1,8 @@
 # FOSSA CLI Changelog
 
+## v3.9.4
+- Reachability: Includes reachability analysis in scan summary [#1379](https://github.com/fossas/fossa-cli/pull/1379)
+
 ## v3.9.3
 - Update error structure [#1364](https://github.com/fossas/fossa-cli/pull/1364)
 

docs/README.md

@@ -59,6 +59,7 @@ Concept guides explain the nuances behind how basic FOSSA primitives work. If yo
   - [Analyzing Specific Submodules](./walkthroughs/analysis-target-configuration.md#target-filtering-for-submodules)
 - [Dynamic Strategy Command Selection](./features/strategy-command-selection.md)
 - [`fossa analyze` Reference](./references/subcommands/analyze.md)
+- [Vulnerable Reachability](./features/vuln_reachability.md)
 
 #### Manually Specifying Dependencies
 

docs/contributing/reachability.md → docs/features/vuln_reachability.md

@@ -1,10 +1,12 @@
 # Reachability
 
 ### What is Reachability?
+
 Reachability Analysis is a security offering designed to enhance FOSSA's security analysis by providing context on vulnerable packages. It alleviates the constraints of traditional CVE assessments through the static analysis of application and dependency code, confirming the presence of vulnerable call paths. 
 
 ### Limitations
-- Reachability currently supports all Maven and Gradle projects dynamically analyzed by fossa-cli. 
+
+- Reachability currently supports all Maven and Gradle projects dynamically analyzed by FOSSA CLI. 
 - The target jar of the project must exist, prior to the analysis. If the jar artifact is not present, or FOSSA CLI fails to
 associate this jar with project, FOSSA CLI will not perform reachability analysis.
 - Reachability requires that `java` is present in PATH, and `java` version must be greater than `1.8` (jdk8+).
@@ -73,27 +75,38 @@ cat fossa.debug.json | jq '.bundleReachabilityEndpoint'
 }
 ```
 
+FOSSA CLI uses [jar-callgraph-1.0.0.jar](../../scripts/jar-callgraph-1.0.0.jar) to infer call path edges. 
+FOSSA CLI uses `java -jar jar-callgraph-1.0.0.jar ./path/to/your/build.jar` command to record edges from
+the your target jar. If you are running into issues with reachability, please confirm that you can execute
+`java -jar jar-callgraph-1.0.0.jar ./path/to/your/build.jar` on your environment.
+
 <!-- 
 ## How do I debug reachability from endpoint?
 
 ```bash
 # get what we sent to endpoint
 cat fossa.debug.json | jq '.bundleReachabilityEndpoint' > rawReachabilityJob.json
 
-# run job in explain mode
-yarn repl
-explainReachability('rawReachabilityJob.json')
-
-# [1] I was provided 'rawReachabilityJob.json'
-# [2] I'm parsing file: 'rawReachabilityJob.json'
-# [3] I found [X] reachability units
-# [4] Working on [0] reachability unit
-# -- 
-# {
-#   ....  
-# }
+# run job in dry mode
+>> yarn repl
+>> performReachabilityInDryMode('rawReachabilityJob.json', 'orgId', 'userRevisionId')
+#
+# [Info] ....
+# [Info] ....
+
+# This will upsert 'rawReachabilityJob.json' to S3, and perform
+# analysis without persisting anything to database. This command is ('orgId', 'userRevisionId')
+# agnostic, meaning that you can run 'rawReachabilityJob.json' for any permutation of ('orgId', 'userRevisionId').
+
+# For example, for any customer's 'rawReachabilityJob.json' (retrieved via debug bundle), you
+# can `performReachabilityInDryMode(...)` in your local environment, for your orgId, and userRevisionId.
 #
+# Since this does not persist any data in cache, nor in exports table - it has no consequences to
+# orgId, and userRevisionId.
+
 ```
+Likewise, you can also inspect analysis done in datadog, by looking at logs associated with the build id. FOSSA
+performs reachability analysis as part of provided build (all variants of provided builds).
 -->
 
 ## F.A.Q. 
@@ -132,3 +145,33 @@ You can inspect the data by running:
 # to endpoint for reachability analysis.
 ; cat fossa.debug.json | jq '.bundleReachabilityRaw'
 ```
+
+2. How do I know if reachability is supported by my organization?
+
+FOSSA requires that `reachability` is enabled for your organization. If `reachability` is not enabled,
+for your organization, you will see following message in output when `fossa analyze` is performed without
+`--output or -o` mode.
+
+```text
+Organization: (your orgId) does not support reachability! skipping reachability analysis upload!
+```
+
+To enable, `reachability` please contact your FOSSA account manager, or [FOSSA support](https://support.fossa.com).
+
+
+3. How do I know if my project was analyzed for reachability by FOSSA CLI?
+
+FOSSA CLI will include scan summary for reachability analysis, as part of `fossa analyze` output.
+
+```text
+Reachability analysis
+  ** maven project in "/Users/dev/code/example-projects/reachability/maven/vuln-function-used/": succeeded
+```
+
+| Summary                          | Meaning                                                                      |
+|----------------------------------|------------------------------------------------------------------------------|
+| succeeded                        | Reachability analysis was successful                                         |
+| skipped (partial graph)          | Project has partial dependency graph (e.g. missing transitive dependencies)  |
+| skipped (not supported)          | Project is not supported for reachability analysis                           |
+| skipped (no dependency analysis) | Project's dependencies were not analyzed, so reachability cannot be computed |
+

integration-test/Reachability/UploadSpec.hs

@@ -6,9 +6,9 @@ module Reachability.UploadSpec (spec) where
 import Analysis.FixtureUtils (FixtureEnvironment (..), TestC, testRunnerWithLogger, withResult)
 import App.Fossa.Analyze.Project (ProjectResult (..))
 import App.Fossa.Analyze.Types (
-  AnalysisScanResult (..),
   DiscoveredProjectIdentifier (..),
   DiscoveredProjectScan (..),
+  SourceUnitReachabilityAttempt (..),
  )
 import App.Fossa.Reachability.Jar (callGraphFromJar)
 import App.Fossa.Reachability.Types (
@@ -20,6 +20,7 @@ import App.Fossa.Reachability.Types (
 import App.Fossa.Reachability.Upload (
   analyzeForReachability,
   callGraphOf,
+  onlyFoundUnits,
  )
 import Data.ByteString.Lazy qualified as LB
 import Data.Foldable (for_)
@@ -83,35 +84,28 @@ spec = describe "Reachability" $ do
     jarFile <- (</> sampleMavenProjectJar) <$> runIO PIO.getCurrentDir
 
     it "should retrieve call graph" $ do
-      let expected = Just (mavenCompleteScanUnit projDir jarFile)
-      resp <- run java8 $ callGraphOf (mavenCompleteScan projDir)
+      let (dpi, dps) = mavenCompleteScan projDir
+      let expected = SourceUnitReachabilityFound dpi (Success [] (mavenCompleteScanUnit projDir jarFile))
+      resp <- run java8 $ callGraphOf dps
       withResult resp $ \_ res -> res `shouldBe` expected
 
   describe "analyzeForReachability" $ do
     projDir <- (</> sampleMavenProjectDir) <$> runIO PIO.getCurrentDir
     jarFile <- (</> sampleMavenProjectJar) <$> runIO PIO.getCurrentDir
 
     it "should return analyzed reachability unit" $ do
+      let (_, dps) = mavenCompleteScan projDir
       let expected = [mavenCompleteScanUnit projDir jarFile]
-      let analysisResult =
-            AnalysisScanResult
-              [mavenCompleteScan projDir]
-              successNothing
-              successNothing
-              successNothing
-              successNothing
-              successNothing
-
-      analyzed <- run java8 $ analyzeForReachability analysisResult
-      withResult analyzed $ \_ analyzed' -> analyzed' `shouldBe` expected
+      analyzed <- run java8 $ analyzeForReachability [dps]
+      withResult analyzed $ \_ analyzed' -> (onlyFoundUnits analyzed') `shouldBe` expected
 
 sampleMavenProjectDir :: Path Rel Dir
 sampleMavenProjectDir = $(mkRelDir "test/Reachability/testdata/maven-default/")
 
 sampleMavenProjectJar :: Path Rel File
 sampleMavenProjectJar = $(mkRelFile "test/Reachability/testdata/maven-default/target/project-1.0.0.jar")
 
-mavenCompleteScan :: Path Abs Dir -> DiscoveredProjectScan
+mavenCompleteScan :: Path Abs Dir -> (DiscoveredProjectIdentifier, DiscoveredProjectScan)
 mavenCompleteScan dir = mkDiscoveredProjectScan MavenProjectType dir Complete
 
 mavenCompleteScanUnit :: Path Abs Dir -> Path Abs File -> SourceUnitReachability
@@ -123,23 +117,25 @@ mavenCompleteScanUnit projDir jarFile =
         (ContentRaw sampleJarParsedContent')
     ]
 
-mkDiscoveredProjectScan :: DiscoveredProjectType -> Path Abs Dir -> GraphBreadth -> DiscoveredProjectScan
+mkDiscoveredProjectScan :: DiscoveredProjectType -> Path Abs Dir -> GraphBreadth -> (DiscoveredProjectIdentifier, DiscoveredProjectScan)
 mkDiscoveredProjectScan projectType dir breadth =
-  Scanned
-    (DiscoveredProjectIdentifier dir projectType)
-    ( Success
-        []
-        (ProjectResult projectType dir empty breadth mempty)
-    )
+  ( dpi
+  , Scanned
+      dpi
+      ( Success
+          []
+          (ProjectResult projectType dir empty breadth mempty)
+      )
+  )
+  where
+    dpi :: DiscoveredProjectIdentifier
+    dpi = DiscoveredProjectIdentifier dir projectType
 
 sampleJarFile :: IO (Path Abs File)
 sampleJarFile = do
   cwd <- PIO.getCurrentDir
   pure (cwd </> $(mkRelFile "test/Reachability/testdata/sample.jar"))
 
-successNothing :: Result (Maybe a)
-successNothing = Success [] Nothing
-
 mkReachabilityUnit :: Path Abs Dir -> [ParsedJar] -> SourceUnitReachability
 mkReachabilityUnit dir jars =
   SourceUnitReachability

src/App/Docs.hs

@@ -10,6 +10,7 @@ module App.Docs (
   pathDependencyDocsUrl,
   staticAndDynamicStrategies,
   apiKeyUrl,
+  vulnReachabilityProductDocsUrl,
 ) where
 
 import App.Version (versionOrBranch)
@@ -53,3 +54,6 @@ staticAndDynamicStrategies = guidePathOf versionOrBranch "/docs/references/strat
 
 apiKeyUrl :: Text
 apiKeyUrl = guidePathOf versionOrBranch "/README.md#generating-an-api-key"
+
+vulnReachabilityProductDocsUrl :: Text
+vulnReachabilityProductDocsUrl = "https://docs.fossa.com/docs/reachability"

src/App/Fossa/Analyze.hs

@@ -54,7 +54,7 @@ import App.Fossa.Lernie.Types (LernieResults (..))
 import App.Fossa.ManualDeps (analyzeFossaDepsFile)
 import App.Fossa.PathDependency (enrichPathDependencies, enrichPathDependencies', withPathDependencyNudge)
 import App.Fossa.PreflightChecks (preflightChecks)
-import App.Fossa.Reachability.Upload (analyzeForReachability)
+import App.Fossa.Reachability.Upload (analyzeForReachability, onlyFoundUnits)
 import App.Fossa.Subcommand (SubCommand)
 import App.Fossa.VSI.DynLinked (analyzeDynamicLinkedDeps)
 import App.Fossa.VSI.IAT.AssertRevisionBinaries (assertRevisionBinaries)
@@ -104,7 +104,7 @@ import Data.Maybe (fromMaybe, mapMaybe)
 import Data.String.Conversion (decodeUtf8, toText)
 import Data.Text.Extra (showT)
 import Diag.Diagnostic as DI
-import Diag.Result (Result (..), resultToMaybe)
+import Diag.Result (resultToMaybe)
 import Discovery.Archive qualified as Archive
 import Discovery.Filters (AllFilters, MavenScopeFilters, applyFilters, filterIsVSIOnly, ignoredPaths, isDefaultNonProductionPath)
 import Discovery.Projects (withDiscoveredProjects)
@@ -400,12 +400,10 @@ analyze cfg = Diag.context "fossa-analyze" $ do
     (False, _) -> traverse (withPathDependencyNudge includeAll) filteredProjects
   logDebug $ "Filtered projects with path dependencies: " <> pretty (show filteredProjects')
 
-  let analysisResult = AnalysisScanResult projectScans vsiResults binarySearchResults manualSrcUnits dynamicLinkedResults maybeLernieResults
-  reachabilityUnitsResult <- Diag.errorBoundaryIO . diagToDebug $ analyzeForReachability analysisResult
-  reachabilityUnits <- case reachabilityUnitsResult of
-    Diag.Result.Failure _ _ -> pure []
-    Diag.Result.Success _ units -> pure units
+  reachabilityUnitsResult <- analyzeForReachability projectScans
+  let reachabilityUnits = onlyFoundUnits reachabilityUnitsResult
 
+  let analysisResult = AnalysisScanResult projectScans vsiResults binarySearchResults manualSrcUnits dynamicLinkedResults maybeLernieResults reachabilityUnitsResult
   renderScanSummary (severity cfg) maybeEndpointAppVersion analysisResult cfg
 
   -- Need to check if vendored is empty as well, even if its a boolean that vendoredDeps exist

src/App/Fossa/Analyze/Debug.hs

@@ -25,7 +25,7 @@ module App.Fossa.Analyze.Debug (
 where
 
 import App.Fossa.EmbeddedBinary (themisVersion)
-import App.Fossa.Reachability.Upload (reachabilityEndpointJson, reachabilityRawJson)
+import App.Fossa.Reachability.Types (reachabilityEndpointJson, reachabilityRawJson)
 import App.Version (fullVersionDescription)
 import Control.Applicative (asum)
 import Control.Carrier.Debug (

src/App/Fossa/Analyze/ScanSummary.hs

@@ -15,10 +15,12 @@ import App.Fossa.Analyze.Types (
   AnalysisScanResult (AnalysisScanResult),
   DiscoveredProjectIdentifier (dpiProjectPath, dpiProjectType),
   DiscoveredProjectScan (..),
+  SourceUnitReachabilityAttempt (..),
  )
 import App.Fossa.Config.Analyze (AnalysisTacticTypes (StaticOnly))
 import App.Fossa.Config.Analyze qualified as Config
 import App.Fossa.Lernie.Types (LernieMatch (..), LernieMatchData (..), LernieResults (..))
+import App.Fossa.Reachability.Types (SourceUnitReachability (..))
 import App.Version (fullVersionDescription)
 import Control.Carrier.Lift
 import Control.Effect.Diagnostics qualified as Diag (Diagnostics)
@@ -171,7 +173,7 @@ renderScanSummary severity maybeEndpointVersion analysisResults cfg = do
       logInfo "------------"
 
 summarize :: Config.AnalyzeConfig -> Text -> AnalysisScanResult -> Maybe ([Doc AnsiStyle])
-summarize cfg endpointVersion (AnalysisScanResult dps vsi binary manualDeps dynamicLinkingDeps lernie) =
+summarize cfg endpointVersion (AnalysisScanResult dps vsi binary manualDeps dynamicLinkingDeps lernie reachabilityAttempts) =
   if (numProjects totalScanCount <= 0)
     then Nothing
     else
@@ -195,6 +197,7 @@ summarize cfg endpointVersion (AnalysisScanResult dps vsi binary manualDeps dyna
           <> summarizeSrcUnit "fossa-deps file analysis" (Just getManualVendorDepsIdentifier) manualDeps
           <> summarizeSrcUnit "Keyword Search" (Just getLernieIdentifier) (lernieResultsKeywordSearches <$$> lernie)
           <> summarizeSrcUnit "Custom-License Search" (Just getLernieIdentifier) (lernieResultsCustomLicenses <$$> lernie)
+          <> summarizeReachability "Reachability analysis" reachabilityAttempts
           <> [""]
   where
     vsiResults = summarizeSrcUnit "vsi analysis" (Just (join . map vsiSourceUnits)) vsi
@@ -309,11 +312,31 @@ summarizeProjectScan (Scanned _ (Success wg pr)) = successColorCoded wg $ render
 summarizeProjectScan (SkippedDueToProvidedFilter dpi) = renderDiscoveredProjectIdentifier dpi <> skippedDueFilter
 summarizeProjectScan (SkippedDueToDefaultProductionFilter dpi) = renderDiscoveredProjectIdentifier dpi <> skippedDueNonProductionPathFiltering
 
+summarizeReachability ::
+  Doc AnsiStyle ->
+  [SourceUnitReachabilityAttempt] ->
+  [Doc AnsiStyle]
+summarizeReachability analysisHeader unitAttempts =
+  [analysisHeader] <> itemize ("  *" <> listSymbol) summarizeReachabilityAttempt unitAttempts
+
+summarizeReachabilityAttempt :: SourceUnitReachabilityAttempt -> Doc AnsiStyle
+summarizeReachabilityAttempt (SourceUnitReachabilityFound _ (Success wg unit)) = successColorCoded wg $ renderReachabilityResult unit <> renderSucceeded wg
+summarizeReachabilityAttempt (SourceUnitReachabilityFound dpi (Failure _ _)) = failColorCoded $ renderDiscoveredProjectIdentifier dpi <> renderFailed
+summarizeReachabilityAttempt (SourceUnitReachabilitySkippedPartialGraph dpi) = renderDiscoveredProjectIdentifier dpi <> skippedReachabilityDueToPartialGraph
+summarizeReachabilityAttempt (SourceUnitReachabilitySkippedNotSupported dpi) = renderDiscoveredProjectIdentifier dpi <> skippedReachabilityDueToNotSupported
+summarizeReachabilityAttempt (SourceUnitReachabilitySkippedMissingDependencyAnalysis dpi) = renderDiscoveredProjectIdentifier dpi <> skippedReachabilityDueToMissingAnalysis
+
 ---------- Rendering Helpers
 
 logInfoVsep :: (Has Logger sig m) => [Doc AnsiStyle] -> m ()
 logInfoVsep = traverse_ logInfo
 
+renderReachabilityResult :: SourceUnitReachability -> Doc AnsiStyle
+renderReachabilityResult unit = annotate bold projectTypeDoc <> pathDoc
+  where
+    pathDoc = " project in " <> viaShow (srcUnitManifest unit)
+    projectTypeDoc = pretty $ srcUnitType unit
+
 renderDiscoveredProjectIdentifier :: DiscoveredProjectIdentifier -> Doc AnsiStyle
 renderDiscoveredProjectIdentifier dpi = renderProjectPathAndType (dpiProjectType dpi) (dpiProjectPath dpi)
 
@@ -341,6 +364,15 @@ skippedDueFilter = ": skipped (exclusion filters)"
 skippedDueNonProductionPathFiltering :: Doc AnsiStyle
 skippedDueNonProductionPathFiltering = ": skipped (non-production path filtering)"
 
+skippedReachabilityDueToPartialGraph :: Doc AnsiStyle
+skippedReachabilityDueToPartialGraph = ": skipped (partial graph)"
+
+skippedReachabilityDueToNotSupported :: Doc AnsiStyle
+skippedReachabilityDueToNotSupported = ": skipped (not supported)"
+
+skippedReachabilityDueToMissingAnalysis :: Doc AnsiStyle
+skippedReachabilityDueToMissingAnalysis = ": skipped (no dependency analysis)"
+
 renderSucceeded :: [EmittedWarn] -> Doc AnsiStyle
 renderSucceeded ew =
   if countWarnings ew == 0
@@ -369,8 +401,8 @@ countWarnings ws =
     isIgnoredErrGroup IgnoredErrGroup{} = True
     isIgnoredErrGroup _ = False
 
-dumpResultLogsToTempFile :: (Has (Lift IO) sig m) => Config.AnalyzeConfig -> Data.Text.Text -> AnalysisScanResult -> m (Path Abs File)
-dumpResultLogsToTempFile cfg endpointVersion (AnalysisScanResult projects vsi binary manualDeps dynamicLinkingDeps lernieResults) = do
+dumpResultLogsToTempFile :: (Has (Lift IO) sig m) => Config.AnalyzeConfig -> Text -> AnalysisScanResult -> m (Path Abs File)
+dumpResultLogsToTempFile cfg endpointVersion (AnalysisScanResult projects vsi binary manualDeps dynamicLinkingDeps lernieResults reachabilityAttempts) = do
   let doc =
         stripAnsiEscapeCodes
           . renderStrict
@@ -386,13 +418,14 @@ dumpResultLogsToTempFile cfg endpointVersion (AnalysisScanResult projects vsi bi
               , renderSourceUnit "fossa-deps analysis" manualDeps
               , renderSourceUnit "Custom-license scan & Keyword Search" lernieResults
               ]
+            ++ renderReachability
 
   tmpDir <- sendIO getTempDir
   sendIO $ TIO.writeFile (fromAbsFile $ tmpDir </> scanSummaryFileName) doc
   pure (tmpDir </> scanSummaryFileName)
   where
     scanSummary :: [Doc AnsiStyle]
-    scanSummary = maybeToList (vsep <$> summarize cfg endpointVersion (AnalysisScanResult projects vsi binary manualDeps dynamicLinkingDeps lernieResults))
+    scanSummary = maybeToList (vsep <$> summarize cfg endpointVersion (AnalysisScanResult projects vsi binary manualDeps dynamicLinkingDeps lernieResults reachabilityAttempts))
 
     renderSourceUnit :: Doc AnsiStyle -> Result (Maybe a) -> Maybe (Doc AnsiStyle)
     renderSourceUnit header (Failure ws eg) = Just $ renderFailure ws eg $ vsep $ summarizeSrcUnit header Nothing (Failure ws eg)
@@ -404,5 +437,13 @@ dumpResultLogsToTempFile cfg endpointVersion (AnalysisScanResult projects vsi bi
     renderDiscoveredProjectScanResult (Scanned dpi (Success ws res)) = renderSuccess ws $ summarizeProjectScan (Scanned dpi (Success ws res))
     renderDiscoveredProjectScanResult _ = Nothing
 
+    renderReachability :: [Doc AnsiStyle]
+    renderReachability = ["Reachability"] <> (mapMaybe renderReachabilityDetailed reachabilityAttempts)
+
+    renderReachabilityDetailed :: SourceUnitReachabilityAttempt -> Maybe (Doc AnsiStyle)
+    renderReachabilityDetailed (SourceUnitReachabilityFound _ (Success wg unit)) = renderSuccess wg $ renderReachabilityResult unit <> renderSucceeded wg
+    renderReachabilityDetailed (SourceUnitReachabilityFound dpi (Failure ws eg)) = Just $ renderFailure ws eg $ renderDiscoveredProjectIdentifier dpi <> renderFailed
+    renderReachabilityDetailed res = Just $ summarizeReachabilityAttempt res
+
 scanSummaryFileName :: Path Rel File
 scanSummaryFileName = $(mkRelFile "fossa-analyze-scan-summary.txt")