Add support for Calculating Risk in VulnerableCode

[ziadhany]

issue: #1543

[DennisClark]

it would be really great to see a review of this PR soon!

[TG1999]

@ziadhany this generally looks good to me, can we have some logs for the pipeline and some UI and API screenshots so we can merge it.

[keshav-space]

Thanks @ziadhany, some nits for your consideration.
Also, please use LoopProgress while iterating over large records, as we do here

vulnerablecode/vulnerabilities/pipelines/enhance_with_exploitdb.py

Line 61 in 590c91a

progress = LoopProgress(total_iterations=fetched_exploit_count, logger=self.log)

[ziadhany]

@TG1999 @keshav-space
Thank you for reviewing this PR. Here are the pipeline logs:

/home/ziad-hany/PycharmProjects/vulnerablecode/venv/bin/python /home/ziad-hany/PycharmProjects/vulnerablecode/manage.py improve localhost:8000 --all 
Improving data using compute_package_risk
INFO 2024-10-28 13:10:07.074 Pipeline [ComputePackageRiskPipeline] starting
INFO 2024-10-28 13:10:07.074 Step [add_risk_package] starting
INFO 2024-10-28 13:10:07.755 Calculating risk for 84,396 affected package records
INFO 2024-10-28 13:14:57.360 Progress: 10% (8440/84396) ETA: 2601 seconds (43.4 minutes)
INFO 2024-10-28 13:17:30.519 Progress: 20% (16880/84396) ETA: 1768 seconds (29.5 minutes)
INFO 2024-10-28 13:19:28.477 Progress: 30% (25319/84396) ETA: 1307 seconds (21.8 minutes)
INFO 2024-10-28 13:22:06.047 Progress: 40% (33759/84396) ETA: 1076 seconds (17.9 minutes)
INFO 2024-10-28 13:25:15.062 Progress: 50% (42198/84396) ETA: 907 seconds (15.1 minutes)
INFO 2024-10-28 13:30:57.706 Progress: 60% (50638/84396) ETA: 833 seconds (13.9 minutes)
INFO 2024-10-28 13:36:04.109 Progress: 70% (59078/84396) ETA: 667 seconds (11.1 minutes)
INFO 2024-10-28 13:39:06.350 Progress: 80% (67517/84396) ETA: 434 seconds (7.2 minutes)
INFO 2024-10-28 13:42:57.214 Progress: 90% (75957/84396) ETA: 219 seconds (3.6 minutes)
INFO 2024-10-28 13:48:50.555 Progress: 100% (84396/84396)
INFO 2024-10-28 13:48:50.694 Successfully added risk score for 84,396 package
INFO 2024-10-28 13:48:50.721 Step [add_risk_package] completed in 2324 seconds (38.7 minutes)
INFO 2024-10-28 13:48:50.721 Pipeline completed in 2324 seconds (38.7 minutes)

Process finished with exit code 0

[tdruez]

@ziadhany @TG1999 Could we make a push on this one as this is a key addition for the VEX support in DejaCode.
Also, make sure it is available everywhere in the relevant API endpoint as we need to capture this data.

[keshav-space]

Thanks @ziadhany, see some suggestion.
Also, please rename the pipeline file from risk_package.py to compute_package_risk.py and the corresponding test file to test_compute_package_risk.py.

[ziadhany]

@tdruez @keshav-space, I've completed the requested changes and I hope we can move forward and merge this PR.

Additionally, please review the weight_config.json file to ensure an accurate view of the risk. The default weight is set to one and may need adjustment to reflect typical risk values if we plan to use it in Dejacode.