Update promptfoo tests

backend/promptfoo/report_agent_config.yaml

@@ -1,7 +1,7 @@
 description: "Test Report Agent Prompts"
 
 providers:
-  - id: mistral:mistral-large-latest
+  - id: openai:gpt-4o-mini
     config:
       temperature: 0
 
@@ -12,102 +12,7 @@ tests:
     vars:
       user_prompt_template: "create-report-user-prompt"
       system_prompt_template: "create-report-system-prompt"
-      user_prompt_args:
-        document_text: "Published September 2024  Carbon Reduction Plan 
-Supplier name: Amazon Web Services EU SARL (UK Branch)  (“AWS UK”)
-Publication date: September 30, 2024
-Commitment to Achieving Net Zero
-AWS UK, as part of Amazon.com, Inc. (“Amazon”), is committed to achieving net -zero
-emissions by 2040. In 2019, Amazon co -founded The Climate Pledge, a public commitment
-to innovate, use our scale for good and go faster to address the urgency of the climate crisis
-to reach net -zero carbon across the entire organization by 2040. Since committing to the
-Pledge, we’ve changed how we conduct our business and the running of our operations, and
-we’ve increased funding and implementation of new technologies and services that
-decarbonize and help preserve the natural world, alon gside the ambitious goals outlined in
-The Climate Pledge. We’re fully committed to our goals and our work to build a better planet.
-Baseline Emissions Footprint
-Base Year emissions are a record of the greenhouse gases that have been produced in the
-past an d are the reference point against which emissions reduction can be measured.
-Baseline Year: 2020
-Additional Details relating to the Baseline Emissions calculations:
-AWS UK utilized January 1, 2020 to December 31, 2020 as the baseline year for emissions
-reporting under this Carbon Reduction Plan. Our plan includes emissions data from relevant
-affiliate companies helping to provide AWS UK’s services to our customers. We ’ve included both
-location -based and market -based method Scope 2 emissions in the following tables. AWS UK
-benefits from contractual arrangements entered into by our affiliate(s) for renewable electricity
-and/or renewable attributes that are reflected in t he market -based data set. More information
-about our corporate carbon footprint and methodology can be found on our website .
-Our baseline year does not include Scope 1 emissions. In 2022 we updated our methodology
-and Scope 1 emissions are now included in total emissions for AWS UK
-
- Published September 2024  Baseline year emissions:
-EMISSIONS  TOTAL (tCO 2e)
-Scope 1  0
-Scope 2  61,346  – Location -based method
-2,813  – Market -based method
-Scope 3 (Included
-Sources)  3,770
-Total Emissions  65,116  – Location -based method
-6,583  – Market -based method
-Current Emissions Reporting
-Reporting Year: 202 3 (January 1, 202 3 to December 31, 202 3)
-EMISSIONS  TOTAL (tCO 2e)
-Scope 1  2,23 3
-Scope 2  126,755  – Location -based method
-0 – Market -based method
-Scope 3 (Included
-Sources)  13,188
-Total Emissions  142,17 6 – Location -based method
-15,42 1 – Market -based method
-
- Published September 2024  Emissions Reduction Targets
-In 2019, we set an ambitious goal to match 100% of the electricity we use with renewable
-energy by 2030. This goal includes all data centres , logistics facilities, physical stores, and
-corporate offices, as well as on -site charg ing points and our financially integrated subsidiaries.
-We are proud to have achieved this goal in 2023, seven years early, with 100% of the electricity
-consum ed by Amazon matched with renewable energy sources.
-Amazon continue s to be transparent and share our progress  to reach net -zero carbon  in our
-annual  Sustainability Report , which also includes details on how we  measure carbon .
-Carbon Reduction Projects
-Completed Carbon Reduction Initiatives
-Amazon continues to take actions across our operations to drive carbon reduction around  the
-world, including in the UK. As of January 202 4, Amazon’s renewable energy portfolio includes
-243 wind and solar farms and 2 70 rooftop solar projects, totalling 513 projects  and 28
-gigawatts of renewable energy capacity.  This includes several utility -scale renewable energy
-projects located within the UK:
-•In 2019, Amazon  announced our first power purchase agreement in the UK, located in
-Kintyre Peninsula, Scotland. The “Amazon Wind Farm Scotland – Beinn an Tuirc 3”
-began o perating in 2021, providing 50 megawatts  (MW)  of new renewable capacity to
-the electricity grid with expected generation of 168,000 megawatt hours (MWh) of
-clean energy annually. That’s enough to power 46,000 UK homes every year.
-•In December 2020,  Amazon a nnounced a two -phase renewable energy project located
-in South Lanarkshire, Scotland, the Kennoxhead wind farm. Kennoxhead will be the
-largest single -site onshore wind project in the UK, enabled through corporate
-procurement.  Once fully operational, Kenno xhead will produce 129 MW of renewable
-capacity and is expected to generate 439,000 MWh of clean energy annually. Phase 1
-(60 MW) began operating in 2022, and Phase 2 (69  MW) will begin operations in 2024 .
-•In 2022, Amazon announced its first project in Nor thern Ireland, a 16  MW onshore
-windfarm in Co Antrim.
-•In 2022, Amazon  also announced  a new 473 MW offshore wind farm, Moray West,
-located off the coast of Scotland . Amazon expects completion of Moray West in 2024.
-This is Amazon’s largest project in Scotland and the largest corporate renewable
-energy deal announced by any company in the UK to date.
-•In 2023, Amazon announced a new 47  MW solar farm, Warl ey located in Essex.
-This project is expected to be operational in 2024.
-
- Published September 2024  Declaration and Sign Off
-This Carbon Reduction Plan has been completed in accordance with PPN 06/21 and
-associated guidance and reporting standard for Carbon Reduction Plans.
-Emiss ions have been reported and recorded in accordance with  the published reporting
-standard for Carbon Reduction Plans and the GHG Reporting Protocol corporate standard1
-and uses the appropri ate Government emission conversion factors for greenhouse gas
-company reporting2.
-Scope 1 and Scope 2 emissions have been reported in accordance with S ECR requirements,
-and the required subset of Scope 3 emissions have been reported in accordance with the
-published reporting standard for Carbon Reduction Plans and the Corporate Value Chain
-(Scope 3) Standard3.
-This Carbon Reduction Plan has been reviewed and signed off by the board of directors (or
-equivalent management body)."
+      file_attachment: "../library/AstraZeneca-Sustainability-Report-2023.pdf"
     assert:
       - type: contains-all
         value:
@@ -122,106 +27,11 @@ equivalent management body)."
     vars:
       user_prompt_template: "find-company-name-from-file-user-prompt"
       system_prompt_template: "find-company-name-from-file-system-prompt"
-      user_prompt_args:
-        file_content: "Published September 2024  Carbon Reduction Plan 
-Supplier name: Amazon Web Services EU SARL (UK Branch)  (“AWS UK”)  
-Publication date: September 30, 2024 
-Commitment to Achieving Net Zero  
-AWS UK, as part of Amazon.com, Inc. (“Amazon”), is committed to achieving net -zero 
-emissions by 2040. In 2019, Amazon co -founded The Climate Pledge, a public commitment 
-to innovate, use our scale for good and go faster to address the urgency of the climate crisis 
-to reach net -zero carbon across the entire organization by 2040. Since committing to the 
-Pledge, we’ve changed how we conduct our business and the running of our operations, and 
-we’ve increased funding and implementation of new technologies and services that 
-decarbonize and help preserve the natural world, alon gside the ambitious goals outlined in 
-The Climate Pledge. We’re fully committed to our goals and our work to build a better planet.  
-Baseline Emissions Footprint  
-Base Year emissions are a record of the greenhouse gases that have been produced in the 
-past an d are the reference point against which emissions reduction can be measured.  
-Baseline Year: 2020  
-Additional Details relating to the Baseline Emissions calculations: 
-AWS UK utilized January 1, 2020 to December 31, 2020 as the baseline year for emissions 
-reporting under this Carbon Reduction Plan. Our plan includes emissions data from relevant 
-affiliate companies helping to provide AWS UK’s services to our customers. We ’ve included both 
-location -based and market -based method Scope 2 emissions in the following tables. AWS UK 
-benefits from contractual arrangements entered into by our affiliate(s) for renewable electricity 
-and/or renewable attributes that are reflected in t he market -based data set. More information 
-about our corporate carbon footprint and methodology can be found on our website . 
-Our baseline year does not include Scope 1 emissions. In 2022 we updated our methodology 
-and Scope 1 emissions are now included in total emissions for AWS UK
-
- Published September 2024  Baseline year emissions:  
-EMISSIONS  TOTAL (tCO 2e) 
-Scope 1  0 
-Scope 2  61,346  – Location -based method  
-2,813  – Market -based method  
-Scope 3 (Included 
-Sources)  3,770  
-Total Emissions  65,116  – Location -based method  
-6,583  – Market -based method  
-Current Emissions Reporting
-Reporting Year: 202 3 (January 1, 202 3 to December 31, 202 3) 
-EMISSIONS  TOTAL (tCO 2e) 
-Scope 1  2,23 3 
-Scope 2  126,755  – Location -based method  
-0 – Market -based method  
-Scope 3 (Included 
-Sources)  13,188  
-Total Emissions  142,17 6 – Location -based method  
-15,42 1 – Market -based method  
-
- Published September 2024  Emissions Reduction Targets
-In 2019, we set an ambitious goal to match 100% of the electricity we use with renewable 
-energy by 2030. This goal includes all data centres , logistics facilities, physical stores, and 
-corporate offices, as well as on -site charg ing points and our financially integrated subsidiaries. 
-We are proud to have achieved this goal in 2023, seven years early, with 100% of the electricity 
-consum ed by Amazon matched with renewable energy sources.  
-Amazon continue s to be transparent and share our progress  to reach net -zero carbon  in our 
-annual  Sustainability Report , which also includes details on how we  measure carbon . 
-Carbon Reduction Projects  
-Completed Carbon Reduction Initiatives  
-Amazon continues to take actions across our operations to drive carbon reduction around  the 
-world, including in the UK. As of January 202 4, Amazon’s renewable energy portfolio includes 
-243 wind and solar farms and 2 70 rooftop solar projects, totalling 513 projects  and 28 
-gigawatts of renewable energy capacity.  This includes several utility -scale renewable energy 
-projects located within the UK:  
-•In 2019, Amazon  announced our first power purchase agreement in the UK, located in
-Kintyre Peninsula, Scotland. The “Amazon Wind Farm Scotland – Beinn an Tuirc 3”
-began o perating in 2021, providing 50 megawatts  (MW)  of new renewable capacity to
-the electricity grid with expected generation of 168,000 megawatt hours (MWh) of
-clean energy annually. That’s enough to power 46,000 UK homes every year.
-•In December 2020,  Amazon a nnounced a two -phase renewable energy project located
-in South Lanarkshire, Scotland, the Kennoxhead wind farm. Kennoxhead will be the
-largest single -site onshore wind project in the UK, enabled through corporate
-procurement.  Once fully operational, Kenno xhead will produce 129 MW of renewable
-capacity and is expected to generate 439,000 MWh of clean energy annually. Phase 1
-(60 MW) began operating in 2022, and Phase 2 (69  MW) will begin operations in 2024 .
-•In 2022, Amazon announced its first project in Nor thern Ireland, a 16  MW onshore
-windfarm in Co Antrim.
-•In 2022, Amazon  also announced  a new 473 MW offshore wind farm, Moray West,
-located off the coast of Scotland . Amazon expects completion of Moray West in 2024.
-This is Amazon’s largest project in Scotland and the largest corporate renewable
-energy deal announced by any company in the UK to date.
-•In 2023, Amazon announced a new 47  MW solar farm, Warl ey located in Essex.
-This project is expected to be operational in 2024.
-
- Published September 2024  Declaration and Sign Off 
-This Carbon Reduction Plan has been completed in accordance with PPN 06/21 and 
-associated guidance and reporting standard for Carbon Reduction Plans.  
-Emiss ions have been reported and recorded in accordance with  the published reporting 
-standard for Carbon Reduction Plans and the GHG Reporting Protocol corporate standard1 
-and uses the appropri ate Government emission conversion factors for greenhouse gas 
-company reporting2. 
-Scope 1 and Scope 2 emissions have been reported in accordance with S ECR requirements, 
-and the required subset of Scope 3 emissions have been reported in accordance with the 
-published reporting standard for Carbon Reduction Plans and the Corporate Value Chain 
-(Scope 3) Standard3. 
-This Carbon Reduction Plan has been reviewed and signed off by the board of directors (or 
-equivalent management body)."
+      file_attachment: "../library/AstraZeneca-Sustainability-Report-2023.pdf"
     assert:
       - type: is-json
         value:
           required: ["company_name"]
           type: object
       - type: javascript
-        value: JSON.parse(output).company_name === "Amazon"
+        value: JSON.parse(output).company_name === "AstraZeneca"