From 4f938e043e69e5718ca4c8b2c2d0f8ef41f8e87f Mon Sep 17 00:00:00 2001 From: Aditya Date: Wed, 24 Jul 2024 18:44:07 +0200 Subject: [PATCH 1/8] Update README.md Signed-off-by: Aditya --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 8dc9482..89ddc07 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ Carbon Efficiency: The amount of carbon dioxide equivalent (CO2e) emitted per US ### Scope and Objectives * Software Carbon Efficiency Rating (SCER) Standard: Develop a standardized software carbon efficiency rating system that can assess and rate the carbon efficiency of software products. This standard will provide a framework for evaluating and comparing the environmental impact of different software applications by including a benchmark and a rating algorithm that can be used for decision-making around software applications' carbon efficiency. -* Creating a Green Software Certification Lab (GSCL), a standardized testing and certification lab for green software, similar to UL (Underwriters Laboratories) for hardware and electrical systems, involves defining specific criteria, processes, and requirements to evaluate and certify the environmental sustainability and efficiency of software applications. Here is a set of specifications for the lab: Certification Framework, Energy Efficiency, Environmental Impact, Resource Optimization, Power Efficiency, Green Data Management, Renewable Energy Compatibility, Sustainable Development Practices, Certification Levels, etc. +* Creating a Green Software Certification Lab (GSCL), a standardized testing and certification lab for green software, similar to UL (Underwriters Laboratories) for hardware and electrical systems, involves defining specific criteria, processes, and requirements to evaluate and certify the environmental sustainability and efficiency of software applications. Here is a set of specifications for the lab: Certification Framework, Energy Efficiency, Environmental Impact, Resource Optimization, Power Efficiency, Green Data Management, Renewable Energy Compatibility, Sustainable Development Practices, Certification Levels, Ratings, etc. * Documentation and Guidelines: Create comprehensive documentation and guidelines that detail the carbon-efficient software development process and the SCER rating system. ### Deliverables: From b27175dc5c8b6d5e537a4ae201960b6fad2f4df6 Mon Sep 17 00:00:00 2001 From: Aditya Date: Wed, 24 Jul 2024 18:50:37 +0200 Subject: [PATCH 2/8] Create events.md Signed-off-by: Aditya --- Resources/events.md | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 Resources/events.md diff --git a/Resources/events.md b/Resources/events.md new file mode 100644 index 0000000..50e5989 --- /dev/null +++ b/Resources/events.md @@ -0,0 +1,3 @@ +Upcoming events for participation: +- [Open Source Summit Japan](https://events.linuxfoundation.org/open-source-summit-japan/) +- [Open Source Summit Asia](https://events.linuxfoundation.org/kubecon-cloudnativecon-open-source-summit-ai-dev-china/) From 5cba1663c245631fb8640e3779ceed8ba2a9c84b Mon Sep 17 00:00:00 2001 From: Aditya Date: Wed, 24 Jul 2024 18:52:25 +0200 Subject: [PATCH 3/8] Add Gosim China Signed-off-by: Aditya --- Resources/events.md | 1 + 1 file changed, 1 insertion(+) diff --git a/Resources/events.md b/Resources/events.md index 50e5989..fe97af1 100644 --- a/Resources/events.md +++ b/Resources/events.md @@ -1,3 +1,4 @@ Upcoming events for participation: - [Open Source Summit Japan](https://events.linuxfoundation.org/open-source-summit-japan/) - [Open Source Summit Asia](https://events.linuxfoundation.org/kubecon-cloudnativecon-open-source-summit-ai-dev-china/) +- [Gosim China 2024](https://china2024.gosim.org/) From 5ef1959ef95193a051077c8148741c3cad44e9c9 Mon Sep 17 00:00:00 2001 From: adityamanglik Date: Tue, 6 Aug 2024 16:33:17 +0200 Subject: [PATCH 4/8] Initialize GSCL --- .gitmodules | 3 +++ GSCL/kepler | 1 + 2 files changed, 4 insertions(+) create mode 100644 .gitmodules create mode 160000 GSCL/kepler diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 0000000..9e4d55a --- /dev/null +++ b/.gitmodules @@ -0,0 +1,3 @@ +[submodule "GSCL/kepler"] + path = GSCL/kepler + url = https://github.com/sustainable-computing-io/kepler diff --git a/GSCL/kepler b/GSCL/kepler new file mode 160000 index 0000000..859d9bf --- /dev/null +++ b/GSCL/kepler @@ -0,0 +1 @@ +Subproject commit 859d9bf154ba276938446707ec0220f3d54782d0 From cfeb569e7706206f96cd6471e261f104f6a029d8 Mon Sep 17 00:00:00 2001 From: adityamanglik Date: Tue, 6 Aug 2024 19:33:32 +0200 Subject: [PATCH 5/8] Extract per process energy values for local --- GSCL/kepler | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GSCL/kepler b/GSCL/kepler index 859d9bf..73f5439 160000 --- a/GSCL/kepler +++ b/GSCL/kepler @@ -1 +1 @@ -Subproject commit 859d9bf154ba276938446707ec0220f3d54782d0 +Subproject commit 73f543905b6d53510bc7fc77ce38f309d49b401d From 814283df9f7b9696e534877db56fe644a55f7afe Mon Sep 17 00:00:00 2001 From: Aditya Date: Wed, 21 Aug 2024 17:51:48 +0200 Subject: [PATCH 6/8] Meeting_Questions.md Signed-off-by: Aditya --- GSCL/Questions.md | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 GSCL/Questions.md diff --git a/GSCL/Questions.md b/GSCL/Questions.md new file mode 100644 index 0000000..5b5264d --- /dev/null +++ b/GSCL/Questions.md @@ -0,0 +1,7 @@ +1. Should we define a universal hardware baseline, or allow for flexibility based on regional or organizational resources? + +3. How can we incentivize major LLM providers to contribute their internal workflows to an open-source standard? + +5. What conformance programs or validation processes should be put in place to maintain the integrity of the certification? + +7. What role should community-driven initiatives, like crowd-sourced testing, play in the GSCL ecosystem? From 4d3fa593df0c51bb9b779ab2509615a293a94b46 Mon Sep 17 00:00:00 2001 From: Aditya Date: Wed, 21 Aug 2024 17:56:36 +0200 Subject: [PATCH 7/8] Create Initial Draft.md Signed-off-by: Aditya --- GSCL/Initial Draft.md | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 GSCL/Initial Draft.md diff --git a/GSCL/Initial Draft.md b/GSCL/Initial Draft.md new file mode 100644 index 0000000..1edacf6 --- /dev/null +++ b/GSCL/Initial Draft.md @@ -0,0 +1,34 @@ +## Objective +To establish a standardized methodology for measuring and comparing the carbon efficiency of software applications, ensuring consistent, reproducible results across different platforms and environments. + +## Components: + +### Hardware Specifications ++ Define a baseline hardware setup that includes processor type, memory capacity, storage type, and network configuration. This baseline should represent mid-tier, widely accessible hardware to ensure broad applicability. +For cloud-based testing, specify a standard cloud instance type (e.g., AWS m5.large, Azure D4s_v3) with fixed resource allocation. + +### Software Configuration ++ Operating System: Use a standardized, widely supported operating system (e.g., Ubuntu 22.04 LTS) with minimal additional services running to reduce background noise in energy consumption measurements. ++ Middleware and Dependencies: Specify versions and configurations for any middleware or software dependencies, ensuring consistency across all test environments. + +### Benchmark Workload Definition + ++ Unit of Software Function (USF): Define the core functionality to be measured (e.g., number of API requests processed, number of data sets analyzed). ++ Execution Frequency: Specify the number of times the USF should be executed during the benchmark to account for variability in short-term performance. + +### Measurement Metrics + ++ SCI: Tool? ++ Energy Consumption (kWh): Measure the total energy consumed during the benchmark using precise energy monitoring tools (e.g., Kepler, or direct power measurement at the hardware level). ++ Carbon Emission (CO2e): Calculate the carbon emissions based on the energy consumption data, using region-specific carbon intensity factors for electricity (e.g., grams of CO2e per kWh). +Test Methodology: + +### Setup Instructions ++ Provide detailed, step-by-step instructions for setting up the software and test environment, including installation scripts and configuration files. ++ Execution Procedure: Outline a sequential list of actions to perform during the benchmark, ensuring that each step is repeatable and documented. ++ Result Collection: Use automated logging tools to collect performance data (e.g., energy consumption, processing time). Ensure data integrity by validating logs against expected outputs. +Certification and Reporting: + ++ SCER (Software Carbon Efficiency Rating): Assign a carbon efficiency rating based on the measured CO2e per USF, normalized to account for any variations in hardware performance. ++ Certification Levels: Define certification levels (e.g., Platinum, Gold, Silver) based on SCER thresholds, providing a clear and intuitive indication of software carbon efficiency. ++ Transparency and Reproducibility: All benchmark results, methodologies, and environmental configurations should be published openly to allow independent verification and reproducibility of results by third parties. From 10b28463e80e0317570ca1388b58e34e9285dbe7 Mon Sep 17 00:00:00 2001 From: Aditya Date: Wed, 21 Aug 2024 17:59:38 +0200 Subject: [PATCH 8/8] Example.md Signed-off-by: Aditya --- GSCL/Example.md | 77 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 GSCL/Example.md diff --git a/GSCL/Example.md b/GSCL/Example.md new file mode 100644 index 0000000..c534d89 --- /dev/null +++ b/GSCL/Example.md @@ -0,0 +1,77 @@ +# Example Specification: Measuring Carbon Efficiency of PostgreSQL on AWS m5.large + +## Objective +To benchmark the carbon efficiency of PostgreSQL running on an AWS m5.large instance by measuring its energy consumption during a typical database workload. + +## Test Environment + +### 1. Hardware Specifications +- **Instance Type:** AWS m5.large +- **vCPU:** 2 vCPUs (Intel Xeon Platinum 8175M) +- **Memory:** 8 GiB RAM +- **Storage:** 100 GiB EBS General Purpose SSD (gp3) +- **Network:** Up to 10 Gbps network bandwidth +- **Operating System:** Ubuntu 22.04 LTS + +### 2. Software Configuration +- **PostgreSQL Version:** 14.4 +- **Database Configuration:** + - **Max Connections:** 100 + - **Shared Buffers:** 2 GiB + - **Work Mem:** 16 MB + - **Maintenance Work Mem:** 64 MB +- **Workload Tool:** `pgbench` (PostgreSQL benchmarking tool) +- **pgbench Parameters:** + - **Scale Factor:** 100 (creates a 1 GB database) + - **Number of Clients:** 10 + - **Number of Transactions:** 100,000 + +## Benchmark Workload Definition + +### 1. Unit of Software Function (USF) +- **Definition:** Execution of 100,000 transactions by 10 concurrent clients on the PostgreSQL instance. TPC-C or TPC-H dataset? + +### 2. Execution Frequency +- The USF will be executed 10 times to ensure consistency and account for any variability in performance. + +## Measurement Metrics + +### 1. Energy Consumption (kWh) +- **Tool:** Use AWS CloudWatch and Kepler to measure the energy consumption of the m5.large instance during the benchmark. + +### 2. Carbon Emission (CO2e) +- **Calculation:** Multiply the measured energy consumption (in kWh) by the region-specific carbon intensity factor (grams of CO2e per kWh) for the AWS data center region where the instance is located. + +## Test Methodology + +### 1. Setup Instructions +1. Launch an AWS m5.large instance in the desired region with Ubuntu 22.04 LTS. +2. Install PostgreSQL 14.4 using the official PostgreSQL repository. +3. Configure PostgreSQL with the parameters specified above. +4. Install `pgbench` on the instance. +5. Install `kepler` on the instance. +6. Create a test database and initialize `pgbench` with a scale factor of 100. + +### 2. Execution Procedure +1. Start monitoring energy consumption using kepler and AWS CloudWatch. +2. Run `pgbench` with 10 clients executing 100,000 transactions. +3. Repeat the benchmark 10 times, recording the energy consumption and transaction performance for each run. + +### 3. Result Collection +- Collect energy consumption data from the monitoring script and CloudWatch logs. +- Calculate the average energy consumed per USF across the 10 runs. +- Calculate the total CO2e emissions using the energy consumption and carbon intensity factor. + +## Certification and Reporting + +### 1. SCER (Software Carbon Efficiency Rating) +- ?? + +### 3. Transparency and Reproducibility +- Publish the benchmark methodology, configuration details, and results openly to allow third-party verification and reproducibility. + +## Discussion Points +- Different stages: Disclosure, Categorization, Benchmarking (this), Rating +- Feasibility of applying this benchmarking methodology across different cloud environments and regions. +- How to incentivize other database management systems to adopt a similar standardized benchmarking process. +- Potential challenges in maintaining the consistency of energy measurements across various AWS regions.