Add optimization options in OR-Tools / XPRESS, refactor

docs/CHANGELOG.md

@@ -8,6 +8,7 @@ toc_depth: 2
 --------------------
 ### Improvements
 * STS groups are now "dynamic" : group names are no longer fixed by code, user is free to define these groups.
+* When xpress or scip solver is selected (using ortools), solver specific parameters can be specified in the generaldata.ini (#1837)
 
 ## 9.0.0
 --------------------

docs/reference-guide/10-command_line.md

@@ -41,6 +41,7 @@ _In all cases, arguments " –h" or "–help" can be used to get help_
 |-m, --mps-export | Export anonymous mps weekly or daily optimal UC+dispatch linear |
 |-s, --named-mps-problems | Export named mps weekly or daily optimal UC+dispatch linear |
 |--solver-logs | Print solver logs |
+|--solver-parameters | Set solver specific parameters: <ul><li>With Xpress : `--solver-parameters="param1 value1 param2 value2"`</li><li>With SCIP : `--solver-parameters="param1 value1, param2 value2"`</li></ul> Command-line parameters override those specified in generaldata.ini within `xpress-parameters` or `scip-parameters` fields |
 
 - Misc.
 

docs/reference-guide/13-file-format.md

@@ -1,8 +1,8 @@
 # Study format changes
 This is a list of all recent changes that came with new Antares Simulator features. The main goal of this document is to lower the costs of changing existing interfaces, both GUI and scripts.
 ## v9.1.0
-## Input 
-### Hydro Maximum Generation/Pumping Power
+### Input 
+#### Hydro Maximum Generation/Pumping Power
 * For time series ![Migration diagram](migration.png "Migration diagram"), for more details, see [this Python script](migration.py)
 * In the existing file **settings/scenariobuilder.dat**, under **<ruleset>** section following properties added: **hgp,<area>,<year> = <hgp-value>**
 
@@ -15,7 +15,7 @@ This implies that, inside one of the previous categories, the number of availabl
 
 * [Logic changes](17-v91.md)
 
-### Short term storage groups
+#### Short term storage groups
 STS groups in input are now "dynamic" : group names are no longer fixed by code, user is free to define these groups.
 
 In the "thematic trimming" section, the new dynamic variable `STS by group` is now used to enable/disable all variables (level/injection/withdrawal) for all groups. The following variables are obsolete and must not be provided
@@ -38,18 +38,27 @@ Other5_level
 
 The default value for group is "OTHER1".
 
-## Output
-### Hydro maximum generation/pumping power
+#### Optimization : solver parameters (expert option)
+
+In file **settings/generaldata.ini**, in section `optimization` add properties `xpress-parameters` and `scip-parameters` [str]. Default value = `empty string`. Allows to specify solver specific parameters in the following format : 
+
+- `xpress-parameters = param1 value1 param2 value2`,
+- `scip-parameters = param1 value1, param2 value2`.
+
+These parameters can also be set from the command-line using `--solver-parameters`. In this case command-line parameters override settings from `generaldata.ini`
+
+### Output
+#### Hydro maximum generation/pumping power
 In existing directory `ts-numbers`, add sub-directory `hgp` containing TS numbers for hydro max pumping/generation, for each area.
 
-### ST Storage
+#### ST Storage
 - Output columns for ST storage are capitalized. For any STS group name my_group, 3 output columns are created : `MY_GROUP_INJECTION`, `MY_GROUP_WITHDRAWAL`, `MY_GROUP_LEVEL`.
 - If a group is empty, no column is produced.
 - There is now a variable number of columns in files values-XXX.txt, depending on the groups of ST storages provided by the user. Note that groups are case-insensitive, for example `battery`, `Battery` and `BATTERY` all represent the same group. If no ST storage exist for a given area, no variables associated to ST storages will be produced.
 
 ## v9.0.0
 ### Input
-### Study version
+#### Study version
 Breaking change in the study format, file **study.antares**
 ```
 version = 900

src/CMakeLists.txt

@@ -464,4 +464,4 @@ endif()
 include(CPack)
 
 #Last. We need all target defined
-add_subdirectory(packaging)
+# add_subdirectory(packaging)

src/libs/antares/CMakeLists.txt

@@ -27,6 +27,8 @@ add_subdirectory(sys)
 add_subdirectory(utils)
 add_subdirectory(writer)
 
+add_subdirectory(optimization-options)
+
 set(HEADERS
         include/antares/antares/antares.h
         include/antares/antares/constants.h

src/libs/antares/InfoCollection/StudyInfoCollector.cpp

@@ -141,17 +141,17 @@ void StudyInfoCollector::solverVersionToFileContent(FileContent& file_content)
 
 void StudyInfoCollector::ORToolsUsed(FileContent& file_content)
 {
-    const bool& ortoolsUsed = study_.parameters.ortoolsUsed;
+    const bool& ortoolsUsed = study_.parameters.optOptions.ortoolsUsed;
     file_content.addItemToSection("study", "ortools used", ortoolsUsed ? "true" : "false");
 }
 
 void StudyInfoCollector::ORToolsSolver(FileContent& file_content)
 {
-    const bool& ortoolsUsed = study_.parameters.ortoolsUsed;
+    const bool& ortoolsUsed = study_.parameters.optOptions.ortoolsUsed;
     std::string ortoolsSolver = "none";
     if (ortoolsUsed)
     {
-        ortoolsSolver = study_.parameters.ortoolsSolver;
+        ortoolsSolver = study_.parameters.optOptions.ortoolsSolver;
     }
     file_content.addItemToSection("study", "ortools solver", ortoolsSolver);
 }

src/libs/antares/checks/checkLoadedInputData.cpp

@@ -58,7 +58,7 @@ void checkStudyVersion(const AnyString& optStudyFolder)
         throw Error::InvalidStudy(optStudyFolder);
 
     if (version > StudyVersion::latest())
-        throw Error::InvalidVersion(version.toString(),StudyVersion::latest().toString());
+        throw Error::InvalidVersion(version.toString(), StudyVersion::latest().toString());
 }
 // CHECK incompatible de choix simultané des options « simplex range= daily » et « hydro-pricing
 // = MILP ».
@@ -150,8 +150,9 @@ void checkMinStablePower(bool tsGenThermal, const Antares::Data::AreaList& areas
 // Number of columns for Fuel & CO2 cost in thermal clusters must be one, or same as the number of
 // TS
 template<class ExceptionT>
-static void checkThermalColumnNumber(const Antares::Data::AreaList& areas,
-        Antares::Data::TimeSeries::TS Antares::Data::EconomicInputData::*matrix)
+static void checkThermalColumnNumber(
+  const Antares::Data::AreaList& areas,
+  Antares::Data::TimeSeries::TS Antares::Data::EconomicInputData::*matrix)
 {
     ExceptionT exception;
     bool error = false;
@@ -178,14 +179,14 @@ static void checkThermalColumnNumber(const Antares::Data::AreaList& areas,
 
 void checkFuelCostColumnNumber(const Antares::Data::AreaList& areas)
 {
-    checkThermalColumnNumber<Antares::Error::IncompatibleFuelCostColumns>(areas,
-                             &Antares::Data::EconomicInputData::fuelcost);
+    checkThermalColumnNumber<Antares::Error::IncompatibleFuelCostColumns>(
+      areas, &Antares::Data::EconomicInputData::fuelcost);
 }
 
 void checkCO2CostColumnNumber(const Antares::Data::AreaList& areas)
 {
-    checkThermalColumnNumber<Antares::Error::IncompatibleCO2CostColumns>(areas,
-                             &Antares::Data::EconomicInputData::co2cost);
+    checkThermalColumnNumber<Antares::Error::IncompatibleCO2CostColumns>(
+      areas, &Antares::Data::EconomicInputData::co2cost);
 }
 
 } // namespace Antares::Check

src/libs/antares/checks/include/antares/checks/checkLoadedInputData.h

@@ -18,12 +18,20 @@
 ** You should have received a copy of the Mozilla Public Licence 2.0
 ** along with Antares_Simulator. If not, see <https://opensource.org/license/mpl-2-0/>.
 */
+#include "antares/study/fwd.h"
+#include "antares/optimization-options/options.h"
+
 namespace Antares::Check
 {
+
 void checkOrtoolsUsage(Antares::Data::UnitCommitmentMode ucMode,
                        bool ortoolsUsed,
                        const std::string& solverName);
 
+void checkOrtoolsSolverSpecificParameters(
+  const Antares::Data::UnitCommitmentMode& ucMode,
+  const Antares::Solver::Optimization::OptimizationOptions& optOptions);
+
 void checkStudyVersion(const AnyString& optStudyFolder);
 
 void checkSimplexRangeHydroPricing(Antares::Data::SimplexOptimization optRange,

src/libs/antares/exception/LoadingError.cpp

@@ -62,18 +62,34 @@ InvalidSimulationMode::InvalidSimulationMode() :
 {
 }
 
-static std::string InvalidSolverHelper(const std::string& solver, const std::string& availableSolvers)
+static std::string InvalidSolverHelper(const std::string& solver,
+                                       const std::string& availableSolvers)
 {
-  std::ostringstream message;
-  message
-    << "Can't use solver '"
-    << solver
-    << "' : it does not match any available OR-Tools solver. Possible choices are "
-    << availableSolvers;
-  return message.str();
+    std::ostringstream message;
+    message << "Can't use solver '" << solver
+            << "' : it does not match any available OR-Tools solver. Possible choices are "
+            << availableSolvers;
+    return message.str();
 }
 
-InvalidSolver::InvalidSolver(const std::string& solver, const std::string& availableSolvers) : LoadingError(InvalidSolverHelper(solver, availableSolvers))
+InvalidSolver::InvalidSolver(const std::string& solver, const std::string& availableSolvers) :
+ LoadingError(InvalidSolverHelper(solver, availableSolvers))
+{
+}
+
+static std::string InvalidSolverSpecificParametersHelper(const std::string& solver,
+                                                         const std::string& solverParameters)
+{
+    std::ostringstream message;
+    message << "Specific parameters '" << solverParameters << "' are not valid or not supported for solver "
+            << solver;
+    return message.str();
+}
+
+InvalidSolverSpecificParameters::InvalidSolverSpecificParameters(
+  const std::string& solver,
+  const std::string& solverParameters) :
+ LoadingError(InvalidSolverSpecificParametersHelper(solver, solverParameters))
 {
 }
 

src/libs/antares/exception/include/antares/exception/LoadingError.hpp

@@ -135,6 +135,12 @@ class InvalidSolver : public LoadingError
     explicit InvalidSolver(const std::string& solver, const std::string& availableSolverList);
 };
 
+class InvalidSolverSpecificParameters : public LoadingError
+{
+public:
+    explicit InvalidSolverSpecificParameters(const std::string& solver, const std::string& solverParameters);
+};
+
 class InvalidStudy : public LoadingError
 {
 public:

src/libs/antares/optimization-options/CMakeLists.txt

@@ -0,0 +1,10 @@
+set(PROJ optimization-options)
+
+add_library(${PROJ} INTERFACE)
+
+target_include_directories(${PROJ}
+        INTERFACE
+        $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
+)
+
+add_library(Antares::${PROJ} ALIAS ${PROJ})

src/libs/antares/optimization-options/include/antares/optimization-options/options.h

@@ -0,0 +1,34 @@
+/*
+** Copyright 2007-2024, RTE (https://www.rte-france.com)
+** See AUTHORS.txt
+** SPDX-License-Identifier: MPL-2.0
+** This file is part of Antares-Simulator,
+** Adequacy and Performance assessment for interconnected energy networks.
+**
+** Antares_Simulator is free software: you can redistribute it and/or modify
+** it under the terms of the Mozilla Public Licence 2.0 as published by
+** the Mozilla Foundation, either version 2 of the License, or
+** (at your option) any later version.
+**
+** Antares_Simulator is distributed in the hope that it will be useful,
+** but WITHOUT ANY WARRANTY; without even the implied warranty of
+** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+** Mozilla Public Licence 2.0 for more details.
+**
+** You should have received a copy of the Mozilla Public Licence 2.0
+** along with Antares_Simulator. If not, see <https://opensource.org/license/mpl-2-0/>.
+*/
+
+#pragma once
+
+namespace Antares::Solver::Optimization {
+struct OptimizationOptions
+{
+    //! Force ortools use
+    bool ortoolsUsed = false;
+    //! The solver name, sirius is the default
+    std::string ortoolsSolver = "sirius";
+    bool solverLogs = false;
+    std::string solverParameters = "";
+};
+}

src/libs/antares/study/include/antares/study/load-options.h

@@ -23,6 +23,8 @@
 
 #include <yuni/yuni.h>
 #include <yuni/core/string.h>
+#include <antares/optimization-options/options.h>
+
 #include "parameters.h"
 
 namespace Antares
@@ -62,8 +64,6 @@ class StudyLoadOptions
     bool mpsToExport;
     //! named problems
     bool namedProblems = false;
-    //! enable solver logs
-    bool solverLogs = false;
     //! Ignore all constraints
     bool ignoreConstraints;
     //! Simulation mode
@@ -79,10 +79,8 @@ class StudyLoadOptions
     //! A non-zero value if the data will be used for a simulation
     bool usedByTheSolver;
 
-    //! Force ortools use
-    bool ortoolsUsed;
-    //! THe solver name, sirius is the default
-    std::string ortoolsSolver = "sirius";
+    //! All options related to optimization
+    Antares::Solver::Optimization::OptimizationOptions optOptions;
 
     //! Temporary string for passing log message
     mutable Yuni::String logMessage;

src/libs/antares/study/include/antares/study/parameters.h

@@ -37,6 +37,7 @@
 #include "version.h"
 
 #include <antares/study/UnfeasibleProblemBehavior.hpp>
+#include <antares/optimization-options/options.h>
 
 namespace Antares::Data
 {
@@ -511,25 +512,19 @@ class Parameters final
     uint seed[seedMax];
     //@}
 
-    //! \name Ortools configuration
-    //@{
-    //! Define if ortools is used
-    bool ortoolsUsed;
-    //! Ortool solver used for simulation
-    std::string ortoolsSolver;
-    //@}
+
     // Format of results. Currently, only single files or zip archive are supported
     ResultFormat resultFormat = legacyFilesDirectories;
 
     // Naming constraints and variables in problems
     bool namedProblems;
 
-    // solver logs
-    bool solverLogs;
+    // All options related to optimization
+    Antares::Solver::Optimization::OptimizationOptions optOptions;
 
 private:
     //! Load data from an INI file
-    bool loadFromINI(const IniFile& ini, StudyVersion& version, const StudyLoadOptions& options);
+    bool loadFromINI(const IniFile& ini, const StudyVersion& version, const StudyLoadOptions& options);
 
     void resetPlayedYears(uint nbOfYears);
 

src/libs/antares/study/load-options.cpp

@@ -40,8 +40,7 @@ StudyLoadOptions::StudyLoadOptions() :
  enableParallel(false),
  forceParallel(false),
  maxNbYearsInParallel(0),
- usedByTheSolver(false),
- ortoolsUsed(false)
+ usedByTheSolver(false)
 {
 }