Merge pull request #27 from alchem0x2A/socket

Socket

.github/workflows/installation_test.yml

@@ -88,4 +88,61 @@ jobs:
       env:
         GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         RID: ${{ github.run_id }}
-
+
+  test-socket:
+    defaults:
+      run:
+        shell: bash -l {0}
+    runs-on: ubuntu-latest
+    strategy:
+      max-parallel: 5
+
+    steps:
+    - uses: actions/checkout@v3
+    - uses: conda-incubator/setup-miniconda@v2
+      with:
+        python-version: "3.10"
+        mamba-version: "*"
+        channels: conda-forge,alchem0x2a,defaults
+        channel-priority: true
+        activate-environment: sparc-api-test
+    - name: Install dependencies
+      run: |
+        # mamba install -c conda-forge ase>=3.22 pymatgen flake8 pytest
+        mamba install -c conda-forge make compilers openmpi fftw scalapack openblas
+    - name: Install package
+      run: |
+        pip install -e ".[test]"
+        # Download the external psp data
+        python -m sparc.download_data
+    - name: Download SPARC output files to SPARC-master
+      run: |
+        # TODO: merge to master
+        wget -O SPARC-socket.zip https://codeload.github.com/alchem0x2A/SPARC/zip/refs/heads/socket
+        unzip SPARC-socket.zip
+    - name: Compile SPARC with socket
+      run: |
+        cd SPARC-socket/src
+        make clean
+        make -j2 USE_SOCKET=1 USE_MKL=0 USE_SCALAPACK=1 DEBUG_MODE=1
+        ls ../lib
+    - name: Test with pytest
+      run: |
+        ls ./SPARC-socket/lib/sparc
+        PWD=$(pwd)
+        export SPARC_TESTS_DIR="${PWD}/SPARC-socket/tests"
+        export ASE_SPARC_COMMAND="mpirun -n 1 ${PWD}/SPARC-socket/lib/sparc"
+        export SPARC_DOC_PATH="${PWD}/SPARC-socket/doc/.LaTeX"
+        coverage run -a -m pytest -svv tests/
+        coverage json --omit="tests/*.py"
+        coverage html --omit="tests/*.py"
+        COVERAGE=`cat coverage.json | jq .totals.percent_covered | xargs printf '%.*f' 0`
+        echo "Current coverage is $COVERAGE"
+        echo "COVPERCENT=$COVERAGE" >> $GITHUB_ENV
+        
+    - name: Lint with flake8
+      run: |
+        echo $CONDA_PREFIX
+        conda info
+        flake8 sparc/ --count --select=E9,F63,F7,F82 --show-source --statistics
+        flake8 sparc/ --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics

.gitignore

@@ -780,3 +780,5 @@ examples/ex1-ase/
 /*.inpt*
 /*.log
 /*.psp8
+/test-1/
+/test-2/

README.md

@@ -320,7 +320,7 @@ unit system:
 |                |                                 | `xc=vdwdf1` or `xc=vdw-df` | `EXCHANGE_CORRELATION: vdWDF1` |
 |                |                                 | `xc=vdwdf2` or `xc=vdw-df2` | `EXCHANGE_CORRELATION: vdWDF2` |
 |                |                                 | `xc=scan` | `EXCHANGE_CORRELATION: SCAN` |
-| `h`            | Real grid spacing    (Å)           | `h=0.2`        | `FD_GRID: Nx Ny Nz`  (nearest int value)         |
+| `h`            | Real grid spacing    (Å)           | `h=0.2`        | `MESH_GRID: 0.38`  (in Bohr)         |
 | `gpts`         | Explicit grid points |   `gpts=[10, 10, 10]` | `FD_GRID: 10 10 10` |
 | `kpts`         | Kpoint mesh          |   `kpts=[3, 3, 3]`    | `KPOINT_GRID: 3 3 3` |
 | `convergence`  | Dict of convergence criteria (see below) |  | |
@@ -357,6 +357,139 @@ from sparc.calculator import SPARC
 calc = SPARC(xc="PBE", kpts=(9, 9, 9), h=0.25, directory="sparc-calc.sparc", convergence={"energy": 1.e-4})
 ```
 
+## Advanced Usage: SPARC-X-API as a Socket Interface
+
+*Disclaimer: The socket communication feature in SPARC and SPARC-X-API are experimental and subject to changes until the release of v2.0 SPARC-X-API.*
+
+### Overview
+Experienced users can now harness the power of SPARC and SPARC-X-API's
+socket communication layer to build efficient and flexible
+computational workflows. By integrating a socket communication
+interface directly into SPARC, users can significantly reduce the
+overhead typically associated with file I/O during calculation
+restarts. This feature is particularly beneficial for tasks involving
+repetitive operations like structural optimization and saddle point
+searches, where traditional file-based communication can become a
+bottleneck.
+
+**Requirements**: the SPARC binary must be manually compiled from the source
+code with [socket
+support](https://github.com/alchem0x2A/SPARC/tree/socket) and with the
+`USE_SOCKET=1` flag enabled (see the [installation
+instructions](https://github.com/alchem0x2A/SPARC/tree/socket).
+
+### Usage
+The socket communication layer in SPARC and SPARC-X-API are designed for:
+- **Efficiency:** Eliminates the need for intermediate file I/O, directly streaming data between processes.
+- **Speed:** Enhances the performance of iterative calculations, crucial for large-scale simulations.
+- **Flexibility:** Allows dynamic modification of calculation parameters without the need to restart the process.
+
+The communication protocol implemented in SPARC and SPARC-X-API
+adheres to the [i-PI protocol](https://github.com/i-pi/i-pi)
+standard. Specifically, we implement the original i-PI protocol within
+the SPARC C-source code, while the python SPARC-X-API uses a
+backward-compatible protocol based on i-PI. The dual-mode design is aimed for both low-level and
+high-level interfacing of the DFT codes, providing the following features:
+
+1. **Unified Interface**: A consistent interface for both client and server codes, simplifying user interaction across different modes.
+2. **Versatile Operation Modes:** Supports various modes (Local-only, Client, Server) to cater to diverse computational needs.
+3. **Seamless Calculation Restart:** Automates the restarting process on the client side, enhancing user convenience.
+
+Based on the user scenarios, the socket communication layer can be accessed via the following approaches:
+
+1. **SPARC binary only**
+
+   SPARC binary with socket support can be readily coupled with any i-PI compatible socker server, such as
+   `ase.calculators.socketio.SocketIOCalculator`, for example
+
+   ```python
+   from ase.calculators.socketio import SocketIOCalculator
+   from subprocess import Popen
+   calc = SocketIOCalculator(port=31415)
+   with calc:
+       # Start sparc at background
+	   process = Popen("mpirun -n 8 sparc -name SPARC -socket localhost:31415", shell=True)
+	   # Single point calculations
+	   process.kill()
+   ```
+
+   The end user is responsible for generating the input files and
+   making sure the same atoms structures are used by
+   `SocketIOCalculator` and the SPARC binary. The mode is also limited
+   to be run on a single computer system. 
+
+
+2. **Local-only Mode**
+
+   Ideal for standalone calculations, this mode simulates a conventional calculator while benefiting from socket-based efficiency.
+
+   ```python
+   with SPARC(use_socket=True, **normal_parameters) as calc:
+       # Execute single-point calculations
+   ```
+   For most users we recommend using this mode when performing a calculation on a single HPC node.
+
+3. **Client (Relay) Mode** 
+
+   In this mode, the `sparc.SPARC` calculator servers as a passive
+   client which listens to a remote i-PI-compatible server. When
+   messages are received on the client side, it relays the relevant
+   message to a local SPARC binary and send results back through the
+   socket pipe. The server side can either be a normal i-PI compatible
+   server (such as `SocketIOCalculator`) or server-mode `sparc.SPARC` (see 4).
+
+   Start the client by:
+   ```python
+   client = SPARC(use_socket=True,
+	              socket_params=dict(host="host.address.com", port=31415))
+   with client:
+       client.run()
+   ```
+
+   Or via Command-Line:
+   ```bash
+   python -m sparc.client -s host:port
+   ```
+
+   Note: when running SPARC-X-API as a socket client, the atoms object
+   can be ommitted (is the server also runs the SPARC protocol). When
+   new atoms positions and parameters arrive, the client will
+   automatically determine if it is necessary to restart the SPARC
+   subprocess.
+
+4. **Server Mode**
+
+   Paired with the client mode in (3), SPARC-X-API can be run as a
+   socket server, isolated from the node that performs the
+   computation. This can be useful for highly-distributed
+   computational workflows.
+
+   On the server node, run:
+   ```python
+   server_calc = SPARC(use_socket=True, socket_params=dict(port=31415, server_only=True), **normal_parameters)
+   with server_calc:
+       # Execute single point calculations for atoms_1
+	   # Execute single point calculations for atoms_2
+   ```
+
+   In this case, the server will opens `0.0.0.0:31415` for
+   connection. Make sure your server is directly accessible from the
+   clients and the port is not occupied. The socker server is capable
+   of receiving `raw_results` directly from the clients, making it
+   possible to access `server_calc.raw_results` without access to the
+   file systems on the client side.
+
+
+### (In-progress) Controlling SPARC routines from socket interface
+
+Our SPARC socket protocol designs allows bidirectional control of
+internal SPARC routines. Local- or server-mode `sparc.SPARC`
+calculators can communicate with the SPARC binary via functions like
+`set_params`. This can be useful for applications like on-the-fly
+force field learning, electron density fitting, setting up boundary
+conditions etc. Applications will be updated in both SPARC and
+SPARC-X-API repositories.
+
 
 
 ## Troubleshooting

setup.py

@@ -17,18 +17,14 @@
 
 setup(
     name="sparc-x-api",
-    version="1.0.2",
+    version="1.0.3",
     python_requires=">=3.8",
     description="Python API for the SPARC DFT Code",
     author="Tian Tian, Ben Comer",
     author_email="[email protected], [email protected]",
     url="https://github.com/SPARC-X/SPARC-X-API",
     packages=find_packages(),
-    install_requires=[
-        "ase>=3.22.0",
-        "numpy>=1.22",
-        "packaging>=20.0"
-    ],
+    install_requires=["ase>=3.22.0", "numpy>=1.22", "packaging>=20.0", "psutil>=5.0.0"],
     entry_points={
         "ase.io": [
             "sparc = sparc.io",