Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/SK-1102 | Simplify ClientAPI and add support for inference #726

Merged
merged 20 commits into from
Nov 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .ci/tests/examples/print_logs.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash
service = $1
example = $2
helper = $3
service="$1"
example="$2"
helper="$3"

if [ "$service" == "minio" ]; then
echo "Minio logs"
Expand Down
2 changes: 1 addition & 1 deletion .ci/tests/examples/wait_for.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ def _test_rounds(n_rounds):
def _test_nodes(n_nodes, node_type, reducer_host='localhost', reducer_port='8092'):
try:

endpoint = "list_clients" if node_type == "client" else "list_combiners"
endpoint = "api/v1/clients/" if node_type == "client" else "api/v1/combiners/"

response = requests.get(
f'http://{reducer_host}:{reducer_port}/{endpoint}', verify=False)
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ services:
- ${HOST_REPO_DIR:-.}/fedn:/app/fedn
entrypoint: [ "sh", "-c" ]
command:
- "/venv/bin/pip install --no-cache-dir -e . && /venv/bin/fedn client start --init config/settings-client.yaml"
- "/venv/bin/pip install --no-cache-dir -e . && /venv/bin/fedn client start --api-url http://api-server:8092"
deploy:
replicas: 0
depends_on:
Expand Down
1 change: 1 addition & 0 deletions fedn/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
from os.path import basename, dirname, isfile

from fedn.network.api.client import APIClient
from fedn.network.clients.fedn_client import FednClient

# flake8: noqa

Expand Down
6 changes: 2 additions & 4 deletions fedn/cli/client_cmd.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ def list_clients(ctx, protocol: str, host: str, port: str, token: str = None, n_
click.echo(f"Error: Could not connect to {url}")


@client_cmd.command("start")
@client_cmd.command("start-v1")
@click.option("-d", "--discoverhost", required=False, help="Hostname for discovery services(reducer).")
@click.option("-p", "--discoverport", required=False, help="Port for discovery services (reducer).")
@click.option("--token", required=False, help="Set token provided by reducer if enabled")
Expand Down Expand Up @@ -208,7 +208,7 @@ def _complement_client_params(config: dict):
click.echo(f"Protocol missing, complementing api_url with protocol: {result}")


@client_cmd.command("start-v2")
@client_cmd.command("start")
@click.option("-u", "--api-url", required=False, help="Hostname for fedn api.")
@click.option("-p", "--api-port", required=False, help="Port for discovery services (reducer).")
@click.option("--token", required=False, help="Set token provided by reducer if enabled")
Expand Down Expand Up @@ -239,8 +239,6 @@ def client_start_v2_cmd(
helper_type: str,
init: str,
):
click.echo(click.style("\n*** fedn client start-v2 is experimental ***\n", blink=True, bold=True, fg="red"))

package = "local" if local_package else "remote"

config = {
Expand Down
121 changes: 121 additions & 0 deletions fedn/network/clients/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
Creating Your Own Client
========================

This guide will help you create your own client for the FEDn network.

Step-by-Step Instructions
-------------------------

1. **Create a virutal environment**: Start by creating a virtual environment and activating it.

.. code-block:: bash

python3 -m venv fedn-env
source fedn-env/bin/activate


2. **Install FEDn**: Install the FEDn package by running the following command:

.. code-block:: bash

pip install fedn

3. **Create your client**: Copy and paste the code Below into a new file called `client_example.py`.

.. code-block:: python

import argparse

from fedn.network.clients.fedn_client import FednClient, ConnectToApiResult


def get_api_url(api_url: str, api_port: int):
url = f"{api_url}:{api_port}" if api_port else api_url
if not url.endswith("/"):
url += "/"
return url

def on_train(in_model):
training_metadata = {
"num_examples": 1,
"batch_size": 1,
"epochs": 1,
"lr": 1,
}

metadata = {"training_metadata": training_metadata}

# Do your training here, out_model is your result...
out_model = in_model
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is better to give a minimal, but working example. This could be confusing.


return out_model, metadata

def on_validate(in_model):
# Calculate metrics here...
metrics = {
"test_accuracy": 0.9,
"test_loss": 0.1,
"train_accuracy": 0.8,
"train_loss": 0.2,
}
return metrics

def on_predict(in_model):
# Do your prediction here...
prediction = {
"prediction": 1,
"confidence": 0.9,
}
return prediction


def main(api_url: str, api_port: int, token: str = None):
fedn_client = FednClient(train_callback=on_train, validate_callback=on_validate, predict_callback=on_predict)

url = get_api_url(api_url, api_port)

name = input("Enter Client Name: ")
fedn_client.set_name(name)

client_id = str(uuid.uuid4())
fedn_client.set_client_id(client_id)

controller_config = {
"name": name,
"client_id": client_id,
"package": "local",
"preferred_combiner": "",
}

result, combiner_config = fedn_client.connect_to_api(url, token, controller_config)

if result != ConnectToApiResult.Assigned:
print("Failed to connect to API, exiting.")
return

result: bool = fedn_client.init_grpchandler(config=combiner_config, client_name=client_id, token=token)

if not result:
return

fedn_client.run()

if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Client Example")
parser.add_argument("--api-url", type=str, required=True, help="The API URL")
parser.add_argument("--api-port", type=int, required=False, help="The API Port")
parser.add_argument("--token", type=str, required=False, help="The API Token")

args = parser.parse_args()
main(args.api_url, args.api_port, args.token)


4. **Run the client**: Run the client by executing the following command:

.. code-block:: bash

python client_example.py --api-url <full-api-url> --token <api-token>

Replace `<api-url>` and `<api-token>` with the URL and token of the FEDn API. *Example when running a local FEDn instance: python client_example.py --api-url http://localhost:8092*

5. **Start training**: Create a session and start training by using either the FEDn CLI or the FEDn UI.
Loading
Loading