Docs

Additional details for Federated Learning implementation

Last updated: May 18, 2022
Additional details for Federated Learning implementation
Additional details for Federated Learning implementation

This document provides more examples of Federated Learning configuration that require extensive code implementation.

Party yml configuration

In addition to the party connector script, you can also download a yml configuration file to connect each of the remote training parties to the aggregator as an alternative to the party connector script.

  1. Modify the yml file as follows:
     aggregator:
     ip: [CPD_HOSTNAME]/ml/v4/trainings/[TRAINING_ID]
 connection:
     info:
     id: [REMOTE_TRAINING_SYSTEM_ID]
     # Supply the name of the data handler class and path to it.
     # The info section may be used to pass information to the 
     # data handler.
     # For example,
     #     "data": {    
     #         "name": "MnistSklearnDataHandler",
     #         "path": "example.mnist_sklearn_data_handler"
     #         "info": {
     #             "npz_file":"./example_data/example_data.npz"
     #         },
     #     },
     "data": {        
         "name": "<data handler>",
         "path": "<path to data handler>",
         "info": {
             <information to pass to data handler>
             # For example:
             # "train_file": "./mnist-keras-train.pkl",
             # "test_file": "./mnist-keras-test.pkl"
         },
     },
 local_training:
     name: LocalTrainingHandler
     path: ibmfl.party.training.local_training_handler
 protocol_handler:
     name: PartyProtocolHandler
     path: ibmfl.party.party_protocol_handler
  2. Have each party run the yml file with this command:
    python -m ibmfl.party.party <config file> <Bearer token> <log_level>. Note: When the party runs the yml file from the CLI, the -s flag needs to be passed, like this command: python -m ibmfl.party.party -s <config file> <Bearer token> <log_level> where <config file> refers to the party yml file path. More information on getting the bearer token and log level.

Return a data generator defined by Keras or Tensorflow 2

The following is a code example that needs to be included as part of the get_data function in the data handler class to return data in the form of a data generator defined by Keras or Tensorflow 2: 

train_gen = ImageDataGenerator(rotation_range=8,
                                width_sht_range=0.08,
                                shear_range=0.3,
                                height_shift_range=0.08,
                                zoom_range=0.08)

train_datagenerator = train_gen.flow(
    x_train, y_train, batch_size=64)

return train_datagenerator

Saving the Scikit-learn model

If you chose Scikit-learn (SKLearn) as the model framework, you need to configure your settings to save the model trained in Federated Learning as a pickle file. Specify your model by the following code example of methods to implement as part of your model file, which depends on the model type that you select for SKLearn.

SKLearn classification

# SKLearn classification
# Specify your model. Users need to provide the classes used in classification problems.
# In the example, there are 10 classes.

from sklearn.linear_model import SGDClassifier
import numpy as np
import joblib

model = SGDClassifier(loss='log', penalty='l2')
model.classes_ = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

joblib.dump(model, "./model_architecture.pickle")

SKLearn regression

# Sklearn regression

from sklearn.linear_model import SGDRegressor
import pickle


model = SGDRegressor(loss='huber', penalty='l2')

with open("./model_architecture.pickle", 'wb') as f:
    pickle.dump(model, f)

SKLearn Kmeans

# SKLearn Kmeans
from sklearn.cluster import KMeans
import joblib

model = KMeans()
joblib.dump(model, "./model_architecture.pickle")

You will need to create a zip file containing your model in pickle format by running the command zip mymodel.zip model_architecture.pickle. The contents of your zip file should contain:

mymodel.zip
└── model_architecture.pickle

Saving the Tensorflow 2 model

If you chose Tensorflow 2 as the model framework, you need to save a Keras model using the SavedModel format. A Keras model can be saved in SavedModel format using tf.keras.model.save().

import tensorflow as tf
from tensorflow.keras import *
from tensorflow.keras.layers import *
import numpy as np
import os

class MyModel(Model):
    def __init__(self):
        super(MyModel, self).__init__()
        self.conv1 = Conv2D(32, 3, activation='relu')
        self.flatten = Flatten()
        self.d1 = Dense(128, activation='relu')
        self.d2 = Dense(10)

    def call(self, x):
        x = self.conv1(x)
        x = self.flatten(x)
        x = self.d1(x)
        return self.d2(x)

# Create an instance of the model

model = MyModel()
loss_object = tf.keras.losses.SparseCategoricalCrossentropy(
    from_logits=True)
optimizer = tf.keras.optimizers.Adam()
acc = tf.keras.metrics.SparseCategoricalAccuracy(name='accuracy')
model.compile(optimizer=optimizer, loss=loss_object, metrics=[acc])
img_rows, img_cols = 28, 28
input_shape = (None, img_rows, img_cols, 1)
model.compute_output_shape(input_shape=input_shape)

dir = "./model_architecture"
if not os.path.exists(dir):
    os.makedirs(dir)

model.save(dir)

You will need to create a zip file containing your model in SavedModel format by running the command zip -r mymodel.zip model_architecture. The contents of your zip file should resemble:


mymodel.zip
└── model_architecture
    ├── assets
    ├── keras_metadata.pb
    ├── saved_model.pb
    └── variables
        ├── variables.data-00000-of-00001
        └── variables.index

Saving the PyTorch model

If you chose PyTorch as the model framework, use torch.save() to save the model. The PyTorch model is a zip file containing the pickled model, but it needs to be zipped once again for Federated Learning.

import torch
import torch.nn as nn

model = nn.Sequential(
    nn.Flatten(start_dim=1, end_dim=-1),
    nn.Linear(in_features=784, out_features=256, bias=True),
    nn.ReLU(),
    nn.Linear(in_features=256, out_features=256, bias=True),
    nn.ReLU(),
    nn.Linear(in_features=256, out_features=256, bias=True),
    nn.ReLU(),
    nn.Linear(in_features=256, out_features=100, bias=True),
    nn.ReLU(),
    nn.Linear(in_features=100, out_features=50, bias=True),
    nn.ReLU(),
    nn.Linear(in_features=50, out_features=10, bias=True),
    nn.LogSoftmax(dim=1),
).double()

torch.save(model, "./model_architecture.pt")

You will need to create a zip file containing your model by running the command zip mymodel.zip model_architecture.pt. The contents of your zip file should contain:

mymodel.zip
└── model_architecture.pt



Parent topic: Federated learning