If you have this notebook as a local copy on your platform, it may become outdated. Download the latest version of this notebook or download the latest version of the project.

Part 1 - WML Federated Learning with MNIST for Admin

With IBM Federated Learning, you can combine data from multiple sources to train a model from the collective data without having to actually share them. This allows enterprises to train data with other companies without delegating resources for security. Another advantage is the remote data does not have to be centralized in one location, eliminates the needs to move potentially large datasets. This notebook demonstrates how to start Federated Learning with the Python client. For more details setting up Federated Learning, terminology, and running Federated Learning from the UI, see Federated Learning documentation.

Learning Goals

When you complete the Part 1 - WML Federated Learning with MNIST for Admin notebook, you should know how to:

Once you complete this notebook, please open Part 2 - WML Federated Learning with MNIST for Party.

This notebook is intended to be run by the administrator of the Federated Learning experiment.

Table of Contents

1. Prequisites

Before you proceed, you need to have:

1.1 Define variables

1.2 Define tags

Used to identify the assets created by this notebook

1.3 Import libraries

2. Obtain Cloud authentication token

3. Load the model

You need an untrained model asset for Federated Learning to work with. In this tutorial, an untrained Tensorflow 2 Keras model is provided for you. Federated Learning supports Scikit-learn and Tensorflow 2, which are free machine learning packages with tutorials. Additionally IBM docs provide some details on how to configure an untrained model for Federated Learning. See:

3.1 Create Untrained Model Asset

Creates an untrained model asset in your project.

3.2 Upload Initial Model

You need to upload the initial model into your project. In this tutorial, an untrained model example is provided and and downloaded from GitHub.

4. Create Remote Training System Asset

Now you will learn to create a Remote Training System (RTS). An RTS handles receiving your multiple parties' call to the aggregator to run the training.

5. Create FL Training Job

In this section, you will launch the Federated Learning experiment.

5.1 Get Training Job Status

Before you run the following code, please make your that your project is associated with a Watson Machine Learning service. For more details on associating services, please see: Associating services

6. Get Variables And Paste Into Party Notebook

Run the following cell and copy the output.

As the Admin, you have now launched a Federated Learning experiment. Copy the output from the previous cell. Open Part 2 - WML Federated Learning with MNIST for Party and paste the output into the first code cell. Run the Part 2 - Party notebook to the end.

7. Save Trained Model To Project

Once training has completed, run the cells below to save the trained model into your project.

7.1 Connection to COS

This information is located in your Watson Studio project, under the Manage tab, on the General page.

  1. The bucket name is listed inside the Storage pane.
  2. To obtain the credentials click on the Manage in IBM Cloud link located inside the Storage pane. From your COS instance click Service Credentials. You can use an existing or create a new credential if needed.
  1. The COS endpoints are listed in your COS instance under Endpoints.

7.2 Install pre-req

7.3 Save model to project

8. Clean Up Project

Use this section to delete the training jobs and assets created by this notebook.

8.1 List all training jobs in project

8.2 Delete all training jobs in this project created by this notebook

This will stop all running aggregators created using this notebook.

8.3 List all remote training systems in project

8.4 Delete all remote training systems in this project created by this notebook

8.5 List all models in project

8.6 Delete all untrained models in this project created by this notebook

As the Admin, you have now launched a Federated Learning experiment. Copy the output from the previous cell. Open Part 2 - WML Federated Learning with MNIST for Party and paste the output into the first code cell.


Copyright © 2020 IBM. This notebook and its source code are released under the terms of the MIT License. 


Love this notebook? Don't have an account yet?
Share it with your colleagues and help them discover the power of Watson Studio! Sign Up