About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Last updated: Oct 03, 2024
Import a JSON file to create and configure all deployments of your asset for model evaluations. Export the configuration file to configure other assets and their deployments.
Note:
You can also import and export configurations for model evaluations by using the import subscription and export subscription API methods.
Load the JSON file content as a Python dictionary
For this example, the file
defines configuration data for a model that predicts cancer type.sagemaker_native_multiclass_breast-cancer_all_monitors_sub_configuration.json
To load the file in Python, run the following command:
configuration_file_path = 'sagemaker_native_multiclass_breast-cancer_all_monitors_sub_configuration.json'
with open(configuration_file_path, 'r') as fp:
subscription_configuration = json.load(fp)
The file contains configuration data. Refer to the following example. See the notebook for a complete example of the configuration content.
{'asset': {'asset_id': '0530ab0cd4f4dd5486b19c08df8b6914',
'asset_type': 'model',
'created_at': '2018-10-10T14:31:44.348Z',
'name': 'DEMO-multi-classification-2018-10-10-14-26-26',
'url': 's3://sagemaker-us-east-1-014862798213/sagemaker/DEMO-breast-cancer-prediction/DEMO-multi-classification-2018-10-10-14-26-26/output/model.tar.gz'},
'asset_properties': {'categorical_fields': [],
'feature_fields': ['radius_mean',
'texture_mean',
. . .
'input_data_schema': {'fields': [{'metadata': {'modeling_role': 'feature'},
'name': 'radius_mean',
'nullable': True,
'type': 'double'},
{'metadata': {'modeling_role': 'feature'},
'name': 'texture_mean',
'nullable': True,
'type': 'double'},
. . .
'input_data_type': 'structured',
'label_column': 'diagnosis',
'output_data_schema': {'fields': [{'metadata': {'modeling_role': 'feature'},
'name': 'radius_mean',
'nullable': True,
'type': 'double'},
{'metadata': {'modeling_role': 'feature'},
'name': 'texture_mean',
'nullable': True,
'type': 'double'},
. . .
'prediction_field': 'predicted_label',
'prediction_probability_field': 'score',
'problem_type': 'multiclass',
'training_data_schema': {'fields': [{'metadata': {'modeling_role': 'feature'},
'name': 'radius_mean',
'nullable': True,
'type': 'double'},
{'metadata': {'modeling_role': 'feature'},
'name': 'texture_mean',
'nullable': True,
'type': 'double'},
. . .
'configurations': {'explainability': {'training_statistics': {'base_values': {'0': 13.37,
'1': 18.84,
'10': 0.3242,
. . .
'fairness_monitoring': {'class_label': 'predicted_label',
'distributions': [{'attribute': 'radius_mean',
'class_labels': [{'counts': [{'class_value': 'B', 'count': 1}],
'label': '[6.8, 7.2]'},
{'counts': [{'class_value': 'B', 'count': 3}], 'label': '[7.6, 8.0]'},
{'counts': [{'class_value': 'B', 'count': 2}], 'label': '[8.0, 8.4]'},
. . .
'favourable_class': ['M'],
'features': [{'feature': 'radius_mean',
'majority': [[0.0, 10.0], [19.0, 20.0]],
'minority': [[15.0, 16.0]],
'threshold': 0.8,
'type': 'float'}],
'min_records': 5,
'perform_debias': True,
'run_status': 'INITIATED',
'training_data_class_label': None,
'unfavourable_class': ['B']},
'payload_logging': {'dynamic_schema_update': True,
'output_data_schema': {'fields': [{'metadata': {'modeling_role': 'feature'},
'name': 'radius_mean',
'nullable': True,
'type': 'double'},
{'metadata': {'modeling_role': 'feature'},
'name': 'texture_mean',
'nullable': True,
'type': 'double'},
. . .
'performance_monitoring': {},
'quality_monitoring': {'evaluation_definition': {'method': 'multiclass',
'threshold': 0.8},
'min_feedback_data_size': 5,
'scheduleId': '63c7f400-aa29-4539-91ad-8a4b9d2b9a51'}},
'deployments': [{'created_at': '2018-10-10T14:39:21.421Z',
'deployment_id': '37a83f399e6dc3b9d08d7d01fe690665',
'deployment_rn': 'arn:aws:sagemaker:us-east-1:014862798213:endpoint/demo-multi-classification-endpoint-201810101439',
'deployment_type': 'online',
'name': 'DEMO-multi-classification-endpoint-201810101439',
'scoring_endpoint': {'request_headers': {'Content-Type': 'application/json'},
'url': 'DEMO-multi-classification-endpoint-201810101439'},
'url': 'DEMO-multi-classification-endpoint-201810101439'}],
'export_info': {'api_version': 'v1',
'origin': '/v1/data_marts/b73545e6-0a6e-466c-8cd0-c47c044c5702/service_bindings/bf44cc7f-990d-4942-bfc6-cbcf71a1b78c/subscriptions/0530ab0cd4f4dd5486b19c08df8b6914',
'timestamp': '2019-02-11T11:41:01.613Z'}}
Import from the configuration file
Now, run the call to add and configure the asset deployment for the sample breast cancer prediction model deployment.
subscription = client.data_mart.subscriptions.import_configuration(binding_uid=binding_uid, configuration_data=subscription_configuration)
The
parameter is optional if only one ML engine is bound.binding_uid
Exporting to the configuration file
You can also export the configuration file as JSON:
exported_configuration = client.data_mart.subscriptions.export_configuration(binding_uid=binding_uid, subscription_uid=subscription.uid)
Results
The asset deployment is created and ready to use for model evaluations.
Parent topic: Preparing to evaluate a model
Was the topic helpful?
0/1000