AutoAI libraries for Python
The autoai-libautoai-libautoai-ts-libs
Installing autoai-lib or autoai-ts-libs for Python
Follow the instructions in Installing custom libraries to install autoai-libautoai-ts-libs
Using autoai-lib and autoai-ts-libs for Python
The autoai-libautoai-ts-libsautoai-libautoai-ts-libs
Installing autoai-lib and autoai-ts-libs for Python
Follow the instructions in Installing custom libraries to install autoai-lib and autoai-ts-libs.
Configuring library-level logging
You can configure the library logger autoai_libs
If you do not update the configuration you will get the default behavior for logging. For example, messages of severity warning and higher (that is warning, error, and critical) will be printed to stderr
The autoai-lib functions
The instantiated project object that is created after you import the autoai-lib
autoai_libs.transformers.exportable.NumpyColumnSelector()
Selects a subset of columns of a numpy array
Usage:
autoai_libs.transformers.exportable.NumpyColumnSelector(columns=None)
| Option | Description |
|---|---|
| columns | list of column indexes to select |
autoai_libs.transformers.exportable.CompressStrings()
Removes spaces and special characters from string columns of an input numpy array X.
Usage:
autoai_libs.transformers.exportable.CompressStrings(compress_type='string', dtypes_list=None, misslist_list=None, missing_values_reference_list=None, activate_flag=True)
| Option | Description |
|---|---|
|
type of string compression. 'string' for removing spaces from a string and 'hash' for creating an int hash. Default is 'string'. 'hash' is used for columns with strings and cat_imp_strategy='most_frequent' |
|
list containing strings that denote the type of each column of the input numpy array X (strings are among 'char_str','int_str','float_str','float_num', 'float_int_num','int_num','Boolean','Unknown'). If None, the column types are discovered. Default is None. |
|
list contains lists of missing values of each column of the input numpy array X. If None, the missing values of each column are discovered. Default is None. |
|
reference list of missing values in the input numpy array X |
|
flag that indicates that this transformer is active. If False, transform(X) outputs the input numpy array X unmodified. |
autoai_libs.transformers.exportable.NumpyReplaceMissingValues()
Given a numpy array and a reference list of missing values for it, replaces missing values with a special value (typically a special missing value such as np.nan).
Usage:
autoai_libs.transformers.exportable.NumpyReplaceMissingValues(missing_values, filling_values=np.nan)
| Option | Description |
|---|---|
|
reference list of missing values |
|
special value that is assigned to unknown values |
autoai_libs.transformers.exportable.NumpyReplaceUnknownValues()
Given a numpy array and a reference list of known values for each column, replaces values that are not part of a reference list with a special value (typically np.nan). This method is typically used to remove labels for columns in a test data set that has not been seen in the corresponding columns of the training data set.
Usage:
autoai_libs.transformers.exportable.NumpyReplaceUnknownValues(known_values_list=None, filling_values=None, missing_values_reference_list=None)
| Option | Description |
|---|---|
|
reference list of lists of known values for each column |
|
special value that is assigned to unknown values |
|
reference list of missing values |
autoai_libs.transformers.exportable.boolean2float()
Converts a NumPy boolean array to floats, replacing missing values with ones. Also changes type of array from 'object' to 'float'.
Usage:
autoai_libs.transformers.exportable.boolean2float(activate_flag=True)
| Option | Description |
|---|---|
|
flag that indicates that this transformer is active. If False, transform(X) outputs the input numpy array X unmodified. |
autoai_libs.transformers.exportable.CatImputer()
This transformer is a wrapper for categorical imputer. Internally it currently uses sklearn SimpleImputer
Usage:
autoai_libs.transformers.exportable.CatImputer(strategy, missing_values, sklearn_version_family=global_sklearn_version_family, activate_flag=True)
| Option | Description |
|---|---|
|
string, optional, default=”mean”. The imputation strategy for missing values. - - - - |
|
number, string, np.nan (default) or None. The placeholder for the missing values. All occurrences of missing_values are imputed. |
|
str indicating the sklearn version for backward compatibiity with versions 019, and 020dev. Currently unused. Default is None. |
|
flag that indicates that this transformer is active. If False, transform(X) outputs the input numpy array X unmodified. |
autoai_libs.transformers.exportable.CatEncoder()
This method is a wrapper for categorical encoder. If encoding parameter is 'ordinal', internally it currently uses sklearn OrdinalEncoder. If encoding parameter is 'onehot', or 'onehot-dense' internally it uses sklearn OneHotEncoder
Usage:
autoai_libs.transformers.exportable.CatEncoder(encoding, categories, dtype, handle_unknown, sklearn_version_family=global_sklearn_version_family, activate_flag=True)
| Option | Description |
|---|---|
|
str, 'onehot', 'onehot-dense' or 'ordinal'. The type of encoding to use (default is 'ordinal') 'onehot': encode the features by using a one-hot aka one-of-K scheme (or also called 'dummy' encoding). This encoding creates a binary column for each category and returns a sparse matrix. 'onehot-dense': the same as 'onehot' but returns a dense array instead of a sparse matrix. 'ordinal': encode the features as ordinal integers. The result is a single column of integers (0 to n_categories - 1) per feature. |
|
'auto' or a list of lists/arrays of values. Categories (unique values) per feature: 'auto' : Determine categories automatically from the training data. |
|
number type, default np.float64 Desired dtype of output. |
|
'error' (default) or 'ignore'. Whether to raise an error or ignore if a unknown categorical feature is present during transform (default is to raise). When this parameter is set to 'ignore' and an unknown category is encountered during
transform, the resulting one-hot encoded columns for this feature are all zeros. In the inverse transform, an unknown category are denoted as None. Ignoring unknown categories is not supported for |
|
str indicating the sklearn version for backward compatibiity with versions 019, and 020dev. Currently unused. Default is None. |
|
flag that indicates that this transformer are active. If False, transform(X) outputs the input numpy array X unmodified. |
autoai_libs.transformers.exportable.float32_transform()
Transforms a float64 numpy array to float32.
Usage:
autoai_libs.transformers.exportable.float32_transform(activate_flag=True)
| Option | Description |
|---|---|
|
flag that indicates that this transformer is active. If False, transform(X) outputs the input numpy array X unmodified. |
autoai_libs.transformers.exportable.FloatStr2Float()
Given numpy array X and dtypes_list that denotes the types of its columns, it replaces columns of strings that represent floats (type 'float_str' in dtypes_list) to columns of floats and replaces their missing values with np.nan.
Usage:
autoai_libs.transformers.exportable.FloatStr2Float(dtypes_list, missing_values_reference_list=None, activate_flag=True)
| Option | Description |
|---|---|
|
list contains strings that denote the type of each column of the input numpy array X (strings are among 'char_str','int_str','float_str','float_num', 'float_int_num','int_num','Boolean','Unknown'). |
|
reference list of missing values |
|
flag that indicates that this transformer is active. If False, transform(X) outputs the input numpy array X unmodified. |
autoai_libs.transformers.exportable.NumImputer()
This method is a wrapper for numerical imputer.
Usage:
autoai_libs.transformers.exportable.NumImputer(strategy, missing_values, activate_flag=True)
| Option | Description |
|---|---|
|
num_imp_strategy: string, optional (default=”mean”). The imputation strategy: - If “mean”, then replace missing values by using the mean along the axis. - If “median”, then replace missing values by using the median along the axis. - If “most_frequent”, then replace missing by using the most frequent value along the axis. |
|
integer or “NaN”, optional (default=”NaN”). The placeholder for the missing values. All occurrences of missing_values are imputed: - For missing values encoded as np.nan, use the string value “NaN”. - |
autoai_libs.transformers.exportable.OptStandardScaler()
This parameter is a wrapper for scaling of numerical variables. It currently uses sklearn StandardScaler internally.
Usage:
autoai_libs.transformers.exportable.OptStandardScaler(use_scaler_flag=True, num_scaler_copy=True, num_scaler_with_mean=True, num_scaler_with_std=True)
| Option | Description |
|---|---|
|
Boolean, optional, default True. If False, try to avoid a copy and do in-place scaling instead. This action is not guaranteed to always work. With in-place, for example, if the data is not a NumPy array or scipy.sparse CSR matrix, a copy might still be returned. |
|
Boolean, True by default. If True, center the data before scaling. An exception is raised when attempted on sparse matrices because centering them entails building a dense matrix, which in common use cases is likely to be too large to fit in memory. |
|
Boolean, True by default. If True, scale the data to unit variance (or equivalently, unit standard deviation). |
|
Boolean, flag that indicates that this transformer is active. If False, transform(X) outputs the input numpy array X unmodified. Default is True. |
autoai_libs.transformers.exportable.NumpyPermuteArray()
Rearranges columns or rows of a numpy array based on a list of indexes.
Usage:
autoai_libs.transformers.exportable.NumpyPermuteArray(permutation_indices=None, axis=None)
| Option | Description |
|---|---|
|
list of indexes based on which columns are rearranged |
|
0 permute along columns. 1 permute along rows. |
Feature transformation
These methods apply to the feature transformations described in AutoAI implementation details.
autoai_libs.cognito.transforms.transform_utils.TA1(fun, name=None, datatypes=None, feat_constraints=None, tgraph=None, apply_all=True, col_names=None, col_dtypes=None)
For unary stateless functions, such as square or log, use TA1.
Usage:
autoai_libs.cognito.transforms.transform_utils.TA1(fun, name=None, datatypes=None, feat_constraints=None, tgraph=None, apply_all=True, col_names=None, col_dtypes=None)
| Option | Description |
|---|---|
|
the function pointer |
|
a string name that uniquely identifies this transformer from others |
|
a list of datatypes either of which are valid input to the transformer function (numeric, float, int, and so on) |
|
all constraints, which must be satisfied by a column to be considered a valid input to this transform |
|
tgraph object must be the starting TGraph( ) object. This parameter is optional and you can pass None, but that can result in some failure to detect some inefficiencies due to lack of caching |
|
only use applyAll = True. It means that the transformer enumerates all features (or feature sets) that match the specified criteria and apply the provided function to each. |
|
names of the feature columns in a list |
|
list of the datatypes of the feature columns |
autoai_libs.cognito.transforms.transform_utils.TA2()
For binary stateless functions, such as sum, product, use TA2.
Usage:
autoai_libs.cognito.transforms.transform_utils.TA2(fun, name, datatypes1, feat_constraints1, datatypes2, feat_constraints2, tgraph=None, apply_all=True, col_names=None, col_dtypes=None)
| Option | Description |
|---|---|
|
the function pointer |
|
|
|
a list of datatypes either of which are valid inputs (first parameter) to the transformer function (numeric, float, int, and so on) |
|
all constraints, which must be satisfied by a column to be considered a valid input (first parameter) to this transform |
|
a list of data types either of which are valid inputs (second parameter) to the transformer function (numeric, float, int, and so on) |
|
all constraints, which must be satisfied by a column to be considered a valid input (second parameter) to this transform |
|
tgraph object must be the invoking TGraph( ) object. Note this parameter is optional and you can pass None, but that results in some missing inefficiencies due to lack of caching |
|
only use applyAll = True. It means that the transformer enumerates all features (or feature sets) that match the specified criteria and apply the provided function to each. |
|
names of the feature columns in a list |
|
list of the data types of the feature columns |
autoai_libs.cognito.transforms.transform_utils.TB1()
For unary state-based transformations (with fit/transform) use, such as frequent count.
Usage:
autoai_libs.cognito.transforms.transform_utils.TB1(tans_class, name, datatypes, feat_constraints, tgraph=None, apply_all=True, col_names=None, col_dtypes=None)
| Option | Description |
|---|---|
|
a class that implements |
|
a string name that uniquely identifies this transformer from others |
|
list of datatypes either of which are valid input to the transformer function (numeric, float, int, and so on) |
|
all constraints, which must be satisfied by a column to be considered a valid input to this transform |
|
tgraph object must be the invoking TGraph( ) object. Note that this is optional and you might pass None, but that results in some missing inefficiencies due to lack of caching |
|
only use applyAll = True. It means that the transformer enumerates all features (or feature sets) that match the specified criteria and apply the provided function to each. |
|
names of the feature columns in a list. |
|
list of the data types of the feature columns. |
autoai_libs.cognito.transforms.transform_utils.TB2()
For binary state-based transformations (with fit/transform) use, such as group-by.
Usage:
autoai_libs.cognito.transforms.transform_utils.TB2(tans_class, name, datatypes1, feat_constraints1, datatypes2, feat_constraints2, tgraph=None, apply_all=True)
| Option | Description |
|---|---|
|
a class that implements fit( ) and transform( ) in accordance with the transformation function definition |
|
a string name that uniquely identifies this transformer from others |
|
a list of data types either of which are valid inputs (first parameter) to the transformer function (numeric, float, int, and so on) |
|
all constraints, which must be satisfied by a column to be considered a valid input (first parameter) to this transform |
|
a list of data types either of which are valid inputs (second parameter) to the transformer function (numeric, float, int, and so on) |
|
all constraints, which must be satisfied by a column to be considered a valid input (second parameter) to this transform |
|
tgraph object must be the invoking TGraph( ) object. This parameter is optional and you might pass None, but that results in some missing inefficiencies due to lack of caching |
|
only use applyAll = True. It means that the transformer enumerates all features (or feature sets) that match the specified criteria and apply the provided function to each. |
autoai_libs.cognito.transforms.transform_utils.TAM()
For a transform that applies at the data level, such as PCA, use TAM.
Usage:
autoai_libs.cognito.transforms.transform_utils.TAM(tans_class, name, tgraph=None, apply_all=True, col_names=None, col_dtypes=None)
| Option | Description |
|---|---|
|
a class that implements |
|
a string name that uniquely identifies this transformer from others |
|
tgraph object must be the invoking TGraph( ) object. This parameter is optional and you can pass None, but that results in some missing inefficiencies due to lack of caching |
|
only use applyAll = True. It means that the transformer enumerates all features (or feature sets) that match the specified criteria and apply the provided function to each. |
|
names of the feature columns in a list |
|
list of the datatypes of the feature columns |
autoai_libs.cognito.transforms.transform_utils.TGen()
TGen is a general wrapper and can be used for most functions (might not be most efficient though).
Usage:
autoai_libs.cognito.transforms.transform_utils.TGen(fun, name, arg_count, datatypes_list, feat_constraints_list, tgraph=None, apply_all=True, col_names=None, col_dtypes=None)
| Option | Description |
|---|---|
|
the function pointer |
|
a string name that uniquely identifies this transformer from others |
|
number of inputs to the function, in this example it is 1, for binary, it is 2, and so on |
|
a list of arg_count lists that correspond to the acceptable input data types for each parameter. In the previous example, since `arg_count=1``, the result is one list within the outer list, and it contains a single type called 'numeric'. In another case, it might be a specific case 'int' or even more specific 'int64'. |
|
a list of arg_count lists that correspond to some constraints that can be imposed on selection of the input features |
|
tgraph object must be the invoking TGraph( ) object. Note this parameter is optional and you can pass None, but that results in some missing inefficiencies due to lack of caching |
|
only use applyAll = True. It means that the transformer enumerates all features (or feature sets) that match the specified criteria and apply the provided function to each. |
|
names of the feature columns in a list |
|
list of the data types of the feature columns |
autoai_libs.cognito.transforms.transform_utils.FS1()
Feature selection, type 1 (using pairwise correlation between each feature and target.)
Usage:
autoai_libs.cognito.transforms.transform_utils.FS1(cols_ids_must_keep, additional_col_count_to_keep, ptype)
| Option | Description |
|---|---|
|
serial numbers of the columns that must be kept irrespective of their feature importance |
|
how many columns need to be retained |
|
classification or regression |
autoai_libs.cognito.transforms.transform_utils.FS2()
Feature selection, type 2.
Usage:
autoai_libs.cognito.transforms.transform_utils.FS2(cols_ids_must_keep, additional_col_count_to_keep, ptype, eval_algo)
| Option | Description |
|---|---|
|
serial numbers of the columns that must be kept irrespective of their feature importance |
|
how many columns need to be retained |
|
classification or regression |
The autoai-ts-libs functions
The combination of transformers and estimators are designed and chosen for each pipeline by the AutoAI Time Series system. Changing the transformers or the estimators in the generated pipeline notebook can cause unexpected results or even failure.
We do not recommend you change the notebook for generated pipelines, thus we do not currently offer the specification of the functions for the autoai-ts-libs
Learn more
Parent topic: Saving an AutoAI generated notebook