Bruce Wu's Blog

Greater model parameters and network design are always preferred for better performance during model training. Manually tuning these parameters and architecture can be difficult and time-consuming, so in this post, I’ll attempt to use an AutoML tool called Autokeras to accelerate this process.

AutoKeras

AutoKeras is an open-source AutoML tool based on Keras that automates designing and tuning machine learning models. It allows users to quickly and easily train high-performance models without extensive expertise in machine learning. By using Autokeras, we can save time and effort compared to manual tuning and train high-quality models for a variety of tasks.

AutoKeras uses Efficient Neural Architecture Search (ENAS) to automate the process of designing and tuning machine learning models. It also includes support for a variety of preprocessing methods, such as normalization, vectorization, and encoding. AutoKeras can be used for a wide range of tasks, including structured data, image data, time series prediction, and multitasking.

auto_keras Source: AutoKeras

The official website of AutoKeras provides examples that show how to use the tool in just a few lines of code.

import autokeras as ak

# Initialize the image classifier.
clf = ak.ImageClassifier()

# Feed the image classifier with training data.
clf.fit(x_train, y_train)

# Predict with the trained model.
predicted_y = clf.predict(x_test)

Environment

With Colab Pro, some Python modules and GPU are already installed and configured in Colab so that we can focus on coding.

colab_pro

Example 1 - Fraud detection

The first case is a credit card fraud detection on Kaggle, which is a structured data classification problem

dataset: Kaggle-Data-Credit-Card-Fraud-Detection (284807 rows × 31 columns)

fraud_data

The StructuredDataClassifier is used here to handle the structured data classification problem.

# StructuredDataClassifier
cbs = [
    # Switch to the next model after three consecutive training cycles without improvement
    # monitor default: val_loss
    tf.keras.callbacks.EarlyStopping(patience=3,monitor='val_loss'), 
    # histogram_freq=1 means to count once in each training cycle, and display the weight and bias distribution of each layer of neurons during training in a histogram
    tf.keras.callbacks.TensorBoard(log_dir=logdir, histogram_freq=1)
]
# max_trials is the number of models to try, the default value is 100, you can find the structure of the default model in oracle.json in the directory
clf=ak.StructuredDataClassifier(max_trials=3)
# The default value of epoch is 1000, and it may end early when using earlystopping
history=clf.fit(x_train,y_train,callbacks=cbs,epochs=25)

In addition to directly using StructuredDataClassifier, if you need to customize the model architecture, search space, etc., you can also use AutoModel to further design the model, as shown below.

input_node = ak.StructuredDataInput()
output_node = ak.StructuredDataBlock()(input_node)
output_node = ak.ClassificationHead()(output_node)
clf = ak.AutoModel(inputs=input_node, outputs=output_node, max_trials=3)

Save and load models:

# Save model
try:
  # TensorFlow SavedModel format
  model.save('model_autokeras',save_format='tf')
except:
  # keras h5 format
  model.save('model_autokeras.h5')

# Load AutoKeras model
from tensorflow.keras.models import load_model
loaded_model = load_model('./auto_model/best_model', 
custom_objects=ak.CUSTOM_OBJECTS)

Training process (Colab) fraud_trainjpg

If the training is interrupted and executed again, it will continue the training based on the records
After the training is completed, AutoKeras will use all training set + validation set for training again with the best model.
After all trials are completed, the model with the best performance will be automatically stored in best_model/*
Due to the problem of category imbalance in this data set, Autokeras also supports setting class_weight in fit() for weighting the loss function, so that the model will pay more attention to minority categories.

Training Visualization - TensorBoard

To understand the training process, you can use tensorboard to read the log data stored in callback to visualize the training process; use the tensorboard.dev to generate a public page for sharing with others.

import tensorflow as tf
import datetime
from tensorboard.plugins.hparams import api as hp

!tensorboard dev upload --logdir logs \
  --name "Simple experiment" \
  --description "fraud detect" \

Execute the above program fragment and check “Reload data” before training, and the data will be updated during training.

Accuracy and loss changes during training tbdev

Model architecture tbdev2

ClearML

If you want to monitor and track the results and share the experiments with people, you can also use the MLOps platform ClearML that supports AutoKeras to present it on an interactive web page.

from clearML import Task
# register at https://app.community.clear.ml
# get the 'key' and 'secret'：
Task.set_credentials(
    api_host='https://api.community.clear.ml',
    web_host='https://app.community.clear.ml',
    files_host='https://files.community.clear.ml',
    key='xxxxx',
    secret='xxxxxx'
)
# Create a new ClearML task
# Visit https://app.community.clear.ml/dashboard
task = Task.init(project_name='autokeras age', task_name='age regressor')

The project overview is on the ClearML main page clearml0

In addition to the loss and accuracy during training, it also includes the machine’s status. clearml2 clearml

Changes in weight and bias in the model clearml3

ClearML will also show the results through the library matplotlib and seaborn

Example 2 - Age prediction

This example is about age prediction, which is a regression problem to predict age from photos.

(Dataset: wiki_crop)

After image filtering and pre-processing, the images were converted to 52984 128*128 images.

Training data: age_train

Use AutoModel to customize model details, which is similar to Keras functional API. AutoKeras then finds the best result from the conditions we give.

input_node = ak.ImageInput()
# If just want to try EfficientNet b0~b7
output_node = ak.ImageBlock(block_type='efficient', normalize=True, augment=True)(input_node)
output_node = ak.RegressionHead(droupout=0.25)(output_node)
reg = ak.AutoModel(inputs=input_node,outputs=output_node, max_trials=3)

Testing results after a few epochs of simple training age_result

AutoKeras also supports dealing with Multi-modal data and multi-task models, using AutoModel to customize complex model architecture. For example, the figure below contains two types of input data, and finally, the classification label and regression value have to be predicted at the same time.

multi

Source: Multi-Modal and Multi-Task

AutoKeras (AutoML)