Code examples
Computer Vision Image classification from scratch Simple MNIST convnet Image classification via fine-tuning with EfficientNet Image classification with Vision Transformer Classification using Attention-based Deep Multiple Instance Learning Image classification with modern MLP models A mobile-friendly Transformer-based model for image classification Pneumonia Classification on TPU Compact Convolutional Transformers Image classification with ConvMixer Image classification with EANet (External Attention Transformer) Involutional neural networks Image classification with Perceiver Few-Shot learning with Reptile Semi-supervised image classification using contrastive pretraining with SimCLR Image classification with Swin Transformers Train a Vision Transformer on small datasets A Vision Transformer without Attention Image Classification using Global Context Vision Transformer When Recurrence meets Transformers Using the Forward-Forward Algorithm for Image Classification Image segmentation with a U-Net-like architecture Multiclass semantic segmentation using DeepLabV3+ Highly accurate boundaries segmentation using BASNet Image Segmentation using Composable Fully-Convolutional Networks Object Detection with RetinaNet Keypoint Detection with Transfer Learning Object detection with Vision Transformers Efficient Object Detection with YOLOV8 and KerasCV 3D Multimodal Brain Tumor Segmentation 3D image classification from CT scans Monocular depth estimation 3D volumetric rendering with NeRF Point cloud segmentation with PointNet Point cloud classification OCR model for reading Captchas Handwriting recognition Convolutional autoencoder for image denoising Low-light image enhancement using MIRNet Image Super-Resolution using an Efficient Sub-Pixel CNN Enhanced Deep Residual Networks for single-image super-resolution Zero-DCE for low-light image enhancement CutMix data augmentation for image classification MixUp augmentation for image classification RandAugment for Image Classification for Improved Robustness Image captioning Natural language image search with a Dual Encoder Visualizing what convnets learn Model interpretability with Integrated Gradients Investigating Vision Transformer representations Grad-CAM class activation visualization Near-duplicate image search Semantic Image Clustering Image similarity estimation using a Siamese Network with a contrastive loss Image similarity estimation using a Siamese Network with a triplet loss Metric learning for image similarity search Metric learning for image similarity search using TensorFlow Similarity Self-supervised contrastive learning with NNCLR Self-supervised contrastive learning with SimSiam Video Classification with a CNN-RNN Architecture Next-Frame Video Prediction with Convolutional LSTMs Video Classification with Transformers Video Vision Transformer Image Classification using BigTransfer (BiT) Gradient Centralization for Better Training Performance Learning to tokenize in Vision Transformers Knowledge Distillation FixRes: Fixing train-test resolution discrepancy Class Attention Image Transformers with LayerScale Augmenting convnets with aggregated attention Learning to Resize Semi-supervision and domain adaptation with AdaMatch Barlow Twins for Contrastive SSL Consistency training with supervision Distilling Vision Transformers Focal Modulation: A replacement for Self-Attention Masked image modeling with Autoencoders Supervised Contrastive Learning Natural Language Processing Structured Data Timeseries Generative Deep Learning Audio Data Reinforcement Learning Graph Data Quick Keras Recipes
► Code examples / Computer Vision / Image classification via fine-tuning with EfficientNet

Image classification via fine-tuning with EfficientNet

Author: Yixing Fu
Date created: 2020/06/30
Last modified: 2026/07/13
Description: Use EfficientNet with weights pre-trained on imagenet for Stanford Dogs classification.

ⓘ This example uses Keras 3

View in Colab • GitHub source

Introduction: what is EfficientNet

EfficientNet, first introduced in Tan and Le, 2019 is among the most efficient models (i.e. requiring least FLOPS for inference) that reaches State-of-the-Art accuracy on both imagenet and common image classification transfer learning tasks.

The smallest base model is similar to MnasNet, which reached near-SOTA with a significantly smaller model. By introducing a heuristic way to scale the model, EfficientNet provides a family of models (B0 to B7) that represents a good combination of efficiency and accuracy on a variety of scales. Such a scaling heuristics (compound-scaling, details see Tan and Le, 2019) allows the efficiency-oriented base model (B0) to surpass models at every scale, while avoiding extensive grid-search of hyperparameters.

B0 to B7 variants of EfficientNet

(This section provides some details on "compound scaling", and can be skipped if you're only interested in using the models)

Based on the original paper people may have the impression that EfficientNet is a continuous family of models created by arbitrarily choosing scaling factor in as Eq.(3) of the paper. However, choice of resolution, depth and width are also restricted by many factors:

  • Resolution: Resolutions not divisible by 8, 16, etc. cause zero-padding near boundaries of some layers which wastes computational resources. This especially applies to smaller variants of the model, hence the input resolution for B0 and B1 are chosen as 224 and 240.
  • Depth and width: The building blocks of EfficientNet demands channel size to be multiples of 8.
  • Resource limit: Memory limitation may bottleneck resolution when depth and width can still increase. In such a situation, increasing depth and/or width but keep resolution can still improve performance.

As a result, the depth, width and resolution of each variant of the EfficientNet models are hand-picked and proven to produce good results, though they may be significantly off from the compound scaling formula. Therefore, the keras implementation (detailed below) only provide these 8 models, B0 to B7, instead of allowing arbitrary choice of width / depth / resolution parameters.

Keras implementation of EfficientNet

An implementation of EfficientNet B0 to B7 has been shipped with Keras since v2.3. To use EfficientNetB0 for classifying 1000 classes of images from ImageNet, run:

from keras.applications import EfficientNetB0
model = EfficientNetB0(weights='imagenet')

This model takes input images of shape (224, 224, 3), and the input data should be in the range [0, 255]. Normalization is included as part of the model.

Because training EfficientNet on ImageNet takes a tremendous amount of resources and several techniques that are not a part of the model architecture itself. Hence the Keras implementation by default loads pre-trained weights obtained via training with AutoAugment.

For B0 to B7 base models, the input shapes are different. Here is a list of input shape expected for each model:

Base model resolution
EfficientNetB0 224
EfficientNetB1 240
EfficientNetB2 260
EfficientNetB3 300
EfficientNetB4 380
EfficientNetB5 456
EfficientNetB6 528
EfficientNetB7 600

When the model is intended for transfer learning, the Keras implementation provides a option to remove the top layers:

model = EfficientNetB0(include_top=False, weights='imagenet')

This option excludes the final Dense layer that turns 1280 features on the penultimate layer into prediction of the 1000 ImageNet classes. Replacing the top layer with custom layers allows using EfficientNet as a feature extractor in a transfer learning workflow.

Another argument in the model constructor worth noticing is drop_connect_rate which controls the dropout rate responsible for stochastic depth. This parameter serves as a toggle for extra regularization in finetuning, but does not affect loaded weights. For example, when stronger regularization is desired, try:

model = EfficientNetB0(weights='imagenet', drop_connect_rate=0.4)

The default value is 0.2.

Example: EfficientNetB0 for Stanford Dogs.

EfficientNet is capable of a wide range of image classification tasks. This makes it a good model for transfer learning. As an end-to-end example, we will show using pre-trained EfficientNetB0 on Stanford Dogs dataset.

Setup and data loading

import tarfile
import urllib.request
from pathlib import Path
import numpy as np
import matplotlib.pyplot as plt
from PIL import Image
import scipy.io
import keras
from keras import layers
from keras.applications import EfficientNetB0

# IMG_SIZE is determined by EfficientNet model choice
IMG_SIZE = 224
BATCH_SIZE = 64

Loading data

We download the Stanford Dogs dataset directly from Stanford's servers using the built-in urllib and tarfile modules. The dataset contains 20,580 images belonging to 120 classes of dog breeds (12,000 for training and 8,580 for testing).

The dataset is downloaded, extracted, and loaded into lists of NumPy arrays. Images have variable dimensions and will be resized to a uniform size in the data pipeline.

Note: This direct download approach eliminates dependency conflicts that can occur with tensorflow_datasets in some environments (particularly Google Colab with protobuf version incompatibilities).

dataset_url = "http://vision.stanford.edu/aditya86/ImageNetDogs/images.tar"
lists_url = "http://vision.stanford.edu/aditya86/ImageNetDogs/lists.tar"
data_dir = Path("./stanford_dogs_data")
data_dir.mkdir(exist_ok=True)


def download_and_extract(url, extract_to):
    filename = url.split("/")[-1]
    filepath = data_dir / filename
    if not filepath.exists():
        print(f"Downloading {filename}...")
        urllib.request.urlretrieve(url, filepath)
        print(f"Extracting {filename}...")
        with tarfile.open(filepath, "r") as tar:
            tar.extractall(extract_to, filter="data")
    return extract_to


# Download dataset
images_dir = download_and_extract(dataset_url, data_dir)
lists_dir = download_and_extract(lists_url, data_dir)

# Parse train/test splits
def load_file_list(filepath):
    mat = scipy.io.loadmat(filepath)
    return [item[0][0] for item in mat["file_list"]]


train_files = load_file_list(data_dir / "train_list.mat")
test_files = load_file_list(data_dir / "test_list.mat")

# Build class name mapping
all_files = train_files + test_files
class_names = sorted(set([f.split("/")[0] for f in all_files]))
class_to_idx = {name: idx for idx, name in enumerate(class_names)}
NUM_CLASSES = len(class_names)

print(
    f"Found {NUM_CLASSES} classes, {len(train_files)} training images, {len(test_files)} test images"
)


# Prepare image paths and labels (lazy loading - no images loaded into memory yet)
def prepare_paths_and_labels(file_list, base_dir):
    image_paths, labels = [], []
    for file_path in file_list:
        class_name = file_path.split("/")[0]
        img_path = base_dir / "Images" / file_path
        if img_path.exists():
            image_paths.append(str(img_path))
            labels.append(class_to_idx[class_name])
    return image_paths, np.array(labels)


print("Preparing dataset paths...")
train_image_paths, train_labels = prepare_paths_and_labels(train_files, data_dir)
test_image_paths, test_labels = prepare_paths_and_labels(test_files, data_dir)
print(f"Found {len(train_image_paths)} train and {len(test_image_paths)} test images")

Each image can have a different shape, so we resize them to a shared input size for EfficientNet. In Keras 3, we do this in a backend-agnostic PyDataset pipeline using keras.ops.image.resize, which works across TensorFlow, JAX, and PyTorch backends, rather than the TensorFlow-specific tf.data mapping steps.

Images are loaded lazily from disk in __getitem__ to avoid loading all 20,580 images into memory at once, which would consume several gigabytes of RAM and cause OOM issues.

class ResizeOnlyDataset(keras.utils.PyDataset):
    def __init__(self, image_paths, labels, img_size, batch_size=1, **kwargs):
        super().__init__(**kwargs)
        self.image_paths = image_paths
        self.labels = labels
        self.img_size = img_size
        self.batch_size = batch_size
        self.indices = np.arange(len(labels))

    def __len__(self):
        return int(np.ceil(len(self.labels) / self.batch_size))

    def __getitem__(self, idx):
        batch_indices = self.indices[
            idx * self.batch_size : (idx + 1) * self.batch_size
        ]
        batch_images = []
        for i in batch_indices:
            img = Image.open(self.image_paths[i]).convert("RGB")
            img_array = np.array(img, dtype="float32")
            img_resized = keras.ops.image.resize(
                img_array, (self.img_size, self.img_size)
            )
            batch_images.append(np.array(img_resized))
        batch_images = np.stack(batch_images)
        batch_labels = self.labels[batch_indices]
        if self.batch_size == 1:
            return batch_images[0], batch_labels[0]
        return batch_images, batch_labels


# Preview stream with resized images for visualization below
preview_train = ResizeOnlyDataset(
    train_image_paths, train_labels, IMG_SIZE, batch_size=1
)
preview_test = ResizeOnlyDataset(test_image_paths, test_labels, IMG_SIZE, batch_size=1)

Visualizing the data

The following code shows the first 9 images with their labels.

def format_label(label):
    class_name = class_names[int(label)]
    return class_name.split("-")[1]  # Extract breed name from "n02085620-Chihuahua"


for i, (image, label) in enumerate(preview_train):
    if i >= 9:
        break
    ax = plt.subplot(3, 3, i + 1)
    plt.imshow(np.array(image).astype("uint8"))
    plt.title("{}".format(format_label(label)))
    plt.axis("off")

png

Data augmentation

We can use Keras preprocessing layers for image augmentation. These layers are backend-agnostic and can be used during both training and inference.

img_augmentation_layers = [
    layers.RandomRotation(factor=0.15),
    layers.RandomTranslation(height_factor=0.1, width_factor=0.1),
    layers.RandomFlip(),
    layers.RandomContrast(factor=0.1),
]


def img_augmentation(images):
    for layer in img_augmentation_layers:
        images = layer(images)
    return images

The img_augmentation function can be used both as a part of the model we later build, and as a standalone function to preprocess data before feeding into the model. Using it as a function makes it easy to visualize the augmentation results. Here we plot 9 examples of augmentation applied to a single image.

first_image, first_label = preview_train[0]

for i in range(9):
    ax = plt.subplot(3, 3, i + 1)
    aug_img = img_augmentation(np.expand_dims(np.array(first_image), axis=0))
    aug_img = np.array(aug_img)
    plt.imshow(aug_img[0].astype("uint8"))
    plt.title("{}".format(format_label(first_label)))
    plt.axis("off")

png

Prepare inputs

Once we verify the input data and augmentation are working correctly, we prepare backend-agnostic datasets for training.

The input images are resized to uniform IMG_SIZE, labels are converted to one-hot (categorical) encoding, and batches are produced by keras.utils.PyDataset.

Compared to the original tf.data version, this Keras 3 setup is backend-agnostic and works seamlessly across TensorFlow, JAX, and PyTorch backends.

class StanfordDogsDataset(keras.utils.PyDataset):
    def __init__(
        self,
        image_paths,
        labels,
        num_classes,
        img_size,
        batch_size,
        augment=False,
        shuffle=False,
        **kwargs,
    ):
        super().__init__(**kwargs)
        self.image_paths = image_paths
        self.labels = labels
        self.num_classes = num_classes
        self.img_size = img_size
        self.batch_size = batch_size
        self.augment = augment
        self.shuffle = shuffle
        self.indices = np.arange(len(labels))
        self.on_epoch_end()

    def __len__(self):
        # Match previous drop_remainder=True behavior.
        return len(self.indices) // self.batch_size

    def on_epoch_end(self):
        if self.shuffle:
            np.random.shuffle(self.indices)

    def __getitem__(self, idx):
        batch_indices = self.indices[
            idx * self.batch_size : (idx + 1) * self.batch_size
        ]

        # Load images lazily from disk
        batch_images = []
        for i in batch_indices:
            img = Image.open(self.image_paths[i]).convert("RGB")
            img_array = np.array(img, dtype="float32")
            img_resized = keras.ops.image.resize(
                img_array, (self.img_size, self.img_size)
            )
            batch_images.append(np.array(img_resized))
        batch_images = np.stack(batch_images)

        if self.augment:
            batch_images = np.array(img_augmentation(batch_images))

        batch_labels = np.array(
            keras.ops.one_hot(self.labels[batch_indices], self.num_classes)
        )

        return batch_images, batch_labels


ds_train = StanfordDogsDataset(
    train_image_paths,
    train_labels,
    num_classes=NUM_CLASSES,
    img_size=IMG_SIZE,
    batch_size=BATCH_SIZE,
    augment=True,
    shuffle=True,
    workers=2,
    use_multiprocessing=False,
)

ds_test = StanfordDogsDataset(
    test_image_paths,
    test_labels,
    num_classes=NUM_CLASSES,
    img_size=IMG_SIZE,
    batch_size=BATCH_SIZE,
    augment=False,
    shuffle=False,
)

Training a model from scratch

We build an EfficientNetB0 with 120 output classes, initialized from scratch (no pretrained weights).

Note: Training from scratch typically shows slower convergence and may overfit on smaller datasets like Stanford Dogs.

model = EfficientNetB0(
    include_top=True,
    weights=None,
    classes=NUM_CLASSES,
    input_shape=(IMG_SIZE, IMG_SIZE, 3),
)
model.compile(optimizer="adam", loss="categorical_crossentropy", metrics=["accuracy"])

model.summary()

epochs = 40  # @param {type: "slider", min:10, max:100}
hist = model.fit(ds_train, epochs=epochs, validation_data=ds_test)

Model: "efficientnetb0"

┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
┃ Layer (type)        ┃ Output Shape      ┃ Param # ┃ Connected to         ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
│ input_layer         │ (None, 224, 224,  │       0 │ -                    │
│ (InputLayer)        │ 3)                │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ rescaling           │ (None, 224, 224,  │       0 │ input_layer[0][0]    │
│ (Rescaling)         │ 3)                │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ normalization       │ (None, 224, 224,  │       7 │ rescaling[0][0]      │
│ (Normalization)     │ 3)                │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ stem_conv_pad       │ (None, 225, 225,  │       0 │ normalization[0][0]  │
│ (ZeroPadding2D)     │ 3)                │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ stem_conv (Conv2D)  │ (None, 112, 112,  │     864 │ stem_conv_pad[0][0]  │
│                     │ 32)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ stem_bn             │ (None, 112, 112,  │     128 │ stem_conv[0][0]      │
│ (BatchNormalizatio… │ 32)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ stem_activation     │ (None, 112, 112,  │       0 │ stem_bn[0][0]        │
│ (Activation)        │ 32)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block1a_dwconv      │ (None, 112, 112,  │     288 │ stem_activation[0][… │
│ (DepthwiseConv2D)   │ 32)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block1a_bn          │ (None, 112, 112,  │     128 │ block1a_dwconv[0][0] │
│ (BatchNormalizatio… │ 32)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block1a_activation  │ (None, 112, 112,  │       0 │ block1a_bn[0][0]     │
│ (Activation)        │ 32)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block1a_se_squeeze  │ (None, 32)        │       0 │ block1a_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block1a_se_reshape  │ (None, 1, 1, 32)  │       0 │ block1a_se_squeeze[… │
│ (Reshape)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block1a_se_reduce   │ (None, 1, 1, 8)   │     264 │ block1a_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block1a_se_expand   │ (None, 1, 1, 32)  │     288 │ block1a_se_reduce[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block1a_se_excite   │ (None, 112, 112,  │       0 │ block1a_activation[… │
│ (Multiply)          │ 32)               │         │ block1a_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block1a_project_co… │ (None, 112, 112,  │     512 │ block1a_se_excite[0… │
│ (Conv2D)            │ 16)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block1a_project_bn  │ (None, 112, 112,  │      64 │ block1a_project_con… │
│ (BatchNormalizatio… │ 16)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2a_expand_conv │ (None, 112, 112,  │   1,536 │ block1a_project_bn[… │
│ (Conv2D)            │ 96)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2a_expand_bn   │ (None, 112, 112,  │     384 │ block2a_expand_conv… │
│ (BatchNormalizatio… │ 96)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2a_expand_act… │ (None, 112, 112,  │       0 │ block2a_expand_bn[0… │
│ (Activation)        │ 96)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2a_dwconv_pad  │ (None, 113, 113,  │       0 │ block2a_expand_acti… │
│ (ZeroPadding2D)     │ 96)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2a_dwconv      │ (None, 56, 56,    │     864 │ block2a_dwconv_pad[… │
│ (DepthwiseConv2D)   │ 96)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2a_bn          │ (None, 56, 56,    │     384 │ block2a_dwconv[0][0] │
│ (BatchNormalizatio… │ 96)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2a_activation  │ (None, 56, 56,    │       0 │ block2a_bn[0][0]     │
│ (Activation)        │ 96)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2a_se_squeeze  │ (None, 96)        │       0 │ block2a_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2a_se_reshape  │ (None, 1, 1, 96)  │       0 │ block2a_se_squeeze[… │
│ (Reshape)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2a_se_reduce   │ (None, 1, 1, 4)   │     388 │ block2a_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2a_se_expand   │ (None, 1, 1, 96)  │     480 │ block2a_se_reduce[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2a_se_excite   │ (None, 56, 56,    │       0 │ block2a_activation[… │
│ (Multiply)          │ 96)               │         │ block2a_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2a_project_co… │ (None, 56, 56,    │   2,304 │ block2a_se_excite[0… │
│ (Conv2D)            │ 24)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2a_project_bn  │ (None, 56, 56,    │      96 │ block2a_project_con… │
│ (BatchNormalizatio… │ 24)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_expand_conv │ (None, 56, 56,    │   3,456 │ block2a_project_bn[… │
│ (Conv2D)            │ 144)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_expand_bn   │ (None, 56, 56,    │     576 │ block2b_expand_conv… │
│ (BatchNormalizatio… │ 144)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_expand_act… │ (None, 56, 56,    │       0 │ block2b_expand_bn[0… │
│ (Activation)        │ 144)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_dwconv      │ (None, 56, 56,    │   1,296 │ block2b_expand_acti… │
│ (DepthwiseConv2D)   │ 144)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_bn          │ (None, 56, 56,    │     576 │ block2b_dwconv[0][0] │
│ (BatchNormalizatio… │ 144)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_activation  │ (None, 56, 56,    │       0 │ block2b_bn[0][0]     │
│ (Activation)        │ 144)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_se_squeeze  │ (None, 144)       │       0 │ block2b_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_se_reshape  │ (None, 1, 1, 144) │       0 │ block2b_se_squeeze[… │
│ (Reshape)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_se_reduce   │ (None, 1, 1, 6)   │     870 │ block2b_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_se_expand   │ (None, 1, 1, 144) │   1,008 │ block2b_se_reduce[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_se_excite   │ (None, 56, 56,    │       0 │ block2b_activation[… │
│ (Multiply)          │ 144)              │         │ block2b_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_project_co… │ (None, 56, 56,    │   3,456 │ block2b_se_excite[0… │
│ (Conv2D)            │ 24)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_project_bn  │ (None, 56, 56,    │      96 │ block2b_project_con… │
│ (BatchNormalizatio… │ 24)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_drop        │ (None, 56, 56,    │       0 │ block2b_project_bn[… │
│ (Dropout)           │ 24)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block2b_add (Add)   │ (None, 56, 56,    │       0 │ block2b_drop[0][0],  │
│                     │ 24)               │         │ block2a_project_bn[… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3a_expand_conv │ (None, 56, 56,    │   3,456 │ block2b_add[0][0]    │
│ (Conv2D)            │ 144)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3a_expand_bn   │ (None, 56, 56,    │     576 │ block3a_expand_conv… │
│ (BatchNormalizatio… │ 144)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3a_expand_act… │ (None, 56, 56,    │       0 │ block3a_expand_bn[0… │
│ (Activation)        │ 144)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3a_dwconv_pad  │ (None, 59, 59,    │       0 │ block3a_expand_acti… │
│ (ZeroPadding2D)     │ 144)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3a_dwconv      │ (None, 28, 28,    │   3,600 │ block3a_dwconv_pad[… │
│ (DepthwiseConv2D)   │ 144)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3a_bn          │ (None, 28, 28,    │     576 │ block3a_dwconv[0][0] │
│ (BatchNormalizatio… │ 144)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3a_activation  │ (None, 28, 28,    │       0 │ block3a_bn[0][0]     │
│ (Activation)        │ 144)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3a_se_squeeze  │ (None, 144)       │       0 │ block3a_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3a_se_reshape  │ (None, 1, 1, 144) │       0 │ block3a_se_squeeze[… │
│ (Reshape)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3a_se_reduce   │ (None, 1, 1, 6)   │     870 │ block3a_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3a_se_expand   │ (None, 1, 1, 144) │   1,008 │ block3a_se_reduce[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3a_se_excite   │ (None, 28, 28,    │       0 │ block3a_activation[… │
│ (Multiply)          │ 144)              │         │ block3a_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3a_project_co… │ (None, 28, 28,    │   5,760 │ block3a_se_excite[0… │
│ (Conv2D)            │ 40)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3a_project_bn  │ (None, 28, 28,    │     160 │ block3a_project_con… │
│ (BatchNormalizatio… │ 40)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_expand_conv │ (None, 28, 28,    │   9,600 │ block3a_project_bn[… │
│ (Conv2D)            │ 240)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_expand_bn   │ (None, 28, 28,    │     960 │ block3b_expand_conv… │
│ (BatchNormalizatio… │ 240)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_expand_act… │ (None, 28, 28,    │       0 │ block3b_expand_bn[0… │
│ (Activation)        │ 240)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_dwconv      │ (None, 28, 28,    │   6,000 │ block3b_expand_acti… │
│ (DepthwiseConv2D)   │ 240)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_bn          │ (None, 28, 28,    │     960 │ block3b_dwconv[0][0] │
│ (BatchNormalizatio… │ 240)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_activation  │ (None, 28, 28,    │       0 │ block3b_bn[0][0]     │
│ (Activation)        │ 240)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_se_squeeze  │ (None, 240)       │       0 │ block3b_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_se_reshape  │ (None, 1, 1, 240) │       0 │ block3b_se_squeeze[… │
│ (Reshape)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_se_reduce   │ (None, 1, 1, 10)  │   2,410 │ block3b_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_se_expand   │ (None, 1, 1, 240) │   2,640 │ block3b_se_reduce[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_se_excite   │ (None, 28, 28,    │       0 │ block3b_activation[… │
│ (Multiply)          │ 240)              │         │ block3b_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_project_co… │ (None, 28, 28,    │   9,600 │ block3b_se_excite[0… │
│ (Conv2D)            │ 40)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_project_bn  │ (None, 28, 28,    │     160 │ block3b_project_con… │
│ (BatchNormalizatio… │ 40)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_drop        │ (None, 28, 28,    │       0 │ block3b_project_bn[… │
│ (Dropout)           │ 40)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block3b_add (Add)   │ (None, 28, 28,    │       0 │ block3b_drop[0][0],  │
│                     │ 40)               │         │ block3a_project_bn[… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4a_expand_conv │ (None, 28, 28,    │   9,600 │ block3b_add[0][0]    │
│ (Conv2D)            │ 240)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4a_expand_bn   │ (None, 28, 28,    │     960 │ block4a_expand_conv… │
│ (BatchNormalizatio… │ 240)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4a_expand_act… │ (None, 28, 28,    │       0 │ block4a_expand_bn[0… │
│ (Activation)        │ 240)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4a_dwconv_pad  │ (None, 29, 29,    │       0 │ block4a_expand_acti… │
│ (ZeroPadding2D)     │ 240)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4a_dwconv      │ (None, 14, 14,    │   2,160 │ block4a_dwconv_pad[… │
│ (DepthwiseConv2D)   │ 240)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4a_bn          │ (None, 14, 14,    │     960 │ block4a_dwconv[0][0] │
│ (BatchNormalizatio… │ 240)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4a_activation  │ (None, 14, 14,    │       0 │ block4a_bn[0][0]     │
│ (Activation)        │ 240)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4a_se_squeeze  │ (None, 240)       │       0 │ block4a_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4a_se_reshape  │ (None, 1, 1, 240) │       0 │ block4a_se_squeeze[… │
│ (Reshape)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4a_se_reduce   │ (None, 1, 1, 10)  │   2,410 │ block4a_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4a_se_expand   │ (None, 1, 1, 240) │   2,640 │ block4a_se_reduce[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4a_se_excite   │ (None, 14, 14,    │       0 │ block4a_activation[… │
│ (Multiply)          │ 240)              │         │ block4a_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4a_project_co… │ (None, 14, 14,    │  19,200 │ block4a_se_excite[0… │
│ (Conv2D)            │ 80)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4a_project_bn  │ (None, 14, 14,    │     320 │ block4a_project_con… │
│ (BatchNormalizatio… │ 80)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_expand_conv │ (None, 14, 14,    │  38,400 │ block4a_project_bn[… │
│ (Conv2D)            │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_expand_bn   │ (None, 14, 14,    │   1,920 │ block4b_expand_conv… │
│ (BatchNormalizatio… │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_expand_act… │ (None, 14, 14,    │       0 │ block4b_expand_bn[0… │
│ (Activation)        │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_dwconv      │ (None, 14, 14,    │   4,320 │ block4b_expand_acti… │
│ (DepthwiseConv2D)   │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_bn          │ (None, 14, 14,    │   1,920 │ block4b_dwconv[0][0] │
│ (BatchNormalizatio… │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_activation  │ (None, 14, 14,    │       0 │ block4b_bn[0][0]     │
│ (Activation)        │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_se_squeeze  │ (None, 480)       │       0 │ block4b_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_se_reshape  │ (None, 1, 1, 480) │       0 │ block4b_se_squeeze[… │
│ (Reshape)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_se_reduce   │ (None, 1, 1, 20)  │   9,620 │ block4b_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_se_expand   │ (None, 1, 1, 480) │  10,080 │ block4b_se_reduce[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_se_excite   │ (None, 14, 14,    │       0 │ block4b_activation[… │
│ (Multiply)          │ 480)              │         │ block4b_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_project_co… │ (None, 14, 14,    │  38,400 │ block4b_se_excite[0… │
│ (Conv2D)            │ 80)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_project_bn  │ (None, 14, 14,    │     320 │ block4b_project_con… │
│ (BatchNormalizatio… │ 80)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_drop        │ (None, 14, 14,    │       0 │ block4b_project_bn[… │
│ (Dropout)           │ 80)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4b_add (Add)   │ (None, 14, 14,    │       0 │ block4b_drop[0][0],  │
│                     │ 80)               │         │ block4a_project_bn[… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_expand_conv │ (None, 14, 14,    │  38,400 │ block4b_add[0][0]    │
│ (Conv2D)            │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_expand_bn   │ (None, 14, 14,    │   1,920 │ block4c_expand_conv… │
│ (BatchNormalizatio… │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_expand_act… │ (None, 14, 14,    │       0 │ block4c_expand_bn[0… │
│ (Activation)        │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_dwconv      │ (None, 14, 14,    │   4,320 │ block4c_expand_acti… │
│ (DepthwiseConv2D)   │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_bn          │ (None, 14, 14,    │   1,920 │ block4c_dwconv[0][0] │
│ (BatchNormalizatio… │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_activation  │ (None, 14, 14,    │       0 │ block4c_bn[0][0]     │
│ (Activation)        │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_se_squeeze  │ (None, 480)       │       0 │ block4c_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_se_reshape  │ (None, 1, 1, 480) │       0 │ block4c_se_squeeze[… │
│ (Reshape)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_se_reduce   │ (None, 1, 1, 20)  │   9,620 │ block4c_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_se_expand   │ (None, 1, 1, 480) │  10,080 │ block4c_se_reduce[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_se_excite   │ (None, 14, 14,    │       0 │ block4c_activation[… │
│ (Multiply)          │ 480)              │         │ block4c_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_project_co… │ (None, 14, 14,    │  38,400 │ block4c_se_excite[0… │
│ (Conv2D)            │ 80)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_project_bn  │ (None, 14, 14,    │     320 │ block4c_project_con… │
│ (BatchNormalizatio… │ 80)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_drop        │ (None, 14, 14,    │       0 │ block4c_project_bn[… │
│ (Dropout)           │ 80)               │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block4c_add (Add)   │ (None, 14, 14,    │       0 │ block4c_drop[0][0],  │
│                     │ 80)               │         │ block4b_add[0][0]    │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5a_expand_conv │ (None, 14, 14,    │  38,400 │ block4c_add[0][0]    │
│ (Conv2D)            │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5a_expand_bn   │ (None, 14, 14,    │   1,920 │ block5a_expand_conv… │
│ (BatchNormalizatio… │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5a_expand_act… │ (None, 14, 14,    │       0 │ block5a_expand_bn[0… │
│ (Activation)        │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5a_dwconv      │ (None, 14, 14,    │  12,000 │ block5a_expand_acti… │
│ (DepthwiseConv2D)   │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5a_bn          │ (None, 14, 14,    │   1,920 │ block5a_dwconv[0][0] │
│ (BatchNormalizatio… │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5a_activation  │ (None, 14, 14,    │       0 │ block5a_bn[0][0]     │
│ (Activation)        │ 480)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5a_se_squeeze  │ (None, 480)       │       0 │ block5a_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5a_se_reshape  │ (None, 1, 1, 480) │       0 │ block5a_se_squeeze[… │
│ (Reshape)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5a_se_reduce   │ (None, 1, 1, 20)  │   9,620 │ block5a_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5a_se_expand   │ (None, 1, 1, 480) │  10,080 │ block5a_se_reduce[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5a_se_excite   │ (None, 14, 14,    │       0 │ block5a_activation[… │
│ (Multiply)          │ 480)              │         │ block5a_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5a_project_co… │ (None, 14, 14,    │  53,760 │ block5a_se_excite[0… │
│ (Conv2D)            │ 112)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5a_project_bn  │ (None, 14, 14,    │     448 │ block5a_project_con… │
│ (BatchNormalizatio… │ 112)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_expand_conv │ (None, 14, 14,    │  75,264 │ block5a_project_bn[… │
│ (Conv2D)            │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_expand_bn   │ (None, 14, 14,    │   2,688 │ block5b_expand_conv… │
│ (BatchNormalizatio… │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_expand_act… │ (None, 14, 14,    │       0 │ block5b_expand_bn[0… │
│ (Activation)        │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_dwconv      │ (None, 14, 14,    │  16,800 │ block5b_expand_acti… │
│ (DepthwiseConv2D)   │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_bn          │ (None, 14, 14,    │   2,688 │ block5b_dwconv[0][0] │
│ (BatchNormalizatio… │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_activation  │ (None, 14, 14,    │       0 │ block5b_bn[0][0]     │
│ (Activation)        │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_se_squeeze  │ (None, 672)       │       0 │ block5b_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_se_reshape  │ (None, 1, 1, 672) │       0 │ block5b_se_squeeze[… │
│ (Reshape)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_se_reduce   │ (None, 1, 1, 28)  │  18,844 │ block5b_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_se_expand   │ (None, 1, 1, 672) │  19,488 │ block5b_se_reduce[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_se_excite   │ (None, 14, 14,    │       0 │ block5b_activation[… │
│ (Multiply)          │ 672)              │         │ block5b_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_project_co… │ (None, 14, 14,    │  75,264 │ block5b_se_excite[0… │
│ (Conv2D)            │ 112)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_project_bn  │ (None, 14, 14,    │     448 │ block5b_project_con… │
│ (BatchNormalizatio… │ 112)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_drop        │ (None, 14, 14,    │       0 │ block5b_project_bn[… │
│ (Dropout)           │ 112)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5b_add (Add)   │ (None, 14, 14,    │       0 │ block5b_drop[0][0],  │
│                     │ 112)              │         │ block5a_project_bn[… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_expand_conv │ (None, 14, 14,    │  75,264 │ block5b_add[0][0]    │
│ (Conv2D)            │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_expand_bn   │ (None, 14, 14,    │   2,688 │ block5c_expand_conv… │
│ (BatchNormalizatio… │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_expand_act… │ (None, 14, 14,    │       0 │ block5c_expand_bn[0… │
│ (Activation)        │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_dwconv      │ (None, 14, 14,    │  16,800 │ block5c_expand_acti… │
│ (DepthwiseConv2D)   │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_bn          │ (None, 14, 14,    │   2,688 │ block5c_dwconv[0][0] │
│ (BatchNormalizatio… │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_activation  │ (None, 14, 14,    │       0 │ block5c_bn[0][0]     │
│ (Activation)        │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_se_squeeze  │ (None, 672)       │       0 │ block5c_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_se_reshape  │ (None, 1, 1, 672) │       0 │ block5c_se_squeeze[… │
│ (Reshape)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_se_reduce   │ (None, 1, 1, 28)  │  18,844 │ block5c_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_se_expand   │ (None, 1, 1, 672) │  19,488 │ block5c_se_reduce[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_se_excite   │ (None, 14, 14,    │       0 │ block5c_activation[… │
│ (Multiply)          │ 672)              │         │ block5c_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_project_co… │ (None, 14, 14,    │  75,264 │ block5c_se_excite[0… │
│ (Conv2D)            │ 112)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_project_bn  │ (None, 14, 14,    │     448 │ block5c_project_con… │
│ (BatchNormalizatio… │ 112)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_drop        │ (None, 14, 14,    │       0 │ block5c_project_bn[… │
│ (Dropout)           │ 112)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block5c_add (Add)   │ (None, 14, 14,    │       0 │ block5c_drop[0][0],  │
│                     │ 112)              │         │ block5b_add[0][0]    │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6a_expand_conv │ (None, 14, 14,    │  75,264 │ block5c_add[0][0]    │
│ (Conv2D)            │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6a_expand_bn   │ (None, 14, 14,    │   2,688 │ block6a_expand_conv… │
│ (BatchNormalizatio… │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6a_expand_act… │ (None, 14, 14,    │       0 │ block6a_expand_bn[0… │
│ (Activation)        │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6a_dwconv_pad  │ (None, 17, 17,    │       0 │ block6a_expand_acti… │
│ (ZeroPadding2D)     │ 672)              │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6a_dwconv      │ (None, 7, 7, 672) │  16,800 │ block6a_dwconv_pad[… │
│ (DepthwiseConv2D)   │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6a_bn          │ (None, 7, 7, 672) │   2,688 │ block6a_dwconv[0][0] │
│ (BatchNormalizatio… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6a_activation  │ (None, 7, 7, 672) │       0 │ block6a_bn[0][0]     │
│ (Activation)        │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6a_se_squeeze  │ (None, 672)       │       0 │ block6a_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6a_se_reshape  │ (None, 1, 1, 672) │       0 │ block6a_se_squeeze[… │
│ (Reshape)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6a_se_reduce   │ (None, 1, 1, 28)  │  18,844 │ block6a_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6a_se_expand   │ (None, 1, 1, 672) │  19,488 │ block6a_se_reduce[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6a_se_excite   │ (None, 7, 7, 672) │       0 │ block6a_activation[… │
│ (Multiply)          │                   │         │ block6a_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6a_project_co… │ (None, 7, 7, 192) │ 129,024 │ block6a_se_excite[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6a_project_bn  │ (None, 7, 7, 192) │     768 │ block6a_project_con… │
│ (BatchNormalizatio… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_expand_conv │ (None, 7, 7,      │ 221,184 │ block6a_project_bn[… │
│ (Conv2D)            │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_expand_bn   │ (None, 7, 7,      │   4,608 │ block6b_expand_conv… │
│ (BatchNormalizatio… │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_expand_act… │ (None, 7, 7,      │       0 │ block6b_expand_bn[0… │
│ (Activation)        │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_dwconv      │ (None, 7, 7,      │  28,800 │ block6b_expand_acti… │
│ (DepthwiseConv2D)   │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_bn          │ (None, 7, 7,      │   4,608 │ block6b_dwconv[0][0] │
│ (BatchNormalizatio… │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_activation  │ (None, 7, 7,      │       0 │ block6b_bn[0][0]     │
│ (Activation)        │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_se_squeeze  │ (None, 1152)      │       0 │ block6b_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_se_reshape  │ (None, 1, 1,      │       0 │ block6b_se_squeeze[… │
│ (Reshape)           │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_se_reduce   │ (None, 1, 1, 48)  │  55,344 │ block6b_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_se_expand   │ (None, 1, 1,      │  56,448 │ block6b_se_reduce[0… │
│ (Conv2D)            │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_se_excite   │ (None, 7, 7,      │       0 │ block6b_activation[… │
│ (Multiply)          │ 1152)             │         │ block6b_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_project_co… │ (None, 7, 7, 192) │ 221,184 │ block6b_se_excite[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_project_bn  │ (None, 7, 7, 192) │     768 │ block6b_project_con… │
│ (BatchNormalizatio… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_drop        │ (None, 7, 7, 192) │       0 │ block6b_project_bn[… │
│ (Dropout)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6b_add (Add)   │ (None, 7, 7, 192) │       0 │ block6b_drop[0][0],  │
│                     │                   │         │ block6a_project_bn[… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_expand_conv │ (None, 7, 7,      │ 221,184 │ block6b_add[0][0]    │
│ (Conv2D)            │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_expand_bn   │ (None, 7, 7,      │   4,608 │ block6c_expand_conv… │
│ (BatchNormalizatio… │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_expand_act… │ (None, 7, 7,      │       0 │ block6c_expand_bn[0… │
│ (Activation)        │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_dwconv      │ (None, 7, 7,      │  28,800 │ block6c_expand_acti… │
│ (DepthwiseConv2D)   │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_bn          │ (None, 7, 7,      │   4,608 │ block6c_dwconv[0][0] │
│ (BatchNormalizatio… │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_activation  │ (None, 7, 7,      │       0 │ block6c_bn[0][0]     │
│ (Activation)        │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_se_squeeze  │ (None, 1152)      │       0 │ block6c_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_se_reshape  │ (None, 1, 1,      │       0 │ block6c_se_squeeze[… │
│ (Reshape)           │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_se_reduce   │ (None, 1, 1, 48)  │  55,344 │ block6c_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_se_expand   │ (None, 1, 1,      │  56,448 │ block6c_se_reduce[0… │
│ (Conv2D)            │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_se_excite   │ (None, 7, 7,      │       0 │ block6c_activation[… │
│ (Multiply)          │ 1152)             │         │ block6c_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_project_co… │ (None, 7, 7, 192) │ 221,184 │ block6c_se_excite[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_project_bn  │ (None, 7, 7, 192) │     768 │ block6c_project_con… │
│ (BatchNormalizatio… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_drop        │ (None, 7, 7, 192) │       0 │ block6c_project_bn[… │
│ (Dropout)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6c_add (Add)   │ (None, 7, 7, 192) │       0 │ block6c_drop[0][0],  │
│                     │                   │         │ block6b_add[0][0]    │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_expand_conv │ (None, 7, 7,      │ 221,184 │ block6c_add[0][0]    │
│ (Conv2D)            │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_expand_bn   │ (None, 7, 7,      │   4,608 │ block6d_expand_conv… │
│ (BatchNormalizatio… │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_expand_act… │ (None, 7, 7,      │       0 │ block6d_expand_bn[0… │
│ (Activation)        │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_dwconv      │ (None, 7, 7,      │  28,800 │ block6d_expand_acti… │
│ (DepthwiseConv2D)   │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_bn          │ (None, 7, 7,      │   4,608 │ block6d_dwconv[0][0] │
│ (BatchNormalizatio… │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_activation  │ (None, 7, 7,      │       0 │ block6d_bn[0][0]     │
│ (Activation)        │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_se_squeeze  │ (None, 1152)      │       0 │ block6d_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_se_reshape  │ (None, 1, 1,      │       0 │ block6d_se_squeeze[… │
│ (Reshape)           │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_se_reduce   │ (None, 1, 1, 48)  │  55,344 │ block6d_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_se_expand   │ (None, 1, 1,      │  56,448 │ block6d_se_reduce[0… │
│ (Conv2D)            │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_se_excite   │ (None, 7, 7,      │       0 │ block6d_activation[… │
│ (Multiply)          │ 1152)             │         │ block6d_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_project_co… │ (None, 7, 7, 192) │ 221,184 │ block6d_se_excite[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_project_bn  │ (None, 7, 7, 192) │     768 │ block6d_project_con… │
│ (BatchNormalizatio… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_drop        │ (None, 7, 7, 192) │       0 │ block6d_project_bn[… │
│ (Dropout)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block6d_add (Add)   │ (None, 7, 7, 192) │       0 │ block6d_drop[0][0],  │
│                     │                   │         │ block6c_add[0][0]    │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block7a_expand_conv │ (None, 7, 7,      │ 221,184 │ block6d_add[0][0]    │
│ (Conv2D)            │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block7a_expand_bn   │ (None, 7, 7,      │   4,608 │ block7a_expand_conv… │
│ (BatchNormalizatio… │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block7a_expand_act… │ (None, 7, 7,      │       0 │ block7a_expand_bn[0… │
│ (Activation)        │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block7a_dwconv      │ (None, 7, 7,      │  10,368 │ block7a_expand_acti… │
│ (DepthwiseConv2D)   │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block7a_bn          │ (None, 7, 7,      │   4,608 │ block7a_dwconv[0][0] │
│ (BatchNormalizatio… │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block7a_activation  │ (None, 7, 7,      │       0 │ block7a_bn[0][0]     │
│ (Activation)        │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block7a_se_squeeze  │ (None, 1152)      │       0 │ block7a_activation[… │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block7a_se_reshape  │ (None, 1, 1,      │       0 │ block7a_se_squeeze[… │
│ (Reshape)           │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block7a_se_reduce   │ (None, 1, 1, 48)  │  55,344 │ block7a_se_reshape[… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block7a_se_expand   │ (None, 1, 1,      │  56,448 │ block7a_se_reduce[0… │
│ (Conv2D)            │ 1152)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block7a_se_excite   │ (None, 7, 7,      │       0 │ block7a_activation[… │
│ (Multiply)          │ 1152)             │         │ block7a_se_expand[0… │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block7a_project_co… │ (None, 7, 7, 320) │ 368,640 │ block7a_se_excite[0… │
│ (Conv2D)            │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ block7a_project_bn  │ (None, 7, 7, 320) │   1,280 │ block7a_project_con… │
│ (BatchNormalizatio… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ top_conv (Conv2D)   │ (None, 7, 7,      │ 409,600 │ block7a_project_bn[… │
│                     │ 1280)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ top_bn              │ (None, 7, 7,      │   5,120 │ top_conv[0][0]       │
│ (BatchNormalizatio… │ 1280)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ top_activation      │ (None, 7, 7,      │       0 │ top_bn[0][0]         │
│ (Activation)        │ 1280)             │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ avg_pool            │ (None, 1280)      │       0 │ top_activation[0][0] │
│ (GlobalAveragePool… │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ top_dropout         │ (None, 1280)      │       0 │ avg_pool[0][0]       │
│ (Dropout)           │                   │         │                      │
├─────────────────────┼───────────────────┼─────────┼──────────────────────┤
│ predictions (Dense) │ (None, 120)       │ 153,720 │ top_dropout[0][0]    │
└─────────────────────┴───────────────────┴─────────┴──────────────────────┘

 Total params: 4,203,291 (16.03 MB)

 Trainable params: 4,161,268 (15.87 MB)

 Non-trainable params: 42,023 (164.16 KB)

Epoch 1/40
   1/187 ━━━━━━━━━━━━━━━━━━━━  5:30:13 107s/step - accuracy: 0.0000e+00 - loss: 5.1065

WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1700241724.682725 1549299 device_compiler.h:187] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.

 187/187 ━━━━━━━━━━━━━━━━━━━━ 200s 501ms/step - accuracy: 0.0097 - loss: 5.0567 - val_accuracy: 0.0100 - val_loss: 4.9278
Epoch 2/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 95s 507ms/step - accuracy: 0.0214 - loss: 4.6918 - val_accuracy: 0.0141 - val_loss: 5.5380
Epoch 3/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 89s 474ms/step - accuracy: 0.0298 - loss: 4.4749 - val_accuracy: 0.0375 - val_loss: 4.4576
Epoch 4/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 90s 479ms/step - accuracy: 0.0423 - loss: 4.3206 - val_accuracy: 0.0391 - val_loss: 4.9898
Epoch 5/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 89s 473ms/step - accuracy: 0.0458 - loss: 4.2312 - val_accuracy: 0.0416 - val_loss: 4.3210
Epoch 6/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 141s 470ms/step - accuracy: 0.0579 - loss: 4.1162 - val_accuracy: 0.0540 - val_loss: 4.3371
Epoch 7/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 89s 476ms/step - accuracy: 0.0679 - loss: 4.0150 - val_accuracy: 0.0786 - val_loss: 3.9759
Epoch 8/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 89s 477ms/step - accuracy: 0.0828 - loss: 3.9147 - val_accuracy: 0.0651 - val_loss: 4.1641
Epoch 9/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 142s 475ms/step - accuracy: 0.0932 - loss: 3.8297 - val_accuracy: 0.0928 - val_loss: 3.8985
Epoch 10/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 88s 472ms/step - accuracy: 0.1092 - loss: 3.7321 - val_accuracy: 0.0946 - val_loss: 3.8618
Epoch 11/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 89s 476ms/step - accuracy: 0.1245 - loss: 3.6451 - val_accuracy: 0.0880 - val_loss: 3.9584
Epoch 12/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 92s 493ms/step - accuracy: 0.1457 - loss: 3.5514 - val_accuracy: 0.1096 - val_loss: 3.8184
Epoch 13/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 88s 471ms/step - accuracy: 0.1606 - loss: 3.4654 - val_accuracy: 0.1118 - val_loss: 3.8059
Epoch 14/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 87s 464ms/step - accuracy: 0.1660 - loss: 3.3826 - val_accuracy: 0.1472 - val_loss: 3.5726
Epoch 15/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 146s 485ms/step - accuracy: 0.1815 - loss: 3.2935 - val_accuracy: 0.1154 - val_loss: 3.8134
Epoch 16/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 87s 466ms/step - accuracy: 0.1942 - loss: 3.2218 - val_accuracy: 0.1540 - val_loss: 3.5051
Epoch 17/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 88s 471ms/step - accuracy: 0.2131 - loss: 3.1427 - val_accuracy: 0.1381 - val_loss: 3.7206
Epoch 18/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 87s 467ms/step - accuracy: 0.2264 - loss: 3.0461 - val_accuracy: 0.1707 - val_loss: 3.4122
Epoch 19/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 88s 470ms/step - accuracy: 0.2401 - loss: 2.9821 - val_accuracy: 0.1515 - val_loss: 3.6481
Epoch 20/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 88s 469ms/step - accuracy: 0.2613 - loss: 2.8815 - val_accuracy: 0.1783 - val_loss: 3.4767
Epoch 21/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 91s 485ms/step - accuracy: 0.2741 - loss: 2.8102 - val_accuracy: 0.1927 - val_loss: 3.3183
Epoch 22/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 90s 477ms/step - accuracy: 0.2892 - loss: 2.7408 - val_accuracy: 0.1859 - val_loss: 3.4887
Epoch 23/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 91s 485ms/step - accuracy: 0.3093 - loss: 2.6526 - val_accuracy: 0.1924 - val_loss: 3.4622
Epoch 24/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 92s 491ms/step - accuracy: 0.3201 - loss: 2.5750 - val_accuracy: 0.2253 - val_loss: 3.1873
Epoch 25/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 95s 508ms/step - accuracy: 0.3280 - loss: 2.5150 - val_accuracy: 0.2148 - val_loss: 3.3391
Epoch 26/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 92s 490ms/step - accuracy: 0.3465 - loss: 2.4402 - val_accuracy: 0.2270 - val_loss: 3.2679
Epoch 27/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 93s 494ms/step - accuracy: 0.3735 - loss: 2.3199 - val_accuracy: 0.2080 - val_loss: 3.5687
Epoch 28/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 89s 476ms/step - accuracy: 0.3837 - loss: 2.2645 - val_accuracy: 0.2374 - val_loss: 3.3592
Epoch 29/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 142s 474ms/step - accuracy: 0.3962 - loss: 2.2110 - val_accuracy: 0.2008 - val_loss: 3.6071
Epoch 30/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 87s 466ms/step - accuracy: 0.4175 - loss: 2.1086 - val_accuracy: 0.2302 - val_loss: 3.4161
Epoch 31/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 87s 465ms/step - accuracy: 0.4359 - loss: 2.0610 - val_accuracy: 0.2231 - val_loss: 3.5957
Epoch 32/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 148s 498ms/step - accuracy: 0.4463 - loss: 1.9866 - val_accuracy: 0.2234 - val_loss: 3.7263
Epoch 33/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 92s 489ms/step - accuracy: 0.4613 - loss: 1.8821 - val_accuracy: 0.2239 - val_loss: 3.6929
Epoch 34/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 139s 475ms/step - accuracy: 0.4925 - loss: 1.7858 - val_accuracy: 0.2238 - val_loss: 3.8351
Epoch 35/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 91s 485ms/step - accuracy: 0.5105 - loss: 1.7074 - val_accuracy: 0.1930 - val_loss: 4.1941
Epoch 36/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 140s 474ms/step - accuracy: 0.5334 - loss: 1.6256 - val_accuracy: 0.2098 - val_loss: 4.1464
Epoch 37/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 87s 464ms/step - accuracy: 0.5504 - loss: 1.5603 - val_accuracy: 0.2306 - val_loss: 4.0215
Epoch 38/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 90s 480ms/step - accuracy: 0.5736 - loss: 1.4419 - val_accuracy: 0.2240 - val_loss: 4.1604
Epoch 39/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 91s 486ms/step - accuracy: 0.6025 - loss: 1.3612 - val_accuracy: 0.2344 - val_loss: 4.0505
Epoch 40/40
 187/187 ━━━━━━━━━━━━━━━━━━━━ 89s 474ms/step - accuracy: 0.6199 - loss: 1.2889 - val_accuracy: 0.2151 - val_loss: 4.3660

Training the model is relatively fast (a few minutes per epoch on modern hardware). However, training EfficientNet on smaller datasets, especially those with lower resolution like CIFAR-100, faces the significant challenge of overfitting.

Training from scratch requires very careful choice of hyperparameters and suitable regularization. It is also much more demanding in computational resources. Plotting the training and validation accuracy makes it clear that validation accuracy stagnates at a low value.

import matplotlib.pyplot as plt


def plot_hist(hist):
    plt.plot(hist.history["accuracy"])
    plt.plot(hist.history["val_accuracy"])
    plt.title("model accuracy")
    plt.ylabel("accuracy")
    plt.xlabel("epoch")
    plt.legend(["train", "validation"], loc="upper left")
    plt.show()


plot_hist(hist)

png

Transfer learning from pretrained weights

Here we initialize the model with pretrained ImageNet weights and fine-tune it on our own dataset. This is the recommended approach for most applications.

def build_model(num_classes):
    inputs = layers.Input(shape=(IMG_SIZE, IMG_SIZE, 3))
    model = EfficientNetB0(include_top=False, input_tensor=inputs, weights="imagenet")

    # Freeze the pretrained weights
    model.trainable = False

    # Rebuild top
    x = layers.GlobalAveragePooling2D(name="avg_pool")(model.output)
    x = layers.BatchNormalization()(x)

    top_dropout_rate = 0.2
    x = layers.Dropout(top_dropout_rate, name="top_dropout")(x)
    outputs = layers.Dense(num_classes, activation="softmax", name="pred")(x)

    # Compile
    model = keras.Model(inputs, outputs, name="EfficientNet")
    optimizer = keras.optimizers.Adam(learning_rate=1e-2)
    model.compile(
        optimizer=optimizer, loss="categorical_crossentropy", metrics=["accuracy"]
    )
    return model

The first step in transfer learning is to freeze all base layers and train only the top layers. For this step, a relatively large learning rate (1e-2) can be used.

Note: Validation accuracy and loss will usually be better than training accuracy and loss. This is because the regularization is strong, which only suppresses training-time metrics.

The convergence may take up to 50 epochs depending on the choice of learning rate. If image augmentation layers were not applied, the validation accuracy may only reach ~60%.

model = build_model(num_classes=NUM_CLASSES)

epochs = 25  # @param {type: "slider", min:8, max:80}
hist = model.fit(ds_train, epochs=epochs, validation_data=ds_test)
plot_hist(hist)

Epoch 1/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 108s 432ms/step - accuracy: 0.2654 - loss: 4.3710 - val_accuracy: 0.6888 - val_loss: 1.0875
Epoch 2/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 119s 412ms/step - accuracy: 0.4863 - loss: 2.0996 - val_accuracy: 0.7282 - val_loss: 0.9072
Epoch 3/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 78s 416ms/step - accuracy: 0.5422 - loss: 1.7120 - val_accuracy: 0.7411 - val_loss: 0.8574
Epoch 4/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 77s 412ms/step - accuracy: 0.5509 - loss: 1.6472 - val_accuracy: 0.7451 - val_loss: 0.8457
Epoch 5/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 81s 431ms/step - accuracy: 0.5744 - loss: 1.5373 - val_accuracy: 0.7424 - val_loss: 0.8649
Epoch 6/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 78s 417ms/step - accuracy: 0.5715 - loss: 1.5595 - val_accuracy: 0.7374 - val_loss: 0.8736
Epoch 7/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 81s 432ms/step - accuracy: 0.5802 - loss: 1.5045 - val_accuracy: 0.7430 - val_loss: 0.8675
Epoch 8/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 77s 411ms/step - accuracy: 0.5839 - loss: 1.4972 - val_accuracy: 0.7392 - val_loss: 0.8647
Epoch 9/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 77s 411ms/step - accuracy: 0.5929 - loss: 1.4699 - val_accuracy: 0.7508 - val_loss: 0.8634
Epoch 10/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 82s 437ms/step - accuracy: 0.6040 - loss: 1.4442 - val_accuracy: 0.7520 - val_loss: 0.8480
Epoch 11/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 78s 416ms/step - accuracy: 0.5972 - loss: 1.4626 - val_accuracy: 0.7379 - val_loss: 0.8879
Epoch 12/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 79s 421ms/step - accuracy: 0.5965 - loss: 1.4700 - val_accuracy: 0.7383 - val_loss: 0.9409
Epoch 13/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 82s 420ms/step - accuracy: 0.6034 - loss: 1.4533 - val_accuracy: 0.7474 - val_loss: 0.8922
Epoch 14/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 81s 435ms/step - accuracy: 0.6053 - loss: 1.4170 - val_accuracy: 0.7416 - val_loss: 0.9119
Epoch 15/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 77s 411ms/step - accuracy: 0.6059 - loss: 1.4125 - val_accuracy: 0.7406 - val_loss: 0.9205
Epoch 16/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 82s 438ms/step - accuracy: 0.5979 - loss: 1.4554 - val_accuracy: 0.7392 - val_loss: 0.9120
Epoch 17/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 77s 411ms/step - accuracy: 0.6081 - loss: 1.4089 - val_accuracy: 0.7423 - val_loss: 0.9305
Epoch 18/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 82s 436ms/step - accuracy: 0.6041 - loss: 1.4390 - val_accuracy: 0.7380 - val_loss: 0.9644
Epoch 19/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 79s 417ms/step - accuracy: 0.6018 - loss: 1.4324 - val_accuracy: 0.7439 - val_loss: 0.9129
Epoch 20/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 81s 430ms/step - accuracy: 0.6057 - loss: 1.4342 - val_accuracy: 0.7305 - val_loss: 0.9463
Epoch 21/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 77s 410ms/step - accuracy: 0.6209 - loss: 1.3824 - val_accuracy: 0.7410 - val_loss: 0.9503
Epoch 22/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 78s 419ms/step - accuracy: 0.6170 - loss: 1.4246 - val_accuracy: 0.7336 - val_loss: 0.9606
Epoch 23/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 85s 455ms/step - accuracy: 0.6153 - loss: 1.4009 - val_accuracy: 0.7334 - val_loss: 0.9520
Epoch 24/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 82s 438ms/step - accuracy: 0.6051 - loss: 1.4343 - val_accuracy: 0.7435 - val_loss: 0.9403
Epoch 25/25
 187/187 ━━━━━━━━━━━━━━━━━━━━ 138s 416ms/step - accuracy: 0.6065 - loss: 1.4131 - val_accuracy: 0.7456 - val_loss: 0.9307

png

The second step is to unfreeze a number of layers and fine-tune the model using a smaller learning rate. In this example we unfreeze the last 20 layers, but depending on the specific dataset it may be desirable to only unfreeze a fraction of all layers.

Advanced usage: The unfreeze_model() function also supports unfreezing by block name (e.g., unfreeze_model(model, layers_to_unfreeze="block7")) to respect EfficientNet's residual block boundaries. See the "Tips for fine-tuning EfficientNet" section below for why this matters.

When feature extraction with the pretrained model works well enough, this step provides only a limited gain in validation accuracy. In our case we only see a small improvement, as ImageNet pretraining already exposed the model to a good amount of dog images.

On the other hand, when we use pretrained weights on a dataset that is more different from ImageNet, this fine-tuning step can be crucial as the feature extractor also needs to be adjusted by a considerable amount. Such a situation can be demonstrated if choosing CIFAR-100 dataset instead, where fine-tuning boosts validation accuracy by about 10% to pass 80% on EfficientNetB0.

Note on freezing/unfreezing models: Setting trainable of a Model will simultaneously set all layers belonging to the Model to the same trainable attribute. Each layer is trainable only if both the layer itself and the model containing it are trainable. Hence when we need to partially freeze/unfreeze a model, we need to make sure the trainable attribute of the model is set to True.

def unfreeze_model(
    model,
    layers_to_unfreeze=20,
    learning_rate=1e-5,
    loss="categorical_crossentropy",
    metrics=None,
):
    """Unfreeze part of `model` and recompile it for fine-tuning.

    Args:
        model: A [`keras.Model`](/api/models/model#model-class) instance to unfreeze in place.
        layers_to_unfreeze: Either an `int` giving the number of layers,
            counted from the end of the base model's layers, to unfreeze, or a `str`
            substring to match against layer names -- the first matching
            layer and every layer after it (in the base model's layers order) are
            unfrozen. Use a string like `"block7"` to respect EfficientNet's
            residual block boundaries instead of an arbitrary layer count.
            Defaults to `20`.
        learning_rate: Learning rate for the fine-tuning `Adam` optimizer.
            Defaults to `1e-5`.
        loss: Loss function passed to `model.compile()`. Defaults to
            `"categorical_crossentropy"`.
        metrics: List of metrics passed to `model.compile()`. Defaults to
            `["accuracy"]`.

    Returns:
        `model`, with the selected layers unfrozen (except
        `BatchNormalization` layers, which are always kept frozen) and
        recompiled with the new optimizer/loss/metrics.
    """
    if metrics is None:
        metrics = ["accuracy"]

    # Access the nested EfficientNet base model by finding the first layer
    # with 'efficientnet' in its name (case-insensitive)
    base_model = None
    for layer in model.layers:
        if "efficientnet" in layer.name.lower():
            base_model = layer
            break
    if base_model is None:
        raise ValueError(
            "Could not find EfficientNet base model in the model. "
            "Expected a layer with 'efficientnet' in its name."
        )
    base_model.trainable = True

    # First, freeze all layers in the base model
    for layer in base_model.layers:
        layer.trainable = False

    if isinstance(layers_to_unfreeze, str):
        unfreeze_from = None
        for i, layer in enumerate(base_model.layers):
            if layers_to_unfreeze in layer.name:
                unfreeze_from = i
                break
        if unfreeze_from is None:
            raise ValueError(f"No layer name contains {layers_to_unfreeze!r}.")
        layers_to_process = base_model.layers[unfreeze_from:]
    elif isinstance(layers_to_unfreeze, int):
        if layers_to_unfreeze <= 0:
            raise ValueError(
                f"layers_to_unfreeze must be > 0, got {layers_to_unfreeze}"
            )
        n_layers_to_unfreeze = min(layers_to_unfreeze, len(base_model.layers))
        layers_to_process = base_model.layers[-n_layers_to_unfreeze:]
    else:
        raise TypeError(
            "layers_to_unfreeze must be an int or str, received: "
            f"{type(layers_to_unfreeze)}"
        )

    # We keep BatchNorm layers frozen -- see "Tips for fine-tuning
    # EfficientNet" in the tutorial for why.
    for layer in layers_to_process:
        if not isinstance(layer, layers.BatchNormalization):
            layer.trainable = True

    optimizer = keras.optimizers.Adam(learning_rate=learning_rate)
    model.compile(optimizer=optimizer, loss=loss, metrics=metrics)
    return model


model = unfreeze_model(model)

epochs = 4  # @param {type: "slider", min:4, max:10}
hist = model.fit(ds_train, epochs=epochs, validation_data=ds_test)
plot_hist(hist)

Epoch 1/4
 187/187 ━━━━━━━━━━━━━━━━━━━━ 111s 442ms/step - accuracy: 0.6310 - loss: 1.3425 - val_accuracy: 0.7565 - val_loss: 0.8874
Epoch 2/4
 187/187 ━━━━━━━━━━━━━━━━━━━━ 77s 413ms/step - accuracy: 0.6518 - loss: 1.2755 - val_accuracy: 0.7635 - val_loss: 0.8588
Epoch 3/4
 187/187 ━━━━━━━━━━━━━━━━━━━━ 82s 437ms/step - accuracy: 0.6491 - loss: 1.2426 - val_accuracy: 0.7663 - val_loss: 0.8419
Epoch 4/4
 187/187 ━━━━━━━━━━━━━━━━━━━━ 79s 419ms/step - accuracy: 0.6625 - loss: 1.1775 - val_accuracy: 0.7701 - val_loss: 0.8284

png

Tips for fine-tuning EfficientNet

On unfreezing layers:

  • The BatchNormalization layers need to be kept frozen (more details). If they are also turned to trainable, the first epoch after unfreezing will significantly reduce accuracy.
  • In some cases it may be beneficial to unfreeze only a portion of layers instead of unfreezing all. This will make fine-tuning much faster when going to larger models like B7.
  • Each block needs to be all turned on or off. This is because the architecture includes a shortcut from the first layer to the last layer for each block. Not respecting blocks also significantly harms the final performance.

Some other tips for utilizing EfficientNet:

  • Larger variants of EfficientNet do not guarantee improved performance, especially for tasks with less data or fewer classes. In such a case, the larger the variant of EfficientNet chosen, the harder it is to tune hyperparameters.
  • EMA (Exponential Moving Average) is very helpful in training EfficientNet from scratch, but not so much for transfer learning.
  • Do not use the RMSprop setup as in the original paper for transfer learning. The momentum and learning rate are too high for transfer learning. It will easily corrupt the pretrained weights and blow up the loss. A quick check is to see if loss (as categorical cross entropy) is getting significantly larger than log(NUM_CLASSES) after the same epoch. If so, the initial learning rate/momentum is too high.
  • Smaller batch sizes benefit validation accuracy, possibly due to effectively providing regularization.

Relevant Chapters from Deep Learning with Python

Image classification via fine-tuning with EfficientNet
Introduction: what is EfficientNet
B0 to B7 variants of EfficientNet
Keras implementation of EfficientNet
Example: EfficientNetB0 for Stanford Dogs.
Setup and data loading
Loading data
Visualizing the data
Data augmentation
Prepare inputs
Training a model from scratch
Transfer learning from pretrained weights
Tips for fine-tuning EfficientNet
Relevant Chapters from Deep Learning with Python