TensorFlow Lite is a powerful tool that allows developers to train and deploy machine learning models on mobile and embedded devices. One of the most popular uses for TensorFlow Lite is object detection, which allows an application to identify and classify objects in real-time. In this article, we will walk through the process of training a TensorFlow Lite object detection model using Google Colab and SSD MobileNet.
To get started, you will need a Google account and access to Google Colab, a cloud-based Jupyter notebook environment that allows you to write and execute Python code. Once you have access to Google Colab, you can begin training your TensorFlow Lite object detection model by following these steps:
Step 1: Import the necessary libraries
In a new Google Colab notebook, begin by importing the necessary libraries for training your object detection model. This will include TensorFlow, TensorFlow Lite, and any other libraries that you may need for data processing and model evaluation.
Step 2: Download and prepare the training data
Next, you will need to download and prepare the training data for your object detection model. This will typically involve gathering a dataset of images and annotations, and then formatting the data in a way that is compatible with the TensorFlow Lite object detection model.
Step 3: Define the model architecture
Once your training data is prepared, you can define the architecture of your object detection model. For this tutorial, we will use the SSD MobileNet architecture, which is a lightweight and efficient model that is well-suited for mobile and embedded devices.
Step 4: Train the model
With your model architecture defined, you can begin training your object detection model using the training data that you prepared in step 2. This may involve running multiple epochs of training in order to optimize the model’s performance and accuracy.
Step 5: Convert the model to TensorFlow Lite
Once your model is trained, you can convert it to the TensorFlow Lite format using the TensorFlow Lite Converter. This will allow you to deploy the model on mobile and embedded devices using the TensorFlow Lite runtime.
By following these steps, you can easily train a TensorFlow Lite object detection model using Google Colab and the SSD MobileNet architecture. This will allow you to create powerful and efficient object detection applications that can be deployed on a wide range of devices. With the growing popularity of object detection in mobile and embedded applications, TensorFlow Lite provides a valuable tool for developers to create innovative and efficient solutions.
hi, is there any way to open the verbose so I'm able to track the epoch on each iteration and know the mAP score during specific epoch?
Good news! Training on GPU is working again. Thank you @markwassef8643 for finding the solution.
Sigh, sorry guys. It looks like something changed in Colab or in TensorFlow, and now it's not able to run training on the GPU. I get several error messages like this when I try to run training in Step 5: " Could not load dynamic library 'libcudart.so.11.0' ". This means it's not able to load the CUDA libraries needed to run on the GPU. I'm not sure why. If anyone can figure out the problem, please post a solution. I'll dive into it after New Year's if no one has found anything by then.
EDIT: I'll send $50 USD to the first person who can figure out a reliable solution and post it. And I'd also be highly interested in hiring you for a paid internship at https://ejtech.io 😃
hello Edje seems like the code is not utilizing the GPU any fix?
I have 8 sec per step time. GPU memory is 0. I didn't change anything – only my classes name. Each image is 600kb (2000*1500). Any tips for this problem?
How can we use txt file instead of xml file? Is it possible?
Amazing work, but I noticed that the camera is very slow when you launch the test. Can you please tell me why? Thanks
After training with my custom dataset I get the .tflite model. the output size is [1,10] but i want output size to be [1,10,4]. What should i do to get the output size to be [1,10,4]
what if i made my data for train , test and validation?
Amazing tutorial! One of the most useful videos I've ever watched. Keep up the good work!
Thanks a lot, Edje!
Amazing!! Incredible to see you using the model on another device.
can anyone provide me link to download coin dataset?
sir thanks amazing video, sir I just want to know is the notebook still working until now ? cuz right now I tried many included your coin dataset and dataset from roboflow this what i got /usr/local/lib/python3.10/dist-packages/tensorflow_io/python/ops/_init__.py:98: UserWarning: unable to load libtensorflow_ioplugins.so: unable to open file: libtensorflow_io_plugins.so, from paths: ['/usr/local/lib/python3.10/dist-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so']
caused by: ['/usr/local/lib/python3.10/dist-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so: undefined symbol: _ZN3tsl5mutexC1Ev']
warnings.warn(f"unable to load libtensorflow_io_plugins.so: {e}")
/usr/local/lib/python3.10/dist-packages/tensorflow_io/python/ops/_init__.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflowio.so, from paths: ['/usr/local/lib/python3.10/dist-packages/tensorflow_io/python/ops/libtensorflow_io.so']
caused by: ['/usr/local/lib/python3.10/dist-packages/tensorflow_io/python/ops/libtensorflow_io.so: undefined symbol: _ZNK10tensorflow4data11DatasetBase8FinalizeEPNS_15OpKernelContextESt8functionIFN3tsl8StatusOrINS5_4core11RefCountPtrIS1_EEEEvEE']
warnings.warn(f"file system plugins are not loaded: {e}")
/usr/local/lib/python3.10/dist-packages/tensorflow_addons/utils/tfa_eol_msg.py:23: UserWarning:
TensorFlow Addons (TFA) has ended development and introduction of new features.
TFA has entered a minimal maintenance and release mode until a planned end of life in May 2024.
Please modify downstream libraries to take dependencies from other repositories in our TensorFlow community (e.g. Keras, Keras-CV, and Keras-NLP).
For more information see: https://github.com/tensorflow/addons/issues/2807
warnings.warn(
/usr/local/lib/python3.10/dist-packages/tensorflow_addons/utils/ensure_tf_install.py:53: UserWarning: Tensorflow Addons supports using Python ops for all Tensorflow versions above or equal to 2.13.0 and strictly below 2.16.0 (nightly versions are not supported).
The versions of TensorFlow you are currently using is 2.8.0 and is not supported.
Some things might work, some things might not.
If you were to encounter a bug, do not file an issue.
If you want to make sure you're using a tested and supported configuration, either change the TensorFlow version or the TensorFlow Addons's version.
You can find the compatibility matrix in TensorFlow Addon's readme:
https://github.com/tensorflow/addons
warnings.warn(
2023-12-07 19:06:56.326747: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/lib/python3.10/dist-packages/cv2/../../lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64
2023-12-07 19:06:56.326801: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
WARNING:tensorflow:There are non-GPU devices in `tf.distribute.Strategy`, not using nccl allreduce.
W1207 19:06:56.328734 137622082002944 cross_device_ops.py:1386] There are non-GPU devices in `tf.distribute.Strategy`, not using nccl allreduce.
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0',)
I1207 19:06:56.342269 137622082002944 mirrored_strategy.py:374] Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0',)
INFO:tensorflow:Maybe overwriting train_steps: 40000
I1207 19:06:56.347236 137622082002944 config_util.py:552] Maybe overwriting train_steps: 40000
INFO:tensorflow:Maybe overwriting use_bfloat16: False
I1207 19:06:56.347490 137622082002944 config_util.py:552] Maybe overwriting use_bfloat16: False
WARNING:tensorflow:From /usr/local/lib/python3.10/dist-packages/object_detection/model_lib_v2.py:563: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version.
Instructions for updating:
rename to distribute_datasets_from_function
W1207 19:06:56.385059 137622082002944 deprecation.py:337] From /usr/local/lib/python3.10/dist-packages/object_detection/model_lib_v2.py:563: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version.
Instructions for updating:
rename to distribute_datasets_from_function
INFO:tensorflow:Reading unweighted datasets: ['/content/train.tfrecord']
I1207 19:06:56.390673 137622082002944 dataset_builder.py:162] Reading unweighted datasets: ['/content/train.tfrecord']
INFO:tensorflow:Reading record datasets for input file: ['/content/train.tfrecord']
I1207 19:06:56.391010 137622082002944 dataset_builder.py:79] Reading record datasets for input file: ['/content/train.tfrecord']
INFO:tensorflow:Number of filenames to read: 1
I1207 19:06:56.391136 137622082002944 dataset_builder.py:80] Number of filenames to read: 1
WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards.
W1207 19:06:56.391219 137622082002944 dataset_builder.py:86] num_readers has been reduced to 1 to match input file shards.
WARNING:tensorflow:From /usr/local/lib/python3.10/dist-packages/object_detection/builders/dataset_builder.py:100: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.deterministic`.
W1207 19:06:56.394553 137622082002944 deprecation.py:337] From /usr/local/lib/python3.10/dist-packages/object_detection/builders/dataset_builder.py:100: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.deterministic`.
WARNING:tensorflow:From /usr/local/lib/python3.10/dist-packages/object_detection/builders/dataset_builder.py:235: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map()
W1207 19:06:56.646597 137622082002944 deprecation.py:337] From /usr/local/lib/python3.10/dist-packages/object_detection/builders/dataset_builder.py:235: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map()
WARNING:tensorflow:From /usr/local/lib/python3.10/dist-packages/tensorflow/python/util/dispatch.py:1082: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.
W1207 19:07:07.259489 137622082002944 deprecation.py:337] From /usr/local/lib/python3.10/dist-packages/tensorflow/python/util/dispatch.py:1082: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.
WARNING:tensorflow:From /usr/local/lib/python3.10/dist-packages/tensorflow/python/util/dispatch.py:1082: sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version.
Instructions for updating:
`seed2` arg is deprecated.Use sample_distorted_bounding_box_v2 instead.
W1207 19:07:10.986947 137622082002944 deprecation.py:337] From /usr/local/lib/python3.10/dist-packages/tensorflow/python/util/dispatch.py:1082: sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version.
Instructions for updating:
`seed2` arg is deprecated.Use sample_distorted_bounding_box_v2 instead.
WARNING:tensorflow:From /usr/local/lib/python3.10/dist-packages/tensorflow/python/util/dispatch.py:1082: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W1207 19:07:13.084192 137622082002944 deprecation.py:337] From /usr/local/lib/python3.10/dist-packages/tensorflow/python/util/dispatch.py:1082: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
2023-12-07 19:07:18.215632: W tensorflow/core/framework/dataset.cc:768] Input of GeneratorDatasetOp::Dataset will not be optimized because the dataset does not implement the AsGraphDefInternal() method needed to apply optimizations.
will the datasets from roboflow work with this colab notebook?
Great video! One issue tho, I noticed that there seemed to be no process for adding metadata to the file, which is crucial for running the tflite model on a raspberry pi or edgetpu, for me my fix was just to just run a quick program I found online to add metadata to the file
from tflite_support.metadata_writers import object_detector
from tflite_support.metadata_writers import writer_utils
ObjectDetectorWriter = object_detector.MetadataWriter
_MODEL_PATH = "edgetpu.tflite"
_LABEL_FILE = "labelmap.txt"
_SAVE_TO_PATH = "edgetpu_fixed.tflite"
_INPUT_NORM_MEAN = 127.5
_INPUT_NORM_STD = 127.5
writer = ObjectDetectorWriter.create_for_inference(
writer_utils.load_file(_MODEL_PATH), [_INPUT_NORM_MEAN], [_INPUT_NORM_STD],
[_LABEL_FILE])
# Optional to verify the metadata generated by metadata writer.
print(writer.get_metadata_json())
writer_utils.save_file(writer.populate(), _SAVE_TO_PATH)
Can you please show how to run it on bookworm or bullseye on a Pi 4 or 5
Good Job !
Hello Sir, Thanks for your great tutorial. I try to run the code you shared, but I found error while running at 10:37 part or in "set up training configuration part". it returns "NotFoundError: /content/labelmap.pbtxt; No such file or directory". What should I do to fix that sir?
has anyone managed to do this with a free account? I've run the training for hours twice now and it's disconnected both times and I've lost all progress…
I want to deploy custom detection model on jetson nano…………..