Image Classification Android App with TensorFlow Lite for Beginner
Today, Machine Learning (ML) is all over the place. The ML reduces human works, suppose I’m trying to identify birds, as a human what we generally do we search for every bird that is present in this world and identify the particular one. But Machine Learning reduces that, in ML we will simply create one model which involves every bird. Then when I tried to search for any birds, that model will simply tell me which bird it is.
So in this blog, we will see how to connect a tflite model in android or how to use a tflite model in the android app.
In this tutorial, we’ll look at how to use TensorFlow Lite to integrate machine learning into an Android app by creating a basic app.
TensorFlow Lite
TensorFlow Lite is a set of tools that enables on-device machine learning by helping developers run their models on mobile, embedded, and IoT devices.
Key Features of TF-Lite:
- Multiple platform support, covering Android and iOS devices, embedded Linux, and microcontrollers.
- Multiple language support, which includes Java, Swift, Objective-C, C++, and Python.
- High performance, with hardware acceleration and model optimization.
In order to start, you will require a trained .tflite model that you can download from here.
Note: after downloading name this file as BirdsModel.tflite.
Add TensorFlow Lite to the Android app.
Step 1: Head over to android studio & Create a new android project.
Step 2: Add TensorFlow Lite to the Android App.
Right-click on the package name in my case it is com.yourpackagename or click on File, then New > Other > TensorFlow Lite Model. Select the model location where you have downloaded the custom trained BirdsModel.tflite earlier
Step 3: Click finish.
Note that the tooling will automatically configure the module’s dependencies for you using ML Model binding and all requirements will be added into your Android module’s build.gradle file.
Step 4: At the end, you’ll see the following. The BirdModel.tflite file has been successfully imported, and it displays high-level model information like as input and output, as well as some sample code to get you started.
Processing image and showing result
following are the simple steps to implement the bird classification model.
1. Create Tensor Flow Lite Model variable and initialize it.
val birdModel = BirdsModel.newInstance(this)
2. Converting input image into tensor flow image. as you can see input image is require to be in bitmap format.
val tfImage = TensorImage.fromBitmap(bitmap)
3. Processing the image and then sorting output results into descending order then picking top result (pick out highest probability one’s).
val outputs = birdModel.process(tfImage)
.probabilityAsCategoryList.apply {
sortByDescending { it.score }
}
//getting result having high probability
val highProbabilityOutput = outputs[0]
4. Dispalying the result into TextView
tvOutput.text = highProbabilityOutput.label
Prepare layout for Your app
This layout contains following views,
- Image view, for displaying our captured bird photo.
- Button, for launching the camera to take the photo.
- Text view, to display output result.
<?xml version=”1.0" encoding=”utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android=”http://schemas.android.com/apk/res/android"
xmlns:app=”http://schemas.android.com/apk/res-auto"
xmlns:tools=”http://schemas.android.com/tools"
android:layout_width=”match_parent”
android:layout_height=”match_parent”
tools:context=”.MainActivity”>
<Button
android:layout_width=”wrap_content”
android:layout_height=”wrap_content”
android:id=”@+id/btn_load_image”
android:text=”Load Image”
app:layout_constraintEnd_toStartOf=”@id/guideline2"
app:layout_constraintStart_toStartOf=”parent”
app:layout_constraintTop_toBottomOf=”@+id/imageView”
android:layout_marginTop=”18dp”/>
<ImageView
android:layout_width=”300dp”
android:layout_height=”450dp”
android:id=”@+id/imageView”
android:layout_marginTop=”16dp”
app:layout_constraintEnd_toEndOf=”parent”
app:layout_constraintStart_toStartOf=”parent”
app:layout_constraintTop_toTopOf=”parent”
app:layout_constraintHorizontal_bias=”0.5"
android:src=”@drawable/place_holder”/>
<Button
android:layout_width=”wrap_content”
android:layout_height=”wrap_content”
android:id=”@+id/btn_capture_image”
android:text=”Take Image”
app:layout_constraintEnd_toEndOf=”parent”
app:layout_constraintStart_toStartOf=”@id/guideline2"
app:layout_constraintTop_toBottomOf=”@id/imageView”
android:layout_marginTop=”18dp”/>
<TextView
android:layout_width=”wrap_content”
android:layout_height=”wrap_content”
android:id=”@+id/textview”
android:text=”Output: “
android:textSize=”21sp”
android:layout_marginTop=”24dp”
app:layout_constraintStart_toStartOf=”parent”
app:layout_constraintTop_toBottomOf=”@id/btn_capture_image”
android:layout_marginStart=”36dp”
android:textColor=”@android:color/black”/>
<TextView
android:layout_width=”wrap_content”
android:layout_height=”wrap_content”
android:id=”@+id/tv_output”
android:text=”Result here”
android:textSize=”21sp”
android:textColor=”@android:color/holo_red_dark”
app:layout_constraintStart_toEndOf=”@id/textview”
app:layout_constraintTop_toBottomOf=”@id/btn_capture_image”
android:layout_marginTop=”24dp” />
<TextView
android:layout_width=”wrap_content”
android:layout_height=”wrap_content”
android:text=”Click on the result text to search on Google”
app:layout_constraintTop_toBottomOf=”@id/textview”
android:layout_marginTop=”8dp”
android:layout_marginStart=”36dp”
app:layout_constraintStart_toStartOf=”parent”
android:textSize=”13sp”/><androidx.constraintlayout.widget.Guideline
android:layout_width=”wrap_content”
android:layout_height=”wrap_content”
android:id=”@+id/guideline2"
app:layout_constraintGuide_percent=”0.5"
android:orientation=”vertical”/>
</androidx.constraintlayout.widget.ConstraintLayout>
Complete MainActivity.Kt
package com.codewithgolap.birdclassificationkotlin
import android.app.Activity
import android.app.Instrumentation
import android.content.ContentValues
import android.content.Intent
import android.content.pm.PackageManager
import android.graphics.Bitmap
import android.graphics.BitmapFactory
import android.graphics.drawable.BitmapDrawable
import android.net.Uri
import androidx.appcompat.app.AppCompatActivity
import android.os.Bundle
import android.provider.MediaStore
import android.util.Log
import android.widget.Button
import android.widget.ImageView
import android.widget.TextView
import android.widget.Toast
import androidx.activity.result.ActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.appcompat.app.AlertDialog
import androidx.core.content.ContextCompat
import com.codewithgolap.birdclassificationkotlin.databinding.ActivityMainBinding
import com.codewithgolap.birdclassificationkotlin.ml.BirdsModel
import org.tensorflow.lite.support.image.TensorImage
import java.io.IOException
class MainActivity : AppCompatActivity() {
private lateinit var binding: ActivityMainBinding
private lateinit var imageView: ImageView
private lateinit var button: Button
private lateinit var tvOutput: TextView
private val GALLERY_REQUEST_CODE = 123
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
binding = ActivityMainBinding.inflate(layoutInflater)
val view = binding.root
setContentView(view)
imageView = binding.imageView
button = binding.btnCaptureImage
tvOutput = binding.tvOutput
val buttonLoad = binding.btnLoadImage
button.setOnClickListener {
if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.CAMERA)
== PackageManager.PERMISSION_GRANTED
) {
takePicturePreview.launch(null)
}
else {
requestPermission.launch(android.Manifest.permission.CAMERA)
}
}
buttonLoad.setOnClickListener {
if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.READ_EXTERNAL_STORAGE)
== PackageManager.PERMISSION_GRANTED){
val intent = Intent(Intent.ACTION_PICK, MediaStore.Images.Media.EXTERNAL_CONTENT_URI)
intent.type = “image/*”
val mimeTypes = arrayOf(“image/jpeg”,”image/png”,”image/jpg”)
intent.putExtra(Intent.EXTRA_MIME_TYPES, mimeTypes)
intent.flags = Intent.FLAG_GRANT_READ_URI_PERMISSION
onresult.launch(intent)
}else {
requestPermission.launch(android.Manifest.permission.READ_EXTERNAL_STORAGE)
}
}
//to redirct user to google search for the scientific name
tvOutput.setOnClickListener {
val intent = Intent(Intent.ACTION_VIEW, Uri.parse(“https://www.google.com/search?q=${tvOutput.text}"))
startActivity(intent)
}
// to download image when longPress on ImageView
imageView.setOnLongClickListener {
requestPermissionLauncher.launch(android.Manifest.permission.WRITE_EXTERNAL_STORAGE)
return@setOnLongClickListener true
}
}
//request camera permission
private val requestPermission = registerForActivityResult(ActivityResultContracts.RequestPermission()){granted->
if (granted){
takePicturePreview.launch(null)
}else {
Toast.makeText(this, “Permission Denied !! Try again”, Toast.LENGTH_SHORT).show()
}
}
//launch camera and take picture
private val takePicturePreview = registerForActivityResult(ActivityResultContracts.TakePicturePreview()){bitmap->
if(bitmap != null){
imageView.setImageBitmap(bitmap)
outputGenerator(bitmap)
}
}
//to get image from gallery
private val onresult = registerForActivityResult(ActivityResultContracts.StartActivityForResult()){result->
Log.i(“TAG”, “This is the result: ${result.data} ${result.resultCode}”)
onResultReceived(GALLERY_REQUEST_CODE,result)
}
private fun onResultReceived(requestCode: Int, result: ActivityResult?){
when(requestCode){
GALLERY_REQUEST_CODE ->{
if (result?.resultCode == Activity.RESULT_OK){
result.data?.data?.let{uri ->
Log.i(“TAG”, “onResultReceived: $uri”)
val bitmap = BitmapFactory.decodeStream(contentResolver.openInputStream(uri))
imageView.setImageBitmap(bitmap)
outputGenerator(bitmap)
}
}else {
Log.e(“TAG”, “onActivityResult: error in selecting image”)
}
}
}
}
private fun outputGenerator(bitmap: Bitmap){
//declearing tensor flow lite model variable
val birdsModel = BirdsModel.newInstance(this)
// converting bitmap into tensor flow image
val newBitmap = bitmap.copy(Bitmap.Config.ARGB_8888, true)
val tfimage = TensorImage.fromBitmap(newBitmap)
//process the image using trained model and sort it in descending order
val outputs = birdsModel.process(tfimage)
.probabilityAsCategoryList.apply {
sortByDescending { it.score }
}
//getting result having high probability
val highProbabilityOutput = outputs[0]
//setting ouput text
tvOutput.text = highProbabilityOutput.label
Log.i(“TAG”, “outputGenerator: $highProbabilityOutput”)
}
// to download image to device
private val requestPermissionLauncher = registerForActivityResult(ActivityResultContracts.RequestPermission()){
isGranted: Boolean ->
if (isGranted){
AlertDialog.Builder(this).setTitle(“Download Image?”)
.setMessage(“Do you want to download this image to your device?”)
.setPositiveButton(“Yes”){_, _ ->
val drawable:BitmapDrawable = imageView.drawable as BitmapDrawable
val bitmap = drawable.bitmap
downloadImage(bitmap)
}
.setNegativeButton(“No”) {dialog, _ ->
dialog.dismiss()
}
.show()
}else {
Toast.makeText(this, “Please allow permission to download image”, Toast.LENGTH_LONG).show()
}
}
//fun that takes a bitmap and store to user’s device
private fun downloadImage(mBitmap: Bitmap):Uri? {
val contentValues = ContentValues().apply {
put(MediaStore.Images.Media.DISPLAY_NAME,”Birds_Images”+ System.currentTimeMillis()/1000)
put(MediaStore.Images.Media.MIME_TYPE,”image/png”)
}
val uri = MediaStore.Images.Media.EXTERNAL_CONTENT_URI
if (uri != null){
contentResolver.insert(uri, contentValues)?.also {
contentResolver.openOutputStream(it).use { outputStream ->
if (!mBitmap.compress(Bitmap.CompressFormat.PNG, 100, outputStream)){
throw IOException(“Couldn’t save the bitmap”)
}
else{
Toast.makeText(applicationContext, “Image Saved”, Toast.LENGTH_LONG).show()
}
}
return it
}
}
return null
}
}
