Image Classification Android App with TensorFlow Lite for Beginner

Today, Machine Learning (ML) is all over the place. The ML reduces human works, suppose I’m trying to identify birds, as a human what we generally do we search for every bird that is present in this world and identify the particular one. But Machine Learning reduces that, in ML we will simply create one model which involves every bird. Then when I tried to search for any birds, that model will simply tell me which bird it is.

So in this blog, we will see how to connect a tflite model in android or how to use a tflite model in the android app.

In this tutorial, we’ll look at how to use TensorFlow Lite to integrate machine learning into an Android app by creating a basic app.

TensorFlow Lite

TensorFlow Lite is a set of tools that enables on-device machine learning by helping developers run their models on mobile, embedded, and IoT devices.

Key Features of TF-Lite:

  • Multiple platform support, covering Android and iOS devices, embedded Linux, and microcontrollers.
  • Multiple language support, which includes Java, Swift, Objective-C, C++, and Python.
  • High performance, with hardware acceleration and model optimization.

In order to start, you will require a trained .tflite model that you can download from here.

Note: after downloading name this file as BirdsModel.tflite.

Add TensorFlow Lite to the Android app.

Step 1: Head over to android studio & Create a new android project.

Step 2: Add TensorFlow Lite to the Android App.

Right-click on the package name in my case it is com.yourpackagename or click on File, then New > Other > TensorFlow Lite Model. Select the model location where you have downloaded the custom trained BirdsModel.tflite earlier

Step 3: Click finish.

Note that the tooling will automatically configure the module’s dependencies for you using ML Model binding and all requirements will be added into your Android module’s build.gradle file.

Step 4: At the end, you’ll see the following. The BirdModel.tflite file has been successfully imported, and it displays high-level model information like as input and output, as well as some sample code to get you started.

Processing image and showing result

following are the simple steps to implement the bird classification model.

1. Create Tensor Flow Lite Model variable and initialize it.

val birdModel = BirdsModel.newInstance(this)

2. Converting input image into tensor flow image. as you can see input image is require to be in bitmap format.

val tfImage = TensorImage.fromBitmap(bitmap)

3. Processing the image and then sorting output results into descending order then picking top result (pick out highest probability one’s).

val outputs = birdModel.process(tfImage)

.probabilityAsCategoryList.apply {

sortByDescending { it.score }

}

//getting result having high probability

val highProbabilityOutput = outputs[0]

4. Dispalying the result into TextView

tvOutput.text = highProbabilityOutput.label

Prepare layout for Your app

This layout contains following views,

  • Image view, for displaying our captured bird photo.
  • Button, for launching the camera to take the photo.
  • Text view, to display output result.

<?xml version=”1.0" encoding=”utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android=”http://schemas.android.com/apk/res/android"
xmlns:app=”http://schemas.android.com/apk/res-auto"
xmlns:tools=”http://schemas.android.com/tools"
android:layout_width=”match_parent”
android:layout_height=”match_parent”
tools:context=”.MainActivity”>
<Button
android:layout_width=”wrap_content”
android:layout_height=”wrap_content”
android:id=”@+id/btn_load_image”
android:text=”Load Image”
app:layout_constraintEnd_toStartOf=”@id/guideline2"
app:layout_constraintStart_toStartOf=”parent”
app:layout_constraintTop_toBottomOf=”@+id/imageView”
android:layout_marginTop=”18dp”/>
<ImageView
android:layout_width=”300dp”
android:layout_height=”450dp”
android:id=”@+id/imageView”
android:layout_marginTop=”16dp”
app:layout_constraintEnd_toEndOf=”parent”
app:layout_constraintStart_toStartOf=”parent”
app:layout_constraintTop_toTopOf=”parent”
app:layout_constraintHorizontal_bias=”0.5"
android:src=”@drawable/place_holder”/>
<Button
android:layout_width=”wrap_content”
android:layout_height=”wrap_content”
android:id=”@+id/btn_capture_image”
android:text=”Take Image”
app:layout_constraintEnd_toEndOf=”parent”
app:layout_constraintStart_toStartOf=”@id/guideline2"
app:layout_constraintTop_toBottomOf=”@id/imageView”
android:layout_marginTop=”18dp”/>
<TextView
android:layout_width=”wrap_content”
android:layout_height=”wrap_content”
android:id=”@+id/textview”
android:text=”Output: “
android:textSize=”21sp”
android:layout_marginTop=”24dp”
app:layout_constraintStart_toStartOf=”parent”
app:layout_constraintTop_toBottomOf=”@id/btn_capture_image”
android:layout_marginStart=”36dp”
android:textColor=”@android:color/black”/>
<TextView
android:layout_width=”wrap_content”
android:layout_height=”wrap_content”
android:id=”@+id/tv_output”
android:text=”Result here”
android:textSize=”21sp”
android:textColor=”@android:color/holo_red_dark”
app:layout_constraintStart_toEndOf=”@id/textview”
app:layout_constraintTop_toBottomOf=”@id/btn_capture_image”
android:layout_marginTop=”24dp” />
<TextView
android:layout_width=”wrap_content”
android:layout_height=”wrap_content”
android:text=”Click on the result text to search on Google”
app:layout_constraintTop_toBottomOf=”@id/textview”
android:layout_marginTop=”8dp”
android:layout_marginStart=”36dp”
app:layout_constraintStart_toStartOf=”parent”
android:textSize=”13sp”/>

<androidx.constraintlayout.widget.Guideline
android:layout_width=”wrap_content”
android:layout_height=”wrap_content”
android:id=”@+id/guideline2"
app:layout_constraintGuide_percent=”0.5"
android:orientation=”vertical”/>
</androidx.constraintlayout.widget.ConstraintLayout>

Complete MainActivity.Kt

package com.codewithgolap.birdclassificationkotlin

import android.app.Activity

import android.app.Instrumentation

import android.content.ContentValues

import android.content.Intent

import android.content.pm.PackageManager

import android.graphics.Bitmap

import android.graphics.BitmapFactory

import android.graphics.drawable.BitmapDrawable

import android.net.Uri

import androidx.appcompat.app.AppCompatActivity

import android.os.Bundle

import android.provider.MediaStore

import android.util.Log

import android.widget.Button

import android.widget.ImageView

import android.widget.TextView

import android.widget.Toast

import androidx.activity.result.ActivityResult

import androidx.activity.result.contract.ActivityResultContracts

import androidx.appcompat.app.AlertDialog

import androidx.core.content.ContextCompat

import com.codewithgolap.birdclassificationkotlin.databinding.ActivityMainBinding

import com.codewithgolap.birdclassificationkotlin.ml.BirdsModel

import org.tensorflow.lite.support.image.TensorImage

import java.io.IOException

class MainActivity : AppCompatActivity() {

private lateinit var binding: ActivityMainBinding

private lateinit var imageView: ImageView

private lateinit var button: Button

private lateinit var tvOutput: TextView

private val GALLERY_REQUEST_CODE = 123

override fun onCreate(savedInstanceState: Bundle?) {

super.onCreate(savedInstanceState)

binding = ActivityMainBinding.inflate(layoutInflater)

val view = binding.root

setContentView(view)

imageView = binding.imageView

button = binding.btnCaptureImage

tvOutput = binding.tvOutput

val buttonLoad = binding.btnLoadImage

button.setOnClickListener {

if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.CAMERA)

== PackageManager.PERMISSION_GRANTED

) {

takePicturePreview.launch(null)

}

else {

requestPermission.launch(android.Manifest.permission.CAMERA)

}

}

buttonLoad.setOnClickListener {

if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.READ_EXTERNAL_STORAGE)

== PackageManager.PERMISSION_GRANTED){

val intent = Intent(Intent.ACTION_PICK, MediaStore.Images.Media.EXTERNAL_CONTENT_URI)

intent.type = “image/*”

val mimeTypes = arrayOf(“image/jpeg”,”image/png”,”image/jpg”)

intent.putExtra(Intent.EXTRA_MIME_TYPES, mimeTypes)

intent.flags = Intent.FLAG_GRANT_READ_URI_PERMISSION

onresult.launch(intent)

}else {

requestPermission.launch(android.Manifest.permission.READ_EXTERNAL_STORAGE)

}

}

//to redirct user to google search for the scientific name

tvOutput.setOnClickListener {

val intent = Intent(Intent.ACTION_VIEW, Uri.parse(“https://www.google.com/search?q=${tvOutput.text}"))

startActivity(intent)

}

// to download image when longPress on ImageView

imageView.setOnLongClickListener {

requestPermissionLauncher.launch(android.Manifest.permission.WRITE_EXTERNAL_STORAGE)

return@setOnLongClickListener true

}

}

//request camera permission

private val requestPermission = registerForActivityResult(ActivityResultContracts.RequestPermission()){granted->

if (granted){

takePicturePreview.launch(null)

}else {

Toast.makeText(this, “Permission Denied !! Try again”, Toast.LENGTH_SHORT).show()

}

}

//launch camera and take picture

private val takePicturePreview = registerForActivityResult(ActivityResultContracts.TakePicturePreview()){bitmap->

if(bitmap != null){

imageView.setImageBitmap(bitmap)

outputGenerator(bitmap)

}

}

//to get image from gallery

private val onresult = registerForActivityResult(ActivityResultContracts.StartActivityForResult()){result->

Log.i(“TAG”, “This is the result: ${result.data} ${result.resultCode}”)

onResultReceived(GALLERY_REQUEST_CODE,result)

}

private fun onResultReceived(requestCode: Int, result: ActivityResult?){

when(requestCode){

GALLERY_REQUEST_CODE ->{

if (result?.resultCode == Activity.RESULT_OK){

result.data?.data?.let{uri ->

Log.i(“TAG”, “onResultReceived: $uri”)

val bitmap = BitmapFactory.decodeStream(contentResolver.openInputStream(uri))

imageView.setImageBitmap(bitmap)

outputGenerator(bitmap)

}

}else {

Log.e(“TAG”, “onActivityResult: error in selecting image”)

}

}

}

}

private fun outputGenerator(bitmap: Bitmap){

//declearing tensor flow lite model variable

val birdsModel = BirdsModel.newInstance(this)

// converting bitmap into tensor flow image

val newBitmap = bitmap.copy(Bitmap.Config.ARGB_8888, true)

val tfimage = TensorImage.fromBitmap(newBitmap)

//process the image using trained model and sort it in descending order

val outputs = birdsModel.process(tfimage)

.probabilityAsCategoryList.apply {

sortByDescending { it.score }

}

//getting result having high probability

val highProbabilityOutput = outputs[0]

//setting ouput text

tvOutput.text = highProbabilityOutput.label

Log.i(“TAG”, “outputGenerator: $highProbabilityOutput”)

}

// to download image to device

private val requestPermissionLauncher = registerForActivityResult(ActivityResultContracts.RequestPermission()){

isGranted: Boolean ->

if (isGranted){

AlertDialog.Builder(this).setTitle(“Download Image?”)

.setMessage(“Do you want to download this image to your device?”)

.setPositiveButton(“Yes”){_, _ ->

val drawable:BitmapDrawable = imageView.drawable as BitmapDrawable

val bitmap = drawable.bitmap

downloadImage(bitmap)

}

.setNegativeButton(“No”) {dialog, _ ->

dialog.dismiss()

}

.show()

}else {

Toast.makeText(this, “Please allow permission to download image”, Toast.LENGTH_LONG).show()

}

}

//fun that takes a bitmap and store to user’s device

private fun downloadImage(mBitmap: Bitmap):Uri? {

val contentValues = ContentValues().apply {

put(MediaStore.Images.Media.DISPLAY_NAME,”Birds_Images”+ System.currentTimeMillis()/1000)

put(MediaStore.Images.Media.MIME_TYPE,”image/png”)

}

val uri = MediaStore.Images.Media.EXTERNAL_CONTENT_URI

if (uri != null){

contentResolver.insert(uri, contentValues)?.also {

contentResolver.openOutputStream(it).use { outputStream ->

if (!mBitmap.compress(Bitmap.CompressFormat.PNG, 100, outputStream)){

throw IOException(“Couldn’t save the bitmap”)

}

else{

Toast.makeText(applicationContext, “Image Saved”, Toast.LENGTH_LONG).show()

}

}

return it

}

}

return null

}

}

Final Result:

Watch the full video on youtube:

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Golap Gunjan Barman

Hi everyone, myself Golap an Android app developer with UI/UX designer.