Text Recognition from Images using Firebase ML Kit in Android

5 min readJul 5, 2021

Text Recognition from Images using Firebase ML Kit in Android

In this blog, we are going to see how to create an android app to recognize text from images using the firebase ml kit. Google Firebase provides us a way to use machine learning in our app to solve real-world problems.

Here we see how to use the Text Recognition ML Kit in Android. Before going to discuss, let’s see what we’re getting:

Step 1: First, create a firebase project and link it with the android project. Make sure to add the google-services.json file in the app folder.

Step 2: Add the dependencies for firebase in the android build.gradle app module.

implementation ‘com.google.firebase:firebase-core:15.0.2’
implementation ‘com.google.firebase:firebase-ml-vision:15.0.0’

update the firebase dependencies, but don’t use above 23.0.0 for ml vision.

Step 3: Add the camera features in the AndroidManifest.xml file

<uses-feature
android:name=”android.hardware.camera”
android:required=”true” />

Step 4: Design the main XML file

<?xml version=”1.0" encoding=”utf-8"?>
<RelativeLayout xmlns:android=”http://schemas.android.com/apk/res/android"
xmlns:app=”http://schemas.android.com/apk/res-auto"
xmlns:tools=”http://schemas.android.com/tools"
android:layout_width=”match_parent”
android:layout_height=”match_parent”
tools:context=”.MainActivity”
android:padding=”16dp”
android:background=”@color/backgroundColor”>
<ImageView
android:layout_width=”80dp”
android:layout_height=”80dp”
android:src=”@drawable/side_top_left”
android:layout_alignParentStart=”true”
android:layout_alignParentTop=”true”/>
<ImageView
android:layout_width=”80dp”
android:layout_height=”80dp”
android:src=”@drawable/side_top_right”
android:layout_alignParentEnd=”true”
android:layout_alignParentTop=”true”/>
<ImageView
android:layout_width=”80dp”
android:layout_height=”80dp”
android:layout_above=”@id/linearButton”
android:src=”@drawable/side_bottom_left”
android:layout_alignParentStart=”true”
android:layout_marginBottom=”16dp”/>
<ImageView
android:layout_width=”80dp”
android:layout_height=”80dp”
android:layout_above=”@id/linearButton”
android:src=”@drawable/side_bottom_right”
android:layout_alignParentEnd=”true”
android:layout_marginBottom=”16dp”/>
<ImageView
android:layout_width=”match_parent”
android:layout_height=”wrap_content”
android:id=”@+id/image_view”
android:padding=”4dp”
android:layout_alignParentStart=”true”
android:layout_alignParentEnd=”true”
android:layout_alignParentTop=”true”
android:layout_marginTop=”20dp”
android:layout_marginStart=”20dp”
android:layout_marginBottom=”32dp”
android:layout_marginEnd=”20dp”
android:layout_centerHorizontal=”true”
android:src=”@drawable/ic_camera”
android:layout_above=”@id/linearButton”/>
<LinearLayout
android:id=”@+id/linearButton”
android:orientation=”horizontal”
android:weightSum=”2"
android:layout_width=”match_parent”
android:layout_height=”wrap_content”
android:layout_alignParentBottom=”true”
android:layout_alignParentEnd=”true”
android:layout_alignParentStart=”true”
android:layout_marginBottom=”24dp”>
<Button
android:id=”@+id/capture_image”
android:layout_weight=”1"
android:layout_width=”wrap_content”
android:layout_height=”wrap_content”
android:background=”@drawable/camera_ripple”
android:padding=”5dp”
android:text=”CAPTURE IMAGE”
android:textColor=”@android:color/white”
android:textAllCaps=”false”
android:textSize=”16sp”
android:layout_marginEnd=”8dp”/>
<Button
android:id=”@+id/detect_text_image”
android:layout_weight=”1"
android:layout_width=”wrap_content”
android:layout_height=”wrap_content”
android:layout_marginBottom=”5dp”
android:background=”@drawable/detect_ripple”
android:padding=”5dp”
android:text=”DETECT TEXT”
android:textColor=”@android:color/white”
android:textAllCaps=”false”
android:textSize=”16sp”
android:layout_marginStart=”8dp”/>
</LinearLayout>
</RelativeLayout>

Step 5: to display the extracted text from an image, we use a custom dialog to show the extracted text, for that we need to create a custom layout file. Create a custom layout (/res/layout/custom_dialog.xml).

<?xml version=”1.0" encoding=”utf-8"?>
<LinearLayout xmlns:android=”http://schemas.android.com/apk/res/android"
android:layout_width=”match_parent”
android:layout_height=”match_parent”
android:orientation=”vertical”>
<LinearLayout
android:layout_width=”match_parent”
android:layout_height=”wrap_content”
android:orientation=”vertical”
android:padding=”16dp”>
<TextView
android:layout_width=”match_parent”
android:layout_height=”wrap_content”
android:text=”@string/extracted_texts”
android:textAlignment=”center”
android:textStyle=”bold”
android:textSize=”22sp”
android:textColor=”@color/colorPrimaryDark”/>
<TextView
android:id=”@+id/text_display”
android:layout_width=”match_parent”
android:layout_height=”wrap_content”
android:layout_marginTop=”16dp”
android:gravity=”center”
android:maxLines=”8"
android:text=””
android:textSize=”14sp”
android:textColor=”@color/colorPrimaryDark” />
<Button
android:id=”@+id/buttonOk”
android:layout_width=”200dp”
android:layout_height=”wrap_content”
android:layout_gravity=”center”
android:layout_marginTop=”16dp”
android:background=”@drawable/detect_ripple”
android:text=”Ok”
android:textSize=”16sp”
android:textColor=”#FFF” />
</LinearLayout>
</LinearLayout>

Step 6: Now add the java codes to recognize the text from images.

First, we capture the image using the camera and display it in the image view.

captureImageBtn.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
dispatchTakePictureIntent();
}
});
private void dispatchTakePictureIntent() {
String fileName = “photo”;
File storageDir = getExternalFilesDir(Environment.DIRECTORY_PICTURES);
try {
File imageFile = File.createTempFile(fileName, “.jpg”, storageDir);
currentPhotoPath = imageFile.getAbsolutePath();
Uri imageUri = FileProvider.getUriForFile(MainActivity.this, “com.example.textrecognitionapp.fileprovider”, imageFile);
Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
intent.putExtra(MediaStore.EXTRA_OUTPUT, imageUri);
startActivityForResult(intent, REQUEST_IMAGE_CAPTURE);
} catch (IOException e) {
e.printStackTrace();
}
}
@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
super.onActivityResult(requestCode, resultCode, data);
if (requestCode == REQUEST_IMAGE_CAPTURE && resultCode == RESULT_OK) {
imageBitmap = BitmapFactory.decodeFile(currentPhotoPath);
imageView.setImageBitmap(imageBitmap);
}
}

Then when we click on the Detect Text button, it will call the custom dialog method as well as to detect text from the image method.

detectTextBtn.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
showCustomDialog();
detectTextFromImage();
}
});
private void showCustomDialog() {
//before inflating the custom alert dialog layout, we will get the current activity viewgroup
ViewGroup viewGroup = findViewById(android.R.id.content);
//then we will inflate the custom alert dialog xml that we created
View dialogView = LayoutInflater.from(this).inflate(R.layout.custom_dialog, viewGroup, false);
textView = (TextView)dialogView.findViewById(R.id.text_display);
okbtn = (Button)dialogView.findViewById(R.id.buttonOk);
//Now we need an AlertDialog.Builder object
AlertDialog.Builder builder = new AlertDialog.Builder(this);
//setting the view of the builder to our custom view that we already inflated
builder.setView(dialogView);
builder.setCancelable(false);
//finally creating the alert dialog and displaying it
AlertDialog alertDialog = builder.create();
alertDialog.show();
okbtn.setOnClickListener(view -> {
alertDialog.dismiss();
});
}

//detect text from image

private void detectTextFromImage() {
FirebaseVisionImage firebaseVisionImage = FirebaseVisionImage.fromBitmap(imageBitmap);
FirebaseVisionTextDetector firebaseVisionTextDetector = FirebaseVision.getInstance().getVisionTextDetector();
firebaseVisionTextDetector.detectInImage(firebaseVisionImage).addOnSuccessListener(new OnSuccessListener<FirebaseVisionText>() {
@Override
public void onSuccess(FirebaseVisionText firebaseVisionText) {
displayTextFromImage(firebaseVisionText);
}
}).addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(@NonNull Exception e) {
Toast.makeText(MainActivity.this, “Error” + e.getMessage(), Toast.LENGTH_SHORT).show();
Log.d(“Error: “,e.getMessage() );
}
});
}

//display text

private void displayTextFromImage(FirebaseVisionText firebaseVisionText) {
List<FirebaseVisionText.Block> blockList = firebaseVisionText.getBlocks();
if (blockList.size() == 0){
Toast.makeText(this, “No Text Found in this image”, Toast.LENGTH_SHORT).show();
}else {
for (FirebaseVisionText.Block block : firebaseVisionText.getBlocks()){
String text = block.getText();
textView.setText(text);
}
}
}

Full Java code of main activity

package com.example.textrecognitionapp;
import androidx.annotation.NonNull;
import androidx.appcompat.app.AlertDialog;
import androidx.appcompat.app.AppCompatActivity;
import androidx.core.content.FileProvider;
import android.content.ContentValues;
import android.content.Intent;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.net.Uri;
import android.os.Bundle;
import android.os.Environment;
import android.provider.MediaStore;
import android.util.Log;
import android.view.LayoutInflater;
import android.view.View;
import android.view.ViewGroup;
import android.widget.Button;
import android.widget.ImageView;
import android.widget.TextView;
import android.widget.Toast;
import com.google.android.gms.tasks.OnFailureListener;
import com.google.android.gms.tasks.OnSuccessListener;
import com.google.firebase.ml.vision.FirebaseVision;
import com.google.firebase.ml.vision.common.FirebaseVisionImage;
import com.google.firebase.ml.vision.text.FirebaseVisionText;
import com.google.firebase.ml.vision.text.FirebaseVisionTextDetector;
import java.io.File;
import java.io.IOException;
import java.util.List;
public class MainActivity extends AppCompatActivity {
//variables
private Button captureImageBtn, detectTextBtn, okbtn;
private ImageView imageView;
private TextView textView;
static final int REQUEST_IMAGE_CAPTURE = 1;
Bitmap imageBitmap;
private String currentPhotoPath;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
captureImageBtn = findViewById(R.id.capture_image);
detectTextBtn = findViewById(R.id.detect_text_image);
imageView = findViewById(R.id.image_view);
captureImageBtn.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
dispatchTakePictureIntent();
}
});
detectTextBtn.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
showCustomDialog();
detectTextFromImage();
}
});
}
private void showCustomDialog() {
//before inflating the custom alert dialog layout, we will get the current activity viewgroup
ViewGroup viewGroup = findViewById(android.R.id.content);
//then we will inflate the custom alert dialog xml that we created
View dialogView = LayoutInflater.from(this).inflate(R.layout.custom_dialog, viewGroup, false);
textView = (TextView)dialogView.findViewById(R.id.text_display);
okbtn = (Button)dialogView.findViewById(R.id.buttonOk);
//Now we need an AlertDialog.Builder object
AlertDialog.Builder builder = new AlertDialog.Builder(this);
//setting the view of the builder to our custom view that we already inflated
builder.setView(dialogView);
builder.setCancelable(false);
//finally creating the alert dialog and displaying it
AlertDialog alertDialog = builder.create();
alertDialog.show();
okbtn.setOnClickListener(view -> {
alertDialog.dismiss();
});
}
private void dispatchTakePictureIntent() {
String fileName = “photo”;
File storageDir = getExternalFilesDir(Environment.DIRECTORY_PICTURES);
try {
File imageFile = File.createTempFile(fileName, “.jpg”, storageDir);
currentPhotoPath = imageFile.getAbsolutePath();
Uri imageUri = FileProvider.getUriForFile(MainActivity.this, “com.example.textrecognitionapp.fileprovider”,
imageFile);
Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
intent.putExtra(MediaStore.EXTRA_OUTPUT, imageUri);
startActivityForResult(intent, REQUEST_IMAGE_CAPTURE);
} catch (IOException e) {
e.printStackTrace();
}
}
@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
super.onActivityResult(requestCode, resultCode, data);
if (requestCode == REQUEST_IMAGE_CAPTURE && resultCode == RESULT_OK) {
imageBitmap = BitmapFactory.decodeFile(currentPhotoPath);
imageView.setImageBitmap(imageBitmap);
}
}
//detect text from image
private void detectTextFromImage() {
FirebaseVisionImage firebaseVisionImage = FirebaseVisionImage.fromBitmap(imageBitmap);
FirebaseVisionTextDetector firebaseVisionTextDetector = FirebaseVision.getInstance().getVisionTextDetector();
firebaseVisionTextDetector.detectInImage(firebaseVisionImage).addOnSuccessListener(new OnSuccessListener<FirebaseVisionText>() {
@Override
public void onSuccess(FirebaseVisionText firebaseVisionText) {
displayTextFromImage(firebaseVisionText);
}
}).addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(@NonNull Exception e) {
Toast.makeText(MainActivity.this, “Error” + e.getMessage(), Toast.LENGTH_SHORT).show();
Log.d(“Error: “,e.getMessage() );
}
});
}
private void displayTextFromImage(FirebaseVisionText firebaseVisionText) {
List<FirebaseVisionText.Block> blockList = firebaseVisionText.getBlocks();
if (blockList.size() == 0){
Toast.makeText(this, “No Text Found in this image”, Toast.LENGTH_SHORT).show();
}else {
for (FirebaseVisionText.Block block : firebaseVisionText.getBlocks()){
String text = block.getText();
textView.setText(text);
}
}
}
}