Table of Contents

  1. What Is an Optical Character Recognition Algorithm and Why Is It Useful?
  2. How Does an OCR Algorithm Work?
    1. Text Attributes for an OCR Algorithm
    2. The Process of Building, Labeling, and Training Text Recognition Algorithms
      1. — Step 1. Collecting
      2. — Step 2. Preprocessing
      3. — Step 3. Data annotation
      4. — Step 4. Training
      5. — Step 5. Post-processing and QA
  3. On a Final Note

This is the second part of a large article that tackles the topic of automated data capturing systems. We already talked about digitization of paper workflows in our previous article on automated data collection. Now, we would like to continue our discussion of automation as a great way to organize and manage your business processes efficiently.

While computer vision is that sexy field of AI that everyone likes to talk about, data annotation types of the NLP family that work with text are actually far more useful for modern business. This is especially true for medium-sized and large companies that routinely deal with massive volumes of data. Think about all the reports and protocols, presentations, corporate letters and legal forms, invoices, receipts, ID cards, and so on, and so forth.

The average document flow of a modern company is truly massive. In fact, research suggests that your employees usually spend a third of their working hours on locating the necessary information. Naturally, a document with searchable text is better than a photo or a scan of a document. You can just hit "Find" to look for the specific word or phrase, which significantly facilitates your work with the data.

Statistical data confirms that paper documents cost more to a business than the process of creating and managing digital copies. Not to mention that reliance on paper documents is one of the largest vulnerabilities of information security. These facts speak for themselves: modern businesses would be saving quite a handsome sum just by switching from analog to digital workflows. Still, while statistics makes its case against using paper, the research suggests it will not disappear in the nearest future. And this is where Optical Character Recognition algorithms appear to save the day.

What Is an Optical Character Recognition Algorithm and Why Is It Useful?

OCR – Optical Character Recognition Algorithm


Optical Character Recognition (OCR) is the type of annotation that allows the transcription of images of typed or handwritten messages into machine-recognizable text.

While it commonly goes unnoticed, OCR is an irreplaceable helper when we talk about automation. It eliminates the flow of unnecessary paper documents. It allows you to classify, organize, store, manage, and share information while avoiding security risks related to the physical nature of paper documents.

And the scope of usability for OCR gets even bigger. You have most definitely seen it in scanners for cinema tickets or at airports and train stations. It is used for data extraction and security surveillance (consider license plates of the cars or street signs). E-signatures are another form of OCR. But arguably the most common use for OCR is to turn images of business documents into digital text that can be searched, edited, and managed.

Let's imagine a situation. You attend an important meeting. Your business partner shows you a document; you take out your smartphone and take a quick photo. It seems that you have the information you need, but it's in the form of an image. You cannot work with the document directly. Instead, you need to translate the pixels of the photo into a readable format so that you could edit and process the information it contains.

In addition, OCR-based automation is not just about sharing information in a digital form. When you have a lot of documents, machines can use them as data entries to look for patterns and trends. Visualization becomes easier too: if you need a graph, a scheme, or a spreadsheet, it's much quicker to use digital documents than compile a visually pleasing report by hand. OCR allows you to spend much less time on processing each new document, to save costs on human resources, and instead focus on value-adding strategies.

How Does an OCR Algorithm Work?

People are great at recognizing text characters, even when they're handwritten. For machines, however, it is a tough task. They require a machine learning algorithm to learn how to read the way people read. For this, the OCR algorithm needs to go through a lot of training to be able to process an image of a text.

In order to understand how an OCR algorithm works, first, we'd like to tell you more about the text and its attributes. Why? Because that's how the machines see text: as part of an image.

Text Attributes for an OCR Algorithm

Text Attributes for an OCR

There is a big difference between the texts that you can find in a business setting and the texts that exist "in the wild": on the streets, in handwritten notes, in the form of CAPTCHAs, etc. A well-structured, neat scan of a quarterly report is miles away from a random graffiti caught on camera by a surveillance drone. Yet these two examples showcase a number of attributes that help to explain the image of a text to a machine learning algorithm.

  1. Density. In a document scan, the text is usually denser than in a text that you can see on the photo of a street corner.
  2. Structure. The difference is between the orderly lines of printed text vs poor structure (or lack thereof) in, say, a handwritten shopping list.
  3. Fonts and size. Rigid font and same-size letters are much easier to recognize compared to the stylistic inconsistency of street signs or a free style of handwriting.
  4. Character type. This attribute indicates the presence of not only letters but also numbers, symbols, and special characters. In addition, language is also important. A document is usually composed in a single language; on the other hand, a sign or a graffiti can include messages in several languages.
  5. Noise. It's important to pay attention to what means were used to get the image (scanning or photocopying for documents; photographing for signs and license plates). Depending on the method, photos tend to get more noise compared to scans.
  6. Location of text on the image and alignment. Scans are usually positioned front and center, with little skew. Photos, on the other hand, do not provide any semblance of a strict layout: text can be located in any part of the image, and it can also be photographed from the side.

As you can see, there's more to text than just a few rows of characters. Naturally, text attributes contribute to the nuance of building an OCR algorithm.

Now that we know how different texts can be, let's see how an OCR algorithm is built.

The Process of Building, Labeling, and Training Text Recognition Algorithms

Building, Labeling, and Training Text Recognition Algorithms

There are quite a few steps that an OCR algorithm goes through when being built from scratch.

Tip: This is a short rundown of the major steps required for building an OCR engine. If you want a more detailed breakdown, follow this link to the long-read on the lifecycle of an AI project.

— Step 1. Collecting

The first thing you need to do is to collect the database of documents. You can already possess the paper documents that you wish to digitize. But, in order to build an Optical Character Recognition algorithm, you need to choose a representative sample of sufficient size. This means that the set of documents you choose should be relevant to your final goal.

In addition, this step includes scanning, photocopying, or photographing the documents. It would greatly benefit and facilitate the training process if the quality of the images is high. Read more about the features of a good dataset in our article.

— Step 2. Preprocessing

Before you can start the recognition of the text, it is essential to prepare the images of documents, to clean and optimize them for an OCR algorithm. There are quite a few issues that lead to poor quality of images: insufficient lighting, paper glints and reflections, bad camera or scanner, skewed angles, missing or poorly printed characters, etc.

If you want to train an OCR algorithm properly, here are the things you should consider doing before the next step:

  • Turn the images to black and white. Removing colors leaves less ambiguity for the detection of the text.
  • Straighten and align. Weird angles significantly complicate the detection process.
  • Cut and center the text. Leave only what's important: the text should be front and center, not hidden somewhere in the corner.
  • Apply filters to reduce the noise. Separate characters should stand out from the background. Keep in mind that scans are usually cleaner when compared to photographs.

— Step 3. Data annotation

This is the key step for an OCR algorithm, and that's what we're here to help you with. The text recognition process consists of two tasks: detecting the text and recognition.

  1. We use boxing to highlight and contour the text areas. This tells the OCR algorithm what to look for in an image.
  2. Then, our labelers transcribe (manually put in the words) on the image. Later, the OCR algorithm will be able to use image classification to find the patterns between the set of pixels and the types of characters.

Additionally, we throw in a couple of rounds of QA. Compared to the machines, people are much better at recognizing text in the images, but even so, we like to make sure we don't miss anything.

This step of data labeling takes up a lot of time and labor, but you don't have to worry about any of it. We'll gladly take this task off your shoulders. Data annotation for OCR tasks is one of the specialties at Label Your Data. We've done it before, and we'll gladly do it again for your OCR project. Give us a call now to learn more!

— Step 4. Training

Now that you have your annotated documents, you can start training the OCR algorithm. This step depends on the type of strategy you use to build your OCR algorithm. The strategies vary widely, from classic computer vision techniques to specialized deep learning approaches based on building neural networks.

Each strategy has its advantages. But whichever method you choose, ML algorithm training usually doesn't work on the first try. Re-training and improving is common practice. Do not be disheartened if the OCR algorithm doesn't deliver the perfect accuracy of text recognition immediately. With practice and persistence, you'll get there!

— Step 5. Post-processing and QA

Actually, QA is needed on every step if you don't want to re-do all the work. But this is the step of the final QA and making your OCR algorithm live. It's time to reap the results of your hard work and finally digitize the document workflow to save time and money for your business.

On a Final Note

OCR works for any type of text

While not often discussed outside the machine learning industry, Optical Character Recognition has one of the highest usability ratings in AI. Businesses still operate based on massive volumes of paper documentation, which is an outdated and almost harmful practice. OCR can help a business deal with it by digitizing the workflow.

Moreover, the scope of application for OCR doesn't end there. Any text, whether a neatly arranged report, a random shop sign, or a handwritten note, can be processed and transformed by OCR into a machine-readable text. This is the step toward automation of the big data.

Curiously, despite building text recognition algorithms not being a new technique, it is as challenging as ever. Sure, the public has access to open-source OCR algorithms. But if you want a state-of-the-art text recognition model that would serve your specific purposes, it's best to build one for yourself. And we can help you with it! Tell us about your project, and we'll expertly annotate the documents to train your OCR algorithm.

Iryna Sydorenko by Iryna Sydorenko
on December 2, 2020.
Infinite Community

Free Infinity Membership.
Subscribe for updates:

Please read our Privacy notice

Count Me In
✔︎ Congrats! You are on the list.

Build Your AI App Faster – Outsource Data Annotation

High Quality + Certified Security

PCI DSS Level 1 Compliance   ISO 27001:213 Security Certification
Get Your Quote →