OCR - Optical character recognition

What is optical character recognition: OCR explained

Optical character recognition is the automated reading of text out of an image or a scanned document, turning pixels that look like characters into characters a computer can store, search and match. The abbreviation OCR is used far more often than the spelled-out form, and both refer to the same task.

OCR is often sold as a solved problem. It is solved for one specific case, and that case is narrower than the demo suggests.

What does OCR mean in practice?

A typical pipeline runs in three stages. First it detects where in the image text is likely to be. Then it segments those regions into lines and characters. Finally a recognition model converts each region into a character sequence, usually with a language model layered on top to prefer real words over letter salad.

Each stage can fail independently, which is why OCR output degrades in strange ways rather than degrading smoothly. Text that is never detected produces no error message at all. It simply is not in the output.

How accurate is optical character recognition?

Accuracy is normally reported as character error rate or as word error rate, the same substitutions plus deletions plus insertions formula used for speech to text. On clean, high-resolution, printed documents, published benchmarks sit below 1 percent character error rate. On handwriting the picture changes: state-of-the-art systems on the IAM handwriting benchmark report character error rates in the low single digits at best, and mixed print-and-cursive documents remain the hardest case in document processing.

The wider research community made this distinction explicit long ago. The ICDAR Robust Reading competitions were created specifically for text "beyond the capabilities of current commercial OCR packages," meaning text photographed in natural scenes rather than scanned off a flatbed. The existence of a separate competition is the tell.

Where OCR actually breaks

Microsoft's published limitations page for its own OCR service is unusually blunt, and it maps closely to what field teams see. Scan quality, resolution, contrast, lighting, rotation, and text size, colour and density all affect accuracy, and the service recommends a minimum of 50 by 50 pixels before it will attempt an image at all.

• Stamped or etched metal. Serial plates, chassis numbers and rating plates have no ink contrast. The characters are shape, not colour, so recognition depends entirely on the angle of the light.
• Curved and non-planar surfaces. A label wrapped round a cylinder distorts character geometry in a way most models were never trained on.
• Glare and specular highlights. A phone flash on a laminated or metal surface removes characters outright rather than dimming them.
• Dirt, corrosion, wear and paint overspray. Partial characters get confidently read as different characters.
• Handwriting. Weaker in every engine, and some engines support handwriting for a limited set of languages only.
• Angle and distance. A person photographing an identifier from a comfortable standing position usually produces text too small and too skewed for reliable reading.

Notice that this list is a good description of every real identifier capture that is not a scanned page. Clean printed text is the demo. Stamped, dirty, curved and badly lit is the job.

OCR explained: a worked example

A technician photographs a machine rating plate at an angle in a dim plant room. The engine returns the model number with a character error rate near zero and returns the serial number as a string in which "8" has become "B" and "0" has become "D". Both fields carry high confidence scores. Downstream, the warranty lookup for that serial fails, and the operator is told the asset does not exist.

How to use OCR without getting burned

The workable pattern is confidence thresholds plus a human. Microsoft's guidance is explicit about this: pick a threshold, send results above it to straight-through processing and route everything below it to human review, and choose the threshold according to what a wrong read actually costs you. A pharmaceutical labelling case and an e-commerce catalogue import should not use the same number.

Three habits do most of the work. Validate every OCR result against a checksum or a database of known-valid identifiers where one exists, since a VIN or an IBAN can be checked arithmetically and a guessed character will usually fail that check. Ask the person capturing the image to fill the frame with the identifier rather than the whole asset. And keep the original photograph attached to the record, because when the read is wrong the image is the only thing that lets a reviewer fix it. That last point is why OCR belongs inside a documented capture flow with the source image retained, rather than as a one-way extraction step, and why identifier photographs are treated as part of equipment verification evidence rather than as disposable input.