Easy on-device text recognition

If you can recognize text on iPhone, you can build handy things like transcribing whiteboards or reading signs.

👁 Image

A 2022 update made Japanese available

Since iOS 16 (2022), Japanese text recognition is possible — using only the built-in framework. The accuracy is quite good; personally I think it's usable in production for many apps.

How to use it

Use Vision's VNRecognizeTextRequest. Set recognitionLanguages to "ja". Requires macOS 13, Xcode 14, iOS 16 or later.

let request = VNRecognizeTextRequest()
request.recognitionLanguages = ["ja"] // specify Japanese
let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer)
do {
 try handler.perform([request])
} catch let error {
 print(error)
}
guard let observations = request.results as? [VNTextObservation] else { return }
for observation in observations {
 let box = observation.boundingBox // bounding box of the position
 let topCandidate = observation.topCandidates(1)
 if let recognizedText = topCandidate.first?.string { // recognized text
 print(recognizedText)
 }
}

That's it. Feed it an image or a camera frame and it recognizes the text.

👁 Image

Originally published in Japanese on Qiita. I build apps with Core ML and write about machine learning. GitHub / X