Multi-lingual Optical Character Recognition Seminar

Abstract: "Mounting" is the UNIX (Linux, Mac OS) term that refers to transparently accessing the storage of a remote system on a local computer. In recent days I explored a particular way of accessing soteria using SSHFS ("Secure Shell File System"). This file system uses an encrypted data protocol (SFTP) to access files on the remote system. There is a great advantage of "mounting" over explicit file transfers, as the workflow for handling remote files is exactly the same as for remote files. There is a version of SSHFS for Windows, too. In this talk I will demonstrate a mount of Soteria file system.

Abstract: Finding strings in text obtained by OCR leads to the problem of "fuzzy matching". One variant can be formulated as follows: Given a string X an a text T, find all substrings of T that approximately match X. The most suitable definition of matching is based on the edit distance (Levenshtein metric). It appears that there is no good solution to the problem in terms of optimal computational complexity. The best solution I found is through Wagner–Fischer algorithm which for free gives the distance from all prefixes of a string. A demo of processing medical forms will be shown.

• UITS shared storage (5TB) to house the data (Marek)
• First look at the data (Marek, Yan)
• Using U of A network for data transfers: data rates, VPN (Marek)

Abstract: This data is our next goal towards extraction of DD data from "PDF attachments". The data features most of the difficulties that we have studied so far. Most of the data require OCR. It features tables with hand-written digits and gesture input (manually encircled text). Thus, this content requires upscaling the techniques we have developed to a larger dataset (est. 15,000 records).

Abstract: A newly funded project with participation of an undergraduate, a graduate student, aims to do OCR on a mobile platform - a drone! Scheduled to be finished by the end of the Spring semester, the project will explore the ability to incorporate OCR and LLM (such as ChatGPT) into the control loop of a drone.

Abstract: This feature offers a low effort way to play with neural networks, statistics and mathematical modeling. This flavor of ChatGPT performs computations directly, using Python (including Pytorch, etc.). I will show how one can solve a classical puzzle in AI, the XOR problem, without any programming. The ChatGPT session can be accessed here: Train 3-Layer Perceptron Network

Abstract: The 9-27-2023 version of ChatGPT features a plugin architecture. Hundreds of plugins are already available, from hundreds of vendors. With the assistance of ChatGPT, we explored the path to writing our own plugin, thus extending ChatGPT functionality to our liking. I will describe the plugin architecture and show some sample code in Python.

Abstract: Medical forms provide interesting examples in the area of OCR. In this segment, I would like to discuss an example in which the English language model results in bad page segmentation (rendering it useless for data extraction) while a custom language model (English + some Unicode characters) fixes the problem. Thus, seemingly independent tasks of spotting characters and words and segmenting text into lines, in reality are closely depending on each other. Sample codes written in MATLAB with the aid from ChatGPT will be discussed.

Abstract: This version of ChatGPT vastly expands the capabilities through the use of plugins. The chats interact with real-time data by using Web browser plugins. The OCR plugins allow inputing scanned documents. Also, documents can be generated in a variety of formats (PowerPoint, image, diagram). A tutorial and examples will be given.

In addition, this capability may be similar to a LLM called a Toolformer. The Toolformer learns to use API calls as tools via the finetuning of a standard LLM. These API calls can range from a simple database query, the query of Wolfram Alpha or a specialized LLM.

Abstract: Medical forms provide interesting examples in the area of OCR. In this segment, I would like to discuss an example in which the English language model results in bad page segmentation (rendering it useless for data extraction) while a custom language model (English + some Unicode characters) fixes the problem. Thus, seemingly independent tasks of spotting characters and words and segmenting text into lines, in reality are closely depending on each other. Sample codes written in MATLAB with the aid from ChatGPT will be discussed.