The main goal of this project is to provide a local web server software solution to automate the extraction and exportation of data information from any short digital documents.
The documents can be receipts, invoices, contact cards, memos, prescriptions, and others.
The solution uses Tesseract OCR software to transform the documents into text data, artificial intelligence Ollama's models to recognize and extract data from the text, and Gradio technology to create the application interface.
The software is accessible in a local browser or on other devices, such as smartphones and tablets.
This software solution server architecture implements two Docker instances, one with the Application and the other with an Ollama's server.
The coding architecture implementation follows the Python module structure. The modules are implemented into three main component layers: a model layer, a controller layer, and a view layer.
Recommended hardware and system:
- Base system: Windows 11 Home system
- Processor: AMD Ryzen 5 5600 6-Core Processor - 3.50 GHz
- RAM: 32 GB
- GPU: NVIDIA GeForce RTX 4060 Ti - 18.00 Gbps - 128 bits - 8188 MB de GDDR6 - 288.032 GB/s - CUDA cores: 4352 - 2565 MHz
- Storage: 1TB SSD
The configuration above supports running the application's server with Ollama's server as one Docker Application Container and Visual Studio Software development.
This solution supports the possibility of splitting the installation of Ollama's server and application server into two separate machines. See custom install possibilities in the Custom install section.
cd <Path-repo-downloaded>
3.3. Choose your appropriate hardware.
3.3.1. Installation for NVIDIA computers.
docker compose -f .\docker-compose-nvidia.yml create
3.3.2. Installation for AMD-ROCm computers.
docker compose -f .\docker-compose-amd.yml create
3.3.3. Installation for computers without a graphic acceleration board (CPU-ONLY).
docker compose -f .\docker-compose-cpu.yml create
By default, this project installs an independent instance of Ollama's server. If you need a private installation of an Ollama server, you must change the URL and port to the server. To do this, change the variable HOST_OLLAMA_ADDRESS at ollamaCore.py source file.
You can install a Linux server without Docker Composer and customize your installation.
-
Install a Linux distribution suitable for you. You can compare different Linux desktop and server distros at Distro Watch website.
-
Install Python environment. Choose one option below:
2.1. Install anaconda. Anaconda is the easiest way to configure a Python environment.
2.2. Install the Python environment with apt software. Some Linux distros have apt-get or apt installation commands. For example:
apt install -y python13.1
- Install the required system packages.
apt update -y && apt upgrade -y && apt install -y --no-install-recommends tesseract-ocr tesseract-ocr-osd tesseract-ocr-por tesseract-ocr-eng libtesseract-dev
- Install Python package requirements
pip install --user --no-cache-dir -r requirements.txt && pip cache purge
- Set up environment variables
export PYTHONDONTWRITEBYTECODE=1
export PYTHONUNBUFFERED=1
export GRADIO_ANALYTICS_ENABLED="False"
- Run the web application.
cd <path-installed>\src
python -u mainApp.py
- Another alternative to run the application server as a service is with the "System Daemon (systemd)." To configure the systemd and read the tutorial, how to set up a service.
If everything was successful during the installation process, the web service of Light Document Manager should be running at http://localhost:7373 or http://your-machine-ip:7373.
The first time you open the web application, you will see a message 'Install at least one Ollama at Configuration tab. After that choose again the import interface.'. Then you will need to install Ollama's models.
If you install the Docker system correctly, the Ollama's models directory will be created at the root of the Light Manager repository with any model downloaded inside it. To download the models, go to the Configuration tab section, and you will see the Ollama's configuration section as shown in the image below.
After a model installation. Ollama's Configuration section should be similar to the image below.
To add or remove a Tesseract OCR language, use the two dropdown textboxes and two buttons in the Other Configuration section at the Configuration Tab. See the image below.
<img src="imgDocumentation/LightDocumentManager/OtherConfiguration.png" alt="Other configuration interface. The image shows two dropdown lists and buttons: "Add Language" and "Remove Language." One of the dropdown lists is to select a language to add to the system, and another is to remove a language from the system." width="100%"/>
The only directory that is necessary to clean in time to time is the temporary directory /tmp/gradio located on the application server. It's good to clean because the system generates results files and those files is not cleaned automatically.
-
Check if Docker Desktop needs to be updated
-
Cleanup Gradio temporary files with the Configuration tab on the Light Document Manager web application. All your data is transferred to the web application or processed.
-
Update the image of Ollama's web server if necessary.
-
Remove unnecessary Ollama's models.
-
Update the version of the Light Document Manager by downloading or cloning the latest version.
-
Check for security vulnerabilities in Docker's images with Docker Scout.
-
If necessary, update the Python Docker image version in the file "manager.Dockerfile".
In the current version, this software has some limitations concerning:
- Ollama's models used to categorize the documents;
- Ollama's models have restrictions on which type of language can operate with the input text.
- This software is only able to process text information with typewriter fonts. Other kinds of graphical information, such as handwriting or a checkbox field in a form, can't be interpreted.
- Don't support data lake chunks;
- Maybe you can get a wrong data interpretation if a document contains too many pages;
- Supported types of digital documents:
- .gif
- .jpe
- .jpeg
- .jpeg2000
- .jpg
- .pbm
- .pgm
- .png
- .ppm
- .tiff
- .webp
- .svg