rikidaniel.
Back to Portfolio

IDP - Intelligent Document Processing System

IDP - Intelligent Document Processing System

IDP – Intelligent Document Processing System

IDP (Intelligent Document Processing) is an enterprise-grade web application built to manage, store, index, and secure company documents in one central place. Powered by Tesseract OCR (Optical Character Recognition) and Elasticsearch, it turns physical scanned documents into indexed digital archives that can be searched in milliseconds. The system also connects directly to NAS (Network-Attached Storage) and provides a real-time monitoring dashboard for operational transparency across the company.

  1. Real-Time Monitoring & Analytics Dashboard
  2. Summary statistics for total documents, number of divisions/work units, and total active users.
  3. Charts showing how document storage is distributed across divisions.
  4. Monitoring of Elasticsearch document indexing status and background queues (background workers).
  5. A real-time user activity panel (recent activity log).
  6. Advanced Search (Full-Text Search, OCR & Hybrid Fallback)
  7. Fast Full-Text Search: An Elasticsearch 8 search engine based on BM25 with an Indonesian language analyzer (indonesian_analyzer), fuzzy matching, and text highlighting.
  8. Automatic Tesseract OCR: Automatic text reading and extraction from scanned PDF files and images.
  9. High-Availability Search: An automatic fallback to PostgreSQL Trigram (pg_trgm) keeps search available 100% of the time without interruption.
  10. File Explorer & NAS (Network-Attached Storage) Integration
  11. Interactive file navigation (Grid View and List View) with a modern layout.
  12. NAS Synchronization: Connects directly to the company's network storage folders (SMB/CIFS protocol).
  13. Automatic Metadata Propagation: Automatically syncs folder privacy and ownership metadata (.folder_metadata.json) from the NAS into the system database.
  14. Flexible User, Division & License Management
  15. Management of the work unit/division hierarchy and user accounts.
  16. Flexible operating modes: supports commercially licensed releases (License Key) as well as a free mode without activation.
  17. Tiered Access Rights & Data Security (Granular RBAC)
  18. Role-based access control (Super Admin, Division Admin, Employee).
  19. A matrix of 6 permission types (View, Upload, Edit, Delete, Download, Preview).
  20. Data Isolation: Strict security filtering so that non-admin users can only access their own documents or documents in public folders.
  21. Safety Confirmation Modal: Destructive actions are protected by a confirmation modal that requires typing an explicit word (such as "HAPUS") before data is deleted.
  22. Audit Log & Compliance Monitoring
  23. Complete recording of all user activity (Create, Read, Update, Delete, Download) for transparency and compliance audits.
  24. Recycle Bin (Trash & Soft Deletes)
  25. Temporary storage for deleted documents (soft delete) with restore and permanent deletion options.
  26. Multi-Language (i18n) & System Notifications
  27. Full multi-language localization (Indonesian and English) via ngx-translate.
  28. A user notification center for system status updates and document activity.

Business Benefits

  1. Efficient Archive Processing: Physical documents are digitized automatically, with no manual data entry thanks to OCR.
  2. Frictionless Information Retrieval: Find documents or specific text clauses in milliseconds.
  3. High-Level Security & Privacy: Precise access restrictions keep confidential company documents from leaking.
  4. Network Integration Without Complex Migration: Integrates directly with the company's existing NAS storage structure.
  5. Transparency & Audit Compliance: Every document's activity history is recorded neatly and ready for internal or external audits.

Technologies Used (Tech Stack)

  1. Frontend: Angular 21 (Standalone Components), PrimeNG 21 (Fluid API), TailwindCSS 4, RxJS, ngx-translate (i18n).
  2. Backend: FastAPI (Python 3.10+ Async), Clean Architecture (Presentation, Service, Repository), Pydantic v2, SQLAlchemy.
  3. Search & Database: Elasticsearch 8, PostgreSQL 15 (uuid-ossp, pg_trgm), Redis 7.
  4. Worker & OCR Engine: Celery (Asynchronous Task Queue & Beat Scheduler), Tesseract OCR Engine.
  5. Infrastructure & Security: Nginx Reverse Proxy, SSL HTTPS (SAN Extension), SMB/CIFS NAS, Systemd Daemons, Single Setup Manager (setup.sh).


Project Metadata

  1. Role: Full Stack Developer
  2. Project Type: Intelligent Document Processing (IDP) System / Enterprise Document Management
  3. Target Users: Super Admin, Division Admin, and all company employees / work units.