Context-Aware Task Automation using Offline LLM

The rapid growth of intelligent automation systems has increased the demand for tools capable of understanding natural language instructions and executing complex tasks autonomously. Many existing solutions rely heavily on cloud-based large language models (LLMs), raising concerns regarding privacy, latency, and internet dependency. This paper presents a context-aware task automation system that operates entirely offline using the LLaMA-2 large language model. The proposed system interprets natural language commands and converts them into structured task plans decomposed into executable subtasks in JSON format. A dual-context management framework maintains both persistent user context and dynamic task context, enabling adaptive decision-making. The system integrates OmniParser for visual UI parsing and Tesseract OCR for text extraction from real-time screenshots. A feedback-driven execution loop continuously monitors the UI state, validates actions, and replans steps when discrepancies are detected. Experimental evaluation across 16 multi-step desktop tasks demonstrates a task success rate of 92.1%, covering application launching, website navigation, form filling, and message automation.