AushadhiAI: AI-Powered Prescription Analysis System

Introduction

AushadhiAI is an innovative solution that leverages artificial intelligence and Azure Computer Vision to decode doctors' handwritten prescriptions, making medication information accessible and understandable for patients. This technical deep dive explores the architecture, implementation details, and key features of the AushadhiAI system.

System Architecture

AushadhiAI employs a modern, scalable architecture that separates frontend and backend concerns while leveraging cloud services for advanced AI capabilities.

flowchart TD
    User[User] -->|Uploads Prescription| Frontend[Frontend UI]
    Frontend -->|HTTP Request| Backend[Backend API]
    Backend -->|Image Analysis| AzureCV[Azure Computer Vision]
    Backend -->|Medication Lookup| MedDB[(Medication Database)]
    AzureCV -->|OCR Results| Backend
    Backend -->|JSON Response| Frontend
    Frontend -->|Display Results| User

Key Components

1. Frontend: HTML, CSS, and JavaScript providing a responsive user interface
2. Backend API: FastAPI application providing RESTful endpoints for image analysis
3. Azure Vision Service: Cloud-based OCR through Azure Computer Vision API
4. Medication Service: Logic for identifying medications from extracted text
5. Medication Database: JSON-based storage of medication information

Technical Implementation Details

Backend System

The backend is built with FastAPI, a modern, high-performance web framework for building APIs with Python. It provides several key endpoints:

classDiagram
    class FastAPIApp {
        +read_root()
        +health_check()
        +get_medications()
        +get_medication_details(name)
        +analyze_prescription(file)
        +get_sample_medications()
        +check_azure()
    }

    class OCRService {
        +extract_text(image_bytes)
    }

    class MedicationService {
        +get_all_medication_names()
        +get_medication_details(name)
    }

    class AzureVisionService {
        +is_available
        +extract_text(image_bytes)
        -_fallback_extract_text(image_bytes)
    }

    class RxNormService {
        +lookup_medication(name)
    }

    FastAPIApp --> OCRService
    FastAPIApp --> MedicationService
    FastAPIApp --> AzureVisionService
    FastAPIApp --> RxNormService

Key Backend Components:

1. app.py: Main FastAPI application that defines all endpoints
2. services/azure_vision_service.py: Handles communication with Azure Computer Vision API
3. services/ocr_service.py: Manages text extraction from images with fallback mechanisms
4. services/med_service.py: Identifies medications from extracted text
5. services/rxnorm_service.py: Integrates with RxNorm for standardized medication information

Prescription Analysis Process

The prescription analysis workflow involves several steps:

sequenceDiagram
    participant User
    participant Frontend
    participant Backend
    participant AzureVision
    participant MedService

    User->>Frontend: Upload prescription image
    Frontend->>Backend: POST /api/analyze
    Backend->>AzureVision: extract_text(image)
    alt Azure available
        AzureVision-->>Backend: OCR text results
    else Azure unavailable
        AzureVision-->>Backend: Use fallback OCR
    end
    Backend->>MedService: Find medications in text
    MedService-->>Backend: Medication matches
    Backend-->>Frontend: Analysis results (JSON)
    Frontend-->>User: Display medication information

Frontend Implementation

The frontend provides an intuitive interface for users to upload and analyze prescriptions:

flowchart LR
    subgraph Frontend
        UI[User Interface] --> Upload
        Upload --> Analysis
        Analysis --> Results
    end

    subgraph Components
        Upload[Upload Component]
        Analysis[Analysis Process]
        Results[Results Display]
    end

    UI -->|User Interaction| Components

The interface includes:

1. Upload Section: For prescription image upload
2. Processing Visualization: Shows analysis progress
3. Results Display: Presents identified medications and details
4. Responsive Design: Works across desktop and mobile devices

Deployment Architecture

AushadhiAI is deployed using a modern cloud-based infrastructure:

flowchart TD
    subgraph "Frontend Deployment"
        GitHubPages[GitHub Pages]
    end

    subgraph "CI/CD Pipeline"
        GitHubActions[GitHub Actions]
    end

    subgraph "Backend Services"
        ElasticBeanstalk[AWS Elastic Beanstalk]
        ECR[Amazon ECR]
        CloudWatch[AWS CloudWatch]
    end

    subgraph "External Services"
        Azure[Azure Computer Vision]
    end

    GitHubActions -->|Deploy Frontend| GitHubPages
    GitHubActions -->|Deploy Backend| ECR
    ECR -->|Container Image| ElasticBeanstalk
    ElasticBeanstalk -->|Monitoring| CloudWatch
    ElasticBeanstalk -->|API Calls| Azure
    GitHubPages -->|API Requests| ElasticBeanstalk

Deployment Components:

1. Frontend: Hosted on GitHub Pages (static hosting)
2. Backend: Containerized with Docker and deployed on AWS Elastic Beanstalk
3. CI/CD: Automated deployment using GitHub Actions
4. Monitoring: AWS CloudWatch for performance and error tracking

System Features

1. Robust OCR Capabilities

The system uses Azure Computer Vision API for high-quality OCR, with a fallback mechanism for offline operation:

flowchart TD
    Start[Receive Image] -->|Process| AzureCheck{Azure Available?}
    AzureCheck -->|Yes| AzureOCR[Use Azure Vision API]
    AzureCheck -->|No| LocalOCR[Use Local OCR Fallback]
    AzureOCR --> TextExtraction[Extract Text]
    LocalOCR --> TextExtraction
    TextExtraction --> MedicationIdentification[Identify Medications]

2. Medication Identification

The system identifies medications using a combination of techniques:

flowchart LR
    OCRText[OCR Text] --> Preprocessing[Text Preprocessing]
    Preprocessing --> NameMatching[Medication Name Matching]
    NameMatching --> Validation[Validation]
    Validation --> DosageExtraction[Dosage Information Extraction]
    DosageExtraction --> Results[Medication Results]

3. Error Handling and Resilience

The system is designed with robust error handling:

flowchart TD
    Request[API Request] --> Validation{Input Valid?}
    Validation -->|Yes| Processing[Process Request]
    Validation -->|No| Error400[Return 400 Error]
    Processing --> ServiceCheck{Services Available?}
    ServiceCheck -->|Yes| SuccessfulResponse[Return Response]
    ServiceCheck -->|No| FallbackMechanism[Use Fallback]
    FallbackMechanism --> LimitedResponse[Return Limited Response]

Performance Considerations

graph LR
    A[Image Upload] --> B[Image Preprocessing]
    B --> C[OCR Processing]
    C --> D[Medication Identification]
    D --> E[Response Generation]

    style A fill:#f9f,stroke:#333,stroke-width:2px
    style C fill:#bbf,stroke:#333,stroke-width:4px
    style D fill:#bbf,stroke:#333,stroke-width:4px

The most computationally intensive parts of the system are:

1. OCR Processing: Handled by Azure Computer Vision to offload processing
2. Medication Identification: Optimized with efficient text matching algorithms
3. Image Preprocessing: Used to enhance OCR accuracy

Security Implementation

flowchart TD
    subgraph "Security Measures"
        CORS[CORS Policy]
        InputValidation[Input Validation]
        APIKeys[API Key Management]
        ErrorHandling[Error Handling]
    end

    Request[User Request] --> CORS
    CORS --> InputValidation
    InputValidation --> Processing[Request Processing]
    Processing --> APIKeys
    APIKeys --> ExternalService[External Services]
    Processing --> ErrorHandling
    ErrorHandling --> Response[Secure Response]

Key security considerations:

1. CORS Configuration: Prevents unauthorized cross-origin requests
2. Input Validation: Sanitizes and validates all user input
3. API Key Management: Securely stores and manages Azure API keys
4. Error Handling: Prevents information leakage in error responses

Future Enhancements

The system is designed for extensibility, with planned enhancements:

timeline
    title Development Roadmap
    Phase 1 : Basic OCR and Medication Identification
    Phase 2 : Detailed Medication Information
    Phase 3 : User Accounts and Prescription History
    Phase 4 : Mobile Application Development
    Phase 5 : Pharmacy System Integration
    Phase 6 : Multi-language Support

Conclusion

AushadhiAI represents a powerful application of AI technology to solve real-world healthcare challenges. By combining Azure Computer Vision's advanced OCR capabilities with custom medication identification algorithms, the system effectively bridges the gap between handwritten prescriptions and patient understanding.

The architecture balances performance, reliability, and user experience, with careful consideration given to fallback mechanisms that ensure the system remains functional even when cloud services are unavailable.

Through its modern deployment architecture and thoughtful technical implementation, AushadhiAI demonstrates how cloud-native applications can deliver meaningful solutions to everyday problems.