1.Abstract

With the rapid increase in vehicles worldwide, traffic violations such as overspeeding, signal jumping, illegal parking, and unauthorized vehicle usage have become a significant concern for law enforcement agencies. Efficient identification of vehicles involved in such violations is essential for ensuring road safety and implementing automated traffic management systems. License Plate Detection and Recognition (LPDR) plays a crucial role in addressing these challenges by enabling automatic extraction of vehicle identification information.

This project presents a fully automated Number Plate Detection and Recognition System that leverages advanced deep learning and vision-language models for accurate and real-time vehicle monitoring. The system integrates the YOLOv8 (You Only Look Once version 8) object detection architecture for detecting and tracking vehicles as well as identifying license plates from input images or video streams. YOLOv8 is chosen due to its high speed, efficiency, and accuracy in real-time object detection tasks.

Once the license plate region is detected, the extracted image is passed to a Vision-Language Model (VLM), specifically the Florence-2 model developed by Microsoft, for Optical Character Recognition (OCR). This model is capable of understanding visual inputs and extracting textual information with high precision. The Florence-2 model supports multiple tasks such as object detection, OCR, and visual question answering, making it highly suitable for license plate text extraction.

To enhance scalability and deployment efficiency, the system utilizes the NVIDIA NIM (NVIDIA Inference Microservices) API, which provides access to a wide range of pretrained models, including large language models and multimodal models. The API-based approach simplifies model inference and enables seamless integration of cloud-based AI capabilities into the system.

The proposed framework follows a pipeline approach: vehicle detection → license plate localization → image cropping → text extraction → result generation. By combining YOLOv8 for detection and Florence-2 for recognition, the system achieves high accuracy in identifying license plates under varying conditions such as different lighting, angles, and occlusions.This system can be effectively applied in traffic surveillance, automated toll collection, parking management, law enforcement, and smart city applications. It reduces human effort, minimizes errors, and ensures faster processing of violations. Overall, the project demonstrates the power of integrating deep learning and multimodal AI models to build an intelligent and scalable vehicle identification system.

2. Objectives

1. To detect vehicles and number plates using YOLOv8.
2. To extract text from license plates using a VLM/OCR model.
3. To automate identification of vehicles for traffic monitoring.
4. To build a real-time detection and recognition system.
5. To reduce manual effort in tracking violations.
6. To integrate detection and recognition into a unified pipeline.

3.Existing System

1. Traditional number plate detection systems rely on image processing techniques such as edge detection, thresholding, and morphological operations.
2. Optical Character Recognition (OCR) is performed using basic methods like template matching or rule-based algorithms.
3. Some systems use classical machine learning models for detection but lack real-time performance.
4. These systems are sensitive to environmental conditions such as lighting variations, shadows, and noise.
5. Manual monitoring is still required in many cases, making the system inefficient for large-scale applications.

4.Proposed System

1. Develop an Automatic Number Plate Detection and Recognition System using advanced computer vision techniques.
2. Use YOLOv8 model for real-time vehicle detection and license plate localization.
3. Implement Vision-Language Model (Florence-2) for accurate text extraction from license plates.
4. Integrate detection and recognition modules into a unified pipeline system.
5. Utilize NVIDIA NIM API for efficient model inference and scalability.
6. Design the system to work in real-time with high accuracy under different environmental conditions.

5. Implementation Procedure

Module 1: Environment Setup

1. Install Python and required libraries.
2. Set up YOLOv8 and API access for VLM (e.g., NVIDIA NIM).

Module 2: Vehicle & Plate Detection

1. Load pre-trained YOLOv8 model.
2. Detect vehicles and license plates in frames.

Module 3: Image Cropping

1. Extract the detected number plate region.
2. Preprocess image (resize, enhance clarity).

Module 4: Text Extraction (OCR/VLM)

1. Pass cropped plate image to Florence (or similar VLM).
2. Extract alphanumeric text from the image.

Module 5: Post-Processing

1. Clean and format extracted text.
2. Validate number plate format.

Module 6: Output & Storage

1. Display detected plate and text.
2. Store results in database or file system.

6.Software Requirements

Operating System                     : Windows 10 / Linux

Programming Language           : Python

Libraries                                    : OpenCV, NumPy, PyTorch / TensorFlow

Model  : YOLOv8, Florence-2 (VLM)

Frontend : HTML, CSS (optional for UI)

Backend : Python / Django / Flask

Database  : MySQL / SQLite

7.Hardware Requirements

Processor                                   : Intel i3 or above

RAM                                           : 4 GB minimum (8 GB recommended)

Hard Disk  : 500 GB

GPU  : Optional (for faster processing)

Camera                                       : For real-time input

8. Advantages of the Project

1. Fully automated number plate detection and recognition.
2. Reduces manual effort and human errors.
3. Real-time monitoring and tracking.
4. High accuracy with deep learning models.
5. Scalable for smart city and traffic systems.
6. Can be integrated with law enforcement databases.