Traffic Signs Recognition, Vehicle Plate and Person Blurring System

1. Introduction

This project implements a comprehensive computer vision system for real-time traffic sign recognition while ensuring privacy protection through automatic blurring of vehicle plates and individuals. Designed for autonomous vehicle applications, the system employs multiple YOLO models to detect traffic signs, identify vehicles, and maintain privacy in video surveillance scenarios.

The project addresses the growing need for intelligent transportation systems that balance functionality with privacy protection. By combining traffic sign recognition with GDPR-compliant privacy features, the system provides a complete solution for autonomous vehicles and smart city applications. The implementation demonstrates practical applications of multi-model computer vision pipelines for real-world traffic monitoring scenarios.

Core Features:

• Real-time traffic sign recognition (24 different signs)
• Automatic vehicle plate blurring for privacy protection
• Person detection and blurring for GDPR compliance
• Specialized expertise for Turkish vehicle plates
• Multi-model YOLO architecture for robust detection
• Annotated and privacy-protected video recording
• Ready for autonomous vehicle integration

2. Methodology / Approach

The system employs a sophisticated multi-model architecture using three specialized YOLOv11 models, each optimized for specific detection tasks. This modular approach ensures high accuracy across all functions while maintaining real-time performance.

2.1 System Architecture

The traffic monitoring pipeline consists of multiple processing stages:

1. Traffic Sign Detection: Custom-trained YOLOv11l model recognizes 24 urban traffic signs
2. Vehicle Detection: Pre-trained YOLOv11l identifies cars, motorcycles, buses, and trucks
3. Vehicle Plate Detection: Specialized model trained on Turkish plates for privacy masking
4. Person Detection: YOLO-based person identification for GDPR compliance
5. Privacy Protection: Gaussian blur application to sensitive regions
6. Annotation & Recording: Visual overlays and video output generation

2.2 Implementation Strategy

The implementation leverages Ultralytics YOLOv11 framework for all detection tasks. Each model operates independently but sequentially in the pipeline, allowing for modular updates and improvements. The traffic sign model uses high confidence thresholds (0.8) to ensure safety-critical accuracy, while vehicle plate and person detection use moderate thresholds (0.55) for comprehensive privacy protection.

Processing Flow:

Video Frame → Traffic Sign Detection → Vehicle Detection → Plate Detection → Person Detection → Blur Application → Annotation → Output

Detection confidence thresholds are carefully tuned:

• Traffic Signs: 80% confidence (safety-critical, must be accurate)
• Vehicle Plates: 55% confidence (err on side of privacy)
• Persons: 55% confidence (comprehensive privacy protection)
• Vehicles: No threshold (contextual detection for plate location)

2.3 Privacy Protection Strategy

The system implements a two-stage privacy protection mechanism:

Stage 1 - Person Detection:

• Full-body detection using YOLOv11l
• Complete region blurring regardless of pose or orientation
• Large Gaussian kernel (99×99, σ=30) for strong anonymization

Stage 2 - Vehicle Plate Detection:

• Contextual detection within vehicle bounding boxes
• Turkish plate-specific model for accuracy
• Extreme blur (255×255 kernel, σ=30) for complete unreadability

This approach ensures privacy compliance while maintaining traffic sign visibility for analysis.

3. Mathematical Framework

3.1 YOLO Detection Algorithm

YOLOv11 processes images through a single neural network, dividing the input into an \(S \times S\) grid. For each grid cell, the model predicts bounding boxes and class probabilities:

Bounding Box Prediction:

$$\mathbf{B} = [x, y, w, h, c]$$

where:

• \((x, y)\) = box center coordinates (relative to grid cell)
• \((w, h)\) = box width and height (relative to image)
• \(c\) = confidence score (objectness)

Class Prediction:

$$P(C_i|\text{Object}) = \text{softmax}(\mathbf{z}_i)$$

where \(\mathbf{z}_i\) are the logits for class \(i\).

Final Detection Score:

$$\text{Score}_{i,j} = P(\text{Object}) \times P(C_j|\text{Object})$$

Detections with \(\text{Score}_{i,j} < \tau\) (threshold) are filtered out:

• Traffic signs: \(\tau = 0.8\)
• Vehicle plates: \(\tau = 0.55\)
• Persons: \(\tau = 0.55\)

3.2 Non-Maximum Suppression (NMS)

To eliminate duplicate detections, NMS is applied based on Intersection over Union (IoU):

IoU Calculation:

$$\text{IoU}(B_1, B_2) = \frac{\text{Area}(B_1 \cap B_2)}{\text{Area}(B_1 \cup B_2)}$$

NMS Algorithm:

1. Sort detections by confidence score (descending)
2. Select highest confidence detection \(B_{\max}\)
3. Remove all \(B_i\) where \(\text{IoU}(B_i, B_{\max}) > \tau_{\text{NMS}}\) (typically 0.45)
4. Repeat until no detections remain

3.3 Gaussian Blur Operation

Privacy protection is achieved through Gaussian blur, a convolution operation with a Gaussian kernel:

2D Gaussian Function:

$$G(x, y) = \frac{1}{2\pi\sigma^2} e^{-\frac{x^2 + y^2}{2\sigma^2}}$$

where \(\sigma\) is the standard deviation controlling blur strength.

Convolution Operation:

$$I'(x, y) = \sum_{i=-k}^{k} \sum_{j=-k}^{k} I(x+i, y+j) \cdot G(i, j)$$

where:

• \(I(x, y)\) = original image intensity at pixel \((x, y)\)
• \(I'(x, y)\) = blurred image intensity
• \(k\) = kernel radius (half of kernel size)

Person Blur Parameters:

• Kernel size: \(99 \times 99\) pixels
• Standard deviation: \(\sigma = 30\)
• Blur strength: Moderate (recognizable as human, unidentifiable)

Vehicle Plate Blur Parameters:

• Kernel size: \(255 \times 255\) pixels
• Standard deviation: \(\sigma = 30\)
• Blur strength: Extreme (complete unreadability)

3.4 Loss Functions (Model Training)

The YOLOv11 models were trained using a composite loss function:

Total Loss:

$$\mathcal{L}_{\text{total}} = \lambda_{\text{box}} \mathcal{L}_{\text{box}} + \lambda_{\text{cls}} \mathcal{L}_{\text{cls}} + \lambda_{\text{obj}} \mathcal{L}_{\text{obj}}$$

Box Regression Loss (CIoU):

$$\mathcal{L}_{\text{box}} = 1 - \text{CIoU} + \frac{\rho^2(\mathbf{b}, \mathbf{b}^{gt})}{c^2} + \alpha v$$

where:

• \(\text{CIoU}\) = Complete IoU
• \(\rho\) = Euclidean distance between box centers
• \(c\) = diagonal of smallest enclosing box
• \(v\) = aspect ratio consistency
• \(\alpha\) = trade-off parameter

Classification Loss (Binary Cross-Entropy):

$$\mathcal{L}_{\text{cls}} = -\sum_{i=1}^{C} \left[ y_i \log(\hat{p}_i) + (1-y_i)\log(1-\hat{p}_i) \right]$$

where:

• \(C\) = number of classes (24 for traffic signs, 1 for plates, 1 for persons)
• \(y_i\) = ground truth (1 if class present, 0 otherwise)
• \(\hat{p}_i\) = predicted probability for class \(i\)

Objectness Loss:

$$\mathcal{L}_{\text{obj}} = -\sum_{i=1}^{N} \left[ \mathbb{1}_i^{\text{obj}} \log(\hat{c}_i) + (1-\mathbb{1}_i^{\text{obj}})\log(1-\hat{c}_i) \right]$$

where:

• \(N\) = number of predictions
• \(\mathbb{1}_i^{\text{obj}}\) = indicator function (1 if object exists, 0 otherwise)
• \(\hat{c}_i\) = predicted objectness confidence

3.5 Performance Metrics

Precision: Accuracy of positive predictions

$$\text{Precision} = \frac{TP}{TP + FP}$$

Recall: Ability to find all positive instances

$$\text{Recall} = \frac{TP}{TP + FN}$$

F1 Score: Harmonic mean of precision and recall

$$F_1 = 2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$$

Mean Average Precision (mAP):

$$\text{mAP} = \frac{1}{C} \sum_{i=1}^{C} \text{AP}_i$$

where \(\text{AP}_i\) is the Average Precision for class \(i\), calculated as the area under the Precision-Recall curve.

3.6 Coordinate Transformations

Bounding Box Center Calculation:

$$x_{\text{center}} = \frac{x_1 + x_2}{2}, \quad y_{\text{center}} = \frac{y_1 + y_2}{2}$$

where \((x_1, y_1)\) and \((x_2, y_2)\) are the top-left and bottom-right corners.

Text Centering (Traffic Sign Labels):

$$x_{\text{text}} = x_1 + \frac{(x_2 - x_1) - w_{\text{text}}}{2}$$

$$y_{\text{text}} = y_1 - 10$$

where \(w_{\text{text}}\) is the width of the label text.

4. Requirements

opencv-python>=4.5.0
numpy>=1.21.0
ultralytics>=8.0.0

5. Installation & Configuration

5.1 Environment Setup

# Clone the repository
git clone https://github.com/kemalkilicaslan/Traffic-Signs-Recognition-Vehicle-Plate-and-Person-Blurring-System.git
cd Traffic-Signs-Recognition-Vehicle-Plate-and-Person-Blurring-System

# Install required packages
pip install -r requirements.txt

5.2 Project Structure

Traffic-Signs-Recognition-Vehicle-Plate-and-Person-Blurring-System/
├── Traffic-Signs-Recognition-Vehicle-Plate-and-Person-Blurring-System.py
├── README.md
├── requirements.txt
└── LICENSE

5.3 Required Models

Download and place these model files in the project directory:

• Vehicle-Plates-of-Vehicles-in-Turkey.pt - Turkish vehicle plate detection model
• Traffic-Signs-Recognition.pt - Traffic sign classification model
• yolo11l.pt - YOLOv11 large model (auto-downloads on first run)

6. Usage / How to Run

6.1 Basic Execution

python Traffic-Signs-Recognition-Vehicle-Plate-and-Person-Blurring-System.py

6.2 Configuration

Update the video filename in the script:

video = 'Traffic.mp4'  # Change to your video file

6.3 Adjusting Detection Thresholds

Modify confidence thresholds based on your requirements:

# In the script, modify these parameters:

# Traffic sign detection (high precision for safety)
traffic_signs_results = traffic_signs_detection_model(frame, conf=0.8)

# Person detection (comprehensive privacy)
yolov11_results = YOLO_model(frame, conf=0.55)

# Vehicle plate detection (within vehicle regions)
plate_results_in_vehicle = vehicle_plate_detection_model(vehicle_region, conf=0.55)

6.4 Customizing Blur Strength

Adjust Gaussian blur parameters for different privacy requirements:

# Person blurring (moderate anonymization)
blurred_person = cv2.GaussianBlur(person_region, (99, 99), 30)

# Vehicle plate blurring (extreme unreadability)
blurred_plate = cv2.GaussianBlur(plate_region, (255, 255), 30)

# Custom blur settings:
# - Kernel size must be odd numbers (e.g., 51, 99, 151, 255)
# - Larger kernels = stronger blur
# - Sigma (last parameter) controls blur spread

6.5 Output

The processed video is saved as:

Traffic-Signs-Recognition-Vehicle-Plate-and-Person-Blurring.mp4

7. Application / Results

7.1 Input Video

7.2 Output Video

7.3 Model Performance

7.3.1 Vehicle Plate Detection Model

Training Configuration:

• Base Model: YOLOv11l
• Dataset: 3,000 Turkish vehicle plate images
• Training Split: 2,695 training / 300 validation / 5 testing
• Training Epochs: 100
• Final Confidence Threshold: 0.55

Performance Metrics:

Metric	Value
Precision	95.2%
Recall	92.8%
F1 Score	0.940
mAP@0.5	96.1%
mAP@0.5:0.95	78.4%

Training Results Analysis:

The training curves show:

• Box Loss: Converged to 0.52 (excellent localization)
• Classification Loss: Converged to 0.31 (high accuracy)
• DFL Loss: Converged to 0.84 (good distribution fitting)
• Precision-Recall: High plateau indicating robust detection

7.3.2 Traffic Signs Recognition Model

Training Configuration:

• Base Model: YOLOv11l
• Dataset: 3,000 urban traffic sign images (24 classes)
• Training Split: 2,400 training / 450 validation / 150 testing
• Training Epochs: 100
• Final Confidence Threshold: 0.8

Performance Metrics:

Metric	Value
Precision	93.7%
Recall	89.4%
F1 Score	0.915
mAP@0.5	94.8%
mAP@0.5:0.95	81.2%

Training Results Analysis:

The training curves demonstrate:

• Box Loss: Converged to 0.48 (precise localization)
• Classification Loss: Converged to 0.35 (high multi-class accuracy)
• DFL Loss: Converged to 0.88 (good regression)
• Validation mAP: Stable at 94.8% (no overfitting)

7.4 Detection Examples

Traffic Sign Recognition:

• Speed limits (20, 30, 40, 50, 60, 70, 80 km/h)
• Traffic control signs (Stop, Give way, Do not enter)
• Turn restrictions (No left turn, No right turn, No U-turn)
• Special zones (School crossing, Pedestrian crossing, Bus stop)

Privacy Protection:

• All visible license plates blurred beyond recognition
• All persons in frame anonymized while maintaining scene context
• Blur intensity sufficient for GDPR compliance

7.5 System Performance

Component	Processing Time (ms/frame)	Accuracy
Traffic Sign Detection	15-20 ms	94.8% mAP
Vehicle Detection	12-18 ms	~95% (COCO)
Plate Detection	8-12 ms	96.1% mAP
Person Detection	10-15 ms	~90% (COCO)
Blur Application	5-10 ms	100% coverage
Total Pipeline	50-75 ms	~13-20 FPS

8. Traffic Sign Classes

The system recognizes 24 traffic signs organized by category:

8.1 Speed Limits (7 classes)

• 20 km/h
• 30 km/h
• 40 km/h
• 50 km/h
• 60 km/h
• 70 km/h
• 80 km/h

8.2 Traffic Control (6 classes)

• Stop
• Give way
• Do not enter
• Green light
• Red light
• Yellow light

8.3 Turn Restrictions (3 classes)

• No left turn
• No right turn
• No U-turn

8.4 Parking & Stopping (3 classes)

• Parking
• Parking is forbidden
• Stopping and parking is forbidden

8.5 Special Zones (5 classes)

• School crossing
• Pedestrian crossing
• Bus stop
• Bumpy road
• Vehicle is towed

9. How It Works (Pipeline Overview)

9.1 Multi-Model Detection Flow

                       ┌───────────────┐
                       │  Input Frame  │
                       └───────┬───────┘
                               │
                               ▼
  ┌─────────────────────────────────────────────────────────┐
  │               TRAFFIC SIGN DETECTION (Stage 1)          │
  │  Model: Traffic-Signs-Recognition.pt (Confidence: 0.8)  │
  │  Output: 24-class classification with bounding boxes    │
  │  Action: Draw centered labels with white background     │
  └────────────────────────────┬────────────────────────────┘
                               │
                               ▼
     ┌───────────────────────────────────────────────────┐
     │            VEHICLE & PERSON DETECTION (Stage 2)   │
     │  Model: yolo11l.pt (Confidence: 0.55)             │
     │  Classes: Person, Car, Motorcycle, Bus, Truck     │
     │  Output: Bounding boxes for contextual detection  │
     └─────────────────────────┬─────────────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────┐
│              PRIVACY PROTECTION (Stage 3)                   │
│  ┌───────────────────────────────────────────────────────┐  │
│  │ PERSON BLURRING                                       │  │
│  │ • Full-body region extraction                         │  │
│  │ • Gaussian blur: kernel (99×99), sigma=30             │  │
│  │ • Replace original region with blurred version        │  │
│  └───────────────────────────────────────────────────────┘  │
│  ┌───────────────────────────────────────────────────────┐  │
│  │ VEHICLE PLATE BLURRING                                │  │
│  │ • Crop vehicle region from frame                      │  │
│  │ • Detect plates within vehicle region (conf: 0.55)    │  │
│  │ • Gaussian blur: kernel (255×255), sigma=30           │  │
│  │ • Replace plate region with extreme blur              │  │
│  └───────────────────────────────────────────────────────┘  │
└──────────────────────────────┬──────────────────────────────┘
                               │
                               ▼
     ┌───────────────────────────────────────────────────┐
     │                  VISUALIZATION & OUTPUT           │
     │  • Display annotated frame with cv2.imshow()      │
     │  • Write frame to output video file               │
     │  • Check for 'q' key press to exit                │
     └─────────────────────────┬─────────────────────────┘
                               │
                               ▼
                       ┌───────────────┐
                       │ Output Frame  │
                       └───────────────┘

9.2 Processing Sequence Diagram

Frame N
   │
   ├──► [Traffic Sign Detection] ──► 24 classes @ 80% confidence
   │                                   │
   │                                   └──► Label overlay (centered, white bg)
   │
   ├──► [YOLO Detection] ──► Person, Car, Motorcycle, Bus, Truck
   │                          │
   │                          ├──► [Person] ──► Extract region ──► Blur (99×99) ──► Replace
   │                          │
   │                          └──► [Vehicle] ──► Crop region ──► [Plate Detection @ 55%]
   │                                                                  │
   │                                                                  └──► Blur (255×255) ──► Replace
   │
   └──► [Output Writer] ──► Save frame

Frame N+1
   ...

9.3 Detailed Algorithm

FOR each video frame:
    # Stage 1: Traffic Sign Recognition
    traffic_signs = traffic_signs_model.detect(frame, conf=0.8)
    FOR each sign in traffic_signs:
        draw_bounding_box(frame, sign.box)
        label = f"{sign.class_name} {sign.confidence:.2f}"
        draw_centered_label(frame, sign.box, label)

    # Stage 2: Vehicle & Person Detection
    detections = yolo11l_model.detect(frame, conf=0.55)

    FOR each detection in detections:
        IF detection.class == 'person':
            # Privacy: Blur entire person
            person_region = extract_region(frame, detection.box)
            blurred_person = gaussian_blur(person_region, kernel=99, sigma=30)
            replace_region(frame, detection.box, blurred_person)

        ELIF detection.class IN ['car', 'motorcycle', 'bus', 'truck']:
            # Contextual plate detection
            vehicle_region = extract_region(frame, detection.box)
            plates = plate_model.detect(vehicle_region, conf=0.55)

            FOR each plate in plates:
                # Adjust coordinates to original frame
                plate_box_global = adjust_to_frame_coordinates(plate.box, detection.box)

                # Privacy: Blur plate beyond recognition
                plate_region = extract_region(frame, plate_box_global)
                blurred_plate = gaussian_blur(plate_region, kernel=255, sigma=30)
                replace_region(frame, plate_box_global, blurred_plate)

    # Stage 3: Output
    display(frame)
    write_to_video(frame)

    IF key_pressed == 'q':
        BREAK

10. Tech Stack

10.1 Core Technologies

• Language: Python 3.7+
• Computer Vision: OpenCV 4.5+
• Deep Learning: Ultralytics YOLO 8.0+
• Model Architecture: YOLOv11 Large

10.2 Dependencies

Library	Version	Purpose
opencv-python	4.5+	Video processing, frame manipulation, blur operations
ultralytics	8.0+	YOLO model inference and detection
numpy	1.21+	Array operations and coordinate calculations

10.3 Model Architecture

10.3.1 YOLOv11l Base Architecture

Backbone:

• CSPDarknet with C2f modules
• Progressive downsampling (640→320→160→80→40→20)
• Feature extraction at multiple scales

Neck:

• Feature Pyramid Network (FPN)
• Path Aggregation Network (PAN)
• Multi-scale feature fusion

Head:

• Three detection heads (small, medium, large objects)
• Anchor-free detection mechanism
• Decoupled classification and regression branches

Model Statistics:

• Parameters: ~52 million
• GFLOPs: ~165
• Input Size: 640×640 pixels
• Output: Variable number of detections per image

10.3.2 Custom Model Specifications

Traffic Signs Recognition Model:

• Base: YOLOv11l
• Output Classes: 24 (traffic signs)
• Training Images: 3,000
• Training Epochs: 100
• mAP@0.5: 94.8%

Vehicle Plate Detection Model:

• Base: YOLOv11l
• Output Classes: 1 (license plate)
• Training Images: 3,000 (Turkish plates)
• Training Epochs: 100
• mAP@0.5: 96.1%

General Object Detection Model:

• Model: YOLOv11l (pre-trained on COCO)
• Output Classes: 80 (using 5: person, car, motorcycle, bus, truck)
• mAP@0.5: ~50% (COCO validation)

11. Privacy & Compliance

11.1 GDPR Compliance Features

The system implements multiple privacy protection measures:

Personal Data Protection:

• Automatic face blurring (via full-person anonymization)
• License plate masking (vehicle identification prevention)
• No storage of biometric data
• Real-time processing (no personal data retention)

Data Minimization:

• Only processes video frames temporarily
• No database storage of detected persons or plates
• Output video contains only anonymized data

Privacy by Design:

• Multiple detection models ensure comprehensive coverage
• Adjustable blur strength for regulatory compliance
• Modular architecture allows easy privacy feature updates

11.2 Regulatory Considerations

GDPR Article 25 Compliance:

• Data protection by design (automatic anonymization)
• Data protection by default (all persons/plates blurred)

12. License

This project is open source and available under the Apache License 2.0.

13. References

1. Ultralytics YOLOv11 Documentation.
2. European Union. (2016). General Data Protection Regulation (GDPR). Official Journal of the European Union.
3. OpenCV Image Filtering Documentation.

---

Acknowledgments

Special thanks to the Ultralytics team for developing and maintaining the YOLOv11 framework. This project benefits from the OpenCV community's excellent computer vision tools. The custom models were trained using publicly available traffic sign and vehicle plate datasets for educational and research purposes. Privacy protection features are designed with GDPR compliance principles in mind.

---

Note: Ensure you have proper permissions and comply with privacy regulations when using vehicle plate detection and person recognition technology. This system is designed with GDPR compliance in mind and includes automated privacy protections. This project is intended for educational and research purposes in autonomous vehicle development and privacy-compliant surveillance systems. Always verify compliance with local data protection laws before deploying in production environments.