Camera Capture and Processing

A C application for real-time camera capture, edge detection, optical flow, scale/rotate/crop, display, and video recording using CUDA, Video4Linux2, SDL2.

Overview

This project provides a camera interface for:

• Edge detection, optical flow, scale/rotate/crop using CUDA
• Real-time video display with SDL2
• Frame capture and saving as PGM image files
• Automatic conversion of captured frames to MP4 video

Requirements

• Linux operating system with Video4Linux2 support
• SDL2 library
• ffmpeg (for video conversion)
• CMake (version 3.10 or higher)
• C compiler with C11 support
• CUDA

Installation of Dependencies

For Ubuntu/Debian-based systems:

# Install V4L2
sudo apt-get update
sudo apt-get install libv4l-dev v4l-utils

# Install SDL2
sudo apt-get install libsdl2-dev

# Install ffmpeg
sudo apt-get install ffmpeg

# Install CMake
sudo apt-get install cmake

# Install GCC (C compiler)
sudo apt-get install build-essential

# Install CUDA
# First, verify your GPU supports CUDA
lspci | grep -i nvidia

# Download and install CUDA from NVIDIA's website
# Visit: https://developer.nvidia.com/cuda-downloads
# Select your Linux distribution and follow the instructions

Building

Default build

mkdir -p build
cd build
cmake ..
make

Build with custom options

mkdir -p build
cd build
cmake .. [OPTIONS]
make

Where [OPTIONS] can include:

• -DLIVE=ON: Enable real-time viewer mode (default: OFF)
• -DRCS=OFF: Disable rotate/crop/scale processing (default: ON)
• -DEDGE_DETECTOR=OFF: Disable edge detection processing (default: ON)
• -DOPTICAL_FLOW=OFF: Disable optical flow processing (default: ON)

Example to build with real-time mode enabled and optical flow disabled:

cmake .. -DLIVE=ON -DOPTICAL_FLOW=OFF
make

The executable will be created in the build/bin directory.

Usage

Real-time viewing mode:

./bin/cuda_project /dev/videoX

Where /dev/videoX is your camera device (e.g., /dev/video0). (Visible using ls /dev)

Frame capture mode:

./bin/cuda_project /dev/videoX NUM_FRAMES [crop_x crop_y crop_width crop_height] [scale] [angle]

Where:

• NUM_FRAMES is the number of frames to capture
• crop_x crop_y crop_width crop_height are optional crop parameters (all 4 must be provided if cropping is desired)
• scale is an optional scale factor (default: 1.0)
• angle is an optional rotation angle in degrees (default: 0.0)

Crop Parameters Explained:

Crop parameters allow you to process only a specific rectangular region of the camera image instead of the entire frame:

• crop_x: X-coordinate of the top-left corner of the crop region (in pixels)
• crop_y: Y-coordinate of the top-left corner of the crop region (in pixels)
• crop_width: Width of the crop region (in pixels)
• crop_height: Height of the crop region (in pixels)

For example, if your camera captures 640x480 images and you want to process only a 320x240 region starting at position (100,100), you would use: 100 100 320 240

If no crop parameters are provided, the entire camera image will be processed.

Example:

./bin/cuda_project /dev/video0 100 100 100 320 240 1.5 15.0

Keyboard Shortcuts (Real-time mode)

• ESC/Q: Close the application
• F: Toggle fullscreen
• S: Take a screenshot
• RCS Commands:
• ARROWUP/ARROWDOWN Scale up/Scale down
• ARROWLEFT/ARROWRIGHT Rotate left/Rotate right
• +/- Crop/Uncrop
• WASD Move crop region

Output

• PGM image files are saved in the build/img directory (format: image-#####.pgm)
• The final video is saved as output.mp4
• The final rcs video is saved as output_processed_rcs.mp4
• The final edge detector video is saved as output_processed_edge.mp4
• The final optical flow video is saved as output_processed_optflow.mp4

Project Structure

• src/: Source code files
  • main.c: Main application entry point
  • capture_camera.c: Camera capture functionality
  • sdl_viewer.c: SDL-based display implementation
  • cuda/: CUDA implementation files
    • cuda_rcs.cu: Rotate/Crop/Scale processing
    • cuda_edgeDetector.cu: Edge detection processing
    • cuda_optflow.cu: Optical flow processing
    • cuda_utils.cu: Utility functions for CUDA operations
  • threading/: Thread implementation files for parallel processing
• includes/: Header files
  • main.h: Main application header
  • capture_camera.h: Camera capture declarations
  • cuda/: CUDA header files
  • SDL/: SDL-related headers
  • utils/: Utility headers
• build/: Build directory for CMake output
  • bin/: Contains the compiled executable
  • img/: Output directory for captured PGM images and output video

Features

Real-time Processing

When built with -DLIVE=ON, the application runs in real-time mode, showing processed camera feed with selected features.

Rotate/Crop/Scale (RCS)

Applies geometric transformations to camera frames using CUDA acceleration. Can be disabled with -DRCS=OFF.

Edge Detection

Applies edge detection algorithms to camera frames using CUDA acceleration. Can be disabled with -DEDGE_DETECTOR=OFF.

Optical Flow

Calculates and visualizes optical flow between consecutive frames using CUDA acceleration. Can be disabled with -DOPTICAL_FLOW=OFF.

Multi-threaded Processing

Uses multiple threads to parallelize frame processing, allowing for efficient use of CPU and GPU resources.