Speedometer: Lab asssignment

Objective

In this project, your goal is to design and implement a vehicle speed measurement program on the NVIDIA Jetson Nano using C++/CUDA. The program should measure the speed of vehicles crossing a camera’s field of view either from left to right or from right to left, providing the result in kilometers per hour (km/h).

Project Details

To achieve the speed estimation, several image processing steps will be necessary. Below is an ordered but non exaustive list of the main stages to implement.

You won’t implement or run any machine learning network, pre- or self-trained. It is not an AI project. Instead, you’re expected to design CUDA kernels.

  1. Camera Input: You will be provided with a pre-written code segment that acquires images from an HD camera. This part of the project is already implemented and provided to you. The code is located in the home directory of the jetson user, more precisely in

~/Documents/NANOCODES/cudavideoproc/video-viewer.cpp

This code gives you the skeleton of the whole processing loop.

  1. Background Subtraction: - Implement background subtraction to isolate moving objects (vehicles) from each image. This is not as easy as you could think. Actually, to be robust especially to changing light conditions, the background estimation must be performed periodically. The time between two consecutive estimations is to be determined. The estimation itself must be performed through a median filter in time, involving several consecutive video frames. The number of frames involved is also to be dertermined by your own experiments. - For computational efficiency, I suggest to process downscaled grayscale images. The code sample you were given already performs the downscaling and the RGB to GRAY conversion.

  2. Binary Image Processing: - After background subtraction, create a binary image where the moving objects (vehicles) are clearly distinguishable from the background.

  3. Bounding Box and Motion Tracking: - Detect and track the vehicle’s bounding box in each frame to measure its position as it moves across the camera’s view. - Assume only one vehicle is present in the frame at any time.

    Note

    I am not sure this step is necessary.

  4. Speed Calculation: - Calculate the vehicle’s speed in pixels per frame based on its bounding box displacement. - Convert the pixel speed into real-world speed (km/h) using the known real-world width represented by the two ends of the image frame.

Implementation Notes

  • Optimization: Since the image processing will run on the Jetson Nano GPU, ensure that image downscaling, grayscale conversion, and background subtraction are optimized for GPU processing using CUDA.

  • Testing Environment: Full-scale testing is impractical in our lab setting. Instead, you will test with toy spring-powered cars, which will simulate vehicle movement within a controlled environment.

  • Real-time computation: it is strongly encouraged to display the actual frame-per-second rate of the whole process.

Deliverables

  1. Workgroups You will work by pair of students. There are two main reasons for this:

    • There probably won’t be enough complete Jetson developping kit (cameras, power supplies, sd cards) as some of them disapeared or happened to malfunction since we received them. We re-ordered the lacking items, but they may not be delivered by the time you will start this assignment.

    • Experimental measurements will be much easier if they are conducted by two students rather than alone.

  2. Source Code: Implemented in C++/CUDA, with appropriate comments for readability. CUDA kernels and C++ function may be defined in one single file (.cu), even if it’s not the best way to organize a projet in the real life.

  3. Documentation: A brief report detailing: - Your approach to background subtraction and motion tracking. - The process of bounding box detection and pixel-to-km/h conversion. - Results of the tests conducted with toy vehicles, including observed speed measurements.

  4. Demonstration: A brief live demonstration of your program using the provided toy cars, showing the vehicle speed calculation in real-time.

Good luck, and remember to leverage GPU capabilities effectively for real-time processing!

For the most seasoned : extra work

The ultimate functionality would be to manage multi-vehicule scene. It is not that complicated and you’ll probably find a lot of examples and help on various websites. Don’t forget: no deep learning network ;)

Warning

Deadline

  • You are invited to send me your source code via e-mail as an attachment. The subject must be [M1IoT Speedometer].

  • I must receive your codes by Wednesday, December 13th. Any code that would not be sent by this date will receive a grade 0/20.

  • On the 15th of december, you will present your design during 5 minutes and then answer a few questions for 5 more minutes.