This is an old revision of the document!
Image-Processing
By analysing the image with we receive from the camera, we can detect boats on the sea. To achieve this, different algorithms and approaches from the image processing and deep learning domain can be used.
Detection by using edgedetector and a 2d tracker in world coordinates
Code
Algorithms
Assumptions for the algorithms
- By flying over the sea with 50-80km/h boats can be assumed to stand almost still in relation to the drone
- Waves appear and dissapear over time
- Approach to detect boats
- Detect parts in the image which dont change over time
- Redect parts by checking the same position in consecutive frames
- If parts look the same, save them
- If something is redetected over 3 frames, we can assume this could be a boat
Proposal / boat detector
- Detection of randomly looking objects in the water
- Classical image processing ( no deep learning)
- Processing chain depicted in the image
- There false positives after inital detection
- Calculation of the 3D Position of the Objects
- In world coordinates
- By using a camera calibration
- Code optimized for ARM Platforms
- By using ARM related flags for OpenCV
- Alternative: Compute Library for even faster processing
- Runtime
- 0,8 sec @ 8 MegaPixel(3240×2480 Pixel) @ Raspberry Pi 3
Tracking
- Redetect boats in consecutive frames
- Get more information/images about the same boats
- See image below
- Reduce false-positive rate
- Valid detections only if we redetect the same boat
- Algorithms
- Association Problem
- Euclidean distance based cost matrix between each possible track and detection
- Solve global neigherest neighbor assignement problem by using Hungarian Algorithm
- Tracking
- Different trackingmodels possible
- Constant position
- Kalman constant position
- Kalman constant velocity
Output
- Save boat images on harddisk
- Metadata for each detection is saved in exif-datablock of each detected boat
- GPS Position
- GPS Time
- Visualization of the detection in digikam
Used software
- ROS
- Module communication
- Coordinatesystem transformations
- Recording and playback of datasets
- Visualisation
- MAVROS
- Communication with the drone via MAVLINK
- OpenCV
- Imageprocessing
- ARM Compute Library
- Imageprocessing
Outlook
- Deep Learning
- Test different approaches
- Runtime evaluation for embedded hardware
- Interfaces to send detections via MAVLINK to basestation
Presentation on the topic
Imagerecognition by using a OpenCV Haar Cascade Classifier
I composed a few images to train a Haar Cascade Classifier. The code is over here:
https://git.etech.fh-augsburg.de/friedrich.beckmann/bilderkennung
Die Bilderkennung markiert in dem Bild mögliche Boote. Das sieht dann so aus:
Und in der Vergrößerung so:
In diesem Beispiel wird das zweite Boot mit der blauen Persenning nicht erkannt. Das Boot darüber ist mit dem blauen Rechteck markiert.
Datasets
For development and test purposes we need datasets. So far we got these datasets:
- We got access to the Seagull Dataset