Differences

This shows you the differences between two versions of the page.

--- searchwing-bilderkennung [2019/05/30 23:16]
wf68spef
+++ searchwing-bilderkennung [2019/06/17 00:22]
wf68spef
@@ Line 3: / Line 3: @@
 By analysing the image with we receive from the camera, we can detect boats on the sea. To achieve this, different algorithms and approaches from the image processing and deep learning domain can be used.
-===== Approaches =====
+===== Detection by using edgedetector and a 2d tracker in world coordinates =====
-==== Boatdetector by using a simple edgedetector and a 2d tracker in world coordinates ====
+{{:ros.jpg?nolink&1311x744|ros.jpg}}
-Features\\
+==== Code ====
-Bootdetektion / Bilderkennung\\
-Erkennung von beliebigen Objekten im Wasser\\
-Klassische Bildverarbeitung\\
-RGB -> HSV -> Sobel-Kantendetektion je Kanal -> Addition -> Konturen/Boundingboxes\\
-Bestimmung der genauen 3D Position der Objekte mittels kalibrierter Kamera\\
-Code für ARM Platform optimiert\\
-Laufzeit: 0,8 sek @ 8 MegaPixel(3240x2480 Pixel) @ Raspberry Pi 3\\
-Tracking\\
-Wiedererkennung der Boote über mehrere Bilder\\
-Dadurch kann sich ein genaueres Lagebild geschaffen werden\\
-Falsch-Positiv Detektionen werden verringert\\
-Speicherung\\
-Abspeicherung der Detektionen als Bilddaten auf dem Flieger\\
-GPS Positionsangabe im Bild als EXIF Datenblock       \\
-Verwendete Software\\
-ROS\\
-Kommunikation zwischen den einzelnen Modulen\\
-Koordinatensystemtransformationen\\
-Aufnahme und abspielen von Flugdaten\\
-Visualisierung der Detektionsergebnisse\\
-MAVROS\\
-Zur Kommunikation mit der Fliegerhardware via MAVLINK\\
-OpenCV\\
-Implementierung der Bootdetektion\\
-ARM Compute Library\\
-Alternative für ARM optimierte Implementierung der Bootdetektion\\
-Ausblick\\
-Deep Learning basierte Objekterkennung\\
-Testen verschiedener Ansätze\\
-Laufzeitevaluation auf den embedded Systemen\\
-Programmierung von Interfaces für die Übertragung der Detektionen mittels MAVLINK zur Basisstation
-Features
+[[https://gitlab.com/searchwing/development/boatdetectorcpp|https://gitlab.com/searchwing/development/boatdetectorcpp]]
-Assumptions
+==== Algorithms ====
-|* Proposal / boat detector
+=== Assumptions for the algorithms ===
-      * Detection of randomly looking objects in the water
-        * Classical image processing ( no deep learning)
+  * By flying over the sea with 50-80km/h boats can be assumed to stand almost still in relation to the drone
-        * Processing chain depicted in the image
+  * Waves appear and dissapear over time
-        * There false positives after inital detection
+  * Approach to detect boats
-      * Calculation of the 3D Position of the Objects
+      * Detect parts in the image which dont change over time
-        * In world coordinates
+      * Redect parts by checking the same position in consecutive frames
-        * By using a camera calibration
+      * If parts look the same, save them
-      * Code optimized for ARM Platforms
+      * If something is redetected over 3 frames, we can assume this could be a boat
-        * By using ARM related flags for OpenCV
-        * Alternative: Compute Library for even faster processing
+=== Proposal / boat detector ===
-      * Runtime
-        * 0,8 sec @ 8 MegaPixel(3240x2480 Pixel) @ Raspberry Pi 3
+  * Detection of randomly looking objects in the water
-|{{:0cae1334da2eb44438be608612fe6628.png}}|
+      * Classical image processing ( no deep learning)
-|* Tracking
+      * Processing chain depicted in the image
-      * Redetect boats in consecutive frames
+  * There false positives after inital detection
-        * Get more information/images about the same boats
+  * Calculation of the 3D Position of the Objects
-        * Reduce false-positive rate
+      * In world coordinates
-          * Valid detections only if we redetect the same boat
+      * By using a camera calibration
-          *
+  * Code optimized for ARM Platforms
+      * By using ARM related flags for OpenCV
+      * Alternative: Compute Library for even faster processing
+  * Runtime
+      * 0,8 sec @ 8 MegaPixel(3240×2480 Pixel) @ Raspberry Pi 3
+{{:0cae1334da2eb44438be608612fe6628.png}}
+|
+=== Tracking ===
+  * Redetect boats in consecutive frames
+      * Get more information/images about the same boats
+        * See image below
+      * Reduce false-positive rate
+        * Valid detections only if we redetect the same boat
       * Algorithms
         * Association Problem
           * Euclidean distance based cost matrix between each possible track and detection
-          * Solve 1-1 assignement problem by using Hungarian Algorithm
+          * Solve global neigherest neighbor assignement problem by using Hungarian Algorithm
         * Tracking
           * Different trackingmodels possible
-            * Constant position
+          * Constant position
-            * Kalman constant position
+          * Kalman constant position
-            * Kalman constant velocity
+          * Kalman constant velocity
-        *
-|{{:5ee30e4d3dc8729b80b2ca22e4b9f3af.png}}|
-| | |
-  *        *
+{{:5ee30e4d3dc8729b80b2ca22e4b9f3af.png}}
-  *        *
+=== Output ===
-==== Imagerecognition by using a OpenCV Haar Cascade Classifier ====
+  * Save boat images on harddisk
+  * Metadata for each detection is saved in exif-datablock of each detected boat
+      * GPS Position
+      * GPS Time
+  * Visualization of the detection in digikam
+{{:digikamdetections.png?nolink&878x498|asd}}
+|
+==== Used software ====
+  * ROS
+      * Module communication
+      * Coordinatesystem transformations
+      * Recording and playback of datasets
+      * Visualisation
+  * MAVROS
+      * Communication with the drone via MAVLINK
+  * OpenCV
+      * Imageprocessing
+  * ARM Compute Library
+      * Imageprocessing
+==== Outlook ====
+  * Deep Learning
+      * Test different approaches
+      * Runtime evaluation for embedded hardware
+  * Interfaces to send detections via MAVLINK to basestation
+==== Presentation on the topic ====
+[[https://media.freifunk.net/v/35c3oio-77-detection-of-refugee-boats-on-the-mediterranean-with-a-drone-by-using-foss|Click to go to the video]]
+===== Imagerecognition by using a OpenCV Haar Cascade Classifier =====
 I composed a few images to train a Haar Cascade Classifier. The code is over here:
@@ Line 87: / Line 104: @@
 [[https://git.etech.fh-augsburg.de/friedrich.beckmann/bilderkennung|https://git.etech.fh-augsburg.de/friedrich.beckmann/bilderkennung]]
-Die Bilderkennung markiert in dem Bild mögliche Boote. Das sieht dann so aus:
+The detected boats can be seen in the following image. They are marked by a blue rectangle.
-{{:bilderkennung-beispiel.jpg|Bilderkennung Beispiel}}
+{{:bilderkennung-beispiel.jpg?1101x1044|Bilderkennung Beispiel}}
-Und in der Vergrößerung so:
+Recified:
-{{:bilderkennung-beispiel-detail.jpg|Bilderkennung Beispiel Detail}}
+{{:bilderkennung-beispiel-detail.jpg?1082x1027|Bilderkennung Beispiel Detail}}
-In diesem Beispiel wird das zweite Boot mit der blauen Persenning nicht erkannt. Das Boot darüber ist mit dem blauen Rechteck markiert.
+In this example the second boat with Persenning does not get detect. The boat above get detected.
 ===== Datasets =====
@@ Line 107: / Line 124: @@
   * [[:searchwing-flug-virus-20181007|Flug am 7.10.2018 mit der Pipistrel Virus zum Bodensee]]
   * [[https://captain-whu.github.io/DOTA/|DOTA Airial Image Dataset (Wuhan Univ./Cornell/DLR)]]
-===   ===