# Enhancing Traffic Oversight with Autonomous Drones and AI Analytics - Event: NestGen ’24 - Published: March 22, 2024 - Length: 30:02 - Watch: https://nestgen.org/library/watch/_ubDzy_MriQ/ > The transcript of a recorded NestGen session, for reading or for use with an AI tool. It comes from the video's > captions; automatic captions can mishear names and terms. ## About this session Discover how autonomous drones and AI analytics redefine traffic management in this insightful session. Petr Dvorak from DataFromSky shares their journey from pioneering traffic surveys with drones to real-world applications like creating digital twins of urban traffic. Through live demonstrations, learn how the platform analyzes real-time traffic data, enhancing road safety measures and enabling law enforcement interventions. Explore the versatility of the technology for security, crowd management, and retail analysis, operating robustly in various conditions. To know more, visit: ===== Subscribe to FlytBase YouTube - Follow Us On - LinkedIn - Facebook - Twitter - Instagram - ## Transcript **[0:20]** hello and welcome to another session of Nan 24 my name is ainka and I'm excited to Welcome All of You today's session is enhancing traffic over site with autonomous drones and AI analytics this session will delve into the transformative potential of drones coupled with AI to redefine the landscape of traffic control ensuring safer more efficient navigation across both bus link cities and secluded areas with that I would like to introduce our speaker Peter vojak from data from Sky pet is head of aial intelligence at data from Sky being responsible for aial applications of the DFS workflow better served as a management Committee Member for EU cost acting as a CE Tech rep for denan avionic USA he gained a detailed insight into avionics for Mann aircraft as well he has been responsible for development of image correlation Optical measurement systems at sobriety and is therefore in a great position for **[1:24]** optical payload integration into U uavs having earned his degree in aerospace engineering better has more than 25 years of experience with unmanned aircraft before we get to the session I would like to share a few housekeeping rules please feel free to type in any questions you might have during the session in the Q&A chat box the session is getting recorded recorded and will be published soon we also have the booth in the Expo section please make sure you visit it during networking and Booth visit slots last but not the least please remember to capture your favorite moments of the session and share them on Twitter and Linkedin by using hash nen I would also like to thank our sponsors title sponsor DJ Enterprise and go sponsors andlower Labs with that over to you Pete better thanks a lot jinka uh I hope that you are able to see my screen uh so let me just switch to the presentation it's a great pleasure to be here uh thanks **[2:29]** a lot for all you to to come to our session and let me just start um basically how we how we started uh this is uh something that happened in 2014 and uh we started as a spin-off from a university project where High number of counts and and uh very high Precision for for traffic survey was needed so that's how we established ourselves and we are basically the first ones to offer uh traffic uh surveys from from a drone uh so we basically defined this Market u 10 years ago since then uh this technology has proven to be very useful for a number of purposes so uh we have created a lot of use cases **[3:30]** and uh some of them Prov to be very uh uh I would say uh big enough so that they um manag to to uh get their own brand name such as parking detection where we are uh monitoring and uh and looking into more than 30,000 parking lots every second all around the globe uh over the years we have been able to work with uh brilliant customers and uh we have been performing very interesting projects for them uh so this is an example of uh a major project that has uh taken place in aens in Greece uh we have cooperated with epfl loan Switzerland on this project and uh uh it's been 10 drones flying simultaneously for several days uh creating **[4:32]** the biggest uninterrupted digital twin of uh of a traffic Urban traffic in in the entire NE neighborhood in Athens more than 100 intersections have been covered uh this is actually uh an open research data set so if you are interested uh uh we can we can give you access to this it's called PMA it's publicly available for researchers right so what we are actually talking about this is a random video feed from from a drone with uh the detection of the objects of Interest enabled I mean nowadays it's nothing special although it it used to be very special in the days when we started but uh this is just the first step right you take the video feed uh and then you basically divide it into every frame and **[5:35]** in each and every frame the artificial intelligence or the Deep neural network that we trained and use uh basically says this is a dog this is a car and this is a truck so that's what we do for each and every uh frame in the video but it's also only the the first step uh for us to be able to connect the the the movement of the of the vehicles or objects into meaningful trajectories we need to then connect these individual um detections together and uh in order to filter out noise and some some movements that wouldn't be feasible in the real world uh we deploy a long developed in-house algorithms that are that are assuring uh or are taking care of uh producing very smooth and uh and connect trajectories so **[6:36]** this is very crucial step because uh if you just connect the different detections without thinking about uh how you connect them uh then you might not be able to do the the third and and I would say most important step why why we are all talking about this is uh that we need to interpret the data and and get meaningful and actionable insights from them that means that uh for example we want to uh see how many yellow cars are turning from the West to the South and if we had the trajectories not connects uh that means that we would have one detection here one detection here but uh the the algorithm or the computer wouldn't know that U it's the same object during the whole time we wouldn't be able to to filter uh the vehicles or or the **[7:38]** objects of interest and uh basically the data would be much less uh useful than they are uh when we deploy all the steps that we have developed in house over the years all right so these are basically the results or what you can do with the data once everything has been processed cessed correctly so as you can see You've Got U counts at different Gates U or virtual induction Loops as you can name them uh you can have velocities accelerations uh obviously we are we are categorizing the the objects into nowadays it's something like 15 categories and uh you are able to produce heat maps of velocity acceleration and stuff like that so it's it's a very rich uh data uh that serves uh for for example redesigning the traffic uh point at this uh for this very example but uh **[8:43]** you can you can do a lot of things with with the this richness of the data what's more since a few years back we are able to um do all these steps in real time that means that you are able to deploy the Drone uh you know near near Highway or your uh scene of interest and you are able to get obviously the counts but also uh the Gap times in real time the velocities and you can also analyze the behavior of the vehicles for uh some unwanted Behavior so basically you can use it for law enforcement in uh in real time and that's actually what I wanted to to show you right now this is what uh Central Bohemian police officers are doing uh and it's the world first **[9:46]** application uh in in this context because uh they are using our technology to detect and measure Gap times between vehicles and they approach the drivers right at the spot where they trespassed uh regulations okay so hopefully you've been able to see the video so uh as you have seen the the deployment scenario is is pretty straightforward in this case uh we've got a unit **[10:50]** that's called traffic drone unit it's basically a very powerful uh field computer uh that sits you know at the deployment side basically the D officers come near to the highway fly their drone they they stream the the video uh to the ground station and directly from there they send it via HDMI cable to our unit and they receive the insights on the spot in real time so they they are able to directly approach the drivers there obviously uh the scenario is is very flexible in terms that you don't have to carry the the computational power uh to the deployment site you can stream the video via internet to your centralized control room and do the processing over there and in the very same manner I think it's **[11:51]** very um relevant to to today's conference you can deploy uh your uh your Solutions where the drones are being deployed automatically from all around the world on multiple sites and do the processing uh in in centralized manner obviously uh you are able to connect from all over the world to the to the insights or to the analyzed results so that's actually what what we are doing to present uh right now so let me just switch to to our environment and this is what U what we've got uh or this is the traffic uh insights or flow insights uh an environment that we uh that we provide to the customers where they can uh access you know the the extracted uh insights from the from the traffic stream **[12:58]** so obviously you can you can have uh more uh scenes or uh or environments predefined so that means that if the Drone is is leaving the dock for for a certain location you just enter a predefined u scenario and you don't have to configure it all over again from from the scratch so this is actually uh a pre-recorded uh stre from Leon in France where we have done uh join demo with our toring supplier Al there and I would like to demonstrate you know the basic functionality of uh of the flow framework uh on this so as you can see uh once once you feed uh the video stream into the system you get all the all the targets annotated and uh **[13:58]** and classified but uh we we want to have some uh some measurements as well apart from only the the visual outcome so this would be the exit gate I'm I'm just uh drawing some some virtual Gates over here so that we can uh we can then uh work with them and don't gu lost uh in the in the data so just like this I drag and drop U these uh these filters basically they are spatial filters uh and we are by by these we are filtering out uh only the trajectories that have passed through these gate **[15:00]** and uh I've got the counts that fulfill the criteria over here so you can see that uh uh in the direction of Leon uh we have we have had uh this number of vehicles passed since since the beginning of U of the analysis okay so that that's that's the very basic uh stuff let me just add some um widgets so that we've got uh a better overview of the situation on the on the dashboard and I'm also interested in in the distribution of the vehicles in the type of vehicles uh that are driving in U each of these three directions so this is what we have done uh we can see the distribution of vehicles for all of the three gates and uh I can also I can see the the counts as well **[16:07]** we can visual visualize it uh uh in in quite a neat way uh so let me just copy the the main outbound gate over here and just by uh dropping the the exit value on it uh I'm um I'm creating this So-Cal Super Widget um and obviously let me just uh color it so the green ones will be those that are uh that are staying on the on the highway and the red ones are uh the vehicles the proportion of the vehicles that's uh departing from the highway so very basic stuff obviously I can sorry for this um I can add a real time overview of the situation it's already been here so let me **[17:08]** just uh and drop it here so in this way we can we can build uh a dashboard for uh for a quick overview of the situation now uh in France uh as you might know uh it's uh a big uh big theme there is uh tailgating or driving very close to the uh preceding vehicle uh it's it's actually one of the most dangerous uh offenses traffic offenses because uh when the vehicle before you starts to break then the vehicle after don't have uh enough time to to react and and it leads to a lot of fatal crashes so me just uh create a a gate where we are going to measure the Gap **[18:14]** it's it's called Gap and we are going to calculate Gap at this gate and we are going also to collect evidence okay so now we have filtered the vehicles that um are uh driving through this Gap uh Gap gate and we are going to filter down only the vehicles that have had U the Gap time uh lower than two seconds that's the safe threshold and all of these vehicles that are D driving dangerously are going to be output into a table so in my dashboard I've got a table with all of the vehicles uh you know obviously there there speeds accelerations and the Gap times and all of these Gap times are lower than two seconds because these are only the vehicles that I filter down the vehicles that **[19:16]** have been D driving dangerously to take it further I can define an Evidence uh event or task so this would be uh get evidence and source of my events would be the table I've just shown you and let me just add an image evidence that's the that's the gate that we have defined earlier and I'll add that one once again with zoomed in uh view like this and for this view we don't need any uh enhancement of the of the image so yeah that uh yeah my my limit is 2 seconds and we are on the highway so it's 130 kilm per hour uh I'll save the the evidence task and uh in my storage uh I immediately get U evidence protocols uh **[20:22]** generated for each and every of the vehicles uh that we have seen in the in the table right so let me just just uh download one of those and um show them to you so that you can see how this evidence protocols look like uh actually this is this is the the official check um or Central Bohemian Police Department protocol so sorry for it being in check but basically you've got all the all the vital Telemetry data here and you also have the uh image evidence present here together with uh with zoomed in view okay so this has been a very quick uh demonstration of of the flow it's a really powerful platform so uh if you are interested to just don't hesitate to contact us and there's a lot that that you can do with with this okay uh so let me just continue with the presentation just just a quick **[21:26]** overview of U of the applications that you can use uh the technology for so obviously the standard one will be uh traffic U traffic surveys uh for for the purposes of redesign of your traffic Network and so on and uh it's been used very widely uh by our partner Ki in Scandinavia for for uh improving uh safety around schools and obviously improving the infrastructure for uh active Mobility cycling paths and so on it's also an Ideal tool for feeding uh the behavioral data into into your simulation uh models such as PTV and stuff like that obviously you can use it for for **[22:27]** security uh security purposes and uh crowd management uh or uh for your retail and um analysis of behavior of the of the customers where they are uh delving more uh you know spending time in front of which shop and stuff like that uh so uh yeah contrary to to popular belief it's the technology is very robust I would say in terms of weather and conditions so the rle Thum is that as long as a human can can see the objects then the neural network can recognize and see and recognize them as well uh that means that we are able to to work even during the night for rain and and so on and if that's not possible uh we do employ uh a special dete of of **[23:29]** such a condition so if you've got a fixed camera installed at the road side uh the system basically sends you a notification that that the conditions aren't fulfilled for for a meaningful analysis and uh it also stops the analysis okay so with that uh I think it's it's the end of the presentation thanks a lot for your presents and I'm open to to any questions if if there are any yep thanks a lot Peter those that was uh I mean thanks for that live demo as well and sharing those different use cases how uh how the solution is being used in different countries uh so that that's really insightful and thanks for highlighting those data points so we we have a lot of questions uh coming in so I I'll just go through the questions one by one so uh first question that we have is U the audience is asking is data DFS **[24:33]** a web based application uh sorry about was that yeah is data from Sky flow a web based web based application well uh we've got uh we've got multiple scenarios U so uh for the Post recording that means that um where you basically record the the situation with your drone uh you are able to upload then the the recording to our uh website and uh we we process it for you and you just download the results and you are able to interpret them and it's very computational resources um light so you can do it on your computer but obviously for the for the Real Time stuff uh you basically need to have uh a dedicated machine for that so we either offer these machines or we are able to or you can you can work with your own computational **[25:36]** Hardware that that's up to the specifications that we provide you and we we just Supply the the software and it's on premise uh deployment then so it's it's really very versatile thanks uh that clarifies the answer and another question we have is can a drone be deployed when an alarm is triggered and upon reaching it it takes the vehicle and send a notification so let's say there's a third party application that gives data that some vehicle is speeding and then the Drone goes to that location and then just to verify the data from sky or the flow platform uh can then be used sure I mean uh as I've as I've uh shown we've got this uh basically we we've got uh wide variety of of U ways how to interact with the outer world uh that means we've got you know API both rest and UDP API um we are able to **[26:41]** send web hooks uh we are uh able to directly control for example traffic lights or traffic controllers uh and we are also able to to directly uh directly control input output relays so that means that uh that you can light a warning light or open and close the gate and stuff like that uh yeah and all I I've I've shown you the the basic way how to generate the the evidence but these uh evidence events are also being uh sent via web HS for example so again it's it's very easy to integrate all the all the data and and inside that we generate right thanks thanks again so there are lot of comments appreciating the accuracy of data I mean detection of the vehicles so on on those lines one question is does the platform also uh is it also able to detect pedestrians or human working sure sure yeah pans are are uh **[27:49]** B one of the basic categories as I've said we we've got more than 15 of them now so you can have not only pedestrians bar but prems and and E scooters and stuff like that so we we even have like a finer resolution within the general pedestrian category but um that's more applicable to to fixed camera views where where you've got your camera looking or you know installed the roadside and you've got I would say a detailed view of of the targets because uh from the Drone uh it might be it might be difficult to to see whether that's standard pedestrian I would say or or a pedestrian riding e scooter so there uh we we restrain only to the pestan category but even from from drones uh the detection of pedestrians is very robust uh **[28:49]** given that that the detection resolution is u high enough that means that basically we need to see the the Target or the the Parra in order to be uh able to recognize it great so yes uh we are out of time but you can again reach out to better on Via the website or directly uh from uh DFS data from Sky social media as well as we also have networking Booth where if you have any specific questions you can talk to better and so thanks again thanks a lot better for that wonderful uh session about how you can enhance traffic oversight using AI analytics so really thank you for joining us today and really appreciate your time today yeah thanks a lot it's been very nice event and uh yeah thanks for having us it's been a pleasure perfect thank you everyone and just one quick note so uh we have started an exclusive invite only Community for all the Drone autonomy enthusiasts so my colleague uh our team **[29:53]** will paste share the link with you it's also available on the reception so feel free to register to it thank you thank you everyone have a good good day bye bye