# CSX's Approach to Mitigating Rail Risks with Autonomous Drone Operations - Event: NestGen ’25 - Published: March 7, 2025 - Length: 34:46 - Watch: https://nestgen.org/library/watch/ZCBXBT_PhuQ/ > The transcript of a recorded NestGen session, for reading or for use with an AI tool. It comes from the video's > captions; automatic captions can mishear names and terms. ## Transcript **[0:20]** All right, hello everyone. Welcome to NextGen 2025. Really sorry we ran into some technical issues, but we're here now and we'll get the session started. Uh ASAP. So, my name is Neil and I'm from the solutions engineering team at FlytBase and absolutely grateful to have you all here. Uh you know, let's start with a quick check-in where you guys joining from. Drop your comments in the chat section and also let us know what you're most excited about today's session. Is it drones, AI, predictive maintenance, free swag? You can let us know in the chat comments. So, as you can see on your screen today, we're going to dive into a world a real road discussion, so to say, but not the steam engine kind. We're talking about CSX's approach to mitigating rail risks with autonomous drone operations. And that might sound complex to a lot of people, but don't worry. We have experts here and we're going to break it down into the what, why, and how. So, what is CSX? **[1:26]** We'll unpack who they are and their role in the rail industry. Why is CSX doing what they're doing? So, the challenges they're trying to solve with autonomous drones and dogs. And how is CSX making this happen? So, we'll get into the nitty-gritties, the technical aspects, the infrastructure, and the real-world applications that make autonomous drone operations work. And before we get started, a couple of housekeeping rules. So, there's a Q&A section on your top right. So, please drop in your questions and we'll get to them at the end end of the session along with the CSX team. Uh don't forget to post your key takeaways on LinkedIn. I've been told that the best posts might even win some exclusive swag. And we're also going to keep the session interactive, so we have polls as well. So, keep an eye out for our mid-session poll. **[2:27]** Last but not the least, a quick word of thanks to our sponsors who make events like NextGen possible, especially the fourth edition. We're grateful for their support. Uh enough from me. I think it's time to get into the real deal. But before that, let's meet the brains behind this innovation and the brains behind what, you know, we're going to learn about today. So, first we have Patrick Barnett, technical director GIS services at CSX. I'll give a quick round of introduction. So, Patrick is the technical director at of GIS services at CSX where he has been shaping the company's approach to spatial technology and data management since 2005. Starting as a GIS analyst, he built an enterprise-wide GIS department that helps drive operational efficiency across CSX. Beyond his work at CSX, he also serves on multiple industry and community boards, always looking to create a positive impact in the areas he's passionate about. **[3:30]** When he's not transforming how CSX leverages geospatial data, he's spending time with his family and their two kids. Hi Patrick, welcome aboard. Good afternoon, everyone. And joining Patrick is Anthony Foisy, or Tony as he's fondly called, a senior GIS engineer at CSX. So, Tony has been leading the charge in drone and lidar technology to automate track inspections. Since 2010, he's been a part of the field services team helping CSX harness drone data for safer and smarter rail operations. Tony has a background in GIS management, photogrammetry, and remote sensing and brings a wealth of expertise to the table. Before CSX Tony has served as a as a meteorological and oceanographer in the US Navy where he developed a deep understanding of environmental environmental data and aviation operations. Tony is also FA FAA certified remote pilot and GIS professional making him a key **[4:35]** player in CSX's innovative innovation and their efforts. So, Patrick and Tony, welcome to NextGen 2025. Super excited to have you here and cannot wait for you to show the world what's been cooking at CSX. Yeah. Thank you, Neil. Good afternoon, everyone. Good morning. Uh good evening. Um so Neil, if you'd like to Yeah. switch to our next slide. Yes, just give me a second. Almost there. Right. All right. So, again thank you everyone for the time. Thank you, FlytBase, for having us. We're super excited today to be here, both myself and my colleague Tony **[5:38]** Foisy. So, who are we? We're CSX Transportation. All right, we're one of the nation's leading suppliers and we are a railroad. So, not not your steam engines. We're we're mostly all diesel engines. We're based out of Jacksonville, Florida. We service about 20,000 route miles across our network. You can see there it's it's the majority of the Eastern United States which serves 26 states in total plus DC, Ontario, and we go up into Quebec up there in Canada. Uh Our network itself serves the largest population across the nation. Nearly 2/3 of all Americans live on our service territory. Our trains move a ton of freight more than 500 miles on one single gallon of fuel. So, that's kind of a fun fact there as we think about 20,000 route miles across this network. On a single gallon of fuel, we can run 500 miles. So, just a fun fact there **[6:45]** on our company. Neil, as we slide to the next slide. Yep. Why are we doing what we're doing that that we're here today? Uh it's it's really about those five bold terms there, safety, automation, streamlining faster, and consistency. And so, we'll start at the top when we think about safety. That's our number one pillar here for CSX Transportation. We want our employees, we want the public, we want everybody to go home the way they came to work that day, the way they were traveling out on the roads. Those sorts of things. We want to take humans out of harm's way as as much as possible. And so, safety is our number one pillar. Through this effort, automation and overall repeatable processes. Right? So, versus eyes looking for defects, we'd like to use technology and overall automation to create that consistent **[7:47]** detection of defects specifically in our rail yards. And we'll get into that a little bit later in the presentation. But what this allows us to do is it allows us to streamline our processes. We can set up ad hoc missions. We can set up um detailed missions that happen on a daily basis and we can also tie into our enterprise systems that tell us when tracks are available to allow us to go out and inspect them from the air. Again, going back to the safety culture that we're trying to preach and have as one of the pillars here at CSX. We're above the track. We are not impacting operations and we're also taking people out of harm's way. What does this allow us to do? Again, it's those last two bullet points. It allows us to generate answers faster and and a more consistent basis, right? We're going to talk about this a little bit **[8:48]** later in the presentation. A lot of AI and machine learning focused tied into this overall operations. So, with that, I will turn it over to my colleague here, Tony Foisy, to provide you with the operations. Hey, Neil, it's always great to talk to you and to the FlytBase team. We really do appreciate the opportunity to share with you our autonomous drone solution. Neil, if you could please the next slide. I'd like to talk to you a little bit about our concept of operations. We have had the opportunity to work with the Federal Aviation Administration here in the United States for several years now and we have now progressed our operations now to the point where we have a remote operation center here at our headquarters in Jacksonville, Florida. So, our primary remote pilot in command, or RPIC, is still on location and we now have a secondary RPIC that's here at the remote operation center, or ROC, **[9:54]** actually doing all of the uh button pushing, if you will. They're interacting with FlytBase. They're monitoring the system as a whole, whereas that primary pick is still on site and responsible for the overall safety of the operation. We do use the automated detect and avoid system, the CASIA G, um to replace the human visual observer. And as part of that certificate of waiver with the FAA, we have established what we call right of way rules around manned aircraft. So, for example, a lot of the railroad yards, if you remember in the previous slide, the picture of the rail yards with a lot of light poles, uh power lines that stretch across our area of operation, we are shielded from those uh and operate within 50 ft of that. So, um being able to descend and stay out of the way of manned aircraft. All of our operations are over CSX property, uh 150 ft below uh the 150 ft is really **[10:56]** the commute to and from the drone dock. Our actual operations, where we're collecting data, is actually at 100 ft AGL, and we'll talk about that uh in a future slide. Uh all of these operations are conducted during daylight hours with weather limitations, as you see here on the side. And we're doing this with enhanced safety measures and really honing in on mitigating those risks and preventing operations over people. So, not flying over highway overpasses, railroad yards by nature are um closed environments with um we do not have lots of uh outside folks that are in the railroad yard. So, being able to prevent ops over people is is kind of inherent with the operation. Neil, if you could, please, the next slide. This is the solution um that we've come to really appreciate at CSX. I'll talk a little bit about the drone. We very much leverage the tried and true DJI **[12:00]** Matrice 350 uh RTK, of course. Um the real-time kinematic approach is really necessary to ensure that we fly directly over the track and that those images that we acquire with the Phase One iXM 100 uh camera is all geotagged uh to a very, very uh highly accurate location. The Phase One 100-megapixel camera with the 80-mm lens. That 80-mm lens, coupled with that sensor, allows us to take really some impressive imagery uh as I mentioned, flying at 100 ft above the tracks. We utilize the Hextronics Atlas, developed by Kurt Larry and the good team down at Hextronics. Um their drone dock actually does battery swapping, which we'll show you a a picture of what that looks like on one of the next slides. Of course, we couldn't do any of this without the good team at FlytBase and their **[13:02]** autonomous software. And as I mentioned previously, we use the CASIA G detect and avoid system, which you can see featured on that picture on the right. The center picture, I'll draw your attention to that for a moment. That is what we really affectionately refer to as our drone zone. That is what we have at 13 sites, as we call them. So, we have 13 of these boxes across 10 locations. So, as I mentioned, part of our risk mitigation is making sure that we have um the ability to not cross highway overpasses or fly over people and cars and things of that nature. So, that is an example of one of our drone zones here in uh at Barr Yard in Chicago, Illinois. Uh next slide, please, Neil. There's a lot of good information that I'd like to try to cram into this one slide. Um to try to do this on one slide and talk about all the good things that FlytBase **[14:04]** has done is kind of challenging, but as Patrick mentioned, we represent the GIS team here at CSX. So, for us to be able to import our enterprise GIS data that we have for our entire network was really a huge thing for us to be able to build those 50 to 80 pre-programmed missions at each of those 10 locations. So, 80 missions * 10 yd, we're talking some 700 to 800 individual missions, some very complex, like you see on the image in the slide here in the upper right with the blue dots. So, the FlytBase team was um very, very well in being able to develop what we'll call advanced editing capabilities, where we can then import our data and ensure that the drone flies over the exact location that we need it to over the track, take pictures using basic photogrammetry principles, **[15:07]** and then being able to um save all of those and pre-program those and develop actual waypoint actions at each of those, in some cases as many as 50, 60, 70 individual waypoints. So, a lot of customizability that we have in our missions. As I mentioned, we have a lot of fail-safes in place. So, the FlytBase system inherently provides us a way, when we lose RF connection, that command and control or the RTK, the real-time kinematic, is lost, the drone knows what to do. And so, having those fail-safes in place is huge for our operations to maintain that overall safety. Of course, as I mentioned with the remote operation center, having that multi-view dashboard will be critical as we move forward into that next regulatory step that we are in the process of doing now. To be able to see all of our missions across these **[16:08]** 13 sites all on one screen is very critical to us, as well as being able to integrate, as Patrick alluded to earlier in the presentation, having our legacy systems that manage operations in our yards to then feed into our Microsoft Azure system using the F Link method or FlytBase links, and then create a series of actions and alerts within FlytBase to then kick off and execute those missions. So, that's really been a a huge milestone and undertaking for us to be able to achieve that higher level of automation. Next slide, please. Some of the key features that I've already mentioned before um that you see, I'll just kind of progress from the upper left towards the right on this slide. Working with the team at Phase One, we were able to uh achieve wireless data transfer. So, **[17:10]** using wireless access points in the Hextronics Atlas box to offload and transfer those images from the Phase One camera through the Atlas drone dock to our edge device that's sitting in what we call a railroad bungalow, which think of it as kind of a giant metal box, like a a little server room out in the wild, out in in the railroad yard. And so, that's what you see pictured here. The command and control is essentially in this locked server box. I mentioned the battery swapping functionality with Hextronics. That's huge. So, if you think about operations and being able to continuously collect imagery, being able to fly a mission, the drone comes back, swaps those batteries in a matter of minutes, and gets back out into the game and collecting more data. That's huge for us, as well as having the extended range of the RF antennas to maintain that integrity and the resiliency of that command and control structure, and also having the pre-flight inspection cameras with **[18:14]** high-resolution cameras looking up at each of the motors, the props, and one centered on the drone looking up as the drone's landing. So, those are huge key features for us. The next slide, please, Neil. At this point, I will turn it back over to Patrick and we'll talk about machine learning. Thanks, Tony. All right. So, one of our last pieces here uh that's highly important is obviously the machine learning piece. And so, for us, um you know, there are there are numerous use cases that are going to that is going to go into this machine learning. Um some upwards of 30 30 total use cases. Right now, um we've got roughly eight or so use cases implemented. And so, as you see here, some of the machine learning that we're doing with the drone and ultimately processing on the edge is things like switch point gaps. So, being able to determine **[19:15]** um where a switch point is greater than 1/8 of an inch where it meets the rail. So, just to be clear, what is a switch? It allows you to go from one track to another track. So, you can continue straight or you can go to an additional track off of it. If if that connection is greater than 1/8 inch of a greater than 1/8 inch of a gap, you can cause a derailment. And so, what is that? That is cars, locomotives toppling over um right there on the side of the side of the rail. If you're thinking about how wide is 1/8 inch of a gap, just take your credit card. So, we're looking at measuring the width of a credit card. So, that's that's what we're trying to do here. And then as we look at some of the other machine learning pieces, joint bar detection and rail gaps, those are big for us. When we think about rail gaps, which is on the right-hand side, the middle image down, we are measuring where those gaps are **[20:19]** greater than 2 inches. So, once we get greater than 2 inches, again, that can cause a negative impact to our operations with potential derailment. So, as you see right there in the image, a detection and then measurement there of 1.06 inches, that's a good that's a good gap. So, anything greater than 2 inches. Then joint bars, so joint bar detection, we're doing numerous things there. Joint bar is where you have two pieces of rail that that again is not not one continuous piece and you're ultimately putting a joint bar in place to put those two individual pieces of rail together. We're looking at a multitude of things. So, we're flying both nader and oblique missions over these tracks. So, at that angle, it allows us to look at these joint bars that are on the side of the rail and also be able to detect missing bolts, also loose bolts. So, we're measuring the threads of the bolts within this machine **[21:22]** learning. Missing bolts as I mentioned, you're able to depict that with shadows. Then we're also looking at if the joint bar is cracked, right? We need to get that out. Also the rail on the side of the rail, if it is if we have broken rail there, we've got cracked rail or anything where we need to do some welding and cut pieces out and put new pieces in. On the top right side, that's some of our latest adds to the use cases. We're on the track gauge both on the static and the loaded side. So, what is that? That is measuring the distance between our two rails. When they get a distance greater than X, where a locomotive can no longer stay in between those two rails, obviously that creates a problem as well. So, we're looking for trends as we sit here and do these inspections to see is the rail starting to drift apart from each other. That would then become a problem for us. So, we'd have to go back in **[22:24]** there, realign the rail, repack the ballast, and those sorts of things to continue to keep our operations flowing. Neil, if you go to the next slide. Sure. And lastly, you know, at the end of the day with the machine learning in this whole program for us as we call it, it's the results, right? It's being able to identify the results, do it in a timely fashion, and get it in the right hands of people on the ground to address any defects that have been identified. So, again, this is a huge collaborated effort both with technology, operations, our engineering workforce, our federal government, and all our tech partners to be able to get everybody on the same page to understand that this is all about safety and continuing our operations in a safe manner, right? So, the more safe we can be, the better everyone is going to be at the end of the day, our employees, our communities, our shareholders, and **[23:28]** all of our partners as well. So, What you see here is is an output of at the end of the day when our machine learning runs on the edge, we produce the results in the cloud. Then we see this back on the dashboard. So, this goes to a tablet that our folks on the ground have to where it'll take them exactly to where you see that red dot right there on the right image. So, that shows you at that switch, we have a switch point gap. And on the image on the left, obviously we're at a hump yard facility. That's where we've got, you know, upwards of 84, 90 tracks in a facility. You know, at that one location right there that you see there in the bowl, all those green dots are switches. So, we're measuring all of those gaps on the fly. And that one red one would have greater than 1/8 inch of a gap. And so, we need to get **[24:30]** somebody on the ground there to take a look at that immediately and provide any kind of mitigation that's needed as soon as possible, right? To mitigate any future negative impacts like derailments of and those sorts of things. Neil, if you would, if you don't mind, go into the last slide. Obviously this is going to be our question time for Q&A, but you know, again, Neil, the entire FlytBase team, myself, Tony Foisy, we'd like to say thank you again for this opportunity for being able to have this discussion, provide this solution to everybody here. It's been a it's been a journey for us, but we've learned a lot. We're looking forward to this year. This is the year that we will turn this system on at numerous locations across our network inside our rail yards. And again, looking forward to the continued partnership with with the FlytBase team, with all of our tech partners, **[25:31]** our communities, and especially the federal government. So, working with the FA. So, with that, we have a few minutes left. I'll open it up for Q&A and Tony and I will try to do our best jobs here. All right. First of all, thank you, Patrick and Tony, for enlightening us with, you know, your entire solution. Lot of great insights, which are very valuable for the entire audience. And not just for them, but for me as well. Every every conversation I have with you, I have something new to learn. And, you know, the groundbreaking stuff that you've been doing, it's been a total pleasure working with you and collaborating with the CSX team, the Xometry team. And yeah, so we can go ahead and come to the Q&A section. So, Okay, so we have one question. What new use cases do you foresee with AI-powered rail inspections **[26:32]** beyond what you just highlighted in your current scope? Yeah, I mean, from a track inspection standpoint, so specifically in the engineering space, I mean, there's going to be things like measuring cross level. Um we're going to be looking for sun kinks. You know, that's a that's a big deal for us. It heats up pretty bad here down in the south once we get into the summer. And sun kinks in the rail cause, obviously, can cause derailments. So, we'll be looking for those as well. And then as we think about long-term, it's all about line of road for us, you know, and and and getting this sort of technology out there on the line of road and helping supplement that 20,000 miles that we have across this network. We have a lot of other use cases that we're kind of working on in other areas outside of just track inspection. Mhm. So, we're working with our teams over in the transportation **[27:35]** space and mechanical space as well. But but we definitely want to turn this on this year. And so, so we're looking forward to it. We've got the majority of the the main heavy hitters as far as the track inspection goes, the ones that cause us the biggest grief every day, I guess you could say, and the ones that provide us the potential derailments. And so, we chose the hardest up front. Measuring the width of a credit card from over 100 feet at 7 miles an hour is a challenge. So, we definitely want to go for the hardest stuff first so that everything else is going to be easy as we move forward. All right. Thanks. Thanks, Patrick. The the fact that, you know, you're trying to detect the crack that's the size of a credit card, it just amazes me every time. So, All right. Go ahead for the next question. You spoke about how data processing is an essential part of your entire workflow. So, closing the loop, figuring out, you know, where the cracks **[28:40]** are. And the key component over there is the real-time image transfer. So, there's a question that are there any latency challenges when it comes to the image transfer and how are you addressing it? Yeah. Yeah, we can Tony will take that one. Pass it back and forth. Yeah, latency with data image transfer, that obviously is a a huge challenge for us. When you think about sometimes we'll have a mission, I mentioned 50 to 80 missions in a yard, and each of those missions could have anywhere from 50 images to as many as 200 images. And these are raw IIQ images off the Phase 1, which are typically anywhere from 60, 70, maybe 80 megabytes apiece. And so, when the drone comes back before we do a battery swap, that imagery has to be offloaded from the camera every **[29:40]** single time. And one of the the things that we designed with the system is to have the access point right smack in the back of the Atlas inches from the back of the phase one camera where that wireless Wi-Fi dongle is on the back of the camera. So, to try to minimize that and and really reduce that latency as much as possible, but inherently we're going to have challenges with that before you can then um battery cycle the the drone and things of that nature. So, yes, absolutely that is a challenge and just trying to work through those as best we can. All right. Thanks. Thanks, Tony. That really helps. The next question is you know with respect to the BVLOS waiver. So, you mentioned your steady progression with the FAA. So, any insights on the waiver process? Any specific challenges that you faced? And because I'm curious we have a lot of people who have joined in from **[30:45]** you know North America and FAA is one of the key components getting that waiver. So, any best practices that you could share would be really valuable. Yeah, absolutely. And just to summarize it, it's been um it it's been a long fun journey. Um we've um I I would like to say we have developed a very positive rapport with the FAA through as I mentioned several years of building that relationship. And really what it comes down to to offer advice to somebody is really building up that that best practice and showing that in the case of the United States where we have the Federal Aviation Administration and um kind of um likewise uh regulatory agencies in other countries, it's it's much the same. It's proving that safety case and you really have to prove and this goes across the board with as railroaders we work directly with the Federal Railroad Administration as drone operators, we also work with the Federal Aviation Administration and the same holds **[31:47]** true. Proving that you have your eyes dotted and your T's crossed, really making that safety case, making sure that you've thought of every bad possible thing that can happen with your drone operation and prove to the FAA or your regulatory body how you're going to deal with that um without even thinking about it. The fail-safes in the FlytBase system is huge, the detect and avoid. I mentioned we make use of the infrastructure masking argument. And just to recap, all of our railroad yards have high mast light poles at 100-150 ft range and that's where the drone is operating. So, using that infrastructure masking as an argument and decreasing our altitude to to leverage that is huge to the point where it it makes it an easier decision for the FAA. So, I hope that helps. Definitely, Tony. You know, lots of great insights there as well. Um so, yeah, I **[32:48]** think um you know, we are a little over time. Uh but just wanted to again thank you and Patrick for sharing these incredible insights. It's been uh amazing watching CSX push the boundaries of drone autonomy, AI, and real safety. And what I know is that we are barely scratching the surface and this is just the beginning. A lot of things are needed to be done and you've already shared that you know vision with us. And it's been a total pleasure working with the CSX team again with our tech partners as well. So, Kurt is right now you know seeing like live at the session as well. So, Kurt has been a true helps, the entire Xtronics team, CSX team. And yes, you know, figuring out your the nitty-gritties that would help you know, uh craft the perfect solution that fits exactly into your workflows is something that you **[33:50]** know is our North Star metric and you know, just pushes us to do the right things for CSX. So, thanks for being valued customers, partners, and yeah, thanks for being here as well. Well, thank you, Neel. And again, on behalf of Patrick and I, we are very grateful for the opportunity to speak to you. It's always a pleasure and thank you to the FlytBase team. All right. Thank you, Patrick, Tony. Thanks to the audience as well for taking out the time and being here. And with that, let's let's call it a wrap. Thanks, Patrick. Thanks, Tony. And Thank you, Neel. Thank you, everyone. Bye-bye. Bye-bye. Close again.