How I accidentally created the most used standards in the drone industry
A decade ago, little did I know that my student project at the Computer Vision and Geometry Lab at ETH Zurich would end up becoming the de facto standard in the drone industry. During this time, my team and I created MAVLink, Pixhawk, PX4, and QGroundcontrol, which are today’s most used flight control hardware and autopilot software in the drone industry. This was the beginning of a story of a very successful open source project outperforming individual corporate development.
In 2008 quadrotor drones were already a known airframe type, rediscovered after the design was first pioneered in 1907 by Louis Breguet. However, I was not interested in designing a drone or a flight controller. It was quite the opposite: I wanted to enable a drone (still called “micro air vehicle” in 2008) to fly autonomously using computer vision. I pursued this research project in parallel to my master’s degree, which I just had started. The plan was very ambitious — after all, it was just the year after the first iPhone launched, smartphones with the needed computing power were not a thing yet and Nokia was still the biggest phone company. ROS, the Robot Operating System, was also in its infancy and just one of the many middleware systems used in robotics. So the wealth of computing power and technology that exists today on drones was not available and still had to be created.
I quickly discovered that before making computer vision work on a drone, I had to make the drone and flight control software and hardware. Four weeks into the project I realized that neither the drone technology was there (MEMS sensors just became low-cost around the same time) nor was I even remotely in a position to complete this within a year, which was the initial timeline for the project. For example, there were no flight controller projects targeted at vision-based navigation. Rather than giving up when facing this challenge, I recruited a team of 14 other students, many more experienced than me, to attempt it. It was very much following the theme of recruiting smarter people than myself so they could tell me what to do.
Second generation Pixhawk drone – Zurich 2009
The team now assembled, we worked day-and-night for 9 months and won the European Micro Air Vehicle competition in the indoor autonomy category in 2009. The complete flight controller was a custom electronics design and we had written the complete flight control code from scratch, but relied on existing open source drivers and frameworks for the infrastructure. This was extremely helpful to get us started and created the desire to contribute our solution back to the community.
The name of the student team was Pixhawk, and because our goal was to work on autonomy and computer vision we released our software as open source in the hope that it would be useful to others.
EMAV competition – Pixhawk team – Delft 2009
And they were. MAVLink, the communication protocol, was the first bit that was picked up by the open source community, used in other autopilot projects such as AutoQuad and Ardupilot. The user interface software QGroundControl (back then not yet running on mobile but on Desktops) also saw adoption, but it would take years and the dedication of two additional members of the core development team who took the lead in the system architecture before it took the position it has today in the industry. The state of these software components from 2008 to 2011 was very limited compared to where they are today and it would have been impossible without the hundreds of contributors that joined the platform since then. There are so many that the best approach to be fair in attribution of their contributions is to refer to the individual projects and their contributor and maintainer lists. They also helped to hone the product-market-fit, adding features that the market was requiring.
In contrast to MAVLink and QGroundControl, I wasn’t quite happy with the flight control software and the autopilot electronics — it felt as if the architecture wasn’t there yet and it wouldn’t scale. So I kept iterating with the student team and eventually sought for external contributors who could bring in industry experience.
We decided for a clean cut in 2011 and essentially threw away all software and hardware we had built in the past three years in favor of a complete rebuild from scratch that had no architectural issues. The fourth rewrite of the PX flight control software (shorthand for Pixhawk) finally yielded the quality I was looking for. PX4 was born! We released the first stable release in 2013 after two years of development. In parallel, we developed the 1st and 2nd generation hardware (Flight Management Unit version 2: FMUv2), which was called Pixhawk as a reference to the original student team where it all started.
Together, these pieces are forming a complete software stack for a drone: Flight control, middleware (modern, publish-subscribe pattern based), communications protocol and ground control station. With the release of a newly-written computer vision library (PX4/avoidance) the open source stack was reaching the same level of completeness as the research systems we had at ETH in 2012 – just that the open source software is now of course much closer to product-readiness compared to the research implementation.
This hardware was made available to the community in collaboration with a manufacturing partner, 3D Robotics, which took over manufacturing and distribution. In addition, we collaborated with ArduPilot to enabled them to run their flight stack on the PX4 middleware on Pixhawk, so users had more choice. The hardware design remained open source and is still hosted on Github. While this made Pixhawk hardware widely available, the sprawling open hardware community also created unanticipated downsides: seemingly small hardware modifications (from the manufacturer perspective) led to user experience issues or even failure, and users did, for the most part, attribute them to the flight control software.
With these components, our team created a full drone technology stack: computer vision, flight control software, autopilot hardware, communication protocol and ground control station software. Everything needed to build and operate a drone. We had built the infrastructure that allowed my research group to test the latest vision-based exploration algorithms – and because everything was still as early and beta, it still included a safety leash:
Pixhawk team – Vision based exploration – Zurich 2012
The success of this platform is no black magic. In retrospect the success factors become quite obvious: I was confronted with a problem which was relevant for my student project but also for a rising and fast-growing industry. By choosing to build an open source community around this, I ensured that development efforts and research results of many years and many talented people worldwide were combined with a full-scale solution which was reusable and standardized. Combined, we had much more development power and skills than any of the well-resourced companies in the field. In addition, the open source development, with its diverse background, required mindfulness about building an ecosystem instead of single solutions — which requires standardization.
All these aspects create one common theme: they define a piece of technology that is very well suited to be reused. This included the choice of a permissive open source license so that anyone could reuse and improve the technology. This is in particular important for academia, where many different cooperative projects and partnerships require as little friction from an intellectual property perspective as possible. It is also important for commercial entities that want to innovate on a platform and protect their differentiation.
One thing that open source communities are not particularly good at in terms of getting external contributions is debugging usability issues or keeping the overall software well organised — this is often up to the maintainers and I found myself spending enormous amounts of time — together with other maintainers — on keeping the core clean and healthy. To overcome this challenge, we established a clearly defined process to debug, test and approve changes. As a result, any code change is evaluated by at least 2 people and validated in automated flight testing before going into the code. The whole flight stack goes through an average of 1000 test flights per month – a value few companies can achieve for their closed source flight software.
While today the number in drone sales and the overall market are still growing, drone companies are facing the challenge of keeping up with the market demand for new features and technology like visual odometry and the still complex and time-consuming development of the flight software itself. It is still hardly possible and very expensive to develop all parts of the software stack, from the basic flight controller up to the autopilot and sensor software. Luckily that is not necessary anymore. Instead of building everything by themselves, drone companies can jump on the open-source platform around PX4 and build customer-relevant features on-top.
In 2014, Dronecode was founded to make sure that all that was created in an open source environment will stay that way and remain non-discriminative. It also provided the platform to create alignment between the different actors in the drone industry together, from drone manufacturers, sensors, and part suppliers, to software service providers. Today, Dronecode is a non-profit organization which belongs to the Linux Foundation and acts as a neutral place where industry and open-source developers can contribute in order to reduce costs and time to market. If you are interested in being part of this development and want to support the creation of standards, Dronecode is a good start to get your foot in.
The success of PX4 means that I personally was able to finish my (Ph.D.) studies, but probably will never be able to really “finish” my student project – a small price to pay. I am grateful to be part of this unbelievable journey and still curious how much more we can develop and create together in an open source community. However, this is not easy to sustain at industrial scale. As the users of PX4 expect product-level user experience and reliability, the community has slowly changed from being academic and enthusiast-driven towards contributors who all use PX4 in some type of product. This requires both the technical skills (most of our contributors today have an MSc. or PhD in control theory/robotics, computer science or computer vision) as well as enough time per day to spend on it. Most of the open source contributors today are employees of companies and consultants who work full-time in the field. They are naturally focused on their specific areas of interest, leaving a gap at the center of the effort to evolve the overall platform.
I co-founded Auterion with Kevin Sartori to make the effort I accidentally started 10 years ago sustainable by creating a company that is committed to maintaining the open source ecosystem as an open source distribution and supports other companies to use it in their products and services. I am convinced that my story is not an outlier but can be a blueprint for others as well. Go find a problem worth solving for you and others and build a strong community around it!
And, I’m not quite done yet – the next step will be a “Pixhawk for computer vision” that I’ve started to discuss high-level with the community. The goal is to build something that is as easy-to-use as the Pixhawk standard. Stay tuned!