Donkey Car Newsletter

Safe, In Person Racing! Donkey Car Roadmap with PyTorch and fast.ai.

In Person Racing Returns to Oakland

Safe, in person racing returns to Oakland on December 5. It will be outdoors and limited to 50 mask wearing people. Unlike most DIYRobocars races, this one will allow GPS! Sign up here

For those of you interested in a online race, sign up here for the December 19th virtual race.

Donkey Car 4.1 roadmap

It feels like just yesterday that Donkey Car 4.0 released, nonetheless 4.1 is coming soon. With he new maintainers we expect to do quarterly releases of Donkey Car. Dirk Prange will be the release captain.

  • PyTorch and fast.ai support - If you have been putting off learning PyTorch and fast.ai, now is the time to get started.

  • Auto Encoder - More below

  • Lots of other fixes…

More on the AutoEncoder

The driving idea behind the auto encoder in Donkey Car is the decoupling of the visual system from the ‘motor neurons’ of the car, called the controller. As it turns out, the visual system which is represented by the CNN layers is far more complex than the controller. And the visual system is not necessarily dependent on a specific track but a component that we will train on a much bigger set of data and then re-use on new tracks, providing an efficient transfer learning for the controller part only. This is very similar to learning motor skills in biological brains. In order to learn how to catch a ball or drive a bicycle we don’t have to learn to see again, this is a skill that we have already acquired at that state. We just learn how to activate the muscles in the right order at the right time, which is a much simpler task.

Cutting the standard donkey linear model after the CNN layers and adding a single dense layer provides us with an image encoder which compresses the high-dimensional image data into a much lower dimensional latent space. Our 120x160x3 size frames are mapped into a 128-dimensional vector. The auto encoder now adds an exact reverse deconvolutional network on its back which inflates the latent vector back to the input image size. A good introduction with Keras code can be found here: https://blog.keras.io/building-autoencoders-in-keras.html.

The auto encoder can be trained completely unsupervised by minimizing the difference of the output and input images in training. However, in this approach we will not get a latent representation of the image information that is most suitable for detecting features relevant for driving, like lane edges. Hence our loss function in training contains also steering and throttle terms.

From the pre-trained auto encoder we subsequently only use the encoder part in our model, where the latent vector is directly fed into the controller. This is a small feed forward network that is about two orders (i.e. factor 100) smaller than the decoder. The training is only performed on the controller, which makes it very fast.

Things to develop further:
1) Training might be so fast that we can possibly train while we are driving. Not necessarily on the car but likely on the PC in the background.
2) We can explore RL on the physical car not only in simulation
3) With the new augmentation we can train a de-noising auto encoder (see example in Keras above). Instead of feeding the original images in training, we feed noisy images into the auto encoder but in the loss function we compare the reproduced images against the original ones. In particular, changes in light levels which are scalar multiplication of the input images can be efficiently factored out. Currently for a trained linear model such an image transformation creates a change in output, i.e. the car over or understeers and drives faster or slower with changes in light levels. Auto encoders are very good at de-noising. If you play with the Keras tutorial above you can create very blurred images that you will find hard to recognize -  the network will generally do a better job than the human eye.

September Donkey Newsletter

New Maintainers, Donkey Car 4.0 Roadmap, a New App and More

New Maintainers

The Donkey Car project is almost 4 years old now; projects like this only survive because of a strong community (you) and the tireless work of open source maintainers. For those who don’t know, our current maintainers are: Tawn Kramer, Ed Murphy, Chris Anderson, Rahul Ravikumar and Adam Conway; Will Roscoe, who co-founded the project has moved on from the project but still checks in with his “maintainer emeritus” status. To those existing 5 maintainers we are adding 6 more maintainers. This will greatly increase the momentum of the project. These are all people who have been very supportive of the Donkey Car Project.

  • Jonathan Tse @Jonathan Tse - Jonathan has been involved with the project for many years. He runs the Robocar store in Hong Kong and has been dedicated to bringing Donkey to education. He is also the creator of the Donkey Car App!

  • Dirk @DGarbonzo has been a long time contributor to Donkey Car, he has done many a PR and helped many people in the community.

  • Thomas Coyle @TCIII - Thomas is a retired systems engineer and has been a huge contributor to many open source projects - in particular he has been a huge supporter of Ardupilot and Dronecode, we are happy to have him now help with Donkey Car.

  • Maxime Ellerbach @redplex - Maxime is a student based out of France and will be focused on the simulator and the map creation. He has been collaborating with Tawn for quite some time and has produced an auto map generator.

  • Jack Silberman @JackSilb - Jack is a professor at UCSD teaching robotics. He has been a long time promoter of the project and even teaches an undergraduate course based on the Donkey Car project. He is famous for bringing a whole gaggle of students up to Oakland from San Diego to compete in the Robocar races (and often cleaning up).

  • Paul Huliganga @Paulh - Paul is a long time electronics hobbyist and enthusiast who enjoys building robots and autonomous machines.  He is an active member of the Ottawa Autonomous Vehicle Group (OAVG) where he learned about Donkey Cars and started diving into Machine Learning.  Paul is also a member of the Parking Lot Nerds team that has been participating in this summer's Virtual Races.  This is a truly international team made up of members from the Stuttgart Connected Autonomous Driving Group, Detroit Autonomous Vehicle Group (DAVG), and Ottawa Autonomous Vehicle Group.

If you are interested in being a maintainer, we are willing to add one or two more right now. What we look for is a history of high quality PRs or other support to the community.

Donkey Car 4.0!

We are doing a major new release of Donkey Car! This is being lead by Rahul and will be released in the next few days!

Datastore v2

We will be replacing the long standing Tub Datastore with Datastore v2. The new Datastore brings many new features including:

  • Arbitrary augmentations. This will allow preprocessing when training. There will be a few preprocessing directives out of the box such as cropping and trapezoidal masking.

  • Improved API

  • Better performance

Tensorflow 2.x and PyTorch Support

  • Support for Tensorflow 2.2, Tensorflow lite and TensorRT upgrades.

  • Support for PyTorch and Fast.ai models.

  • Stretch goal: Transfer learn a Resnet18 based model. 

Technical Debt and Cleanup

  • Cleanup `manage.py` and remove redundant, less used features. 

  • Stretch: Threaded Part v2 design. Deprecate non-threaded parts completely. 

  • Stretch: Organize parts to their new folders. 

  • Improve documentation in 4.x, especially given that some things are going to break.

Robocar Controller

Jonathan Tse, who is one of our new maintainers, created new app for Donkey Car. It is in the Apple’s App Store and Google Play Store. The Robocar Controller is designed to provide a “commandless” user experience to get started with the Donkey Car.

  • Command-less experience - No SSH or text editor

  • Built-in Hotspot

  • Search vehicle on the network

  • Real-time Calibration

  • Virtual joystick

  • Visualize the data

  • Drive Summary

  • Free GPU training

  • Autopilot

  • Advanced configuration

  • Battery level

Other odds and ends

Join the community on Discord! For those of you who are interested in the project or need help it is a great place to get started. The link is here.

Join our virtual races! The link is here to register or watch.

Race in Place!

Online racing with DIYRobocars and other News

DIYRobocars worldwide Event

Usually our DIYRobocars events are regional but with everybody at home, this is our first opportunity to have a worldwide event. You can sign up here. If you have been intending to learn how to use the simulator or just curious to watch, please join us.

A couple of tips:

  • If you plan on racing, please know your lap time in advance (this will help with seeding) and sign up here

  • If you plan on just watching, please watch the twitch stream.

If you want to get started, here is a great tutorial.

Other News:

No longer stocking the Sombrero

The Donkey Car store had been ordering sombrero boards and selling them sometimes break even sometimes at a loss for about 1.5 years now, The design is still open source, and still supported. I am looking at other options to simplify the electronics and battery in the donkey car. For those who want to know, the reason not stocking the cars were this:

  • the hope is that some 3rd party would pick up the design and improve it, but that never happened

  • It was too complex to assemble a Sombrero - often they would have issues after assembly.

  • I had to buy large quantities and I did not like outlaying thousands of dollars.

Donkey Car - 2019 in review

2019 has been a great year for the Donkey Car project. Please read our new end of year blog post here.

New newsletter service

Some of you may notice that the newsletter has a different format. I am moving away from Mailchimp which was getting very expensive and towards Substack which is free. I am also experimenting with the format itself and linking to blog posts rather than putting the content in the email.

We are experimenting with different ways of doing the blog but for now I am using Medium.

Loading more posts…