Code from Sensors Group
See Sensors Group github projects at https://github.com/SensorsINI and https://github.com/NeuromorphicProcessorProject
Please write Tobi Delbruck <tobi@ini.uzh.ch> if you need help to find code associated with a particular paper or project.
Major code projects
jAER
Java tools for Address Event Representation sensors (DVS/DAVIS/DAS event sensors)
jAER is our flagship software project for supporting sensors group event sensors, including DVS, DAVIS, and DAS. Most of the robots in the Sensors Group YouTube playlist were realized in jAER.
Citation: Delbruck, Tobi. 2008. “Frame-Free Dynamic Digital Vision.” In *Proceedings of Intl. Symp. on Secure-Life Electronics, Advanced Electronics for Quality Life and Society*, 1:21–26. Tokyo, Japan: Tokyo. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.192.2794&rep=rep1&type=pdf .
using_jaer_2021-01-22_08-16-47_1_1_1.webmv2e - Video to Events
v2e (see v2e web page) simulates accurately-modeled DVS event camera events from frame based video.
https://github.com/SensorsINI/v2e
Citation: Hu, Yuhuang, Shih-Chii Liu, and Tobi Delbruck. 2021. “v2e: From Video Frames to Realistic DVS Events.” In *2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)*, 1312–1321. doi:10.1109/CVPRW53098.2021.00144. http://dx.doi.org/10.1109/CVPRW53098.2021.00144 .Best paper award from CVPR-W on Event Based Vision.
v2e-tennis-split-screen_1.webmOther useful event camera tools
aedat4to2 converts AEDAT4 files from Inivation's python DV into AEDAT-2.0 files for jAER https://github.com/SensorsINI/jaer/. Based on inivation AEDAT file format specifications AEDAT_file_formats.
pyaer wraps inivation's libcaer to enable simple configuration and capture of event camera data. Used for projects like Dextra (the rock scissors paper robot) and Trixsy (the card finding magic robot).
Neural Control Tools
These projects support our neural control research on data-driven nonlinear optimal control of the physical cartpole robot and our INIvincible F1Tenth race car. See below the chart showing dependencies and selected videos. For more details see our Neural Control Tools Github page.
CartPoleSimulator_1.webmf1tenth inivincible neural controller 20231123_162424_1.mp4DeNoising Dynamic vision sensors 2021
DND21 supplies the jAER algorithms and datasets for low cost DVS event camera denoising.
Contributors: Shasha Guo, Tobi Delbruck
Citation: S. Guo and T. Delbruck, “Low Cost and Latency Event Camera Background Activity Denoising,”, IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2022.
More tools
snn_toolbox - Bodo Rueckauer's ANN to SNN conversion tools.
ADaPTION - Moritz Milde's reduced weight+state precision Caffe CNN training tools.
ddd17-utils - DDD17 DAVIS Driving Dataset tools for collecting real automobile end-to-end data and exporting it to machine learning tools.
Phased LSTM - Danny Neil's PLSTM tools for dramatically speeding up RNN training using "time gates". Great for sensor fusion RNNs that integrate sensors with wide ranging time scales.
More Unpublished resources
Quick start for users of the CAVIAR USB1/2 AER boards (retina and monitor/sequencer boards)
SimpleMonitorUSBXPress board software, PCB design, firmware etc. These single chip boards allow monitoring AER up to 100 keps directly into matlab.
A bias generator design kit, a design kit for semi automatically generating bias current generators with wide dynamic range, including extensive documentation.
A list of surface mount equipment and suppliers that Tobi assembled as a basis for our SMD setup.
The address-event code on our PIC microcontroller AER interface board described in our paper on orientation-selective aVLSI spiking neurons.
Latest Physcmp Archive (Mar 28, 2006) of Tobi's Matlab m-files and MEX for talking to GPIB-controlled instruments using a National Instruments GPIB interface. Evolved from code developed by Dan Naar at Apple and Mead's lab at Caltech in the during the middle and late 1980's. Includes plot182, a matlab plot replacement that allows you to measure slope, intercept, and that does this sensibly for log scales, showing you the e-folding parameter. plot182 was written for matlab4 but it still runs. The GPIB interface amazingly also still runs using a barely modified mex file, thanks to NI's amazing software stability. I use it only because I'm too lazy to figure out Mathwork's instrument control toolbox.
readtspice.m, a matlab script for reading Tanner EDA TSPICE output files into matlab. From the Physcmp archive.
zonechart.m (a generating matlab m-file) and zonechart.tif. These are really useful for measuring the aliasing properties of imagers. This method was suggested to Tobi by Peter Centen of Thomson Broadcast Solutions, makers of high-end video systems.
Notes on practical photometry, from the Pulsed-Bipolar imager project. (An error of a factor of two in the formula relating chip to scene illumination corrected Feb 2006).
Plans for lens mounts for PGA84 and PGA100 packages, with either M12/M13 thread mini lenses or CS-mount lenses.