Description

The SNACE project (Neuromorphic Sensing with Active Efficient Coding for Edge Computing) focuses on developing adaptive neuromorphic sensory systems that dynamically adjust sensing and processing to changing environmental conditions and task requirements. The goal is to enable robust, low-latency, and energy-efficient perception on resource-constrained edge devices.

A key challenge is to achieve reliable perception despite limited computing resources, noisy and changing sensory conditions, and fixed sensor parameters. Multimodal perception introduces additional challenges, as information from different sensory modalities must be combined efficiently while balancing perception performance, computational requirements, and energy consumption.

To address these challenges, SNACE investigates Active Efficient Coding, an approach that adaptively adjusts sensing and processing to the current perception task. The system combines multiple neuromorphic sensing modalities, including an event camera, a neuromorphic auditory sensor (cochlea), and tactile sensing. Multimodal perception and decision-making are implemented using spiking neural networks (SNNs).

The developed methods will be integrated and evaluated on ANYmal D, a quadrupedal robot. The demonstrator focuses on spiking human identification, tracking and following as well as adaptive terrain classification, investigating how information from multiple sensory modalities can enable efficient and reliable perception of changing terrain conditions.