MarketWatch, May 4, 2021. Caspar.AI uses the whole home as a sensor – already existing elements like sensors on switches and appliances. From automated control of lights, shades and temperature to more complex tasks such as fall detection, the Caspar system works without cameras and without “wearables” such as resident pendants.
Deep Multimodal Embedding: Manipulating Novel Objects with Point-clouds, Language and Trajectories. ICRA, 2017. (finalist for ICRA Best cognitive robotics paper award)
A system operating in a real-world environment such as homes needs to perform reasoning with a variety of sensing modalities.
However, manually designing such features is extremely challenging. This work presents an algorithm to learn to embed point-cloud, natural language, and trajectory data into a shared embedding space using a deep neural network.