Microsoft along with Liebherr are collaborating in the development of new generation of the Smart refrigerators and freezers, connecting them to the internet. The system is designed to have a long lifecycle. Modular units can be integrated and upgraded at any time in existing SmartDevice-ready appliances to create value and comfort for customers through new digital features and solutions.

New Smart refrigerators to help you shop online
Image: Microsoft

“With the SmartDevice capability, future refrigerators will help in shopping and planning meals with intelligent food management. Stored groceries can be monitored using internal cameras and object recognition technology. This process not only captures images for viewing but also recognizes individual food items inside the refrigerator. This information flows automatically into an inventory list, which lets the customer see quickly and clearly what is in the refrigerator from anywhere”, says a Microsoft blog.

Using the SmartDeviceBox voice module, additional groceries can be added to a shopping list that customers can access when on the move using an app for iOS, Android or Windows devices.

Microsoft is providing computer vision capability as part of this collaboration. Starting with a general purpose computer vision model learned from millions of generic images, the Microsoft system can learn to recognize new types of objects – milk cartons, ketchup bottles, pickle jars and much more – from example images. When a new image from inside a refrigerator is provided to the newly learned model, it can detect the presence of the objects it has seen before during training. The image result shows an example image with labels of the food products detected (and confidence levels) overlaid.