DOCSF22: Sensors, Sensing, and Sensei

In this episode, Ivan Poupyrev, Director of Google ATAP, Advanced Technologies and Products Division, Nicholas Gillian, lead machine learning engineer at Google Atap, and Stefano Bini, professor of orthopedic surgery at UCSF, talk about a project they are working on together where they use sensors to improve the ability to collect data from patients. This project seeks to explore modern techniques for artificial intelligence in computing to help with the problems of postoperative recovery after orthopedic surgery. 

Ivan Poupyrev starts by talking about the next generation of computing, where the physical world is enhanced by this constantly. He explains how the project uses sensors to scan objects of interest, measure them, and convert this information into data. This type of technology is applied to build digital twins for healthcare. He details how data collection with sensors creates an accurate representation from which you can gather valuable insights for the consumer, be it the patient, the provider, or a payer. Thanks to modern AI advances, it's now possible to integrate as many desired sensors and create clouds without the need for a computer connection.

Nicholas Gillian presents the Google Jacquard tag, which contains an inertial sensor, a small microcontroller, flash memory, and Bluetooth that can directly stream data to the cloud. Using many of these together, the goal of that project is to see and demonstrate that low-cost, non-invasive consumer-grade hardware, Combined with the best of Google's AI and software, can be used to understand and replicate patient outcomes by using motion capture to register and analyze knee angular velocity, total support movement, and hip flexion, among other variables that can help surgeons monitor patients after surgery.

Listen to this conversation about how computation takes data collection into the cloud and visual representation for better insights and outcomes. And also, learn how wearable sensors derive more accurate outcome measures!