On Tech Ethics Podcast – Active Physical Intelligence Explained

To easily navigate through our podcast, simply click on the ☰ icon on the player. This will take you straight to the chapter timestamps, allowing you to jump to specific segments and enjoy the parts you’re most interested in.

1. Introduction and Guest Welcome (00:00:03) Host introduces the guests and outlines the episode’s focus on active physical intelligence.
2. Guest Backgrounds and Company Mission (00:00:57) Tara and Sam introduce themselves, their backgrounds, and KavAI’s mission in industrial safety.
3. Foundations of Information Theory and AI (00:02:05) Tara explains her research in information theory and its relevance to next-generation AI systems.
4. How Active Physical Intelligence Differs from Traditional AI (00:04:16) Discussion on how active physical intelligence is distinct from passive, data-ingesting AI systems.
5. Sensor Data and Signal Modalities (00:06:24) Exploration of the types of data and signals used by active physical intelligence, with practical examples.
6. Real-World Applications and Industrial Safety (00:08:54) Examples of how the technology improves safety and monitoring in industrial environments.
7. Case Studies in Oil, Gas, and Industrial Sites (00:10:21) Sam shares specific use cases and benefits in large-scale industrial settings.
8. User Interface and Experience (00:13:59) Description of what plant staff would see and interact with when using the system.
9. Benefits of Multimodal Sensing (00:15:26) Explanation of how combining different sensor types improves diagnosis and reduces errors.
10. Applications Beyond Industry (00:17:00) Potential uses in public safety, emergency response, healthcare, and other fields.
11. Ethical Challenges in Active Physical Intelligence (00:18:53) Discussion of core ethical issues, including transparency, agency, and data privacy.
12. Responsible Design Principles (00:21:48) Design principles guiding ethical and responsible development, focusing on energy efficiency and privacy.
13. Learning More and Getting Involved (00:24:11) Recommendations for resources and ways to engage with the technology and its development.
14. Final Thoughts and Broader Impact (00:25:25) Reflections on the importance of responsible technology for environmental and human safety.
15. Energy Efficiency and Biological Inspiration (00:27:04) Emphasis on building low-energy, high-efficiency systems inspired by biological intelligence.
16. Closing and Further Resources (00:28:34) Host wraps up, thanks guests, and shares information about related courses and podcasts.

Daniel Smith: Welcome to On Tech Ethics with CITI Program. Our guests today are Tara Javidi and Sam Bigdeli, who are the co-founders of KavAI, a company that is using active physical intelligence to develop solutions that revolutionize monitoring and safety in industrial environments that pose the greatest threats to both industry and the planet. Today we are going to discuss active physical intelligence, which is an adaptive foundation model designed for real-time data collection and acquisition.

Before we get started, I want to quickly note that this podcast is for educational purposes only. It’s not designed to provide legal advice or legal guidance. You should consult with your organization’s attorneys if you have questions or concerns about the relevant laws and regulations that may be discussed in this podcast. In addition, the views expressed in this podcast are solely those of our guests. And on that note, welcome to the podcast Tara and Sam.

Sam Bigdeli: Thank you for having us.

Daniel Smith: It’s wonderful to have you. So I just very briefly introduced you both, but to get started, can you tell us more about yourselves and your work at KavAI?

Sam Bigdeli: My name is Sam Bigdeli. I’m the co-founder and CEO of KavAI, I’ve got an engineering background and a business background, serial entrepreneur. And at KavAI, we are tackling some really hard challenges in the AI field. And as you mentioned, the active physical intelligence scenario that we’re very passionate about and active Physical intelligence is a first sensor, native curiosity-driven AI system that, as you mentioned, actively seeks information in the real and the physical world. And that has many, many applications in many industries. It enables us to have a better understanding of what we are looking for and to prevent some catastrophes and loss of life and safety, and I can go on and on about it.

Tara Javidi: And my name is Tara Javidi and I’m the CTO and co-founder of KavAI, like Sam. From the perspective of education and academic work, I do research in a field that is called information theory, where the main problem of interest is how information can be extracted from data and how data can be informative. And as you can imagine, this is a really important question that goes back centuries. I would say that maybe historically the question of information has been viewed from a philosophical or maybe even linguistic perspective. And then it wasn’t until the middle of last century where people asked the question, can you actually think of information as a quantity? In particular has discrete quantities that we can store or communicate, and that gave a very different view of information and I would call it mechanistically.

And so my own research is focused on understanding this. Earlier on, this was a very important topic for building storage devices and communications devices like cell phones and whatnot. But given the success of AI in the past decade, my research is now entirely focused on how the next generation of AI systems should be designed from this data information centric point of view, and from that first principle. So in particular at KavAI, I am focused on really translating these rather academic questions into actually building this next generation of AI systems that will operate in the physical world and try to basically create a interface between the physical world and the intelligence that we need to interpret and predict and operate the physical environment better.

Daniel Smith: So going off of that a bit, and I know you touched on this, but can you just tell us some more about how active physical intelligence differs from other artificial intelligence or machine learning systems?

Tara Javidi: Yeah, so I like to think about the current AI systems, even the very advanced ones we all like to use these AI assistants that we are getting more and more used to depend on, they all are built on a principle that the data is provided to them and then they sort of ingest it. So the data process, the collection of the data has been done prior to building intelligence, the artificial intelligence on it.

But when we think about natural intelligence in contrast, say intelligence in the physical world, we see an animal that we say, “Ah, that animal is intelligently behaving,” or we see other people. We don’t limit it to this passively sitting there, ingesting data and analyzing it. We think of intelligence as innately coupled with action and curiosity. So basically we turn our head when we hear a noise. We sort of walk towards something we don’t recognize to get a better view. We pick an object to get a sense for its weight to in order to decide what my Amazon deliveries must probably contain today.

So active physical intelligence is basically is our proprietary AI ecosystem that brings this more critical notion of intelligence, this activeness of the intelligence to the physical spaces, especially physical space, the massive industrial physical spaces that absolutely need it, and we can talk about that later, but basically that’s what it really means to emphasize, that it is active in the process of extracting data and then ingesting that for building and training.

Daniel Smith: So in terms of the data and signals that active physical intelligence relies on, can you just talk some more about that? I know it sounds like it’s context specific, so maybe if you want to share an example or two, but just tell me more about the kind of data and signals that it can use and leverage.

Tara Javidi: So the first rule of thumb that we build our system is the more sources and signals you have, the better. So basically we want to be sensor native in the sense that we want to operate with the sensors and their physics in mind, but we also understand that to operate in the physical world, you are limited to your sensing. So we want it to be general, to capture sensing modalities that different customers have the ability to track but also put them all together. The very same way we as humans, we understand our sensors, our sensing capability, if those of us who have bad eyesight would wear glasses. So our intelligence is sort of adapted to our sensing capabilities, this is similar in that sense.

So to give you an example and that would an easy one to build is that we work with a lot of physical spaces that have cameras and now a camera’s angle of view, their field of view, their sensitivity to light and so on can be adapted at all times. So our solution profiles these sensors, now in this case it’s very simple cameras and just sort of orchestrates the collection of information to build, first of all it’s 3D understanding of the world, but also capture other sensing modalities. So let’s say if I have temperature readings to combine these two type of modalities, the world model that we’ve built will capture that input. So the nice thing about, just thinking about information from an abstract point of view is that all of these signals will be digested to be talking about the same information source, which is the physical environment that they’re collecting signals from.

Daniel Smith: So going off of that example some more, can you tell me about some of the real world applications of this and how it can contribute to safety in an industrial or other environment?

Tara Javidi: Yeah, so I can give you a simple example and then pass it to Sam who will talk about it from the perspective of the larger context. But the examples I can think of is, for example, the cameras I mentioned, if you’re running a large site, often you would be interested in safety requirements for the workers in the site, and so you would want the cameras to capture the images and videos that are most relevant to the safety requirements, an object is wobbly, the crane is not behaving as is safe, a person has taken off their helmet because they’re cleaning their face and you need to orchestrate the rest of the site’s operation accordingly. So those are the kind of environments where cameras are useful. And then I will pass it to Sam because we have done this work for other entities that have more interesting sensors in terms of chemical sensors. So I joke that if you think of cameras as eyes, they have noses and ears and so on. So maybe I’ll pass it to Sam if you want to talk about this Sam a little bit more.

Sam Bigdeli: Sure, sure. I mean, having physical awareness and being able to understand and reason and know the sensing all the sensing modality will give you a lot more information. It enables you to get all that information and be able to denoise that information from multiple modalities or multiple sensors. One of the key areas that we have found to be very useful is just think of these large industrial sites like in petrochemical or a refinery. You have miles and miles of pipeline and acres of land. And if you want to just very quickly look at the site and try to identify some areas very quickly that have potential issues, how do you do that? And that’s what we have been using active physical intelligence in order to tackle that problem.

As Tara mentioned, as you look at a scene with all different modalities of sensing in time and space, so at this moment in time and space, I have high mechanical vibration at this tank, I do also see some higher humidity and the insulation is broken and you really don’t need a lot of information and data, trained data to realize that four or five of your channels are showing some kind of abnormality. And that kind of triggers your foundation models looking and starting to curiously ask for more resolution or higher coverage and get close to get more information about the anomalies that we’ve seen.

So this active, being actively focusing on the area of attention enables you to be way more efficient, use less energy and make decisions much faster in the physical world. We’ve done some projects in the oil and gas industry, paid projects that we put that into use. We’ve been able to show that looking at the world in this wider spectrum of sensing and modality, but we’re using our world model, it can identify things that they would haven’t been seen in the past or they don’t even see it today because they usually either do their inspections in one modality, in one sensor, like a thermal camera, and they look at the site and then six months later they come in and do another kind of visual inspection. So you can never look at the same tank, same pipe, same valve, and see at that time what are those other signals that are coming out of that object that is considered abnormal.

Alexa McClellan: I hope you’re enjoying this episode of On Tech Ethics. If you’re interested in hearing conversations about the research industry, join me, Alexa McClellan, for CITI’s other podcast called On Research with CITI Program. You can subscribe wherever you listen to podcasts. Now, back to the episode.

Daniel Smith: So bringing all of that together, if I were a staff member at a plant like that, what would the interface look like for me if I was utilizing artificial intelligence in this way and getting the sensor data and the insights from active physical intelligence?

Sam Bigdeli: Imagine if you have eyes, super sensitive eyes, ears, and nose, and you would walk into the site and not only you would see a visual like RGB signals, you would also see at the same time, you could hear, see the thermal signatures, you could see the humidity, you could hear mechanical vibrations that are coming at the same time, all of that at the same time. So if you’re hearing high mechanical vibration or sounds from the pump and then you can immediately turn around and see, “Oh, that pump is overheated.” So that enables you to quickly make a decision. And that’s the interface that we’re offering to big industrial use cases that we’ve picked, as our beachhead market is they can now look at the entire site and have all the data points in this 3D model that’s not just a digital twin or a 3D reconstruction of images.

You can peel off different sensing modalities and say, “Okay, I want to overlay gas signatures that I see around this tag. I want to overlay heat signatures, humidity.” And that is very important in diagnosing things. It’s just even when you and I go to the doctors, the doctor is going to listen to your heart, it’s going to do some tests. It’s not always single modality that is going to give you the ability to increase your signal-to-noise ratio.

And I’ll give you an example, we did a project that they had high hydrochloric acid availability in an area and that area happened to be a high humidity area due to proximity to the cooling tower. So that immediately changes the, basically, the corrosion ratio. So instead of kind of guessing where in the next plant shutdown, where to go and what to look for, they quickly could zone into that particular kind of pipe and tank and put more effort into that. And as a result you get faster turnaround, which saves you a lot of money and helps you not miss the critical things, the false positives and false negatives that happens all the time in these large industrial sites.

Daniel Smith: It seems like this technology could also be applied in other settings as well beyond industrial sites. So are there other industries or fields that you think would benefit from using this technology now or in the future?

Sam Bigdeli: Yes, absolutely. We are actually in talks with two, three other verticals. Our focus obviously is on these large industrial sites, it’s a very big problem, it’s an environmental issue. But we’re also talking to public safety and security. One of the really interesting use cases, we’re talking to one of the fire departments here in Bay Area, the response time is three minutes usually to an incident. And we are now offering to send a drone of time to use hyperspectral imaging and a couple of other modalities to basically assess the structural integrity and also use other modalities to pinpoint where people are in the building. So by the time the engine gets to the site, they have some information ahead of time about should they enter the building or from what area, from what angle to tackle the fire, things like that. And that’s one area.

And we also are exploring use cases in healthcare and anywhere you interact with people real world and you really need to have an understanding in real time to basically have this dynamically decide what to do next. This technology or a foundation model, that’s what it really shines.

Daniel Smith: That’s excellent. So shifting gears a bit in terms of the ethical challenges that active physical intelligence presents, can you just walk through some of the core ethical issues that you all are thinking about as you deploy this technology in different use cases?

Sam Bigdeli: I mean, with all these systems, you always have those concerns and it’s a topic of conversation every day, but particularly in our approach in physical intelligence, at least at this point, we’re constant to think about it, this intelligence that we are developing, it’s more about going out there and investigating something to prove your hypothesis, you’re curiously looking for information. And if that is in the service of, it’s kind of a hybrid system with a human operator or just assisting humans, that’s what we call, at this point we’re looking at it, but obviously we’re constantly thinking about transparency, how these decisions are made, the agents and the safety around the agency of the system, is it doing what it’s supposed to do or is it basically deviating from its decision making or what it’s intended to do? We definitely are constantly working on those and I think it’s one of our top priorities.

Tara Javidi: Yeah, one of the issues that comes up a lot for us is the questions of siloing data of sites that I think is an interesting conversation. Again, as Sam was mentioning, because we’re translating everything in terms of hypotheses about the actual physical space, some of the knowledge that gets produced are very local, and those are not to be translated to other outside domains. So that’s really important issue that is particularly challenging is what information is local and is to be kept private and what information is native to the sensor, and it gives you information and that the cameras, when they see far the qualities that… That is not private information that needs to be, or private attributes of data collection that needs to be saved at sites. So really working these things out, is that only part of the decision-making when you think about, I think, what you learn and what you carry from one site to the other.

Daniel Smith: Absolutely. And with the ethical issues in mind, are there certain design principles that guide your responsible development of active physical intelligence?

Tara Javidi: Yeah, for sure. The ethics of energy consumption and privacy are the two main philosophical motivation for active learning as opposed to passive learning. So the idea that you will just have this amazing amount of redundant information that has been given to you for free, the way that it’s being tried in legal cases that I’m sure you and your audience are following up that model is really what active learning tries to go away with. We want to be very… We build systems that are very, very careful about which piece of information is needed and how critical it is. So that’s definitely those two, we feel like we already have an edge by relying on these active learning principles as opposed to passive ones. There are other ones that I think, as Sam was talking about, transparency, ability and such, which is things that we need to address with our customers are more generic, I would say, to a lot of AI solutions. Sam, am I missing something there?

Sam Bigdeli: No, fantastic. I think to kind of reiterate, our goal is in the company really deep philosophy or if we want to follow the vision is, the system is built to be very efficient in the use of energy and information, obviously the information we want to process, we need more energy, we need more power and that’s the whole conversation that’s going on right now. But the goal really is to, that we are, from an ethic perspective, we are really, really focusing on what really is the information that we need to process. And we don’t want to digitize the hell out of the world and process all of the digital information. And that is as our core belief, at our core fundamental design of everything we do.

Daniel Smith: That’s really interesting. So I would imagine that a lot of our listeners would like to learn more about this technology or possibly get involved in the development of this technology. So with that in mind, do you have any recommendations for additional resources or just how one can learn more?

Sam Bigdeli: Sure. I mean, they can directly reach out to us or our website or on LinkedIn. We’ll be more than happy to, we’re always interacting and Tara has a lot of talks that she’s doing around the country and internationally. This is an area that I would strongly, strongly recommend people to look into because we have to be way more conscious about how we process information and data. We can’t just support information and gigawatts of power in order to achieve what we’re trying to achieve.

Daniel Smith: And my final question for you both is just do you have any final thoughts that we’ve not already touched on?

Tara Javidi: I definitely would like to emphasize that even though the solution we have is quite general for all physical spaces, we both, Sam and I feel very strongly that really we have built physical and industrial spaces that are much larger than are, so far, than our ability to monitor them and make sure that they don’t end up in places and in states that are causing environment, even human loss of life.

We just really have not built our technologies with the same care that we should give the planet, we both feel. And I tend to think of there are diseases and health conditions that we all view as things of the past, and we have learned to really be smart about what we consider rules of hygiene. And this has really changed human life, humanity’s life should say. And this is what we both in the bigger picture are very committed to really building technology should not come at the cost of destroying our planet or hurting some people in service of delivering that technology. And that’s something that I think in these very large sites, I would very much like, and I know Sam agrees with me, we would very much like to see incidents on plant shutdowns and irreversible environmental damage should become a thing of the past.

Sam Bigdeli: Absolutely, a hundred percent, that is the topic of conversation every week, and that’s what kind of drives us. I also want to add one other kind of aspect of all of this, and I think we’ve kind of touched on it, but it’s very important that this is, we really need to be mindful in those systems that require less energy. We cannot go with the way we’re going with energy and we have to…

And these sites, our needs for energy is not going to be going down anytime soon, but we can definitely build models and solutions of that can achieve greater results and intelligence that comes at less cost. And that’s an area that Tara and our team is like working day and night on how we can be better at processing information. And human beings and animals are a very good example of that. Even a cat has one of the highest physical intelligence that you can find. There are many, many examples in the animal kingdom, but a small model that they have with very little power, five watts of power, they have very good spatial temporal understanding of the world.

Daniel Smith: I think that’s a wonderful place to leave our conversation for today. So thank you again, Tara and Sam.

Tara Javidi: Thank you Daniel.

Sam Bigdeli: Thank you Dan for having us. It was a pleasure.

Daniel Smith: If you enjoyed today’s conversation, I encourage you to check out CITI program’s, other podcasts, courses and webinars. As technology evolves, so does the need for professionals who understand the ethical responsibilities of its development and use. CITI program offers ethics-focused self-paced courses on AI and other emerging technologies, cybersecurity, data management, and more. These courses will help you enhance your skills, deepen your expertise, and lead with integrity. If you’re not currently affiliated with a subscribing organization, you can sign up as an independent learner. Check out the link in this episode’s description to learn more. And I just want to give a last special thanks to our line producer, Evelyn Fornell and production and distribution support provided by Raymond Longaray and Megan Stuart. And with that, I look forward to bringing you all more conversations on all things tech ethics.