Millions of Americans have started to work from home amidst the current COVID-19 pandemic. Retailers have struggled with supply while nervous consumers are hoarding everything from toilet paper to hand soap. Across the globe, Chinese e-commerce giant JD began testing a level-4 autonomous delivery robot in Wuhan and running its automated warehouses 24 hours a day to cope with a surge in demand.

Suddenly, autonomous machines need to be better than just proof of concepts. They can no longer depend on on-site engineering support for edge cases. They must be robust enough to work independently across various real-life situations.

In some ways, the COVID-19 epidemic accelerates an automated future that’s already on its way. It has exposed problems that have long existed in the artificial intelligence (AI) venture scene: buzzwords and hype cloud people’s judgment, making it difficult to see real progress.

The industry needs to take on much-needed reforms towards real-world autonomous systems in the following three areas.

1. Rethink metrics

As more autonomous machines are deployed in the real world, conventional metrics such as speed, cycle time, or success rate can no longer represent the full picture. We need to measure the reliability of the system under uncertainties with robustness metrics such as the average number of human interventions.

We need more tools and industry standards to evaluate overall system performance across a wide range of scenarios because real life, unlike a controlled environment, is unpredictable.

If a delivery robot can reach a max speed of 4 MPH but cannot complete a single delivery without human support, the robot is not creating much value to its users.

DevOps emerged a few years ago to shorten the development cycle and continuously deliver high-quality software. In comparison to software engineering, AI or machine learning (ML) is much less mature. 87% of ML projects never go into production. However, recently we’ve started to see MLOps or AIOps appearing more and more.

This marks a crucial transition from AI/ML research to actual products that are used and tested every day. It requires a significant change in mindset to focus on quality assurance instead of state-of-the-art ML models. I’m not saying we can’t have both at the same time, but to date, we’ve seen much more emphasis on the latter.

2. Redesign error handling and communication

The recent shutdown of Starsky Robotics reminds us that we are still years away from fully autonomous solutions. That doesn’t mean robotics cannot bring immediate values to humans. Even if humans need to handle edge cases 15% of the time, that still means companies can reduce significant labor and integration costs.

However, AI companies currently tend to spend much more resources on building autonomous systems and much less time thinking about error handling and seamless hand-off between machines and humans.

We need a better way of handling and communicating errors, especially for ML products because ML is more probabilistic and less transparent. Therefore, showing the confidence level of model predictions or framing your predictions as suggestions instead of decisions are ways to gain trust with users.

We need to categorize errors into different levels, design different protocols accordingly, and prioritize minimizing fatal errors that stop the system and require human intervention. If fatal errors occur and the system isn’t working anymore, can we respond quickly and troubleshoot remotely?

The most difficult part is to identify the unknown unknowns, errors that systems cannot detect. Therefore, it’s also crucial to have two-way communication and allow users to flag errors or choose to activate the previously agreed fallback plan.

3. Redefine human-machine interaction

The novel coronavirus forces companies to more rapidly adopt automation and shift to the cloud. As fewer people control a larger number of robots, do we have the right tools and technologies to pass all the relevant information to that decision-maker promptly? Are there enough sensors on each robot to provide a full picture?

Today, we rely on tactile input like computers or tablets to control robots. Are these still the best interface as the amount of information soars and response time remains short? Should we reconsider human-machine interfaces that go beyond tactile, for example, voice, VR/AR or brain-machine interface?

We also need to decide who should be in control. As machines get smarter, should we always make the final call?

For example, who should be controlling an autonomous robotaxi? The car itself? The human safety driver? Someone who monitors a fleet of robotaxis remotely? The passengers? Under what situation? Or should it be a co-decision with weighted judgment by both humans and machines? What’s the ethical implication? Can the interface support multi-step co-decision making?

Ultimately, how do we design human-centered AI to make sure autonomous machines make our lives better, not worse? How do we automate the right use cases to augment humans? How do we build a hybrid team that delivers better outcomes and allows humans and machines to learn from each other?

There are still a lot of questions that we need to answer. And the current COVID-19 pandemic is pushing us to answer them more quickly so that would-be autonomous systems can deliver on their promise. If the makers of these systems can focus on the three areas I’ve outlined above, they’ll be better positioned to reach key conclusions more quickly. And that will ensure we’re heading in the right direction.

About the Author

Bastiane Huang is a Product Manager at OSARO, a San Francisco-based startup building AI-defined robotics. She has worked for Amazon in its Alexa group and with Harvard Business Review, as well as the university’s Future of Work Initiative. She also writes about ML, robotics, and product management.