Computer vision is already hard at work
AI-enabled software is keeping us safe on the streets, policing the Internet, and simplifying commercial transactions.
In public safety, law enforcement dash cameras are supported by smart software that can read license plates on moving vehicles, looking for stolen cars or supporting Amber Alerts.
On the Internet, YouTube uses algorithms to scan millions of hours of video content in search of inappropriate language, nudity and copyright violations. Automation and AI are vital to this effort, which would certainly swamp even an army of human observers.
In the financial sector, Spanish bank BBVA allows new customers to open an account by uploading a photo of their ID and taking a selfie. Computer vision analyzes the photos to confirm identity. In healthcare, Gauss Surgical offers a real-time blood monitor that analyzes pictures of surgical sponges in order to track blood loss during surgery.
Accenture has also worked with a major insurer to use computer vision and other aspects of AI to automate vehicle damage claims processing. When receiving photos of a damaged car, the insurer can automatically detect the level of damage and use it, for example, to order spare parts and possibly detect fraud. As a result, this company is realizing 90 percent accuracy in automated claims analysis, reducing travel requirements of adjustors, and using an intelligent claims processing system that allows agents to spend their time on the subset of claims where their expertise is needed. Now the client is expanding this capability to other lines of insurance such as home and valuable personal property.
On the commercial side, Amazon has integrated a computer vision element into its prototype Amazon Go stores. By leveraging computer vision, sensors and other technologies, the retailer enables customers to exit the store without stopping at a cash register, with purchases charged automatically to their Amazon accounts.
Accenture recently worked with a large oil & gas equipment and services firm to use computer vision to optimize how they manage and maintain production equipment. By actively analyzing their operations, they were able to streamline service processes and increase asset throughput across their maintenance facilities to double utilization for $35 billion of operating assets.
It’s not hard to image how these emerging use cases could impact federal agencies on both the military and civilian sides.
Take for instance the field worker documenting a construction site, conducting a restaurant inspection, or charting a family’s domestic conditions. Computer vision can be trained to look for change states, documenting the evolution of a visual landscape over time and alerting a human analyst when things don’t look right.
For example, Accenture is working with a European government agency to create an automated land registry. By comparing property records with aerial images, they can detect discrepancies and non-compliance.
This same capacity might be brought to bear in efforts to assess and remediate in the wake of natural disasters. A change-state assessment might enable computers to tell emergency responders at a glance where the water is deepest and how fast it is rising. Computer vision might likewise be able to determine which roads and bridges are still passable, or how many buildings have collapsed.
Federal agencies also might look to computer vision to augment their physical security. Backed by AI, computers can monitor a facility 24/7 and can be taught to recognize signs of trouble. No one should be on the loading dock at 3 a.m. for example. Computer vision can watch for such anomalies and trigger appropriate responses.
It’s easy to see how the GSA might be interested in this, with over 9,600 owned or leased buildings under its watch. Together with the Federal Protective Service, they must ensure that security and well-being of more than one million tenants and visitors every day. It may be that computer vision, when layered on top of existing security-camera infrastructure, could improve both physical security and facility operations.
Perhaps the most talked-about role of computer vision is in support of self-driving cars, which will need to have eyes on the road – human or otherwise – if they are to navigate safely. Government has skin in the game, with responsibility over highway design and wayfinding. Transportation officials will need to incorporate an understanding of computer vision as they seek to make the roads safe for tomorrow’s auto-piloted vehicles.
As computer vision becomes more sophisticated, it could serve an ever-expanding role in national security.
We’re already seeing iterations of computer vision in use by the military as an important competent of C4ISR (Command, Control, Communications, Computer, Intelligence, Surveillance, and Reconnaissance). Computers can scan endless hours of drone and satellite footage, highlighting changes, discrepancies and suspicious activities. It’s an invaluable service: Unlike human analysts, the computers don’t get drowsy or distracted.
The military has shown early interest in taking such capabilities even further through the application of automation protocols. In late 2017, for instance, a team of researchers from the Air Force Research Laboratory won the Large-Scale Movie Description Challenge at the 2017 International Conference on Computer Vision in Venice, Italy. In that challenge, researchers deployed AI-driven systems to automatically create simple written descriptions of short clips taken from commercial film footage.
From this descriptive capability, the military would like to expand to a predictive capability. As a kid in school you may have seen a sequence of pictures: A child has a ball, child throws the ball, dog chases the ball. What happens next? (Dog brings it back.)
Kids can do this without having to think about it. Computers can’t yet reason – but if they could, the military repercussions might be significant. Computer vision at that level could change the game dramatically, taking AI beyond a surveillance function to provide more enhanced situational awareness. What will it take to get there? Let’s consider some important next steps for government.