An AI-powered network of thermal sensors is helping ships avoid collisions with whales.
What’s new: WhaleSpotter detects gray whales in real time based on their heat signatures and relays images to human experts for validation. Newly deployed in the San Francisco Bay, the system alerts ship captains to the presence of whales, despite glare, darkness, or light fog, with enough lead time for large ships to change course.
How it works: WhaleSpotter’s algorithm takes input from heat-sensing cameras that can be mounted on land or vessels. When the algorithm detects a whale, the system transmits a video excerpt to experts, who can send an alert to ships in the area. Within a week and a half of operation, it had logged 6,600 whales.)
- WhaleSpotter has not disclosed details of its algorithm. Press reports describe a neural network that was trained on hundreds of thousands of thermal images including negative examples including birds, breaking waves, and boats. It runs on undisclosed local hardware to avoid delays incurred by transmitting high-resolution video to a data center for processing.
- Whales are warm-blooded, so the water emitted from their blowholes and exposed surfaces of its bodies are typically at least 3.6 degrees Fahrenheit (2 degrees Celsius) warmer than ocean water. Thermal cameras detect blows and breaches up to 4 nautical miles away. Shoebox-size enclosures protect them from salt water and other environmental hazards, and a stabilization system keeps their view steady. The San Francisco installation includes two: one mounted on a Coast Guard tower on an island in the Bay and another on a passenger ferry, with plans for more.
- When the algorithm classifies a whale, the system sends out a brief video segment as well as vessel telemetry (GPS location, bearing, time of day) to an onshore data center, which relays it to a team of experts who can validate the video within around 30 seconds.
- On a ship, a dashboard receives alerts in real time and displays verified whale locations (see image). The time elapsed between classification and alert can be as little as 1 minute. Human-in-the-loop operation yields in 99 percent accuracy, avoiding fatigue that may be caused by false alarms.
Behind the news: WhaleSpotter’s system is the result of more than a decade of research at the Woods Hole Oceanographic Institution (WHOI). In 2024, the team formed the company to commercialize the technology. The shipping company Matson provided early support, and it put units in vessels that served Alaska and Hawaii, becoming the first container carrier to deploy the technology commercially. Today more than 70 WhaleSpotter systems are deployed in vessels, ports, and offshore-energy operations. The San Francisco installation is the first that includes both stationary and moving cameras.
Why it matters: WhaleSpotter addresses a longstanding problem among mariners. Ships strike and kill 20,000 whales of all kinds, according to the conservation group Ocean Wise. Traditionally, ship operators rely on humans who look for visual cues of whales on the water’s surface (which is effective only if whales surface in conditions of high visibility) and listen for whale vocalizations picked up by hydrophones mounted on buoys or ship hulls (which works only if whales vocalize). In recent years, as ocean temperatures have warmed, greater numbers of gray whales have entered San Francisco Bay in search of food. Of whales that die there, around 40 percent are struck by vessels, according to a study by Marin County’s Marine Mammal Center and California Academy of Sciences.
We’re thinking: A whale of an opportunity awaits AI builders who combine advances in sensors with deep domain knowledge and workflow integration!