River Herring Machine Learning Monitoring Systems

MIT Sea Grant Assistant Director, Advisory Services Robert Vincent and a team from the Institute for Experiential Robotics at Northeastern University – Anuj Shrivatsav Srikanth, Saicharan Thirandas, Dhanush Adithya Balamuguran, Anurag Daga, Taskin Padir, and Dipanjan Saha – published an IEEE OCEANS Conference paper on novel river herring machine learning monitoring systems. The work was also presented at the OCEANS Conference in Singapore.

>>View the Paper: Real-Time Background-Agnostic Fish Localization in Underwater Videos towards Autonomous Species Monitoring

In the Northeast US, adult river herring migrate from the ocean to freshwater for spawning every spring. In turn, juvenile herrings migrate to seawater every summer and fall. This yearly migration pattern makes river herring an essential component of both freshwater and marine ecosystems, as well as coastal fisheries.

The need for continuous monitoring was established more than ten years ago, when a decline in population to less than 3% of the historical peaks led to the closure of many river herring fisheries. Herring populations (alewife, Alosa pseudoharengus; and blueback, Alosa aestivalis) are often counted visually, followed by statistical corrections. Visual counting is labor-intensive and impractical, relying on volunteers counting sporadically during daylight hours, or thousands of hours of monitoring video. Real-time, automated species identification and counting is a complex problem, as underwater environments have multiple sources of uncertainty, including water turbidity, lighting conditions, and fish overlap during periods of high passage rates.

Accurate assessment of fish population and spawning abundance is crucial for providing fishery management staff and marine scientists with valuable data necessary for sustainability, ecosystem monitoring, and understanding patterns of fish abundance and behavior in underwater habitats including near aquaculture farm habitats. A fully automated underwater video monitoring framework may be constructed in three stages, outlined in the paper, Real-Time Background-Agnostic Fish Localization in Underwater Videos towards Autonomous Species Monitoring.

Abstract—This paper investigates a novel machine learning framework for autonomous, real-time fish localization in underwater videos with diverse backgrounds. The framework consists of three different algorithms from the family of deep learning and computer vision. Each of them is a good solution to one or more specific needs; however, each algorithm has its own limitations. Combining these methods using ensemble learning is a way to accomplish background-agnostic fish localization in real-time. A specific combination called weighted voting learns an optimal set of weights, such that the highest weight goes to the algorithm with the highest prediction accuracy. Results presented for two underwater datasets with significantly varying background and illumination demonstrate that weighted voting can produce consistent localization irrespective of the environment.