By: Christian Lopez & Alan Huang

Without a doubt the most difficult aspect of ROV project is finding the—elusive and endangered—Chinook Salmon and Steelhead trout; thankfully, this year’s team can count on the data gathered from previous years to advance the project to the next level. Thanks to the efforts of Jacob Jones, who laid down the foundation to use Tensorflow on the team’s computer, the fish were able to be identified in still images:

The model training conducted on Sep 27th served as a proof of concept for the full-scale training the software team will conduct once more data is gathered from the deployment location. The fish identification was made possible by leveraging Tensorflow’s Object Detection API along with COCO datasets.

Let’s dive a little deeper on the Deep Learning Fish-Model (pun intended)

The training was done on a Dell XPS 15-L502x with a Nvidia GeForce GTX1060 Max-Q with an Intel i7 7th gen CPU for 300 steps, we can observe some of the progress done after the training:

The image on the left depicts a school of trout with only one being recognized as it was part of the training data. After a brief training, the model is able to identify four additional trout. In the coming weeks the team will visit the Guadalupe River in hopes of finding more data to continue improving the model; additionally, the team hopes to deploy the model in live video streams by leveraging OpenCV.

Trip to the Project Site

On September 28, we convened with our community partner, Dr. Allee, at the Guadalupe River to get a better grasp on what the problem is. Video footage was taken for the first two sections of the river system. The river has three sections where the steelhead trout and chinook salmon may swim by.

The first section of the river we explored has the most turbulent waters, it has the most rifles and it’s the shallowest; it has water flowing down from multiple levels through narrow openings in the middle of short walls. These small openings are where the velocity of the water is at its greatest. The color of the water from a top view is brown and murky. This section is the cleanest with the least amount of trash since it is near the edge of the trail.

Section 1:

The second section of the water is where trash becomes more prominent at the river banks because of the homeless but the coloration of the water from a top view is a murky green. This section of the water is calmer than the first, the water flows slower and at a more constant velocity. The water is about as shallow as the first section. As we walked down this section, the river got narrower and narrower until it was only a few feet wide and this part had a lot of branches and natural entities blocking the river flow.

Section 2:

Crossing over the second section, we come to the main section of the Guadalupe River. This section has the deepest waters and least amount of flowing waters but it also has the most amount of trash and dirty clothes laying around the banks of the river. The color of the main section is similar to the second section with a murky green color. Dr. Allee noted that this section will have the least amount of steelheads residing and swimming. We couldn’t get videos of this section of the river because it was too dirty and the walk down was too steep.