New Life after the Rains and the Jetson

The team’s brand new semester starts off by executing the ideas that were discussed to make this project successful. With our deadline set to have deliverable results by the end of March this team is focused on bringing these concepts to life in the time we have left.

A New Year for the Fish


Image 1: A couple of Chinook spotted in December

We have our thanks to Dr. Allee form South Bay Clean Creek Coalition (SBCCC) for consulting us of the current status concerning the spawning of the Salmon. For a female Salmon, it’s important for them to select the right location where there is a good flow of water that can provide enough oxygen to their eggs, they will then die after completing their parental duty of securing the egg nest. The SBCCC organization counts these carcasses for record keeping and also formulates an estimation of Salmon in the river. The organization believes that around 100 fish returned to spawn this year; however, that number is in the best case scenario.

It’s extremely difficult to observe any of the hatchlings at this time of year because of the recent rains that we had in the Bay area. This causes sediment deposits to alter the properties river by decreasing visibility, finding fish at this moment is like trying to find a needle in a haystack. Conditions will return to normal towards the beginning of next month for testing the device.

The Brains of the System

We have acquired the Jetson TX2 and using it to analyze the images and videos collected from the river. This was made for AI application so it should have no problem doing the work we want it to do. The team intends to mount this device on the final product equipped along with a webcam to perform real-time analysis, so we have to be sure that it’s container is air and water tight. Future iteration of the project can take advantage of this device’s features and processing power. It currently has Jetpack and Ubuntu installed to equip it with software tools.


Imge 2: The Jetson TX2 by NVIDIA


Redesign of Rover

We are currently in the process of coming up with new designs for our rover. This rover will float on top of the Guadalupe River and look down in order to locate large populations of fish. The new design needs to be able to withstand highly turbulent water conditions, and move in 4 directions (forward, backward, left, right) while maintaining rotational stability. Below you can see 2 preliminary designs that have been taken into consideration and have been modeled on Solidwork’s CAD software.  Many other hand-drawn sketches have been created that may be finalized later.   The biggest obstacle has been fitting the large battery in the rover alongside other potential hardware that will be implemented by our computer engineering team.  The mechanical team plans to finalize which designs are the best fit for our problem by the end of this week and hope to start creating the first iteration of our prototype by the end of October so that we can work out different issues that we may face along the way.



Preliminary Designs that have been modeled in Solidworks.

Below the preliminary designs, you will see the parts we have to work with (from previous rovers) in order to build the rover.  The team is also currently working on controlling T100 Thrusters with an Electronic Speed Control Module (ESC) using an Arduino Microcontroller.  Alongside, we plan to use adopted toolboxes that are installed within MatLab that can help us to interface with the Arduino and create various control systems. Alongside learning about the thrusters and how they work, the mechanical team has been working on updating themselves with Arduino, Matlab, Solidworks, and other potential software that can help benefit us in providing analysis on various designs for the rover.


Inception of the ROV’s AI

By: Christian Lopez & Alan Huang

Without a doubt the most difficult aspect of ROV project is finding the—elusive and endangered—Chinook Salmon and Steelhead trout; thankfully, this year’s team can count on the data gathered from previous years to advance the project to the next level. Thanks to the efforts of Jacob Jones, who laid down the foundation to use Tensorflow on the team’s computer, the fish were able to be identified in still images:

The model training conducted on Sep 27th served as a proof of concept for the full-scale training the software team will conduct once more data is gathered from the deployment location. The fish identification was made possible by leveraging Tensorflow’s Object Detection API along with COCO datasets.

Let’s dive a little deeper on the Deep Learning Fish-Model (pun intended)

The training was done on a Dell XPS 15-L502x with a Nvidia GeForce GTX1060 Max-Q with an Intel i7 7th gen CPU for 300 steps, we can observe some of the progress done after the training:

The image on the left depicts a school of trout with only one being recognized as it was part of the training data. After a brief training, the model is able to identify four additional trout. In the coming weeks the team will visit the Guadalupe River in hopes of finding more data to continue improving the model; additionally, the team hopes to deploy the model in live video streams by leveraging OpenCV.


Trip to the Project Site

On September 28, we convened with our community partner, Dr. Allee, at the Guadalupe River to get a better grasp on what the problem is. Video footage was taken for the first two sections of the river system. The river has three sections where the steelhead trout and chinook salmon may swim by.

The first section of the river we explored has the most turbulent waters, it has the most rifles and it’s the shallowest; it has water flowing down from multiple levels through narrow openings in the middle of short walls. These small openings are where the velocity of the water is at its greatest. The color of the water from a top view is brown and murky. This section is the cleanest with the least amount of trash since it is near the edge of the trail.

Section 1:

The second section of the water is where trash becomes more prominent at the river banks because of the homeless but the coloration of the water from a top view is a murky green. This section of the water is calmer than the first, the water flows slower and at a more constant velocity. The water is about as shallow as the first section. As we walked down this section, the river got narrower and narrower until it was only a few feet wide and this part had a lot of branches and natural entities blocking the river flow.

Section 2:

Crossing over the second section, we come to the main section of the Guadalupe River. This section has the deepest waters and least amount of flowing waters but it also has the most amount of trash and dirty clothes laying around the banks of the river. The color of the main section is similar to the second section with a murky green color. Dr. Allee noted that this section will have the least amount of steelheads residing and swimming. We couldn’t get videos of this section of the river because it was too dirty and the walk down was too steep.





Strong Alliances

On Friday 14th, 2018 Jacob and Colin visited IBM Research to speak with Industry Advisor and Project Mentor Dr. Tom Zimmerman. During the visit, Dr. Zimmerman entrusted the team a prototype microscope for data collection as well as materials to build a microscope of our own capable of running on the ROV.

On behalf of the Underwater ROV 3.0 team, we would like to thank Dr. Zimmerman and his colleagues for taking the time to provide guidance during this project, their extensive knowledge will prove instrumental in the success of this year’s endeavors.

Colin Schardt at IBM Research, San Jose CA 2018.

Jacob Jones at IBM Research, San Jose CA 2018.