Background

    While at South Carolina, I have been involved with several laboratories off and on campus with diverse and different missions. I have integrated my own mission to implement strong computational applications into empirical ecology projects while producing products in the form of presentations, publications, and repositories. I often recognize a pattern of difficulty in balancing the fields of theoretical and empirical ecology, but have a strong desire to continue constructing this necessary bridge. I find it necessary for someone with my diverse interests to expand myself past a defined ecological problem, but challenge myself in model construction across a diverse set of data types and ecological scenarios.

Problem Statement

     As offshore energy expands across the U.S. exclusive economic zone, federal regulatory agencies and industry are using rapidly advancing sensing technologies for environmental compliance under NEPA and protected species mitigation. Passive acoustic monitoring (PAM) has become a significant contributor to the success of marine protected species by providing an autonomous platform for evaluating dynamic stressors. However, remote sensing approaches like PAM are inhibited in identifying species-specific fish vocalizations.

Problem Solution

    Audio-video classification of species offers a solution but has been limited to the sample of fish that vocalize in direct sight of a standard camera. Utilizing a first-of-its-kind system of surround video and a compact acoustic array, I will identify the source of the sounds in the marine soundscape and visually classify the species. The addition of visual confirmation will identify the sound source of specific bioacoustic events, such that those acoustic events can later be used for evaluating ecosystem health from longer-term PAM surveys. To limit observational bias, enhance efficiency, and identify multidimensional relationships on fish acoustic behavior, a convolutional neural network (CNN) ensemble would classify species of interest and pair specific vocalizations. Thus, fish communication would become an autonomous means in inferring relationships with potential vulnerabilities on coastal ecosystems. These vulnerabilities include increasing water temperatures, development of offshore infrastructure, disease proliferation, and congested shipping lanes. Furthermore, as species and their sounds are discovered, the construction of sound libraries will revolutionize the evaluation of PAM to the resolution of species.

Implementation

1. Improve Constructed System of Surround Video and Compact Acoustic Array

Testing of the system will be undertaken in a pool to adjust any malfunctions or bugs in the system. This includes water proofing the electrical system and ensuring a full range of view is obtainable. Additionally, this is when a procedure for setting down this device will be established as to deminish habitat disturbance. Refer to the figure for information of the system's design (Rice, A, 2021).

2. Select Locations for Testing and Data Collection

Find possible testing locations for the system by investigating areas of ecological importance and a suspected high diversity of fish vocalizations, for example coral reef systems or windmills producing renewable energy. Start creating data libraries of combined video and bioacoustic data for large data storage.

3. Construct a Model to Identify Fish Species in Video

Let data collect over the year to allow ample time to build a strong foundation in the database construction and generalization. This will give me a well rounded data base to create a model that can autonomously identify species in video and localize audio data to possible vocalizations. Model type may be experimental, but I will initially try a type of convolutional neural network model ensemble.

4. Build Sound Libraries of Fish Vocalizations

After model construction, I can store a library of known fish vocalizations to specific species. This library synthesis can then be transferred to spin off studies and be utilized in expanding the libraries.

5. Extrapolate Sound Libraries to Passive Acoustic Monitor Data

Since acoustic data has been collecting for a long time, we can further investigate the health of an ecosystem by matching sounds to specific fish species. This can help us to identify fish occupancy and estimate abundance without visual confirmation in remote or high turbidity areas. An important tool for conservation organizations looking for a dynamic method to assess ecosystem health.

6. Expand and Repeat Study to New Areas

Because we would only have fish in the environemtn where we initially tested and developed this method, we will have to expand this system to other environments. This could lead to new key stone species being identified in determine environmental health and would be an important tool in identifying new anthropogenic marine initiative effects. This includes shipping, energy development, and fishing that results in disease proliferation and population declines.

❮ ❯

Significance

Upon successful CNN construction and matching biological sounds to identified species, fish bioacoustics could revolutionize our understanding of ecological stressors, such as mitigating impacts of renewable energy development on coastal ecosystems. Inferring acoustic communication events provides management organizations with the tools to quickly adapt to vulnerabilities that have significant economic and environmental impacts on our nation.