Autonomous Lifeguard System:

The creation of a system to patrol beaches and

rescue people in aquatic distress

Shlok Shah
8​th ​grade
 Abstract

Although drowning is the third leading cause of world accidental death, most beaches
cannot afford to hire lifeguards, and even if there is a lifeguard, they cannot get to the victim
very quickly. Machine learning is a collection of algorithms that create a model to do a certain
task based on predefined examples known as training data. In this invention, the power of
machine learning is used to create a system that can help save drowning people on beaches,
pools, and other water areas. The invention consists of one or more raspberry pi drones that
patrol separate areas of the beach, and a land-based ground system. All these machines
communicate with each other using weak-link radio signal. Simulated drowning sequences were
recorded and used as training data for machine learning classification system. Based on the GPS
coordinates of the beach given to the ground station, the drones navigate the beach and capture
frames from cameras. By feeding in this data into the convolutional neural network, it will try to
determine whether or not a person is in aquatic distress or not. Once detected, the drone will
navigate to the person based on the position of the person in the frame, and drop a floatie. The
invention was divided into multiple components and each was tested separately. First, the
accuracy of the detection system was tested on the training and testing data. The network
achieved a 94% accuracy on testing images. Then, the drone navigation was tested, its average
loss was 1.2, and its average navigation time was 5.8 seconds. In conclusion the system worked
well, and the detection system succeeded immensely on the given data. But as is always the case
with machine learning, more data will improve its accuracy and scalability.

1
 Table of Contents

Abstract 1

Real-world Problem 3

Engineering Goal 3

Research 3

Materials 7

Methods 8

Challenges Overcome 15

Results 16

Conclusion 18

Suggestions for Improvement/Further Research 19

Acknowledgements 20

References 21

2
Real-world Problem

Drowning is the third highest cause of world accidental death, and so has a huge

magnitude. But, most dangerous and remote beaches cannot afford to have one or two lifeguards.

And, even if they do have a lifeguard, it is hard for them to detect drowning from far away and it

takes a while for them to get to the person. This is why I am creating an autonomous lifeguard

system.

Engineering Goal

The creation of an autonomous lifeguard system for private and public beaches to

autonomously navigate the beach and rescue drowning people.

Research

Imagine a world where machines could think by themselves and perform all the laborious

tasks that humans have to perform currently by themselves. Currently, a branch of AI (Artificial

Intelligence) called machine learning is making this possible. Coupled with standard

programming and electronics, machine learning can make many dreams a reality, and improve

the world. For example, it can be used to reduce world drowning rates, the third leading cause of

world accidental death.

3
 Machine learning is a branch of artificial intelligence that can help machines learn new

algorithms through examples. It first uses a pre-defined mathematical model, such as f(x) =

mx + b for example, with certain unfilled values known as parameters (m and b). It then first fills

these parameters with random values as a starting point. After randomly initializing the model,

training begins, where the algorithm adjusts the model’s parameters based on training data. To

train, a loss function is needed, which evaluates how incorrect the model is when evaluated on

training data. A very popular optimization technique called gradient descent uses derivatives

(form of calculus) to correct each parameter individually. It calculates the derivative of the loss

function in respect to each of the parameters using the chain rule, multiplies them by a constant

factor known as the learning rate to scale them down, and subtracts each from its respective

parameter to correct it. New types of machine learning models (the mathematical function with

parameters that the optimizer trains) can increase accuracy and efficiency.

4
 A convolutional neural network filters through image and sound data, finding patterns in

highly complex sequences, using layers of pooling and softmax at the end (shown in diagram

above). A recurrent neural network is meant to help the model gain some memory and uses past

decisions to influence current ones. Tensorflow is Google’s own machine learning library for

python, that simplifies code and increases efficiency. It uses automatic derivation and complex

multivariate calculus to help compute gradients quickly. It is also well documented and highly

customizable. Machine learning helps machines learn patterns on their own, decreases system

development time dramatically and allowing scientists to do things that were never before

possible.

The Raspberry Pi is a popular computing platform that is small, extremely efficient,

powerful, and good for robots. The raspberry pi is easy to setup and can be easily programmed. It

supports various languages including Python, C, and C++. It only needs a USB power cable and

SD card with Linux to boot up. Coupled with Navio2, a flight controller, it can be used to control

robots and other embedded systems. Its small size and computing power makes it good for

robots. Navio2 has various sensors that help the autopilot keep track of the drone’s state. It can

control up to 15 servo motors that can be used to manipulate the robot’s state in the physical

world. Ardupilot is an open-source autopilot software that helps keep the drone stable, by

performing computations with various sensor readings (barometer, accelerometer, gyroscope,

GPS) as input. Using state of the art motor stabilizing techniques, and position tracking,

ardupilot helps simplify code. Raspberry Pi coupled with Navio2 is very useful for drones and

other robots, as it helps simplify higher-level development.

5
 There are two different stages of drowning, both of which have to be treated differently.

When water hasn’t entered a person’s lungs yet, they will be making noise and calling for help;

this is called aquatic distress. Because they still have the energy and consciousness needed to

make these actions, they can thrash around and try to call attention. To rescue these people, a

floatie will be fine, and a watch should be kept on them. When water has entered the person’s

lungs, they will be making no noise, and will be barely floating in the water; in about twenty

seconds, they will start sinking. After this, they have about a minute left to live. To save these

people, a nearby beach should be contacted and the police informed. CPR must be done to get

the water out of their lungs and save them. Contrary to the general belief, both kinds of drowning

people can be saved differently.

Machine learning, with modern robotics and technology could help save drowning

people. Drowning is the third worst cause of world death, and lifeguard automation can help

reduce drowning deaths.

6
Materials

● 2 raspberry pi 3’s - the computing platform where software is run real-time

● Navio 2 flight controller - rpi attachment to help fly the drone

● Carbon fiber drone frame - chassis of the drone

● 4 1.5 kg servo motors with wires - flight motors

● 4 0503 propellers - help generate thrust

● 4 ESC - to help power and control the motors

● 3C LiPo battery 1500 mAh - to power the drone

● Composite rpi camera - camera for detection

● Quad Core processor desktop computer - for training the neural network

● Navio2 compatible GPS - to get current locations

● Wires, zip ties

7
Methods

Diagram of Drone

8
 Diagram of Drone’s Navigation Path

The arrows show the path the drone will take during its patrol. The green and blue

rectangles show where the drone’s camera can “see” during a certain point in the patrol. “Stable

detection” refers to where the drone can reliably detect and classify; unstable is where the system

cannot do it reliably.

9
 Diagram of angles from Camera and distances

C: angle from camera to a line parallel to the horizon

FOV: vertical field of view of the camera

h: Height of the drone above the ground

d1: Distance from the bottom of the camera frame to the horizontal location of the drone

d2: Distance from the drone’s horizontal location to the top of the frame

10
Diagram of steps to create classification model

11
 Flowchart for Software Running on the Drone

Note: The “ground system” is a device on shore that keeps track of the location of all the

drones and helps them communicate to authorities, first responders, and with each other.

Note: ​In current system implementation, the drone beeps and flashes LED when it reached

drowning person. A mechanism to drop floatie can be added later on.

12
 Instructions for Testing System

1. Gather all the data used to train the neural network.

2. Use this data to test the neural network (without drone).

3. Record the percent of data classified correctly.

4. Test the neural network on additional sequences (testing data) that it was not trained on.

5. Record the percent of data classified correctly.

6. Instead of the drone updating “direction” on the detection of drowning people, program it

to change “direction” upon the detection of any face (this way just the ability of the drone

to navigate to the person and drop the floatie can be tested).

7. Now sit on a flat surface.

8. Next, have a helper activate the drone (after entering into it the GPS coordinates of the

area).

9. Once it is fully activated start a stopwatch and see how long it takes for the drone to stop

and flash led near the person.

10. If it doesn’t do it within one minute, record a failure.

11. Repeat steps 7-10 six times for several different cases and positions.

a. Cases are different simulated situations where the drone and person can end up in

a real-life scenario

b. Some examples used for testing were the person and drone on opposite ends, and

the drone and person close by. Various other scenarios were also used.

12. Record the percent of times that it was a failure and the average speed to rescue.

13
13. Next have a clear area with no humans.

14. Record the optimal path that the drone should take in your computer.

15. Activate your drone (with GPS coordinates).

16. Do not interfere for 5 minutes.

17. Then, compute loss between the optimal path that you recorded and the path that it took.

18. Record this number.

19. Repeat steps 14-18 30 times.

Precautions

Make sure there are no people in the area except for yourself and one helper, so if the drone

malfunctions it cannot harm anyone else. Make sure ​you ​are the one who is pretending to drown.

14
Challenges Overcome

In the beginning, a convolutional neural network was trained to recognize between apples

and oranges as a baby step, but it was not converging. To solve this, the image size was reduced

greatly so that the the neural network could process the data and train on it. But, even after doing

this, it was not scaling well to other images. This was solved by giving the network much more

training data, and it performed really well. Also, the drone came with faulty sensors, so when it

was flown, it started swerving to the left very hard. So, the motors were immediately disarmed

and it fell on the ground. Later, after switching Navio2s and putting a vibration mount, the drone

was able to fly stably, but testing the navigation on the drone was too risky. This is why all the

navigation algorithms had to be tested in simulations. Lastly, the computer didn’t have enough

RAM to train a 3D convolutional network. So, each image was fed in separately into a 2D

network

15
Results

Accuracies of the Classification

16
Navigational Testings

17
Conclusion

Although drowning is the third leading cause of world accidental death, 70 percent of

beaches cannot afford to hire lifeguards, and even in the ones that can, lifeguards cannot

sometimes get to the victim in time. This is why the engineering goal was to create an

autonomous lifeguard system that can autonomously navigate the area and detect and rescue

people in aquatic distress.

The invention was divided up into three aspects: classification for detection of aquatic

distress, patrolling of the area, and navigation to the person in need. To test the classification

aspect, the model was tested on all the data that was used to train the neural network. All these

individual accuracies were averaged, and the result was a 99% accuracy on this data. This was

repeated with testing data, additional sequences of drowning and swimming, and 94% accuracy

was the result. Because these accuracies are fairly close together, it means that the system did

detect an actual pattern, and the fact that it was mid-nineties for testing data means that it

performed excellently.

To test navigation of the area, the software was run in a computer drone simulation.

Every second the algorithm calculated the loss between the drone’s GPS coordinates and the

expected coordinates, and then averaged this at the end. This was repeated thirty times and an

average loss of 1.2 was achieved. This means that the navigation did fairly well. The algorithm

for getting to the person was also in a simulation and an average time of travel of 5.8 seconds

was achieved.

18
 The engineering goal was achieved to a high extent. The network got a higher

classification accuracy than expected, and the drone navigated to the a person in its area ten to

fifteen seconds faster than a lifeguard who is fairly close to the person on a kayak. Also, each

drone would cost no more than one thousand dollars for implementation and a ground control

station no more than a hundred. This makes the system much more cost effective than lifeguards,

where each one costs more than seventy thousand dollars a year.

Suggestions for Improvement/Further Research

As is always the case with machine learning, more data would have increased the

accuracy and scalability immensely. Since actual drowning sequences were not widely available,

simulated drowning sequences were recorded to train on, but this limited the classification

system to only examples of me. For it to be extremely scalable for implementation, it needs to be

trained on recordings of many different kinds of people. To improve the system, the network

could also be trained on recordings from actual beaches, because this is the location the system is

designed for. Additionally, a mechanism to drop the floatie could be added and navigation could

be tested on the drone itself.

Features for auto floatie refill and auto charging could also be developed. These would

use GPS coordinates of certain charging and refill stations in addition to object detection. Lastly,

a RNN could be used to get data from the flattened layer of the CNN to make inferences using

past deductions.

19
Acknowledgements

I want to thank my father for guiding me when I needed help, ordering materials for me,

and taking me to all the locations I needed to go. I would also like to thank my teacher Mrs. Deza

for answering all the questions I had.

20
 References

Bishop, Christopher. (2006). ​Pattern Recognition and Machine Learning. ​Ithaca, NY:Springer.

Branche, C. M. et al.(Ed.). (2001, March 15). ​Lifeguard Effectiveness: A Report of the Working
Group​ (Rep.). Retrieved February 14, 2018, from CDC website:
https://www.cdc.gov/homeandrecreationalsafety/pubs/lifeguardreport-a.pdf

Emlid. (n.d.). Documentation for Navio2. Retrieved November 13, 2017 from
https://docs.emlid.com/navio2/

Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014).
Large-Scale Video Classification with Convolutional Neural Networks. ​2014 IEEE
Conference on Computer Vision and Pattern Recognition​. doi:10.1109/cvpr.2014.223

Reid, J. (2017, October 30). Multirotor Motor Guide. Retrieved November 13, 2017 from
http://www.rotordronemag.com/guide-multirotor-motors/

United States Lifesaving Association. (2013-2017). 2013 - 2017 National Lifesaving Statistics.
Retrieved November 13, 2017 from ​http://arc.usla.org/Statistics/current.asp?Statistics=5

Vittone, M. (2017, May 26). Drowning Doesn’t Look Like Drowning. Retrieved November 13,
2017, from
http://www.slate.com/articles/health_and_science/family/2013/06/rescuing_drowning_ch
ildren_how_to_know_when_someone_is_in_trouble_in_the.html

Vrigkas, M., Nikou, C., & Kakadiaris, I. A. (2015, October 29). A Review of Human Activity
Recognition Methods. Retrieved February 14, 2018, from
https://www.frontiersin.org/articles/10.3389/frobt.2015.00028/full

21