Project Overview:
This project was completed for my undergraduate biomedical engineering senior capstone class. It was done in conjunction with a research laboratory at the University of Cincinnati College of Medicine. The overall objective was to develop a non-invasive way to track the physical movement underwent by the guinea pigs that were being studied for heart failure research. This would allow the researchers to develop software that could predict when a guinea pig would enter cardiac arrest, without needing to surgically implant sensors that recorded physiological data.
Clinical Problem:
Sudden Cardiac Death (SCD) is the largest cause of natural death in the United States, resulting in approximately 325,000 deaths/year. It accounts for half of all heart disease deaths. Investigating the link between Heart Failure and SCD could reveal predictive measures that could provide more effective treatment strategies, from smarter ICD implantation to preventative lifestyle changes. SCD typically occurs with little to no warning, but underlying physiological signals such as heart rate, ECG patterns, and activity may exhibit trends indicative of an oncoming adverse cardiac event.
Doctors typically classify patients' heart failure according to the severity of their symptoms, with the most common scale being the New York Heart Association (NYHA) Functional Classification. The NYHA scale is divided into four classes based on how much a patient’s heart failure limits their ability to engage in physical activity, with Class 1 being no limitation of physical activity and Class 4 being unable to carry on any physical activity without discomfort. The development of a classification system similar to the NYHA scale in the Guinea Pig model that the lab uses would reveal how the occurrence of SCD changes as Heart Failure progresses, and also how different therapeutic strategies effect animals’ ability to exert physical activity.
Objectives:
This project will address improving predictive measures for sudden cardiac death by creating a system for tracking Guinea Pig movement. Tracking movement will enable the identification of trends and patterns between different treatment groups, and in single groups over time. Accomplishing this goal will involve the selection of appropriate hardware for video collection and the development of a camera setup and appropriate fixturing that facilitates accurate collection of data. It will also involve the development of a machine learning model that accurately tracks Guinea Pig movement.
This is significant because there are no published papers describing methods for quantifying Guinea Pig movement. Guinea Pigs exhibit similar cardiac electrophysiology to humans, so they make a desirable animal model for studying cardiac diseases. The development of a system for tracking activity could increase researchers’ ability to learn about the role that physical activity plays in heart disease and sudden cardiac death.
This approach could utilize noninvasive sensors such as those found in wearable fitness tracking devices that already measure various physiological parameters, including heart rate and activity. Once a signal is identified that is indicative of sudden cardiac death, an implanted device could be modulated to restore the heart to its normal rhythm. In order to develop an artificial intelligence system that decides whether a patient is going to experience an adverse cardiac event, these patterns first need to be identified. Identification of trends in activity necessitates long-term acquisition of accurate activity data.
Needs:
This project was completed for my undergraduate biomedical engineering senior capstone class. It was done in conjunction with a research laboratory at the University of Cincinnati College of Medicine. The overall objective was to develop a non-invasive way to track the physical movement underwent by the guinea pigs that were being studied for heart failure research. This would allow the researchers to develop software that could predict when a guinea pig would enter cardiac arrest, without needing to surgically implant sensors that recorded physiological data.
Clinical Problem:
Sudden Cardiac Death (SCD) is the largest cause of natural death in the United States, resulting in approximately 325,000 deaths/year. It accounts for half of all heart disease deaths. Investigating the link between Heart Failure and SCD could reveal predictive measures that could provide more effective treatment strategies, from smarter ICD implantation to preventative lifestyle changes. SCD typically occurs with little to no warning, but underlying physiological signals such as heart rate, ECG patterns, and activity may exhibit trends indicative of an oncoming adverse cardiac event.
Doctors typically classify patients' heart failure according to the severity of their symptoms, with the most common scale being the New York Heart Association (NYHA) Functional Classification. The NYHA scale is divided into four classes based on how much a patient’s heart failure limits their ability to engage in physical activity, with Class 1 being no limitation of physical activity and Class 4 being unable to carry on any physical activity without discomfort. The development of a classification system similar to the NYHA scale in the Guinea Pig model that the lab uses would reveal how the occurrence of SCD changes as Heart Failure progresses, and also how different therapeutic strategies effect animals’ ability to exert physical activity.
Objectives:
This project will address improving predictive measures for sudden cardiac death by creating a system for tracking Guinea Pig movement. Tracking movement will enable the identification of trends and patterns between different treatment groups, and in single groups over time. Accomplishing this goal will involve the selection of appropriate hardware for video collection and the development of a camera setup and appropriate fixturing that facilitates accurate collection of data. It will also involve the development of a machine learning model that accurately tracks Guinea Pig movement.
This is significant because there are no published papers describing methods for quantifying Guinea Pig movement. Guinea Pigs exhibit similar cardiac electrophysiology to humans, so they make a desirable animal model for studying cardiac diseases. The development of a system for tracking activity could increase researchers’ ability to learn about the role that physical activity plays in heart disease and sudden cardiac death.
This approach could utilize noninvasive sensors such as those found in wearable fitness tracking devices that already measure various physiological parameters, including heart rate and activity. Once a signal is identified that is indicative of sudden cardiac death, an implanted device could be modulated to restore the heart to its normal rhythm. In order to develop an artificial intelligence system that decides whether a patient is going to experience an adverse cardiac event, these patterns first need to be identified. Identification of trends in activity necessitates long-term acquisition of accurate activity data.
Needs:
- Some part of animal in frame of video at all times to prevent gaps in activity collection
- Accurately tracks linear movement in the x-y plane
- Camera at same height and angle in each video
- Compliant with IACUC requirements
- Usability - camera can easily and consistently be placed in desired location
- Tracker can redetect object if lost
- Can track circular movement and non-linear movement such as head-turning
- Animal trackable in normal and low-light environments
- Entire cage in field of view
- Automatic data transfer to computer
Selected concept for mounting the video camera to the top of the cage
Selected concept for cage design that would ensure full view of the cage floor
Conclusions:
The video system and machine learning model developed through this project was shown to successfully track Guinea Pig movement over time and enabled the calculation of distance traveled. Of the four locations on the animal’s body that were tracked, the center of the head yielded the highest tracking confidence, with an average of 43.3% confidence after 50,000 training iterations on approx. labeled frames. This was slightly lower than the desired confidence of 50%, but confidence of over 50% was observed in individual videos, indicating it can be achieved by labeling more frames depicting the animal in more diverse scenarios and/or training the algorithm for more iterations.
The cage and camera setup accomplished the design aim of facilitating accurate collection of data. By using the nested cage section approach, the entire cage bottom was in the field-of-view of the camera, eliminating the possibility of the Guinea Pig leaving the camera frame and thus not tracking its movement, which could cause gaps in the data that lead unreliable results.
The camera fixture successfully secured the camera to prevent movement during video recording. Movement of the camera could change the height or angle from the lens to the cage floor. Since distance is measured by converting pixel distance to actual distance using a conversion factor obtained from a test video, the height of the camera lens must be consistent in each video for the conversion factor to remain constant. The camera fixture also is a usability improvement because eliminates the need for tape. The previous method of taping the camera to the cage added significant time to the video collection process, sometimes resulted in useless data if the tape came undone and the camera fell over, and covered up the infrared sensors on the camera, making it impossible to collect video when the lights are off in the room. Using this improved setup, future lab members can collect videos at night and use them to train the tracking model to recognize Guinea Pigs at low light levels, enabling continuous tracking.
There are several ways this system could be improved. To improve the tracking software, DeepLabCut can be run for a greater number of iterations, ideally between 100,000 and 200,000. To do this in a reasonable amount of time (<1 week), a GPU should be configured on the computer on which training is being done, or alternate High-Performance Computing resources should be utilized.
To improve the camera setup, a long-term solution should enable a water bottle to attached to the modified cage. One way to do this could be to drill a large hole in the side of the cage, and the water bottle could be suspended from the outside of the cage lip with the spout sticking inside the cage through the drilled hole. Alternatively, a different camera could be used that has a different focal distance so that it can capture the entire cage bottom while attached to the grill of the cage lid. This would eliminate the need for an insert/camera raising mechanism.
Another important improvement would be the collection of long-term or continuous data. To make this process efficient, videos should be recorded for 24 hours, automatically exported to an external computer, deleted to free up space on the Raspberry Pi for a new video, and a new video would be started automatically. Eventually, the exported videos could be automatically analyzed by DeepLabCut and notify lab personnel when an adverse cardiac event is about to occur, or when an animal is showing worsening mobility due to heart failure.
This project exceeded the project requirements in that it not only tracks gross linear movement, but also more subtle movements such as the head moving or the animal turning in a circle. Additionally, this project did not require any changes to the IACUC protocol, as no new materials were inside the cages and the animals still had access to the entire cage.
The video system and machine learning model developed through this project was shown to successfully track Guinea Pig movement over time and enabled the calculation of distance traveled. Of the four locations on the animal’s body that were tracked, the center of the head yielded the highest tracking confidence, with an average of 43.3% confidence after 50,000 training iterations on approx. labeled frames. This was slightly lower than the desired confidence of 50%, but confidence of over 50% was observed in individual videos, indicating it can be achieved by labeling more frames depicting the animal in more diverse scenarios and/or training the algorithm for more iterations.
The cage and camera setup accomplished the design aim of facilitating accurate collection of data. By using the nested cage section approach, the entire cage bottom was in the field-of-view of the camera, eliminating the possibility of the Guinea Pig leaving the camera frame and thus not tracking its movement, which could cause gaps in the data that lead unreliable results.
The camera fixture successfully secured the camera to prevent movement during video recording. Movement of the camera could change the height or angle from the lens to the cage floor. Since distance is measured by converting pixel distance to actual distance using a conversion factor obtained from a test video, the height of the camera lens must be consistent in each video for the conversion factor to remain constant. The camera fixture also is a usability improvement because eliminates the need for tape. The previous method of taping the camera to the cage added significant time to the video collection process, sometimes resulted in useless data if the tape came undone and the camera fell over, and covered up the infrared sensors on the camera, making it impossible to collect video when the lights are off in the room. Using this improved setup, future lab members can collect videos at night and use them to train the tracking model to recognize Guinea Pigs at low light levels, enabling continuous tracking.
There are several ways this system could be improved. To improve the tracking software, DeepLabCut can be run for a greater number of iterations, ideally between 100,000 and 200,000. To do this in a reasonable amount of time (<1 week), a GPU should be configured on the computer on which training is being done, or alternate High-Performance Computing resources should be utilized.
To improve the camera setup, a long-term solution should enable a water bottle to attached to the modified cage. One way to do this could be to drill a large hole in the side of the cage, and the water bottle could be suspended from the outside of the cage lip with the spout sticking inside the cage through the drilled hole. Alternatively, a different camera could be used that has a different focal distance so that it can capture the entire cage bottom while attached to the grill of the cage lid. This would eliminate the need for an insert/camera raising mechanism.
Another important improvement would be the collection of long-term or continuous data. To make this process efficient, videos should be recorded for 24 hours, automatically exported to an external computer, deleted to free up space on the Raspberry Pi for a new video, and a new video would be started automatically. Eventually, the exported videos could be automatically analyzed by DeepLabCut and notify lab personnel when an adverse cardiac event is about to occur, or when an animal is showing worsening mobility due to heart failure.
This project exceeded the project requirements in that it not only tracks gross linear movement, but also more subtle movements such as the head moving or the animal turning in a circle. Additionally, this project did not require any changes to the IACUC protocol, as no new materials were inside the cages and the animals still had access to the entire cage.
