Being often irregular in shape, accurate measurements of wounds are often challenging. Tracing or length measurement by the longest dimension are often inaccurate and time-consuming to carry out. Yet, proper wound area monitoring is essential for wound care. Instead of multiple clinical visits, which also increase the burden on health care systems and the associated expenses, patients can take the responsibility for their own wound progress. To allow for better decisions based on more quantitatively accurate data and address the problems mentioned, the “APD Skin Monitoring” app was created for patient self-monitoring of wounds through the automated area calculation, area graph plotting, and colourimetric analysis functions. With automated features, patients and even clinical staff can better utilize their smartphones for wound care. In this article, we describe the image processing optimization to improve the automated wound detection and demonstrated the use of the app using wound images from a patient volunteer.
Clinical wound measurements are typically performed using the measuring tape or by the wound tracing method amongst other manual methods (Swezey, 2014). Such measurement methods usually involve a high degree of approximations on the area, and the tracing may touch wounds and be a source of contamination. With the disruption of smartphones in many sectors from clinical (Gan, et al., 2016) and biomedical research (Gan & Poon, 2016), alongside many apps using image processing methods to analyze pictures non-invasively such as GelApp, (Sim, et al., 2015) and APD Colony (Wong, et al., 2016), that can be improved by machine learning methods (Nguyen, et al., 2016) or combined with real-life modifications to facilitate better detection e.g. for protein gels for GelApp (Ling, et al., 2015), there is great promise to change the way that wound area monitoring and size tracking can be revolutionized. Using the smartphone, patients can be empowered to take the responsibility for their own health, reducing the burden on the healthcare system and dealing with associated costs.
The patient is ultimately the main benefactor from improved wound management and tracking. And thus, it is our aim to empower the patient to play a bigger role in their own wound management using smartphone apps. Through more involvement, the patient may make a better judgement on their needs whether to make a pre-scheduled visit with the clinician or not at all. With the penetration of smartphone apps to educate (e.g. Wound Education app, AniCare), monitor health conditions e.g. neurological tremors in VibraTilt (Ng, et al., 2016), general habits in Samsung Health, Apple Health apps), clinical treatment monitoring e.g.ThyroidSPOT (Sim, et al., 2017); empowering patients via the use of apps to manage their own wounds is certainly achievable. This article describes the use of existing OpenCV image processing methods and algorithms to build on previous area/volume calculation apps – APD Areametric (Wu, et al., 2019), APD Volumetric (Budianto & Gan, 2017), and StanXY (Budianto, et al., 2015) apps to aid in the area calculation, monitoring and colourimetric analysis of wounds. Targeted for patient’s own use, this app may be utilized in clinical settings in the future. As a proof-of-concept in the difficulty of accessing medical image libraries, we demonstrated APD Skin Monitoring app using the images from a volunteer with a healing wound.
The APD Skin Monitoring app was developed using Android Studio version 3.1 for the Android version and the XCode 9.4 for the iOS version. The OpenCV library was used in both app versions.
Area measurement
Coin Detection
To calculate the area of the wound while balancing both speed and accuracy requires the use of a reference item. For this purpose, coins from the home (Singapore) and major countries were adopted. Despite its large market, China PRC was not included as Google and Apple app stores are not available there at the time of writing.
The wound image must be taken together with a coin for accurate calculations, and this method was adopted from our previous “APD Areametric App” (Wu, et al., 2019). Briefly, the coin detection algorithm from the OpenCV library was further optimized for our use with additional processing involving the Hue, Saturation, and Value (HSV) spectrum for improved detection.
The Hue channel pertains to the colour tone; while the Saturation channels refers to the colour luminosity, and the Value channel pertains to the colour intensity or brightness (Bradley, 2013). HSV was incorporated prior to the coin detection algorithm to optimize the contrast applied for improving coin detection. Within our customized coin detection algorithm, the average saturation and value channel values of image would first be checked against threshold values, before increasing the contrast. In our optimized algorithm, the Hue channel value was eliminated since colour was not a critical factor for contrast, and that improving the detection of the coin was the focus. Thus, only the Saturation and Value channel were processed.
Segmentation
Image segmentation is the grouping of input images into categories based on certain criteria (Mallick, 2018). In this application, the algorithm was to segregate the foreground object (Wound) from the background in the image. Two approaches were considered to achieve the objective.
Wound Detection
Approach 1: GrabCut algorithm
The first approach was the use of the grabCut algorithm in the OpenCV library which is an interactive algorithm based on the Graph cut algorithm. The grabCut algorithm works by first defining a rectangle around the wound, followed by corrective editing when the first cut was inaccurate. This interactive process is highly dependent on the user input in defining the rectangle background region. The grabCut algorithm provided better cropping when the input bounding rectangle enclosed the target fittingly (Yu, et al., 2017) as also shown in Figures 1 and 2. However, drawing tight-fit rectangles increases human input, and iteration cycles had to be increased to ensure higher accuracy of the grabCut algorithm, resulting in long processing times. Thus, for using this algorithm there was a trade-off between accuracy and processing time.
Figure 1: (a) Image with tight fitting rectangle, (b) output
Figure 2: (a) Image with loose rounding rectangle, (b) output
Approach 2: Wound detection based on colour
The second approach involved the use of wound characteristics for colour and contour. The colour detection thresholding was based on the common wound colours such as red from (0, 0, 120) to (100, 100, 255) based on Blue-Green-Red (BGR). Colour detection on the image first separated the possible wound pixels from the background before thresholding, which was used to speed up the detection of the contours for more accurate area calculation.
Figure 3: Image of wound detection (Green outline depicts the detected wound region)
Histogram
The histogram function was built using the OpenCV histogram feature to show the overall distribution of the intensity. The pixels in the image were separated into subparts called bins, with a range of intensity values from 0 - 255. These values were then calculated using the OpenCV histogram feature to generate the graph.
Image Overlay
The image overlay feature utilized the coin detection algorithm in the area calculation feature. The coin acts as a reference for the resizing of the second image to match that of the first. This allows standardization of images taken in different angles or distances that would affect the object size. For this purpose, the coin reference provided the guide to the changes required. This allowed a more accurate comparison, especially with a dragging of the more transparent second image to overlay the first. This thus allows a visual of the changes to the wound in both size and other parameters.
Application on Android and iOS
The APD Skin monitoring app includes area calculation, histogram of wound image, wound image overlay and the wound tracking graph on both Android and iOS.
Area calculation
Figure 4: a) Selecting the coin. b) Coin detection. c) Wound detection. d) Calculated wound area.
The wound area calculation is easily performed by first uploading a frontal plane wound image including the appropriate coin. When correctly detected, wound detection can be done with the same single button click (Figure 4). In cases where the coin or wound detection is inaccurate, manual edit can be performed.
Histogram
Figure 5: Wound image colour histogram
A histogram of the overall colour distribution of the wound image can be plotted with a button click (Figure 5). This is particularly useful to detect for coloured featural changes such as blood, scabs, pus, etc.
Wound image overlay
Figure 6: Image overlay
This image overlay allows the user to visualize the changes of the wound in terms of colour and size easily by overlaying them. For standardization of sizes, the coin in both pictures allow resizing of the second image to that of the first. Users may then drag and move the second image over the first image to overlay (Figure 6). With one image translucent, it is easy for the user to track for wound changes.
Wound growth graph plotting
Since tracking the changes of the wound can help determine if a wound is healing, the area of the wound taken over time could be plotted as a graph. Recognizing it is difficult for patients to recall calculated areas over weeks or months, APD Skin Monitoring app automatically stores data of the calculated wound areas shown through the “Area Calculation” function. The data is only stored on the user devices and the app does require internet usage to ensure privacy of data. Within this function, the user can specify the dates and data points for the graph plot (Figure 7), allowing an efficient and easy plot to show to a medical professional when necessary
Figure 7: Graph of a volunteer’s wound area overtime. Pictures provided by the patient and the dates are shown in Supplementary Material. Informed consent was obtained.
Comparing Figures 8 and 9, the incorporation of HSV improved the coin detection in our proof-of-concept trial using a volunteer’s images.
Coin detection result after optimization
Figure 8: Coin detection with HSV
As shown in Figure 9, the coins were either not correctly detected or that the sizes were incorrect. With the incorporation of HSV, the coin sizes were also more accurately detected (Figure 8). Should the detection be inaccurate due to image quality or other factors, we incorporated the function to manually edit for correcting the auto-detection. This can be performed by dragging the green circle to the right place of the coin. The size of the green circles can also be decreased or increased by pinching in or out by the user.
Coin detection result without optimization
Figure 9: Coin detection without HSV
Comparison of the two approaches of wound detection
The comparison of the two approaches showed that while the grabCut algorithm enabled reasonable wound detection, it was slower and less accurate when compared to the colour thresholding method in our use case. Of the four tested images, the grabCut algorithm had only 2/4 accurate outlining of the wound (Figure 10) while the colour threshold approach had a full score of 4/4 in outlining the wound (Figure 11). With regards to the time taken in getting the result, the colour threshold second approach was more efficient with an average processing time of 1.70 seconds compared to an average of 23.2 seconds for the grabCut, both using the same Redmi6A smartphone. The processing time will differ from devices to device due to phone camera resolutions, processor speeds and presence of background app operations.
Approach 1: GrabCut algorithm
Figure 10: Wound detection from grabCut algorithm
Based on the results shown, we therefore incorporated the colour threshold second approach for the APD Skin Monitoring app. This improved speed was important for the convenience and implementation as a long processing time would inevitably cause the user to lose patience when using the app.
Thus, in addition to the results of faster processing and improved accuracy, we abandoned further development and optimization of the grabCut algorithm, using only the colour threshold method.
Approach 2: Colour and thresholding algorithm
Figure 11: Wound detection result based on colour and threshold
The APD Skin monitoring app is a handy on-the-go tool leveraging on optimized image processing methods and features to empower wound monitoring beyond clinical settings. The app facilitates a better level of wound care as evidenced by the use of our volunteer, and is freely available on both Google and Apple app stores.
This work was partially funded by the “Wound Care in the Tropics” research programme from A*STAR Singapore. We thank Mr Lua WH for assisting the recruitment volunteer.
To avoid conflict of interest, the article was handled by an independent member of the editorial board. The article-processing-charge for this article was also waived for Bioinformatics Institute, A*STAR. There is no commercial conflict as the app is provided free on the app stores.
WWL drafted the manuscript and made the Android app. KYWY made the iOS version. MAJF contributed to the overlay feature. SKEG conceived the idea and supervised all aspects of the manuscript.
Budianto, I. H. & Gan, S. K.-E., 2017. APD volumetric app: android app for the quantification of reagents. Scientific Phone Apps and Mobile Devices, Volume 3:7.
Budianto, I.-H., Wong, C.-F., Nguyen, P.-V. & Gan, S. K.-E., 2015. StanXY: standard curve app for Android. Scientific Phone Apps and Mobile Devices, Volume 1:2.
Gan, S. K.-E., Koshy, C., Nguyen, P.-V. & Haw, Y.-X., 2016. An overview of clinically and healthcare related apps in Google and Apple app stores: connecting patients, drugs, and clinicians. Scientific Phone Apps and Mobile Devices, 2(8).
Gan, S. K.-E. & Poon, J.-K., 2016. The world of biomedical apps: their uses, limitations, and potential. Scientific Phone Apps and Mobile Devices, 2(6).
Ling, W.-L., Lua, W.-H. & Gan, S. K.-E., 2015. Fast reversible single‐step method for enhanced band contrast of polyacrylamide gels for automated detection. Wiley Online Library, 36(9-10).
Mallick, S., 2018. Learn OpenCV. [Online] Available at: https://www.learnopencv.com/image-segmentation/[Accessed 28 Mar 2019].
Ng, K. M., Nguyen, P.-V. & Gan, S. K.-E., 2016. VibraTilt: Accelerometer & Gyroscope measurement app. Scientific Phone Apps and Mobile Devices, Volume 2:5.
Nguyen, P.-V.et al., 2016. Optimal processing for gel electrophoresis images: Applying Monte Carlo Tree Search in GelApp. Wiley Online Library, 37(15-16).
Sim, J.-Z., Nguyen, P.-V., Lee, H.-K. & Gan, S. K.-E., 2015. GelApp: Mobile gel electrophoresis analyser. Nature Methods Application Notes.
Sim, J.-Z.et al., 2017. Thyroid-SPOT for mobile devices: personalised thyroid treatment management app. Scientific Phone Apps and Mobile Devices, Volume 3:4.Swezey, L., 2014. Wound Source. [Online] Available at: https://www.woundsource.com/blog/5-techniques-accurate-wound-measurements[Accessed 28 March 2019].
Wong, C. F., Yeo, J. & Gan, S. K.-E., 2016. APD Colony Counter App: Using Watershed Algorithm for improved colony counting. Nature Methods Applications Notes.
Wu, W. L., Yong, K. Y. W. & Gan, S. K. E., 2019. Application Notes on APD Areametric App: Automated area quantification for both Android and iOS. APD Trove.
Yu, H. et al., 2017. LooseCut: Interactive Image Segmentation with Loosely Bounded Boxes.
Figure S1: Data from volunteer (with informed consent) with pictures taken by clinician/helper for the APD Skin Monitoring app evaluation.
APD SKEG Pte Ltd is the member of
Scientific Phone Apps and Mobile Devices is in the Research collection of
