TY - GEN
T1 - Implementing Gesture Recognition in a Sign Language Learning Application
AU - Tan, Daphne
AU - Meehan, Kevin
N1 - Publisher Copyright:
© 2020 IEEE.
PY - 2020/6
Y1 - 2020/6
N2 - Artificial Intelligence (AI) has become increasingly prevalent in contemporary times. It has a wide variety of application areas which can almost replicate tasks that humans would normally perform. Many companies that are using this form of technology are making efficiencies by replacing humans with AI agents. However, researchers are still making efforts to find ways to enhance artificial intelligence to be more 'human-like'. Gesture recognition is a form of human computer interaction in which AI has the potential to improve. Similar to humans, AI has the ability to 'see' and recognise gestures. Sign language is a language that a small proportion of the human population know and use. However, it is slowly gaining popularity and more resources are being provided in order to learn the language. Some people tend to go for in-person classes whereas others tend to go online or use applications to self-learn. This research discovers the success of technology such as gesture recognition to assist in learning sign language. The main research aim is to determine whether gesture recognition can assist self-learners in learning the language. The research has explored the use of Convolutional Neural Networks (CNN) to detect shapes that represent sign language form. The research demonstrated different accuracies based on a small sample size of 10 participants using three different types of datasets: non pre-processed, skin mask, and Sobel filtered images. The CNN model trained with the skin mask dataset was overall the most suitable model in identifying gestures from images; however, the CNN model trained with the non pre-processed dataset was slightly more accurate in recognising the American Sign Language (ASL) gestures in realtime. All CNN models demonstrated accuracy levels above 70% proving that the CNN has the ability to recognise sign language gestures.
AB - Artificial Intelligence (AI) has become increasingly prevalent in contemporary times. It has a wide variety of application areas which can almost replicate tasks that humans would normally perform. Many companies that are using this form of technology are making efficiencies by replacing humans with AI agents. However, researchers are still making efforts to find ways to enhance artificial intelligence to be more 'human-like'. Gesture recognition is a form of human computer interaction in which AI has the potential to improve. Similar to humans, AI has the ability to 'see' and recognise gestures. Sign language is a language that a small proportion of the human population know and use. However, it is slowly gaining popularity and more resources are being provided in order to learn the language. Some people tend to go for in-person classes whereas others tend to go online or use applications to self-learn. This research discovers the success of technology such as gesture recognition to assist in learning sign language. The main research aim is to determine whether gesture recognition can assist self-learners in learning the language. The research has explored the use of Convolutional Neural Networks (CNN) to detect shapes that represent sign language form. The research demonstrated different accuracies based on a small sample size of 10 participants using three different types of datasets: non pre-processed, skin mask, and Sobel filtered images. The CNN model trained with the skin mask dataset was overall the most suitable model in identifying gestures from images; however, the CNN model trained with the non pre-processed dataset was slightly more accurate in recognising the American Sign Language (ASL) gestures in realtime. All CNN models demonstrated accuracy levels above 70% proving that the CNN has the ability to recognise sign language gestures.
KW - Computer Vision
KW - Convolutional Neural Network
KW - Gesture Recognition
KW - Sign Language
UR - http://www.scopus.com/inward/record.url?scp=85092733543&partnerID=8YFLogxK
U2 - 10.1109/ISSC49989.2020.9180197
DO - 10.1109/ISSC49989.2020.9180197
M3 - Conference contribution
AN - SCOPUS:85092733543
T3 - 2020 31st Irish Signals and Systems Conference, ISSC 2020
BT - 2020 31st Irish Signals and Systems Conference, ISSC 2020
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 31st Irish Signals and Systems Conference, ISSC 2020
Y2 - 11 June 2020 through 12 June 2020
ER -