Abstract
This study uses Natural Language Processing (NLP) to explore the sentiment and prevalent topics within the PGR (Postgraduate Student Survey) StudentSurvey.ie, analysing data from 2018, 2019 and 2021.
The study focuses on two free response questions to extract insights into what experiences postgraduate students find most valuable about their research programme (Q10) and what can be improved (Q11). The study uses sentiment analysis to examine how different aspects of the student experience relate to the overall positivity reported by postgraduate students.
Innovative Methodology
This report demonstrates the application of Natural Language Processing (NLP) techniques in analysing a large qualitative dataset. Sentiment scoring, which involves determining the emotional tone of each response, was conducted using Microsoft Azure, and statistical analysis was performed to explore the relationship between sentiment scores and characteristic factors. Latent Dirichlet Allocation (LDA), a topic modelling technique, was utilized to uncover key topics in the entire dataset and specific subgroups of respondents. Finally, the report examined specific responses, highlighting the student voice, emotional tone, and perspective by selecting and presenting
respondent’s illustrative quotes.
This methodology has allowed us to broaden the evidence base for aspects known to be important to the postgraduate research experience. Beyond this, we have also identified novel areas of importance, by leveraging topic modelling to reveal latent thematic similarities and patterns. This can be achieved even when different language or terminology is used to describe related concepts. To facilitate transparency and reproducibility, the data has been visualized with an interactive dashboard (see Appendix 1). A comprehensive step-by-step guide illustrates on how to implement this novel approach using both open source and black box solutions (see Appendix 2). The integration of NLP and machine learning offers additional capability and functionality to researchers to expand sample sizes, reduce bias, shorten project time frames, enhance data exploration, and strengthen the rigor of education research.
Insights and Key Findings
The findings relate to the two guiding questions, the sentiments expressed in response are predominantly neutral or positive with only 11% of responses classified as negative for Q10, and 26% negative for Q11.
What aspects/ elements of your research degree programme are most valuable?
The crucial importance of the supervisor's role in the postgraduate research experience is substantiated by an extended body of evidence. As well as being the dominant topic in the respondent’s data, sentiment analysis captured
that the relationship has effects across most aspects of the student experience. The study also shows that a good supervisor – student relationship affects the positive sentiment of the respondents most valuable experience, even for those who did not mention the relationship directly. Institutional acknowledgment of the significance of this relationship would be demonstrated by supporting and resourcing environments(time and space) that cultivate these relationships. The respondents emphasize the critical value of skill development, both in formal research-specific training and transferrable team and communication skills development. Positive sentiment scores for the most valuable experiences are correlated with the evidence of opportunities to develop these skills. In terms of tangible demonstration of the acquisition of transferable and subject specific skills, the publication of papers and presentation at conferences is strongly linked to a positive most valuable experience. Respondents who have positive working relationships, clarity on university processes, coupled with sufficient funding and good life balance report more positive best experience
The study focuses on two free response questions to extract insights into what experiences postgraduate students find most valuable about their research programme (Q10) and what can be improved (Q11). The study uses sentiment analysis to examine how different aspects of the student experience relate to the overall positivity reported by postgraduate students.
Innovative Methodology
This report demonstrates the application of Natural Language Processing (NLP) techniques in analysing a large qualitative dataset. Sentiment scoring, which involves determining the emotional tone of each response, was conducted using Microsoft Azure, and statistical analysis was performed to explore the relationship between sentiment scores and characteristic factors. Latent Dirichlet Allocation (LDA), a topic modelling technique, was utilized to uncover key topics in the entire dataset and specific subgroups of respondents. Finally, the report examined specific responses, highlighting the student voice, emotional tone, and perspective by selecting and presenting
respondent’s illustrative quotes.
This methodology has allowed us to broaden the evidence base for aspects known to be important to the postgraduate research experience. Beyond this, we have also identified novel areas of importance, by leveraging topic modelling to reveal latent thematic similarities and patterns. This can be achieved even when different language or terminology is used to describe related concepts. To facilitate transparency and reproducibility, the data has been visualized with an interactive dashboard (see Appendix 1). A comprehensive step-by-step guide illustrates on how to implement this novel approach using both open source and black box solutions (see Appendix 2). The integration of NLP and machine learning offers additional capability and functionality to researchers to expand sample sizes, reduce bias, shorten project time frames, enhance data exploration, and strengthen the rigor of education research.
Insights and Key Findings
The findings relate to the two guiding questions, the sentiments expressed in response are predominantly neutral or positive with only 11% of responses classified as negative for Q10, and 26% negative for Q11.
What aspects/ elements of your research degree programme are most valuable?
The crucial importance of the supervisor's role in the postgraduate research experience is substantiated by an extended body of evidence. As well as being the dominant topic in the respondent’s data, sentiment analysis captured
that the relationship has effects across most aspects of the student experience. The study also shows that a good supervisor – student relationship affects the positive sentiment of the respondents most valuable experience, even for those who did not mention the relationship directly. Institutional acknowledgment of the significance of this relationship would be demonstrated by supporting and resourcing environments(time and space) that cultivate these relationships. The respondents emphasize the critical value of skill development, both in formal research-specific training and transferrable team and communication skills development. Positive sentiment scores for the most valuable experiences are correlated with the evidence of opportunities to develop these skills. In terms of tangible demonstration of the acquisition of transferable and subject specific skills, the publication of papers and presentation at conferences is strongly linked to a positive most valuable experience. Respondents who have positive working relationships, clarity on university processes, coupled with sufficient funding and good life balance report more positive best experience
| Original language | English (Ireland) |
|---|---|
| Number of pages | 57 |
| Publication status | Published - 31 May 2024 |