Date of Graduation

7-2021

Document Type

Dissertation

Degree Name

Doctor of Philosophy in Computer Science (PhD)

Degree Level

Graduate

Department

Computer Science & Computer Engineering

Advisor/Mentor

Susan Gauch

Committee Member

Brajendra Panda

Second Committee Member

David Andrews

Third Committee Member

Mark Arnold

Keywords

experts, fair ranking, group formation, algorithmic fairness, demographic profiles, expertise models, equity

Abstract

Expert recommendation is the process of identifying individuals who have the appropriate knowledge and skills to achieve a specific task. It has been widely used in the educational environment mainly in the hiring process, paper-reviewer assignment, and assembling conference program committees. In this research, we highlight the problem of diversity and fair representation of underrepresented groups in expertise recommendation, factors that current expertise recommendation systems rarely consider. We introduce a novel way to model experts in academia by considering demographic attributes in addition to skills. We use the h-index score to quantify skills for a researcher and we identify five demographic features with which to represent a researcher's demographic profile. We highlight the importance of these features and their role in bias within the academic environment.

We utilize these demographic features within an expert recommender system in academia to achieve demographic diversity and increase the exposure of the underrepresented groups using two approaches. In the first approach, we present three different algorithms for scholar recommendation: expertise-based, diversity-based, and a hybrid algorithm that uses a tuning parameter to calibrate the balance between expertise loss and diversity gain. To evaluate the ranking produced by these algorithms, we introduce a modified normalized Discounted Cumulative Gain (nDCG) version that supports multi-dimensional features, and we report diversity gain from each method. Our results show that we can achieve the best possible balance between diversity gain and expertise loss when the tuning parameter value is set around 0.4, giving nearly equal weight to both expertise and diversity.

Finally, we explore diversity from the lens of the demographic parity and develop two algorithms to achieve a representative group that reflects the demographics of the recommendation pool. One is inspired by Hill Climbing, a mathematical optimization technique, wherein a solution is built gradually to the problem, and the other one is inspired by the problem of seat allocation in electoral voting systems. We evaluated these algorithms by comparing them to the hybrid algorithm from the previous approach. Our evaluation shows that both approaches provide a better diversity gain as compared to the hybrid algorithm. However, Hill Climbing Diversity is more effective when it comes to expertise savings with a statistically significant result, making it the preferred algorithm to achieve the goal of promoting diversity while maintaining expertise in an expert recommendation process.

Download

Share

COinS