Exploring the Potential of Large Language Models (LLMs) in Analyzing Passengers' Perceptions of Transit Service Quality

Keywords: Transit Service quality; Metro system; Large Language Models; Natural language processing; Social Media
Abstract Type: Paper Abstract

Abstract

Public transit systems are essential to urban mobility, affecting millions of daily commuters. To develop a more responsive, equitable, and efficient public transportation system, it is crucial for transportation planners and policymakers to gain a comprehensive understanding of the diverse travel experiences of transit users. Social media platforms offer valuable, continuous feedback, enabling transit providers to identify issues, make real-time adjustments, and plan long-term improvements. Recently, large language models (LLMs) have attracted significant attention in the urban planning field due to their exceptional performance in natural language processing (NLP) tasks. Using a Weibo dataset related to the Shenzhen metro system (2018-2019) in China, this study developed a two-stage analysis framework to evaluate LLMs' capabilities in transit service management acting as customer experience analyst and transport planner respectively. In particular, we employed LLMs including GPT-3.5 and GPT-4o, utilizing zero-shot, few-shot, and chain-of-thought prompting techniques. Our findings demonstrate that LLMs consistently excel in classification task and policy recommendation task when benchmarked against the traditional Bag of Words (BOW) model. This research underscores the potential of LLMs in enhancing transit service quality assessment and emphasizes the importance of domain expert rationale in designing prompts and interpreting results. Our study offers valuable insights for transportation planners aiming to leverage advanced NLP techniques for more responsive and data-driven service improvements.