It is obvious that railway environments are gaining interest due to their social and economical impact. Today, the main challenge towards seamless connectivity over mobile communications on-board railway vehicles stems from tens of decibels of vehicle penetration loss. To cope with it, frequency selective surfaces are quickly gaining ground as an alternative solution to replace railway vehicle's standard windows which only guarantee protection against infrared and ultraviolet rays. In this study, we performed smartphone-based measurements on-board a frequency selective surface-aided high speed train. All in all, we first evaluated the performance of key performance indicators based on small-scale measurements and then applied nonparametric inferential statistics to assess the service quality of users on the move in various vehicle placements.