The best way to improve English listening skills is by listening a looooot to spoken English, which is widely available on the internet. Go to YouTube, or open up your Spotify and listen to podcasts.
English listening skills are an important part of language learning because English speakers, native or non-native, have different accents. If the only accent you know is yours, you will ask the “what” question.
“I am sorry, but what? Could you repeat that? What do you mean?” Those are harmless, and it says more about you than them. But people want to be understood, and they might feel disheartened if you ask.