Post-hoc analysis of Arabic transformer models.

Analyze the internal representations of Arabic transformer models. While there’ve been an extrinsic evaluation of Arabic transformer (AT) models, no work has been carried out to analyze their internal representations. This<a href="https://arxiv.org/abs/2210.09990" rel="noopener ugc nofollow" target="_blank"> paper</a> probe how Arabic linguistic information is encoded in AT models. The authors performed a layer & neuron analysis on the models using morphological tagging tasks for different dialects and a dialectal identification task. The overall idea is to extract feature vectors from the learned representations and train probing classifiers towards understudied auxiliary tasks (of predicting morphology or identifying dialect). Additionally, they used the Linguistic Correlation Analysis method to identify salient neurons with respect to a downstream task. The analysis enlightens interesting findings such as: <a href="https://medium.com/@Mustafa77/post-hoc-analysis-of-arabic-transformer-models-95790745c712">Website</a>