tokenization using indic NLP library
<p>Hello! I should say नमस्ते since today’s topic is regarding Indian language.</p>
<p><strong>Natural Language Processing</strong> looks fascinating but it’s similar to Machine Learning where we need data cleaning and data pre-processing.</p>
<p>Sounds boring right? But it’s not our mistake…machines never tried to learn human languages . It was us who generously learnt numbers to communicate with them . Jokes apart, when we talk data pre-processing, <strong>Tokenization </strong>is an integral part of this. Basically, we split the text further into units called <strong>tokens </strong>which can be words or characters.</p>
<p><a href="https://mrraghav.medium.com/tokenization-using-indic-nlp-library-257a9a44a272"><strong>Click Here</strong></a></p>