Tag: text

How to Chunk Text Data — A Comparative Analysis

Introduction The ‘Text chunking’ process in Natural Language Processing (NLP) involves the conversion of unstructured text data into meaningful units. This seemingly simple task belies the complexity of the various methods employed to achieve it, each with its strengths and weaknesses...

Making a Text-Adventure Game

My little text-adventure reached a certain point, so I’ve decided to write about it. I’ve already talked about the tech side and the reasoning behind making it, so this time I’d like to describe the overall structure of the game. It is currently deployed and ...

Top 3 Advanced Linux Commands For Text Processing

Linux experience is one of the requirements for almost any engineering position. While many developers work not only with Linux but with macOS and Windows, their knowledge of Linux is highly appreciated. Where would they use them? The majority of web servers are running on Linux. Everybody contri...

Linux — How to Find All Files Containing Specific Text

In the fast-paced world of DevOps, efficiently navigating and managing system configurations, code, logs, and more is crucial. One essential skill for every DevOps engineer working with Linux-based systems is the ability to rapidly locate files containing specific text or configuration settings. ...

Did Christianity find a shocking Biblical text—and keep it quiet?

I look around for news of this incredible find. But I find nothing in Vatican News, or Christianity Today, or anywhere—in any religious outlet, or any popular outlet at all. Only academic notices exist—and these are mostly in a non-Christian journal, the Harvard...

Religious Texts can Lose Something With Translation

When we read religious texts today, most of them have been translated into modern languages, so that we can understand them more easily. Most ancient religious texts were written in languages that are considered to be ‘dead’ today. For example, the Bible was predominately written in G...

The Math Behind the Magic — How Language Models Generate Convincing Text?

In the digital age, Language Models (LLMs) are everywhere, shaping the way we interact with technology. From autocomplete suggestions to virtual assistants, these models play a pivotal role in generating human-like text. But how do they do it? Let’s analyze what’s happening behind-the-...

Text Summarization and Keyword Extraction from customer reviews in French (Part 1 / 3)

Online review community such as Trustpilot gives an NLP enthusiast an infinite dataset as a playgrond to perform NLP tasks. Our aim is that we are leveraging these datasets to perform 2 key tasks: Summarization: to have the pre-trained model capture only the most important parts...

[NLP] Basics: Measuring The Linguistic Complexity of Text

Determining the linguistic complexity of text is one of the first basic steps you learn in natural language processing. In this article, I take you through what linguistic complexity is and how to measure it. The pre-processing steps First, you need to proceed with the tokenisation of your cor...