Text Preprocessing Tool: Clean Text for NLP in One Click

How to use the Text Preprocessing Tool

1. 1. Paste textAdd the sentences or documents you want to clean.
2. 2. Choose stepsPick lowercase, punctuation, numbers, stopwords and stemming.
3. 3. Get clean tokensCopy the result or download it as a text file.

The usual NLP cleaning steps

1. Lowercase so "Python" and "python" count as one word.
2. Remove punctuation and, if they do not matter, numbers.
3. Remove stopwords, very common words like the, are and to that carry little meaning.
4. Stem words to a common root so running and runs both become run.

Worked example: "The students are running quickly to their classes!" becomes 8 lowercase words, loses 4 stopwords (the, are, to, their), and the Porter stemmer turns the rest into student run quickli class.

Stemming, lemmatisation and when not to clean

The Porter stemmer (Martin Porter, 1980) removes suffixes using rules, so it is fast but produces stems like "quickli" and "happi" that are not real words. This implementation follows the original algorithm and was checked against NLTK's Porter stemmer in original mode. Lemmatisation instead maps words to dictionary forms (better → good) but needs a vocabulary and part-of-speech tags, so it is not done here.

• Keep stopwords for sentiment ("not good" needs not) and for modern transformer models, which learn from full sentences.
• Keep numbers when they matter, such as prices or versions.
• Clean the same way for training and new data.

The stopword list has about 170 English words, similar to NLTK's. Everything runs in your browser. Next, score the clean words with the TF-IDF calculator or count phrases with the n-gram generator.

Frequently asked questions

More free tools

• TF-IDF CalculatorCalculate TF-IDF scores for words across your documents.
• N-gram GeneratorGenerate word or character n-grams (bigrams, trigrams and
• Sentiment Analyzer (Lexicon Based)Check if English text is positive, negative or neutral with
• LLM Token CounterEstimate how many tokens your prompt uses, check it against
• Cosine Similarity CalculatorCalculate cosine similarity between two vectors or two
• Word CounterCount words, characters, sentences and reading time.

See all free tools