Showing posts with label languageprocessing. Show all posts
Showing posts with label languageprocessing. Show all posts

Thursday, 23 July 2026

ngrams, ngrams everywhere

An n-gram is a continuous sequence of n items - words, letters or symbols - from text or speech, used to analyze patterns and predict sequences in language.

Suppose we choose the items of our n-gram as "words". We then have the following taxonomy:

  • unigram - consisting of a single word e.g. design
  • bigram - consisting of two words e.g. design experiments
  • trigram - e.g. design of experiments
Google has a great ngram viewer to see the frequency of certain ngrams in books throughout the ages.

Sunday, 21 June 2026

Cut and Paste Metadata in Word

This metadata can conflict with the document's own metadata - creating inconsistent results for example in spell checking, within the same document.