Language Model
Here are all the things I know and used in my working experience and study. Language Model(LM) maybe a long history technology or some kind of old technology. It is first introduced in Speech Automatic Recognition(ASR) to make the recognized sentence more fluent and natural. Then people find LM can be used anywhere when producing sentences, such as Machine Translation(MT), Information Retrieval, Optical Character Recognition(OCR). Before deep learning become popular, LM is just used to score sentences and choose the best sentence according to the LM score. After GPT and Bert emerge with the developing of deep learning, LM is used as pre-trained model to be the base of many other models. This kind of technology highly extend the usage of LM.
Three kind of LMs are included here:

  1. Ngram Language Model
  2. Neural Language Model
  3. Pre-trained Language Model. Also there’s the developing history of LM. For each kind of LM, principle,demo code and mostly used open source code will be included.