As a natural language processing tool, statistical language modeling is proved to be able to process large-scale real text.

To count the large scale language corpus, to find some linguistic phenomena and to establish statistical model are the common method used in linguistic research and computational linguistic research.

Different estimation methods of the probabilities of sparse events for the computation of the entropy in large scale modern Chinese text are applied in this paper.