Word/phrase classification processing method and apparatus

DETAILED DESCRIPTION A first object of the present invention is to provide a word/phrase classification processing apparatus and method thereof which can automatically classify word and phrase as one block.
A second object of the present invention is to provide a phrase extraction apparatus which can extract a phrase from a large amount of text data at a high speed.
A third object of the present invention is to provide a speech recognition apparatus which can perform accurate speech recognition using the correspondence or similarity between word and phrase or between phrases.
A fourth object of the present invention is to provide a machine translation apparatus which can perform accurate machine translation using the correspondence or similarity between word and phrase or between phrases.
To attain the above described first object, word and phrase included in text data are classified together to generate a class in which the word and phrase exist together, according to the present invention.
With such a class, not only words, but word and phrase or phrases can be classified as one block, thereby easily identifying the correspondence or similarity between the word and phrase or between the phrases.
Furthermore, according to an embodiment of the present invention, a one-dimensional sequence of word classes is generated by mapping word classes into which words are classified, into a one-dimensional sequence of words included in text data.
Then, a word class sequence in which all of the degrees of stickiness between contiguous word classes are equal to or more than a predetermined value, is extracted from the one-dimensional sequence of word classes of the text data and has a token attached.
After word and token are classified together, a word class sequence corresponding to the token is replaced with a phrase belonging to that word class sequence.
As described above, a token is attached to a word class sequence to regard that sequence as one word.
As a result, equal handling of a word included in text data and a word class sequence with a token attached allows classification processing to be performed without making a distinction between word and phrase