Class ClassicFilter

All Implemented Interfaces:
Closeable, AutoCloseable, Unwrappable<TokenStream>

public class ClassicFilter extends TokenFilter
Normalizes tokens extracted with ClassicTokenizer.
  • Field Details

    • APOSTROPHE_TYPE

      private static final String APOSTROPHE_TYPE
    • ACRONYM_TYPE

      private static final String ACRONYM_TYPE
    • typeAtt

      private final TypeAttribute typeAtt
    • termAtt

      private final CharTermAttribute termAtt
  • Constructor Details

    • ClassicFilter

      public ClassicFilter(TokenStream in)
      Construct filtering in.
  • Method Details

    • incrementToken

      public final boolean incrementToken() throws IOException
      Returns the next token in the stream, or null at EOS.

      Removes 's from the end of words.

      Removes dots from acronyms.

      Specified by:
      incrementToken in class TokenStream
      Returns:
      false for end of stream; true otherwise
      Throws:
      IOException