net.sourceforge.pmd.cpd

Class AnyTokenizer

public class AnyTokenizer extends Object implements Tokenizer

This class does a best-guess try-anything tokenization.

Author: jheintz

Field Summary
static StringTOKENS
Method Summary
voidtokenize(SourceCode sourceCode, Tokens tokenEntries)

Field Detail

TOKENS

public static final String TOKENS

Method Detail

tokenize

public void tokenize(SourceCode sourceCode, Tokens tokenEntries)