|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.Tokenizer
org.opensolaris.opengrok.analysis.JFlexTokenizer
org.opensolaris.opengrok.analysis.c.CSymbolTokenizer
public class CSymbolTokenizer
This class is a scanner generated by JFlex 1.4.3 on 2/21/10 8:06 AM from the specification file /builddir/build/BUILD/opengrok-0.8.1-src/src/org/opensolaris/opengrok/analysis/c/CSymbolTokenizer.lex
Field Summary | |
---|---|
static int |
COMMENT
|
static int |
QSTRING
|
static int |
SCOMMENT
|
static int |
STRING
lexical states |
static int |
YYEOF
This character denotes the end of file |
static int |
YYINITIAL
|
Fields inherited from class org.opensolaris.opengrok.analysis.JFlexTokenizer |
---|
reuseToken |
Fields inherited from class org.apache.lucene.analysis.Tokenizer |
---|
input |
Constructor Summary | |
---|---|
CSymbolTokenizer(java.io.InputStream in)
Creates a new scanner. |
|
CSymbolTokenizer(java.io.Reader in)
Creates a new scanner There is also a java.io.InputStream version of this constructor. |
Method Summary | |
---|---|
void |
close()
|
void |
reInit(char[] buf,
int len)
|
void |
yybegin(int newState)
Enters a new lexical state |
char |
yycharat(int pos)
Returns the character at position pos from the matched text. |
void |
yyclose()
Closes the input stream. |
int |
yylength()
Returns the length of the matched text region. |
org.apache.lucene.analysis.Token |
yylex()
Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs. |
void |
yypushback(int number)
Pushes the specified amount of characters back into the input stream. |
void |
yyreset(java.io.Reader reader)
Resets the scanner to read from a new input stream. |
int |
yystate()
Returns the current lexical state. |
java.lang.String |
yytext()
Returns the text matched by the current regular expression. |
Methods inherited from class org.opensolaris.opengrok.analysis.JFlexTokenizer |
---|
next |
Methods inherited from class org.apache.lucene.analysis.Tokenizer |
---|
reset |
Methods inherited from class org.apache.lucene.analysis.TokenStream |
---|
next, reset |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static final int YYEOF
public static final int STRING
public static final int YYINITIAL
public static final int COMMENT
public static final int SCOMMENT
public static final int QSTRING
Constructor Detail |
---|
public CSymbolTokenizer(java.io.Reader in)
in
- the java.io.Reader to read input from.public CSymbolTokenizer(java.io.InputStream in)
in
- the java.io.Inputstream to read input from.Method Detail |
---|
public void close()
close
in class org.apache.lucene.analysis.Tokenizer
public void reInit(char[] buf, int len)
public final void yyclose() throws java.io.IOException
java.io.IOException
public final void yyreset(java.io.Reader reader)
reader
- the new input streampublic final int yystate()
public final void yybegin(int newState)
newState
- the new lexical statepublic final java.lang.String yytext()
public final char yycharat(int pos)
pos
- the position of the character to fetch.
A value from 0 to yylength()-1.
public final int yylength()
public void yypushback(int number)
number
- the number of characters to be read again.
This number must not be greater than yylength()!public org.apache.lucene.analysis.Token yylex() throws java.io.IOException
yylex
in class JFlexTokenizer
java.io.IOException
- if any I/O-Error occurs
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |