public class TokSpanEventStream extends AbstractEventStream<TokenSample>
TokenSample
s from the given Iterator
and converts the TokenSample
s into Event
s which
can be used by the maxent library for training.Constructor and Description |
---|
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
boolean skipAlphaNumerics)
Initializes the current instance.
|
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
boolean skipAlphaNumerics,
Pattern alphaNumeric,
TokenContextGenerator cg)
Initializes the current instance.
|
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
boolean skipAlphaNumerics,
TokenContextGenerator cg)
Initializes the current instance.
|
Modifier and Type | Method and Description |
---|---|
protected Iterator<Event> |
createEvents(TokenSample tokenSample)
Adds training events to the event stream for each of the specified tokens.
|
hasNext, next
remove
public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, Pattern alphaNumeric, TokenContextGenerator cg)
tokenSamples
- skipAlphaNumerics
- cg
- public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg)
tokenSamples
- skipAlphaNumerics
- cg
- public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics)
tokenSamples
- skipAlphaNumerics
- protected Iterator<Event> createEvents(TokenSample tokenSample)
createEvents
in class AbstractEventStream<TokenSample>
tokens
- character offsets into the specified text.text
- The text of the tokens.Iterator
of training events or
an empty Iterator
.Copyright © 2016 The Apache Software Foundation. All rights reserved.