public class ThresholdSelector extends RandomizableSingleClassifierEnhancer implements OptionHandler, Drawable
-C <integer> The class for which threshold is determined. Valid values are: 1, 2 (for first and second classes, respectively), 3 (for whichever class is least frequent), and 4 (for whichever class value is most frequent), and 5 (for the first class named any of "yes","pos(itive)" "1", or method 3 if no matches). (default 5).
-X <number of folds> Number of folds used for cross validation. If just a hold-out set is used, this determines the size of the hold-out set (default 3).
-R <integer> Sets whether confidence range correction is applied. This can be used to ensure the confidences range from 0 to 1. Use 0 for no range correction, 1 for correction based on the min/max values seen during threshold selection (default 0).
-E <integer> Sets the evaluation mode. Use 0 for evaluation using cross-validation, 1 for evaluation using hold-out set, and 2 for evaluation on the training data (default 1).
-M [FMEASURE|ACCURACY|TRUE_POS|TRUE_NEG|TP_RATE|PRECISION|RECALL] Measure used for evaluation (default is FMEASURE).
-manual <real> Set a manual threshold to use. This option overrides automatic selection and options pertaining to automatic selection will be ignored. (default -1, i.e. do not use a manual threshold).
-S <num> Random number seed. (default 1)
-D If set, classifier is run in debug mode and may output additional info to the console
-W Full name of base classifier. (default: weka.classifiers.functions.Logistic)
Options specific to classifier weka.classifiers.functions.Logistic:
-D Turn on debugging output.
-R <ridge> Set the ridge in the log-likelihood.
-M <number> Set the maximum number of iterations (default -1, until convergence).Options after -- are passed to the designated sub-classifier.
Modifier and Type | Field and Description |
---|---|
static int |
ACCURACY
accuracy
|
static int |
EVAL_CROSS_VALIDATION
n-fold cross-validation
|
static int |
EVAL_TRAINING_SET
entire training set
|
static int |
EVAL_TUNED_SPLIT
single tuned fold
|
static int |
FMEASURE
F-measure
|
protected double |
m_BestThreshold
The threshold that lead to the best performance
|
protected double |
m_BestValue
The best value that has been observed
|
protected int |
m_ClassMode
Method to determine which class to optimize for
|
protected int |
m_DesignatedClass
Designated class value, determined during building
|
protected int |
m_EvalMode
The evaluation mode
|
protected double |
m_HighThreshold
The upper threshold used as the basis of correction
|
protected double |
m_LowThreshold
The lower threshold used as the basis of correction
|
protected boolean |
m_manualThreshold
True if a manually set threshold is being used
|
protected double |
m_manualThresholdValue
-1 = not used by default
|
protected int |
m_NumXValFolds
The number of folds used in cross-validation
|
protected int |
m_RangeMode
The range correction mode
|
protected static double |
MIN_VALUE
The minimum value for the criterion.
|
static int |
OPTIMIZE_0
first class value
|
static int |
OPTIMIZE_1
second class value
|
static int |
OPTIMIZE_LFREQ
least frequent class value
|
static int |
OPTIMIZE_MFREQ
most frequent class value
|
static int |
OPTIMIZE_POS_NAME
class value name, either 'yes' or 'pos(itive)'
|
static int |
PRECISION
precision
|
static int |
RANGE_BOUNDS
Correct based on min/max observed
|
static int |
RANGE_NONE
no range correction
|
static int |
RECALL
recall
|
static Tag[] |
TAGS_EVAL
The evaluation modes
|
static Tag[] |
TAGS_MEASURE
the measure to use
|
static Tag[] |
TAGS_OPTIMIZE
How to determine which class value to optimize for
|
static Tag[] |
TAGS_RANGE
Type of correction applied to threshold range
|
static int |
TP_RATE
true-positive rate
|
static int |
TRUE_NEG
true-negative
|
static int |
TRUE_POS
true-positive
|
m_Seed
m_Classifier
m_Debug
BayesNet, Newick, NOT_DRAWABLE, TREE
Constructor and Description |
---|
ThresholdSelector()
Constructor.
|
Modifier and Type | Method and Description |
---|---|
void |
buildClassifier(Instances instances)
Generates the classifier.
|
protected String |
defaultClassifierString()
String describing default classifier.
|
String |
designatedClassTipText() |
double[] |
distributionForInstance(Instance instance)
Calculates the class membership probabilities for the given test instance.
|
String |
evaluationModeTipText() |
protected void |
findThreshold(FastVector predictions)
Finds the best threshold, this implementation searches for the
highest FMeasure.
|
Capabilities |
getCapabilities()
Returns default capabilities of the classifier.
|
SelectedTag |
getDesignatedClass()
Gets the method to determine which class value to optimize.
|
SelectedTag |
getEvaluationMode()
Gets the evaluation mode used.
|
double |
getManualThresholdValue()
Returns the value of the manual threshold.
|
SelectedTag |
getMeasure()
get measure used for determining threshold
|
int |
getNumXValFolds()
Get the number of folds used for cross-validation.
|
String[] |
getOptions()
Gets the current settings of the Classifier.
|
protected FastVector |
getPredictions(Instances instances,
int mode,
int numFolds)
Collects the classifier predictions using the specified evaluation method.
|
SelectedTag |
getRangeCorrection()
Gets the confidence range correction mode used.
|
String |
getRevision()
Returns the revision string.
|
String |
globalInfo() |
String |
graph()
Returns graph describing the classifier (if possible).
|
int |
graphType()
Returns the type of graph this classifier
represents.
|
Enumeration |
listOptions()
Returns an enumeration describing the available options.
|
static void |
main(String[] argv)
Main method for testing this class.
|
String |
manualThresholdValueTipText() |
String |
measureTipText()
Tooltip for this property.
|
String |
numXValFoldsTipText() |
String |
rangeCorrectionTipText() |
void |
setDesignatedClass(SelectedTag newMethod)
Sets the method to determine which class value to optimize.
|
void |
setEvaluationMode(SelectedTag newMethod)
Sets the evaluation mode used.
|
void |
setManualThresholdValue(double threshold)
Sets the value for a manual threshold.
|
void |
setMeasure(SelectedTag newMeasure)
set measure used for determining threshold
|
void |
setNumXValFolds(int newNumFolds)
Set the number of folds used for cross-validation.
|
void |
setOptions(String[] options)
Parses a given list of options.
|
void |
setRangeCorrection(SelectedTag newMethod)
Sets the confidence range correction mode used.
|
String |
toString()
Returns description of the cross-validated classifier.
|
getSeed, seedTipText, setSeed
classifierTipText, getClassifier, getClassifierSpec, setClassifier
classifyInstance, debugTipText, forName, getDebug, makeCopies, makeCopy, runClassifier, setDebug
public static final int RANGE_NONE
public static final int RANGE_BOUNDS
public static final Tag[] TAGS_RANGE
public static final int EVAL_TRAINING_SET
public static final int EVAL_TUNED_SPLIT
public static final int EVAL_CROSS_VALIDATION
public static final Tag[] TAGS_EVAL
public static final int OPTIMIZE_0
public static final int OPTIMIZE_1
public static final int OPTIMIZE_LFREQ
public static final int OPTIMIZE_MFREQ
public static final int OPTIMIZE_POS_NAME
public static final Tag[] TAGS_OPTIMIZE
public static final int FMEASURE
public static final int ACCURACY
public static final int TRUE_POS
public static final int TRUE_NEG
public static final int TP_RATE
public static final int PRECISION
public static final int RECALL
public static final Tag[] TAGS_MEASURE
protected double m_HighThreshold
protected double m_LowThreshold
protected double m_BestThreshold
protected double m_BestValue
protected int m_NumXValFolds
protected int m_DesignatedClass
protected int m_ClassMode
protected int m_EvalMode
protected int m_RangeMode
protected boolean m_manualThreshold
protected double m_manualThresholdValue
protected static final double MIN_VALUE
protected String defaultClassifierString()
defaultClassifierString
in class SingleClassifierEnhancer
protected FastVector getPredictions(Instances instances, int mode, int numFolds) throws Exception
instances
- the set of Instances
to generate
predictions for.mode
- the evaluation mode.numFolds
- the number of folds to use if not evaluating on the
full training set.FastVector
containing the predictions.Exception
- if an error occurs generating the predictions.public String measureTipText()
public void setMeasure(SelectedTag newMeasure)
newMeasure
- Tag representing measure to be usedpublic SelectedTag getMeasure()
protected void findThreshold(FastVector predictions)
predictions
- a FastVector
containing the predictions.public Enumeration listOptions()
listOptions
in interface OptionHandler
listOptions
in class RandomizableSingleClassifierEnhancer
public void setOptions(String[] options) throws Exception
-C <integer> The class for which threshold is determined. Valid values are: 1, 2 (for first and second classes, respectively), 3 (for whichever class is least frequent), and 4 (for whichever class value is most frequent), and 5 (for the first class named any of "yes","pos(itive)" "1", or method 3 if no matches). (default 5).
-X <number of folds> Number of folds used for cross validation. If just a hold-out set is used, this determines the size of the hold-out set (default 3).
-R <integer> Sets whether confidence range correction is applied. This can be used to ensure the confidences range from 0 to 1. Use 0 for no range correction, 1 for correction based on the min/max values seen during threshold selection (default 0).
-E <integer> Sets the evaluation mode. Use 0 for evaluation using cross-validation, 1 for evaluation using hold-out set, and 2 for evaluation on the training data (default 1).
-M [FMEASURE|ACCURACY|TRUE_POS|TRUE_NEG|TP_RATE|PRECISION|RECALL] Measure used for evaluation (default is FMEASURE).
-manual <real> Set a manual threshold to use. This option overrides automatic selection and options pertaining to automatic selection will be ignored. (default -1, i.e. do not use a manual threshold).
-S <num> Random number seed. (default 1)
-D If set, classifier is run in debug mode and may output additional info to the console
-W Full name of base classifier. (default: weka.classifiers.functions.Logistic)
Options specific to classifier weka.classifiers.functions.Logistic:
-D Turn on debugging output.
-R <ridge> Set the ridge in the log-likelihood.
-M <number> Set the maximum number of iterations (default -1, until convergence).Options after -- are passed to the designated sub-classifier.
setOptions
in interface OptionHandler
setOptions
in class RandomizableSingleClassifierEnhancer
options
- the list of options as an array of stringsException
- if an option is not supportedpublic String[] getOptions()
getOptions
in interface OptionHandler
getOptions
in class RandomizableSingleClassifierEnhancer
public Capabilities getCapabilities()
getCapabilities
in interface CapabilitiesHandler
getCapabilities
in class SingleClassifierEnhancer
Capabilities
public void buildClassifier(Instances instances) throws Exception
buildClassifier
in class Classifier
instances
- set of instances serving as training dataException
- if the classifier has not been generated successfullypublic double[] distributionForInstance(Instance instance) throws Exception
distributionForInstance
in class Classifier
instance
- the instance to be classifiedException
- if instance could not be classified
successfullypublic String globalInfo()
public String designatedClassTipText()
public SelectedTag getDesignatedClass()
public void setDesignatedClass(SelectedTag newMethod)
newMethod
- the new class selection mode.public String evaluationModeTipText()
public void setEvaluationMode(SelectedTag newMethod)
newMethod
- the new evaluation mode.public SelectedTag getEvaluationMode()
public String rangeCorrectionTipText()
public void setRangeCorrection(SelectedTag newMethod)
newMethod
- the new correciton mode.public SelectedTag getRangeCorrection()
public String numXValFoldsTipText()
public int getNumXValFolds()
public void setNumXValFolds(int newNumFolds)
newNumFolds
- the number of folds used for cross-validation.public int graphType()
public String manualThresholdValueTipText()
public void setManualThresholdValue(double threshold) throws Exception
threshold
- the manual threshold to useException
public double getManualThresholdValue()
public String toString()
public String getRevision()
getRevision
in interface RevisionHandler
getRevision
in class Classifier
public static void main(String[] argv)
argv
- the optionsCopyright © 2015 University of Waikato, Hamilton, NZ. All rights reserved.