skmultiflow.rules.
VeryFastDecisionRulesClassifier
Very Fast Decision Rules classifier.
The Very Fast Decision Rules (VFDR) [1] is an incremental rule learning classifier. The learning process of VFDR is similar to that of Hoeffding Tree, but instead of a tree it uses a collection of rules. The core of VFDR is its rules that aim to create a highly interpretable classifier thanks to their nature. Each rule is a conjunction of conditions based on attribute values and the structure for keeping sufficient statistics. The sufficient statistics will determine the class predicted by the rule
IF \(att_1 < 1\) and \(att_2 = 0\) THEN class 0.
The Adaptive Very Fast Decision Rules (AVFDR) is an extension of VFDR capable to adapt to evolving data streams. To adapt with the concept every rule is equipped with a drift detector to monitor it’s performance. If a change is detected then the rule is removed and a new one will be learned.
‘first_hit’ - Uses the votes of the first rule that fires.
‘weighted_max’ - Uses the votes of the rules with highest weight that fires.
‘weighted_sum’ - Uses the weighted sum of votes of all the rules that fire.
‘info_gain’ - Information Gain
‘hellinger’ - Hellinger Distance
‘foil_gain’ - Foil Gain
Notes
When using an ordered set, the rules need to be read in order as each rule contain a hidden conjunction which is the opposite of all the rules before it. Also only one rule can fire which restricts the prediction to first hit.
References
Petr Kosina and João Gama. 2015. Very fast decision rules for classification in data streams. Data Min. Knowl. Discov. 29, 1 (January 2015), 168-202. DOI=http://dx.doi.org/10.1007/s10618-013-0340-z
Examples
>>> from skmultiflow.rules import VeryFastDecisionRulesClassifier >>> from skmultiflow.data import AGRAWALGenerator >>> # Setup the stream >>> stream = AGRAWALGenerator() >>> X, y = stream.next_sample(20000) >>> # Setup the learner >>> learner = VeryFastDecisionRulesClassifier() >>> # Train >>> learner.partial_fit(X, y) >>> # Print rules >>> print(learner.get_model_description()) Rule 0 :Att (2) <= 36.360| class :0 {0: 13867.746092302219} Rule 1 :Att (2) > 60.450| class :0 {0: 16174.452447424192} Rule 2 :Att (2) <= 39.090| class :0 {0: 2374.5506811568403} Rule 3 :Att (2) <= 58.180| class :1 {1: 15082.368305403843} Rule 4 :Att (2) <= 59.090| class :1 {1: 767.0} Default Rule :| class :0 {0: 837.0} >>> # Predict >>> X, y = stream.next_sample(100) >>> learner.predict(X) [0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 1 0 0 0 0 0 1 1 1 1 1 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 1 0 1 0 0 0 1 1 0 1 0 0 0 1 0 0 0 0 0 1 0 1 0 0 1]
Methods
compute_hoeffding_bound(range_val, confidence, n)
compute_hoeffding_bound
Compute the Hoeffding bound, used to decide how many samples are necessary at each node.
first_hit(self, X)
first_hit
Get class votes from the first rule that fires.
fit(self, X, y[, classes, sample_weight])
fit
Fit the model.
get_info(self)
get_info
Collects and returns the information about the configuration of the estimator
get_model_description(self)
get_model_description
Returns the rules of the model
get_model_measurements(self)
get_model_measurements
Collect metrics corresponding to the current status of the model.
get_model_rules(self)
get_model_rules
Get the rules that describe the model
get_params(self[, deep])
get_params
Get parameters for this estimator.
get_votes_for_instance(self, X)
get_votes_for_instance
Get class votes for a single instance.
measure_model_size(self[, unit])
measure_model_size
new_rule(self[, class_distribution, …])
new_rule
Create a new rule.
partial_fit(self, X, y[, classes, sample_weight])
partial_fit
Incrementally trains the model.
predict(self, X)
predict
Predicts the label of the instance(s).
predict_proba(self, X)
predict_proba
Predicts probabilities of all label of the instance(s).
reset(self)
reset
Resets the model to its initial state.
score(self, X, y[, sample_weight])
score
Returns the mean accuracy on the given test data and labels.
set_params(self, **params)
set_params
Set the parameters of this estimator.
weighted_max(self, X)
weighted_max
Get class votes from the rule with highest vote weight.
weighted_sum(self, X)
weighted_sum
Get class votes from the sum of rules that fires.
Attributes
classes
expand_confidence
grace_period
nb_threshold
nominal_attributes
ordered_rules
remove_poor_atts
rule_prediction
tie_threshold
Rule
Rule class
A rule is collection of predicates(conditions) that make up the conjunction (the IF part of the rule). The conditions are in the form of: \(Att_{idx} > value\), \(Att_{idx} <= value\) and \(Att_{idx} = value\).
The rule can also track the class distribution and use a drift detector to track change in concept.
Class observations collected from the instances seen in the rule.
The drift detector used to signal the change in the concept.
class_idx
covers_instance
Check if the rule covers the instance X.
Instance attributes to test on the rule.
True if the rule covers the instance else False.
disable_attribute
Disable an attribute observer.
Attribute index.
drift_detector
get_best_expand_suggestion
Find possible expand candidates.
The criterion used to chose the best expanding suggestion.
if foil gain is used as a criterion class_idx is the class rule tries to learn.
expand candidates.
get_class_votes
Get the votes per class for a given instance.
Instance attributes.
Very Fast Decision Rules.
Class votes for the given instance.
Configuration of the estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
get_rule
Get the rule
Full description of the rule.
get_weight_seen
Calculate the total weight seen by the node.
Total weight seen.
learn_from_instance
Update the rule with the provided instance. The model class distribution of the model and each attribute are updated. The one for the model is used for prediction and the distributions for the attributes are used for learning Gaussian estimators are used to track distribution of numeric attributes and dict with class count for nominal and the model distributions.
Instance attributes for updating the node.
Instance class.
Instance weight.
Very Fast Decision Rule model.
observed_class_distribution
Provides information about the classification of the rule for the drift detector in order to follow it’s performance.
The true label
1 if the prediction is correct else 0
Resets the estimator to its initial state.
restart
Restarts the rule with initial values
The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
<component>__<parameter>
weight_seen_at_last_expand
Retrieve the weight seen at last expand evaluation.
Weight seen at last expand evaluation.
Range value.
Confidence of choosing the correct attribute.
Number of samples.
The Hoeffding bound.
The Hoeffding bound is defined as:
where:
\(\epsilon\): Hoeffding bound.
\(R\): Range of a random variable. For a probability the range is 1, and for an information gain the range is log c, where c is the number of classes.
\(\delta\): Confidence. 1 minus the desired probability of choosing the correct attribute at any given node.
\(n\): Number of samples.
The class distribution of the fired rule.
The features to train the model.
An array-like with the class labels of all samples in X.
Contains all possible/known class labels. Usage varies depending on the learning method.
Samples weight. If not provided, uniform weights are assumed. Usage varies depending on the learning method.
Description of the rules
A string buffer containing the measurements of the model.
The class the rule is describing.
The created rule.
Train samples (instances) are composed of X attributes and their corresponding targets y.
Tasks performed before training:
Verify instance weight. if not provided, uniform weights (1.0) are assumed.
If more than one instance is passed, loop through X and pass instances one at a time.
Update weight seen by model.
Training tasks:
If the rule_set is empty, update the default_rule and if enough statistics are collected try to create rule.
If rules exist in the rule_set, check if they cover the instance. The statistics of the ones that fire are updated using the instance.
If enough statistics are collected if a rule then attempt to expand it.
Classes (targets) for all samples in X.
Contains the class values in the stream. If defined, will be used to define the length of the arrays returned by predict_proba
Instance weight. If not provided, uniform weights are assumed.
Samples for which we want to predict the labels.
Predicted labels for all instances in X.
Predicted the probabilities of all the labels for all instances in X.
self
In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.
Test samples.
True labels for X.
Sample weights.
Mean accuracy of self.predict(X) wrt. y.
the class distribution from the rule with highest weight.
The rules are weighted.
The class distribution from the sum of the fired rules.