.. automodule:: Orange.statistics.estimate
.. index:: Probability Estimation
Probability estimators compute probabilities of values of class variable. They come in two flavours:
- for unconditional probabilities (p(C=c), where c is a class) and
- for conditional probabilities (p(C=c|V=v), where v is a feature value).
A duality much like the one between learners and classifiers exists between probability estimator constructors and probability estimators: when a probability estimator constructor is called with data, it constructs a probability estimator that can then be called with a value of class variable to obtain a probability of that value. This duality is mainly needed to enable probability estimation for continuous variables, where it is not possible to generate a list of probabilities of all possible values in advance.
First, probability estimation constructors for common probability estimation techniques are enumerated. Base classes, knowledge of which is needed to develop new techniques, are described later in this document.
Bases: :class:`EstimatorConstructor`
Compute distribution using relative frequencies of classes.
| rtype: | :class:`EstimatorFromDistribution` |
|---|
Bases: :class:`EstimatorConstructor`
Use Laplace estimation to compute distribution from frequencies of classes:
p(c) = \\frac{Nc+1}{N+n}
where Nc is number of occurrences of an event (e.g. number of instances in class c), N is the total number of events (instances) and n is the number of different events (classes).
| rtype: | :class:`EstimatorFromDistribution` |
|---|
Bases: :class:`EstimatorConstructor`
.. method:: __init__(m)
:param m: Parameter for m-estimation.
:type m: int
Use m-estimation to compute distribution from frequencies of classes:
p(c) = \\frac{Nc+m*ap(c)}{N+m}
where Nc is number of occurrences of an event (e.g. number of instances in class c), N is the total number of events (instances) and ap(c) is the prior probability of event (class) c.
| rtype: | :class:`EstimatorFromDistribution` |
|---|
Bases: :class:`EstimatorConstructor`
.. method:: __init__(min_impact, smoothing, n_points)
:param min_impact: A requested minimal weight of a point (default:
0.01); points with lower weights won't be taken into account.
:type min_impact: float
:param smoothing: Smoothing factor (default: 1.144).
:type smoothing: float
:param n_points: Number of points for the interpolating curve. If
negative, say -3 (default), 3 points will be inserted between each
data points.
:type n_points: int
Compute probabilities for continuous variable for certain number of points using Gaussian kernels. The resulting point-wise continuous distribution is stored as :class:`~Orange.statistics.distribution.Continuous`.
Probabilities are always computed at all points that are present in the data (i.e. the existing values of the continuous feature). If :obj:`n_points` is positive and greater than the number of existing data points, additional points are inserted between the existing points to achieve the required number of points. Approximately equal number of new points is inserted between each adjacent existing point each data points. If :obj:`n_points` is negative, its absolute value determines the number of points to be added between each two data points.
| rtype: | :class:`EstimatorFromDistribution` |
|---|
Bases: :class:`EstimatorConstructor`
.. method:: __init__(window_proportion, n_points)
:param window_proportion: A proportion of points in a window.
:type window_proportion: float
:param n_points: Number of points for the interpolating curve. If
negative, say -3 (default), 3 points will be inserted between each
data points.
:type n_points: int
Prepare a probability estimator that computes probability at point x
as weighted local regression of probabilities for points in the window
around this point.
The window contains a prescribed proportion of original data points. The
window is as symmetric as possible in the sense that the leftmost point in
the window is approximately as far from x as the rightmost. The
number of points to the left of x might thus differ from the number
of points to the right.
Points are weighted by bi-cubic weight function; a weight of point
at x' is (1-|t|^3)^3, where t is
(x-x'>)/h and h is the distance to the farther
of the two window edge points.
| rtype: | :class:`EstimatorFromDistribution` |
|---|
Bases: :class:`ConditionalEstimatorConstructor`
.. method:: __init__(window_proportion, n_points)
:param window_proportion: A proportion of points in a window.
:type window_proportion: float
:param n_points: Number of points for the interpolating curve. If
negative, say -3 (default), 3 points will be inserted between each
data points.
:type n_points: int
Construct a conditional probability estimator, in other aspects similar to the one constructed by :class:`Loess`.
| rtype: | :class:`ConditionalEstimatorFromDistribution`. |
|---|
All probability estimators are derived from two base classes: one for unconditional and the other for conditional probability estimation. The same is true for probability estimator constructors.
Constructor of an unconditional probability estimator.
.. method:: __call__([distribution[, prior]], [instances[, weight_id]])
:param distribution: input distribution.
:type distribution: :class:`~Orange.statistics.distribution.Distribution`
:param priori: prior distribution.
:type priori: :class:`~Orange.statistics.distribution.Distribution`
:param instances: input data.
:type instances: :class:`Orange.data.Table`
:param weight_id: ID of the weight attribute.
:type weight_id: int
If distribution is given, it can be followed by prior class
distribution. Similarly, instances can be followed by with
the ID of meta attribute with instance weights. (Hint: to pass a
prior distribution and instances, but no distribution,
just pass :obj:`None` for the latter.) When both,
distribution and instances are given, it is up to constructor to
decide what to use.
.. note:: The `instances` and `weight_id` argument are at the moment
only used by :class:`ConditionalByRows`. The rest of the builtin
constructors require that `distribution` is given.
.. attribute:: supports_discrete
Tells whether the estimator can handle discrete attributes.
.. attribute:: supports_continuous
Tells whether the estimator can handle continuous attributes.
.. method:: __call__([value])
If value is given, return the probability of the value.
:rtype: float
If the value is omitted, an attempt is made
to return a distribution of probabilities for all values.
:rtype: :class:`~Orange.statistics.distribution.Distribution`
(usually :class:`~Orange.statistics.distribution.Discrete` for
discrete and :class:`~Orange.statistics.distribution.Continuous`
for continuous) or :obj:`NoneType`
Constructor of a conditional probability estimator.
.. method:: __call__([table[, prior]], [instances[, weight_id]])
:param table: input distribution.
:type table: :class:`Orange.statistics.contingency.Table`
:param prior: prior distribution.
:type prior: :class:`~Orange.statistics.distribution.Distribution`
:param instances: input data.
:type instances: :class:`Orange.data.Table`
:param weight_id: ID of the weight attribute.
:type weight_id: int
If distribution is given, it can be followed by prior class
distribution. Similarly, instances can be followed by with
the ID of meta attribute with instance weights. (Hint: to pass a
prior distribution and instances, but no distribution,
just pass :obj:`None` for the latter.) When both,
distribution and instances are given, it is up to constructor to
decide what to use.
.. note:: The `instances` and `weight_id` argument are at the moment
only used by :class:`ConditionalByRows`. The rest of the builtin
constructors require that `table` is given.
As a counterpart of :class:`Estimator`, this estimator can return conditional probabilities.
.. method:: __call__([[value,] condition_value])
When given two values, it returns a probability of :math:`p(value|condition)`.
:rtype: float
When given only one value, it is interpreted as condition; the estimator
attempts to return a distribution of conditional probabilities for all
values.
:rtype: :class:`~Orange.statistics.distribution.Distribution`
(usually :class:`~Orange.statistics.distribution.Discrete` for
discrete and :class:`~Orange.statistics.distribution.Continuous`
for continuous) or :obj:`NoneType`
When called without arguments, it returns a
matrix containing probabilities :math:`p(value|condition)` for each
possible :math:`value` and :math:`condition` (a contingency table);
condition is used as outer
variable.
:rtype: :class:`Orange.statistics.contingency.Table` or :obj:`NoneType`
If estimator cannot return precomputed distributions and/or
contingencies, it returns :obj:`None`.
Bases: :class:`Estimator`
Probability estimator constructors that compute probabilities for all values in advance return this estimator with calculated quantities in the :obj:`probabilities` attribute.
.. attribute:: probabilities
A precomputed list of probabilities.
.. method:: __call__([value])
If value is given, return the probability of the value. For discrete
variables, every value has an entry in the :obj:`probabilities`
attribute. For continuous variables, a linear interpolation between
two nearest points is used to compute the probability.
:rtype: float
If the value is omitted, a copy of :obj:`probabilities` is returned.
:rtype: :class:`~Orange.statistics.distribution.Distribution`
(usually :class:`~Orange.statistics.distribution.Discrete` for
discrete and :class:`~Orange.statistics.distribution.Continuous`
for continuous).
Bases: :class:`ConditionalEstimator`
Probability estimator constructors that compute the whole contingency table (:class:`Orange.statistics.contingency.Table`) of conditional probabilities in advance return this estimator with the table in the :obj:`probabilities` attribute.
.. attribute:: probabilities
A precomputed contingency table.
.. method:: __call__([[value,] condition_value])
For detailed description of handling of different combinations of
parameters, see the inherited :obj:`ConditionalEstimator.__call__`.
For behaviour with continuous variable distributions,
see the unconditional counterpart :obj:`EstimatorFromDistribution.__call__`.
Bases: :class:`ConditionalEstimatorConstructor`
.. attribute:: estimator_constructor
An unconditional probability estimator constructor.
Computes a conditional probability estimator using an unconditional probability estimator constructor. The result can be of type :class:`ConditionalEstimatorFromDistribution` or :class:`ConditionalEstimatorByRows`, depending on the type of constructor.
.. method:: __call__([table[, prior]], [instances[, weight_id]], estimator)
:param table: input distribution.
:type table: :class:`Orange.statistics.contingency.Table`
:param prior: prior distribution.
:type prior: :class:`~Orange.statistics.distribution.Distribution`
:param instances: input data.
:type instances: :class:`Orange.data.Table`
:param weight_id: ID of the weight attribute.
:type weight_id: int
:param estimator: unconditional probability estimator constructor.
:type estimator: :class:`EstimatorConstructor`
Compute contingency matrix if it has not been computed already. Then
call :obj:`estimator_constructor` for each value of condition attribute.
If all constructed estimators can return distribution of probabilities
for all classes (usually either all or none can), the
:class:`~Orange.statistics.distribution.Distribution` instances are put
in a contingency table
and :class:`ConditionalEstimatorFromDistribution`
is constructed and returned. If constructed estimators are
not capable of returning distribution of probabilities,
a :class:`ConditionalEstimatorByRows` is constructed and the
estimators are stored in its :obj:`estimator_list`.
:rtype: :class:`ConditionalEstimatorFromDistribution` or :class:`ConditionalEstimatorByRows`
Bases: :class:`ConditionalEstimator`
A conditional probability estimator constructors that itself uses a series of estimators, one for each possible condition, stored in its :obj:`estimator_list` attribute.
.. attribute:: estimator_list
A list of estimators; one for each value of :obj:`condition`.
.. method:: __call__([[value,] condition_value])
Uses estimators from :obj:`estimator_list`,
depending on given `condition_value`.
For detailed description of handling of different combinations of
parameters, see the inherited :obj:`ConditionalEstimator.__call__`.
>>> import Orange >>> iris = Orange.data.Table("iris") >>> >>> # discrete class distribution >>> iris_dist = Orange.statistics.distribution.Distribution("iris", iris) >>> # m estimate constructor >>> mest_constructor = Orange.statistics.estimate.M(m=10) >>> >>> # create the estimator >>> mest = mest_constructor(iris_dist) >>> print "%.2f" % mest(iris[0]['iris']) 0.33 >>> # petal length (continuous) distribution >>> plength_dist = Orange.statistics.distribution.Distribution("petal length", iris) >>> plength_dist.normalize() >>> >>> # loess contructor >>> loess_est_constructor = Orange.statistics.estimate.Loess() >>> >>> # create the loess estimator >>> loess_est = loess_est_constructor(plength_dist) >>> >>> print "%.2f" % loess_est(iris[0]['petal length']) 0.04 >>> # contingency matrix for the conditional estimator >>> contingency = Orange.statistics.contingency.VarClass('petal length', iris) >>> conditional_loess_constructor = Orange.statistics.estimate.ConditionalLoess() >>> >>> cloess_est = conditional_loess_constructor(contingency) >>> print cloess_est(iris[0]['petal length']) <0.980, 0.008, 0.012>