arviz_plots.plot_ecdf_pit

Contents

arviz_plots.plot_ecdf_pit#

arviz_plots.plot_ecdf_pit(dt, var_names=None, filter_vars=None, group='prior_sbc', coords=None, sample_dims=None, ci_prob=None, coverage=False, plot_collection=None, backend=None, labeller=None, aes_by_visuals=None, visuals=None, stats=None, **pc_kwargs)[source]#

Plot Δ-ECDF.

Plots the Δ-ECDF, that is the difference between the observed ECDF and the expected CDF. It assumes the values in the DataTree have already been transformed to PIT values, as in the case of SBC analysis or values from arviz_base.loo_pit.

Simultaneous confidence bands are computed using the simulation method described in [1].

Alternatively, we can visualize the coverage of the central posterior credible intervals by setting coverage=True. This allows us to assess whether the credible intervals includes the observed values. We can obtain the coverage of the central intervals from the PIT by replacing the PIT with two times the absolute difference between the PIT values and 0.5.

For more details on how to interpret this plot, see https://arviz-devs.github.io/EABM/Chapters/Prior_posterior_predictive_checks.html#pit-ecdfs.

Parameters:
dtxarray.DataTree

Input data

var_namesstr or list of str, optional

One or more variables to be plotted. Currently only one variable is supported. Prefix the variables by ~ when you want to exclude them from the plot.

filter_vars{None, “like”, “regex”}, optional, default=None

If None (default), interpret var_names as the real variables names. If “like”, interpret var_names as substrings of the real variables names. If “regex”, interpret var_names as regular expressions on the real variables names.

groupstr, optional

Which group to use. Defaults to “prior_sbc”.

coordsdict, optional

Coordinates to plot.

sample_dimsstr or sequence of hashable, optional

Dimensions to reduce unless mapped to an aesthetic. Defaults to rcParams["data.sample_dims"]

ci_probfloat, optional

Indicates the probability that should be contained within the plotted credible interval. Defaults to rcParams["stats.ci_prob"]

coveragebool, optional

If True, plot the coverage of the central posterior credible intervals. Defaults to False.

plot_collectionPlotCollection, optional
backend{“matplotlib”, “bokeh”, “plotly”}, optional
labellerlabeller, optional
aes_by_visualsmapping of {strsequence of str}, optional

Mapping of visuals to aesthetics that should use their mapping in plot_collection when plotted. Valid keys are the same as for visuals except for “remove_axis”

visualsmapping of {strmapping or False}, optional

Valid keys are:

statsmapping, optional

Valid keys are:

  • ecdf_pit -> passed to ecdf_pit. Default is {"n_simulation": 1000}.

**pc_kwargs

Passed to arviz_plots.PlotCollection.wrap

Returns:
PlotCollection

References

[1]

Säilynoja et al. Graphical test for discrete uniformity and its applications in goodness-of-fit evaluation and multiple sample comparison. Statistics and Computing 32(32). (2022) https://doi.org/10.1007/s11222-022-10090-6

Examples

Rank plot for the crabs hurdle-negative-binomial dataset.

>>> from arviz_plots import plot_ecdf_pit, style
>>> style.use("arviz-variat")
>>> from arviz_base import load_arviz_data
>>> dt = load_arviz_data('sbc')
>>> plot_ecdf_pit(dt)
../../_images/arviz_plots-plot_ecdf_pit-1.png

Faceted plot with PIT Δ-ECDF values for each variable The plot_ecdf_pit function assumes the values passed to it has already been transformed to PIT values, as in the case of SBC analysis or values from arviz_base.loo_pit. The distribution should be uniform if the model is well-calibrated. To make the plot easier to interpret, we plot the Δ-ECDF, that is, the difference between the expected CDF from the observed ECDF. As small deviations from uniformity are expected, the plot also shows the credible envelope.

PIT-ECDF

PIT-ECDF

Coverage refers to the proportion of true values that fall within a given prediction interval. For a well-calibrated model, the coverage should match the intended interval width. For example, a 95% credible interval should contain the true value 95% of the time. The distribution should be uniform if the model is well-calibrated. To make the plot easier to interpret, we plot the Δ-ECDF, that is, the difference between the expected CDF from the observed ECDF. As small deviations from uniformity are expected, the plot also shows the credible envelope. We can compute the coverage for equal-tailed intervals (ETI) by passing coverage=True to the plot_ecdf_pit function. This works because ETI coverage can be obtained by transforming the PIT values. However, for other interval types, such as HDI, coverage must be computed explicitly and is not supported by this function.

Coverage ECDF

Coverage ECDF
Contents