AATest

class hypex.AATest(precision_mode=False, control_size=0.5, stratification=False, n_iterations=None, sample_size=None, additional_params=None, random_states=None, equal_variance=None, groups_sizes=None, **kwargs)[source]

Bases: ExperimentShell

A class for conducting A/A tests with configurable parameters.

This class provides functionality to run A/A tests with options for stratification, precision control (fast or with type 1 error control), and sample size specification. It sets up the experiment pipeline with appropriate parameters, performs homogeneity tests for each split in order to evaluate their quality and to identify the best one.

Parameters:

  • precision_mode (bool, optional) – If True, runs more iterations (2000) in order to tuckle type 1 error. If False, runs fewer iterations (10) for quicker results. Defaults to False.

  • control_size (float, optional) – The proportion of data to allocate to control group. Must be between 0 and 1. Defaults to 0.5.

  • stratification (bool, optional) – Whether to use stratified sampling when splitting data. Defaults to False.

  • n_iterations (int, optional) – Number of test iterations to run. If None, determined by precision_mode. Defaults to None.

  • sample_size (float, optional) – Fraction of data to sample for each test. Must be between 0 and 1. If None, uses full dataset. Defaults to None.

  • additional_params (Dict[str, Any], optional) – Additional parameters to pass to the experiment pipeline. Defaults to None.

  • random_states (Iterable[int], optional) – Random seeds to use for each iteration. If None, uses range(n_iterations). Defaults to None.

  • equal_variance (bool, optional) – If True , perform a standard independent 2 sample test that assumes equal population variances. If False (default), perform Welch’s t-test, which does not assume equal population variance.

Examples

# Basic A/A test with default parameters
aa_test = AATest()
results = aa_test.execute(data)

# High precision A/A test with stratification
aa_test = AATest(
    precision_mode=True,
    stratification=True,
    control_size=0.5
)
results = aa_test.execute(data)

# A/A test with custom sample size and iterations
aa_test = AATest(
    sample_size=0.8,
    n_iterations=100
)
results = aa_test.execute(data)