Utils and variables (utils package)

Variables

class gadma.utils.variables.ContinuousVariable(name, domain=None, rand_gen=None)

Bases: Variable

Class of the continuous variable.

Parameters:

  • domain – domain of the variable, if None then default_domain will be taken.

  • rand_gen – random generator for the variable, if None then default_rand_gen will be taken.

apply_logarithm(back=False)

Applies logarithm transform to the variable. Domain and random generator are changed.

correct_value(value)

Check that value is correct for this variable.

default_domain = array([-inf,  inf])
default_rand_gen()

Uniform generator. Runs numpy.random.uniform on domain.

get_bounds()

Returns the domain of the variable.

get_possible_values()
Raises:

AttributeError – it is impossible to get possible values for continuous variable.

class gadma.utils.variables.DiscreteVariable(name, domain=None, rand_gen=None)

Bases: Variable

Class of the discrete variable.

Parameters:

  • domain – domain of the variable, if None then default_domain will be taken.

  • rand_gen – random generator for the variable, if None then default_rand_gen will be taken.

correct_value(value)

Check that value is correct for this variable.

default_domain = array([], dtype=float64)
default_rand_gen(size=None, replace=True, p=None)

See documentation of numpy.random.choice for more information.

property domain
get_bounds()

Returns bounds - minimum and maximum over domain of the variable.

get_possible_values()

Returns domain of the variable.

class gadma.utils.variables.Dynamic

Bases: object

Abstract class for Dynamic value. New class should be instance of this class, should have format_str attribute and implement _inner_func() (staticmethod), __str__() and func_str() methods.

Variables:

Dynamic.format_str – format string for string representation of the dynamic.

format_str = ''
classmethod func_str(y1, y2, x_diff)

Returns string representation of the dynamic via formating format_str with argumets.

class gadma.utils.variables.DynamicVariable(name, domain=None, rand_gen=None)

Bases: DemographicVariable, DiscreteVariable

Variable for keepeing selection parameter of the demographic model.

default_domain = ['Sud', 'Lin', 'Exp']
default_rand_gen()

See documentation of numpy.random.choice for more information.

get_bounds()
Raises:

AttributeError – Dynamic variable has incomparative values.

static get_func_from_value(value)

Returns gadma.utils.variables.Dynamic._inner_func() function from the value.

Parameters:

value (gadma.utils.variables.Dynamic) – value of the variable.

class gadma.utils.variables.Exp

Bases: Dynamic

Exponential dynamic.

  • format_str = “lambda t: {0} * ({1} / {0}) ** (t / {2})”

format_str = 'lambda t: {0} * ({1} / {0}) ** (t / {2})'
class gadma.utils.variables.FractionVariable(name, domain=None, rand_gen=None)

Bases: DemographicVariable, ContinuousVariable

Variable for keepeing fraction parameter of the demographic model.

default_domain = array([0.001, 0.999])
default_rand_gen()

Uniform generator. Runs numpy.random.uniform on domain.

units = 'universal'
class gadma.utils.variables.Lin

Bases: Dynamic

Linear dynamic.

  • format_str = “lambda t: {0} + ({1} - {0}) * (t / {2})”

format_str = 'lambda t: {0} + ({1} - {0}) * (t / {2})'
class gadma.utils.variables.MigrationVariable(name, units='genetic', domain=None, rand_gen=None)

Bases: DemographicVariable, ContinuousVariable

Variable for keepeing migration parameter of the demographic model.

Note:

Values are assumed to be in genetic units.

default_domain = array([ 0., 10.])
default_rand_gen()

Generates random value of the migration. With probability of 0.5 generates zero, otherwise runs gadma.utils.distributions.trunc_normal_sigma_generator().

class gadma.utils.variables.PopulationSizeVariable(name, units='genetic', domain=None, rand_gen=None)

Bases: DemographicVariable, ContinuousVariable

Variable for keepeing size of population in demographic model.

Note:

Values are assumed to be in genetic units.

default_domain = array([1.e-02, 1.e+02])
default_rand_gen()

Generator for trunc_lognormal_3_sigma_rule()

translate_units_to(units, Nanc_domain=None, Nanc_mean=None)
class gadma.utils.variables.SelectionVariable(name, units='genetic', domain=None, rand_gen=None)

Bases: DemographicVariable, ContinuousVariable

Variable for keepeing selection parameter of the demographic model.

Note:

Values are assumed to be in genetic units.

default_domain = array([ 0, 10])
default_rand_gen()

Generator for trunc_normal_3_sigma_rule()

class gadma.utils.variables.Sud

Bases: Dynamic

Sudden (constant) dynamic.

format_str = '{1}'
class gadma.utils.variables.TimeVariable(name, units='genetic', domain=None, rand_gen=None)

Bases: DemographicVariable, ContinuousVariable

Variable for keepeing time parameter of the demographic model.

Note:

Values are assumed to be in genetic units.

default_domain = array([1.e-15, 5.e+00])
default_rand_gen()

Generator for trunc_lognormal_3_sigma_rule()

class gadma.utils.variables.Variable(name, var_type, domain, rand_gen)

Bases: object

Abstract class for keeping parameters of events in demographic model. Any new class for variable must be an instance of this class.

New class should have default_domain, default_rand_gen class attributes and get_bounds(), get_possible_values() methods implemented.

Variables:

  • Variable.default_domain – default domain of the variable.

  • Variable.default_rand_gen – default random generator of the variable. Is used in resample() method.

Parameters:

  • name (str) – unique name of the variable.

  • var_type (str) – type of the variable (usually ‘continuous’ or ‘discrete’).

  • domain – domain of the variable.

  • rand_gen – random generator of the variable, should be a function that takes domain as argument and returns sampled value.

apply_logarithm(back=False)
correct_value(value)

Check that value is correct for this variable.

default_domain = None
default_rand_gen = None
get_bounds()

Returns bounds of the variable domain.

get_possible_values()

Returns list of all possible values of the variable.

property log_transformed
resample(*args, **kwargs)

Returns sampled value of the variable by calling rand_gen on the domain.

Parameters:

  • *args – arguments to pass in rand_gen.

  • **kwargs – kwargs to pass in rand_gen.

Variable Pool

class gadma.utils.variable_pool.VariablePool(lst=None)

Bases: list

Class for list of variables. All variables should have different names.

Parameters:

lst – list of the variables.

append(item)

Append new item to pool.

check_type(item)

Raises ValueError if item is not Variable.

extend(items)

Extend pool with other list.

fix_pickling()

Distributions

class gadma.utils.distributions.DemographicGenerator(var_cls, Nanc_domain, Nanc_mean, combined_generator=True)

Bases: object

gadma.utils.distributions.custom_generator(variables)

Custom generator for demographic model variables.

gadma.utils.distributions.generator_for_Nanc(N_mean, domain)
gadma.utils.distributions.rescale_generator(generator, rescale_function)
gadma.utils.distributions.trunc_lognormal(mean, sigma, lower, upper)

Truncated log-normal distribution.

gadma.utils.distributions.trunc_lognormal_3_sigma_rule(mean, lower, upper)

Truncated log-normal distribution with sigma according by three sigma rule.

gadma.utils.distributions.trunc_lognormal_sigma_generator(domain)

Generator for trunc_lognormal_3_sigma_rule()

gadma.utils.distributions.trunc_normal(mean, sigma, lower, upper)

Truncated normal distribution.

gadma.utils.distributions.trunc_normal_3_sigma_rule(mean, lower, upper)

Truncated normal distribution with sigma according by three sigma rule.

gadma.utils.distributions.trunc_normal_sigma_generator(domain)

Generator for trunc_normal_3_sigma_rule()

gadma.utils.distributions.uniform_generator(domain)

Uniform generator. Runs numpy.random.uniform on domain.

Utils

class gadma.utils.utils.CacheInfo

Bases: object

Class for keeping cache info like one from functools.lru_cache cache_info.

class gadma.utils.utils.StdAndFileLogger(log_filename, silent=False, stderr=False)

Bases: object

Logger for printing output both in file and stdout.

flush()
write(message)
class gadma.utils.utils.WeightedMetaArray(array, dtype=None, order=None)

Bases: ndarray

Array with metadata.

Parameters:

  • array – array to keep.

  • dtype – dtype of elements of the array.

  • order – see numpy.ndarray for more information.

str_as_list()
gadma.utils.utils.abspath(path)
gadma.utils.utils.apply_transform(variables, transform, x)

Applies transform to x. Usually is used for logarithm and exponential transforms. So value if x is transformed if it is float (ContinuousVariable) and does not have 0 in its domain.

class gadma.utils.utils.bcolors

Bases: object

BOLD = '\x1b[1m'
ENDC = '\x1b[0m'
FAIL = '\x1b[91m'
HEADER = '\x1b[95m'
OKBLUE = '\x1b[94m'
OKGREEN = '\x1b[92m'
UNDERLINE = '\x1b[4m'
WARNING = '\x1b[93m'
gadma.utils.utils.cache_func(f, maxeval=None)

Cashes function with one argument. :param f: function such that f(x). :returns: function that is cashed.

gadma.utils.utils.check_dir_existence(path_to_dir)
gadma.utils.utils.check_file_existence(path_to_file)
gadma.utils.utils.choose_by_weight(X, weights, nsample)

Choose nsample samples from X according to weights. The greater weight is the greater the probability to choose sample is.

Note:

if weights is None then choice will be uniform

gadma.utils.utils.deserialize_meta_array(x)

Transforms back pickles version of x.

gadma.utils.utils.ensure_dir_existence(path_to_dir, check_emptiness=False)
gadma.utils.utils.ensure_file_existence(path_to_file)
gadma.utils.utils.eval_wrapper(f, eval_file=None)

Returns good function for optimization. Each evaluation of function will be written in file. If needed function will be cached.

Parameters:

  • f – function. Is called as f(x, *args).

  • args – tuple of arguments.

  • eval_file – file to write evaluations.

  • cache – if True then function will be cached.

gadma.utils.utils.exponent_transform(x)

Transforms x by applying numpy.exp on it.

gadma.utils.utils.fix_args(f, *args)

Fixes argumets of function.

Parameters:

  • f – Function such that f(x, *args)

  • args – Tuple of function arguments.

Returns:

function that will take only x as argument.

gadma.utils.utils.float_repr(value, precision=5)
gadma.utils.utils.get_LOO_score(X_train, Y_train, gp_model, mode='rassmusen', verbose=False, do_optimize=True)
gadma.utils.utils.get_aic_score(n_params, log_likelihood)

Returns AIC score.

Parameters:

  • n_params – Number of parameters of model.

  • log_likelihood – Value of log likelihood.

gadma.utils.utils.get_best_kernel(optimizer, variables, X, Y, kernels=['matern52', 'matern32', 'rbf', 'exponential'], mode='rassmusen', verbose=False)
gadma.utils.utils.get_claic_score(engine, x0, boots, args=(), eps=1e-05, log_likelihood=None, return_eps=False)

Calculate CLAIC score for the model.

Parameters:

  • engine – Engine with model and data.

  • x0 – Parameters of the model.

  • boots – Bootstrap data.

Params args:

Arguments for engine’s evaluate function.

Params log_likelihood:

Value of log-likelihood for x0.

Params return_eps:

If True then tuple (CLAIC, eps) is returned.

gadma.utils.utils.get_correct_dtype(x)
gadma.utils.utils.get_loo_score_for_optimizer(optimizer, variables, X, Y, mode='rassmusen', verbose=False)

Optimizer has the correct kernel

gadma.utils.utils.get_loo_scores_for_kernels(optimizer, variables, X, Y, kernels=['matern52', 'matern32', 'rbf', 'exponential'], mode='rassmusen', verbose=False)
gadma.utils.utils.get_mu_and_sigma_rassmusen(K_inv, Y, i)

Get mu and sigma for missed element i according to Rassmusen.

Parameters:

  • K_inv – Inversed covariance matrix (cov(X, X)).

  • Y – All objectives.

  • i – Index of element that we exclude from X and Y.

Returns:

mean and std of GP prediction for i-th element if GP is trained on all X and Y excluding i-th element.

gadma.utils.utils.get_one_score(mu, sigma, y_true)

Returns LOO score for excluding one element.

gadma.utils.utils.ident_transform(x)

Identical transform. Just returns x.

gadma.utils.utils.is_pickleable(obj)

Returns True if obj could be dumped with pickle.

gadma.utils.utils.logarithm_transform(x)

Transforms x by applying numpy.log on it.

gadma.utils.utils.lru_cache(func)

Our lru cache. We want to get cache itself while functools.lru_cache could not do it. Please, be carefull as it could be some attributes named the same way and it will be ruined. We use it for our decorator cache_func().

gadma.utils.utils.module_name_from_path(path)

Returns name for module that will be imported from given path.

gadma.utils.utils.normalize(Y)
gadma.utils.utils.run_f_and_save_result_into_queue(f, queue, *args, **kwargs)

Runs f with args and kwargs and save the result of run into queue.

gadma.utils.utils.serialize_meta_array(x)

Transforms x to pickle it.

gadma.utils.utils.sort_by_other_list(x, y, reverse=False, key=None)

Sort x and y according to values in y.

gadma.utils.utils.timeout(f, time)

Wraps function so that its execution time has limit of time seconds. The limitation is made via multiprocessing.Process.

gadma.utils.utils.transform_smac(optimizer, variables, X, Y)
gadma.utils.utils.update_by_one_fifth_rule(value, const, was_improved)

Updates value according to ‘one-fifth’ rule and const. Used in genetic algorithm.

Parameters:

  • value – Value to change.

  • const – Const for rule.

  • was_improved – Bool if fitness was improved or not.

gadma.utils.utils.variables_values_repr(variables, values)
gadma.utils.utils.warning_format(message, category, filename, lineno, file=None, line=None)