optimpv.general package

Submodules

optimpv.general.BaseAgent module

BaseAgent class for Agent objects

class optimpv.general.BaseAgent.BaseAgent

Bases: object

Provides general functionality for Agent objects

bounds_descale(bounds, params)

Descale the bounds to match the Fitparam() objects descaling

Parameters:

• bounds (list) – list of bounds to descale

• params (list of Fitparam() objects) – list of Fitparam() objects to descale

Returns:

list of bounds descaled

Return type:

list

create_metrics_list()

Create a list of all metrics from all agents.

Returns:

• list – List of metric names

• list – List of minimize values

descale_dataframe(df, params)

Descale the dataframe to match the Fitparam() objects descaling

Parameters:

• df (DataFrame) – dataframe to descale

• params (list of Fitparam() objects) – list of Fitparam() objects to descale

Returns:

dataframe descaled

Return type:

DataFrame

get_all_agent_metric_names()

Get all metric names from the agent

Returns:

List of all metric names from the agent, formatted as ‘name_exp_format_metric_loss’

Return type:

list

Raises:

ValueError – If no metric or exp_format is defined in the agent

get_all_agent_tracking_metric_names(add_tracking_suffix=True)

Get all tracking metric names from the agent

Parameters:

add_tracking_suffix (bool, optional) – Whether to add a ‘_tracking’ suffix to the metric names, by default True

Returns:

List of all tracking metric names from the agent, formatted as ‘name_exp_format_metric_loss’

Return type:

list

Raises:

ValueError – If no metric or exp_format is defined in the agent

get_bounds_list()

Get the bounds list from the agent’s parameters

Returns:

List of bounds from the agent’s parameters

Return type:

list

params_descale(parameters, params)

Descale the parameters dictionary to match the Fitparam() objects descaling

Parameters:

• parameters (dict) – dictionary of parameter names and values to descale the Fitparam() objects

• params (list of Fitparam() objects) – list of Fitparam() objects to descale

Raises:

ValueError – If the value_type of the parameter is not ‘float’, ‘int’, ‘str’, ‘cat’, ‘sub’ or ‘bool’

Returns:

dictionary of parameter names and values descaled

Return type:

dict

params_rescale(parameters, params)

Rescale the parameters dictionary to match the Fitparam() objects rescaling

Parameters:

• parameters (dict) – dictionary of parameter names and values to rescale the Fitparam() objects

• params (list of Fitparam() objects) – list of Fitparam() objects to rescale

Raises:

ValueError – If the value_type of the parameter is not ‘float’, ‘int’, ‘str’, ‘cat’, ‘sub’ or ‘bool’

Returns:

dictionary of parameter names and values rescaled

Return type:

dict

params_w(parameters, params)

Populate the Fitparam() objects with the values from the parameters dictionary

Parameters:

• parameters (dict) – dictionary of parameter names and values to populate the Fitparam() objects

• params (list of Fitparam() objects) – list of Fitparam() objects to populate

Raises:

ValueError – If the value_type of the parameter is not ‘float’, ‘int’, ‘str’, ‘cat’, ‘sub’ or ‘bool’

Returns:

list of Fitparam() objects populated with the values from the parameters dictionary

Return type:

list of Fitparam() objects

rescale_array(array, params, param_name)

Rescale the array to match the Fitparam() objects rescaling

Parameters:

• array (list) – list of values to rescale

• params (list of Fitparam() objects) – list of Fitparam() objects to rescale

Returns:

list of values rescaled

Return type:

list

rescale_dataframe(df, params)

Rescale the dataframe to match the Fitparam() objects rescaling

Parameters:

• df (DataFrame) – dataframe to rescale

• params (list of Fitparam() objects) – list of Fitparam() objects to rescale

Returns:

dataframe rescaled

Return type:

DataFrame

optimpv.general.FitParams module

FitParams class

class optimpv.general.FitParams.FitParam(name='', type='range', value_type='float', value=None, bounds=None, values=None, start_value=None, log_scale=False, rescale=False, fscale=None, stepsize=None, display_name='', unit='', axis_type=None, std=None, encoding=None, is_ordered=False, is_sorted=False, force_log=False)

Bases: object

optimpv.general.SuggestOnlyAgent module

Provides general functionality for Agent objects for non ideal diode simulations

class optimpv.general.SuggestOnlyAgent.SuggestOnlyAgent(params, exp_format=None, metric=None, loss=None, threshold=100, minimize=True, tracking_metric=None, tracking_loss=None, tracking_exp_format=None, name='suggest', **kwargs)

Bases: BaseAgent

Agent object for suggesting new points in an optimization process without running the agents.

Parameters:

• params (list of Fitparam() objects) – List of Fitparam() objects.

• exp_format (str or list of str, optional) – Format of the experimental data, by default [].

• metric (str or list of str, optional) – Metric to evaluate the model, see optimpv.general.calc_metric for options, by default ‘mse’.

• loss (str or list of str, optional) – Loss function to use, see optimpv.general.loss_function for options, by default ‘linear’.

• threshold (int or list of int, optional) – Threshold value for the loss function used when doing multi-objective optimization, by default 100.

• minimize (bool or list of bool, optional) – If True then minimize the loss function, if False then maximize the loss function (note that if running a fit minize should be True), by default True.

• tracking_metric (str or list of str, optional) – Additional metrics to track and report in run_Ax output, by default None.

• tracking_loss (str or list of str, optional) – Loss functions to apply to tracking metrics, by default None.

• tracking_exp_format (str or list of str, optional) – Experimental formats for tracking metrics, by default None. Weights for tracking metrics, by default None.

• name (str, optional) – Name of the agent, by default ‘diode’.

• **kwargs (dict) – Additional keyword arguments.

optimpv.general.general module

General functions

optimpv.general.general.calc_metric(y, yfit, sample_weight=None, metric_name='mse')

Calculate the metric between the true values and the predicted values

Parameters:

• y (array-like of shape (n_samples,)) – True values

• yfit (array-like of shape (n_samples,)) – Predicted values

• sample_weight (array-like of shape (n_samples,), optional) – Sample weights, by default None

• metric_name (str, optional) –

  Name of the metric to calculate, by default ‘mse’ Possible values are:

  • ’mse’: Mean Squared Error

  • ’mae’: Mean Absolute Error

  • ’mape’: Mean Absolute Percentage Error

  • ’msle’: Mean Squared Log Error

  • ’rmsle’: Root Mean Squared Log Error

  • ’rmse’: Root Mean Squared Error

  • ’medae’: Median Absolute Error

  • ’nrmse’: Normalized Root Mean Squared Error

  • ’rmsre’: Root Mean Squared Relative Error

Returns:

The calculated metric

Return type:

float

Raises:

ValueError – If the metric is not implemented

optimpv.general.general.direct_mean_euclidean_distance(X_true, y_true, X_fit, y_fit)

Calculate the mean euclidean distance between the true and the predicted values

Parameters:

• X_true (array-like of shape (n_samples,)) – True values of the X coordinate

• y_true (array-like of shape (n_samples,)) – True values of the y coordinate

• X_fit (array-like of shape (n_samples,)) – Predicted values of the X coordinate

• y_fit (array-like of shape (n_samples,)) – Predicted values of the y coordinate

Returns:

The average euclidian distance between the true and the predicted values

Return type:

float

optimpv.general.general.interpolation_safe(x, y, xnew, mode='linear', log_base=10)

Interpolate y at xnew.

Invalid source values are dropped before interpolation. Invalid query values are returned as NaN. For log modes, interpolation is performed in log space but the output is returned on the original y scale.

optimpv.general.general.interpolation_safe2(x, y, xnew, mode='linear', method='auto', log_base=10, slope_ratio_threshold=50)

Robust interpolation supporting linear, loglin and loglog modes.

Parameters:

• x (array-like) – Input data.

• y (array-like) – Input data.

• xnew (array-like) – Query points.

• mode ({"linear", "loglin", "loglog"})

• method ({"auto", "pchip", "spline", "linear"})

• log_base ({10, np.e})

• slope_ratio_threshold (float) – Threshold used by auto mode to detect sharp transitions.

Returns:

ynew

Return type:

ndarray

optimpv.general.general.inv_loss_function(value, loss='linear')

Calculate the inverse loss function for the given value. Inspired by the scipy loss functions (https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html). The following loss functions are implemented:

• ‘linear’ (default)rho(z) = z. Gives a standard

  least-squares problem.

• ‘soft_l1’rho(z) = 2 * ((1 + z)**0.5 - 1). The smooth

  approximation of l1 (absolute value) loss. Usually a good choice for robust least squares.

• ‘huber’rho(z) = z if z <= 1 else 2*z**0.5 - 1. Works

  similarly to ‘soft_l1’.

• ‘cauchy’rho(z) = ln(1 + z). Severely weakens outliers

  influence, but may cause difficulties in optimization process.

• ‘arctan’rho(z) = arctan(z). Limits a maximum loss on

  a single residual, has properties similar to ‘cauchy’.

• ‘log’rho(z) = log( z). Logarithmically scales the

  loss, very similar to ‘cauchy’ but not as safe.

• ‘log10’rho(z) = log10(z). Logarithmically scales the

  loss with base 10 log, very similar to ‘cauchy’ but not as safe.

Parameters:

• value (float) – value to calculate the inverse loss function

• loss (str, optional) – loss function to use, by default ‘linear’

Returns:

value of the inverse loss function

Return type:

float

Raises:

ValueError – If the loss function is not implemented

optimpv.general.general.loss_function(value, loss='linear')

Calculate the loss function for the given value. Inspired by the scipy loss functions (https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html). The following loss functions are implemented:

• ‘linear’ (default)rho(z) = z. Gives a standard

  least-squares problem.

• ‘soft_l1’rho(z) = 2 * ((1 + z)**0.5 - 1). The smooth

  approximation of l1 (absolute value) loss. Usually a good choice for robust least squares.

• ‘huber’rho(z) = z if z <= 1 else 2*z**0.5 - 1. Works

  similarly to ‘soft_l1’.

• ‘cauchy’rho(z) = ln(1 + z). Severely weakens outliers

  influence, but may cause difficulties in optimization process.

• ‘arctan’rho(z) = arctan(z). Limits a maximum loss on

  a single residual, has properties similar to ‘cauchy’.

• ‘log’rho(z) = log( z). Logarithmically scales the

  loss, very similar to ‘cauchy’ but not as safe.

• ‘log10’rho(z) = log10(z). Logarithmically scales the

  loss with base 10 log, very similar to ‘cauchy’ but not as safe.

Parameters:

• value (float) – value to calculate the loss function

• loss (str, optional) – loss function to use, by default

Returns:

value of the loss function

Return type:

float

Raises:

ValueError – If the loss function is not implemented

optimpv.general.general.mean_min_euclidean_distance(X_true, y_true, X_fit, y_fit)

Calculate the minimum euclidean distance between the true and the predicted values

Parameters:

• X_true (array-like of shape (n_samples,)) – True values of the X coordinate

• y_true (array-like of shape (n_samples,)) – True values of the y coordinate

• X_fit (array-like of shape (n_samples,)) – Predicted values of the X coordinate

• y_fit (array-like of shape (n_samples,)) – Predicted values of the y coordinate

Returns:

The average minimum euclidian distance between the true and the predicted values

Return type:

float

optimpv.general.general.transform_data(y, y_pred, X=None, X_pred=None, transforms='linear', epsilon=None, do_G_frac_transform=None)

Transform data according to specified transformation type

Parameters:

• y (array-like) – True values to transform

• y_pred (array-like) – Predicted values to transform alongside y

• X (array-like, optional) – X coordinates of true values, by default None

• X_pred (array-like, optional) – X coordinates of predicted/fitted values, by default None

• transform_type (str or list of str, optional) –

  Type of transformation to apply, if a list is provided, transformations are applied sequentially, by default ‘linear’ Possible values are:

  • ’linear’: No transformation

  • ’log’: Log10 transformation of absolute values

  • ’normalize’: Division by maximum value

  • ’sqrt’: Square root transformation

• epsilon (float, optional) – Small value to add to avoid log(0), by default the machine epsilon for float64

• do_G_frac_transform (bool, optional) – Whether to apply a specific transformation based on the second column of X, by default False

Returns:

(y_transformed, y_pred_transformed)

Return type:

tuple of array-like

Raises:

ValueError – If the transformation type is not implemented

optimpv.general.general.transform_data_old(y, y_pred, X=None, X_pred=None, transform_type='linear', epsilon=None, do_G_frac_transform=False)

Transform data according to specified transformation type

Parameters:

• y (array-like) – True values to transform

• y_pred (array-like) – Predicted values to transform alongside y

• X (array-like, optional) – X coordinates of true values, by default None

• X_pred (array-like, optional) – X coordinates of predicted/fitted values, by default None

• transform_type (str, optional) –

  Type of transformation to apply, by default ‘linear’ Possible values are:

  • ’linear’: No transformation

  • ’log’: Log10 transformation of absolute values

  • ’normalized’: Division by maximum value

  • ’normalized_log’: Normalization followed by log transformation

  • ’sqrt’: Square root transformation

• epsilon (float, optional) – Small value to add to avoid log(0), by default the machine epsilon for float64

• do_G_frac_transform (bool, optional) – Whether to apply a specific transformation based on the second column of X, by default False

Returns:

(y_transformed, y_pred_transformed)

Return type:

tuple of array-like

Raises:

ValueError – If the transformation type is not implemented

optimpv.general.logger module

class optimpv.general.logger.ClassDecorator

Bases: object

decorate_callable(func)

class optimpv.general.logger.OptimpvOutputNameFilter(name='')

Bases: Filter

This is a filter which sets the record’s output_name, if not configured

filter(record: LogRecord) → bool

Determine if the specified record is to be logged.

Returns True if the record should be logged, or False otherwise. If deemed appropriate, the record may be modified in-place.

optimpv.general.logger.build_file_handler(filepath: str, level: int = 20) → StreamHandler

Build a file handle that logs entries to the given file, using the same formatting as the stream handler.

Parameters:

• filepath – Location of the file to log output to. If the file exists, output will be appended. If it does not exist, a new file will be created.

• level – The log level. By default, sets level to INFO

Returns:

A logging.FileHandler instance

optimpv.general.logger.build_stream_handler(level: int = 20) → StreamHandler

Build the default stream handler used for most optimpv logging. Sets default level to INFO, instead of WARNING.

Parameters:

level – The log level. By default, sets level to INFO

Returns:

A logging.StreamHandler instance

class optimpv.general.logger.disable_logger(name: str, level: int = 40)

Bases: ClassDecorator

decorate_callable(func: Callable[[...], T]) → Callable[[...], T]

class optimpv.general.logger.disable_loggers(names: list[str], level: int = 40)

Bases: ClassDecorator

decorate_callable(func: Callable[[...], T]) → Callable[[...], T]

optimpv.general.logger.get_logger(name: str, level: int = 20, force_name: bool = False) → Logger

Get an optimpv logger.

To set a human-readable “output_name” that appears in logger outputs, add {“output_name”: “[MY_OUTPUT_NAME]”} to the logger’s contextual information. By default, we use the logger’s name

NOTE: To change the log level on particular outputs (e.g. STDERR logs), set the proper log level on the relevant handler, instead of the logger e.g. logger.handers[0].setLevel(INFO)

Parameters:

• name – The name of the logger.

• level – The level at which to actually log. Logs below this level of importance will be discarded

• force_name – If set to false and the module specified is not ultimately a descendent of the optimpv module specified by name, “optimpv.” will be prepended to name

Returns:

The logging.Logger object.

optimpv.general.logger.make_indices_str(indices: Iterable[int]) → str

Generate a string representation of an iterable of indices; if indices are contiguous, returns a string formatted like like ‘<min_idx> - <max_idx>’, otherwise a string formatted like ‘[idx_1, idx_2, …, idx_n’].

optimpv.general.logger.set_optimpv_logger_levels(level: int) → None

Set the log level for all optimpv loggers, such that logs of given level are printed to STDERR by the root logger

optimpv.general.logger.set_stderr_log_level(level: int) → None

Set the log level for stream handler, such that logs of given level are printed to STDERR by the root logger

Module contents