spliced package

Submodules

spliced.client module

spliced.client.get_parser()

spliced.client.run_spliced()

run_spliced to perform a splice!

spliced.logger module

class spliced.logger.ColorizingStreamHandler(nocolor=False, stream=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>, use_threads=False)

Bases: StreamHandler

BLACK = 0

BLUE = 4

BOLD_SEQ = '\x1b[1m'

COLOR_SEQ = '\x1b[%dm'

CYAN = 6

GREEN = 2

MAGENTA = 5

RED = 1

RESET_SEQ = '\x1b[0m'

WHITE = 7

YELLOW = 3

can_color_tty()

colors = {'CRITICAL': 1, 'DEBUG': 4, 'ERROR': 1, 'INFO': 2, 'WARNING': 3}

decorate(record)

emit(record)

Emit a record.

If a formatter is specified, it is used to format the record. The record is then written to the stream with a trailing newline. If exception information is present, it is formatted using traceback.print_exception and appended to the stream. If the stream has an ‘encoding’ attribute, it is used to determine how to do the output to the stream.

property is_tty

class spliced.logger.Logger

Bases: object

cleanup()

debug(msg)

error(msg)

exit(msg, return_code=1)

handler(msg)

info(msg)

location(msg)

progress(done=None, total=None)

set_level(level)

set_stream_handler(stream_handler)

shellcmd(msg)

text_handler(msg)

The default snakemake log handler. Prints the output to the console. :param msg: the log message dictionary :type msg: dict

warning(msg)

spliced.logger.setup_logger(quiet=False, printshellcmds=False, nocolor=False, stdout=False, debug=False, use_threads=False, wms_monitor=None)

spliced.schemas module

spliced.experiment module

spliced.experiment.base module

class spliced.experiment.base.Experiment

Bases: object

A base Experiment holds information for a splice experiment!

add_splice(result, success=False, splice=None, command=None, different_libs=False, package=None)

Add a splice to the experiment

A splice can either be successful (so it will have libs, binaries, etc) or it can represent a failed state (for any number of reasons)

TODO refactor so we do one splice at a time TODO can we cache the splice setup? # ALSO add cache variable to save cache for smeagle (add to spack experiment)

init(package, splice, experiment, replace=None, validate=True, splice_versions=None)

Init config variables directly

load(config_file, validate=True)

Load a config from a yaml file

property name

property package

property package_so_prefix

predict(names=None, skip=None, predict_type=None)

Given a single named predictor (or a list to skip) make predictions.

property replace

run()

run the experiment.

property splice

property splice_so_prefix

property splice_versions

to_dict()

Return a dictionary of results

to_json()

Return a json dump of results

validate()

class spliced.experiment.base.Splice(package=None, splice=None, experiment=None, replace=None, result=None, success=False, different_libs=False)

Bases: object

A Splice holds the metadata for a splice, and if successful (or possible) will hold a result. A default splice result is not successful

add_identifier(key, identifier)

Add some experiment specific identifier (e.g., dag hash for spack)

add_spec(key, spec)

Add a spec (with a key) to metadata

match_libs()

Try to match dependencies between spliced/original library for comparison

to_dict()

Return the result as a dictionary

to_json()

Return the result as json

spliced.experiment.spack module

class spliced.experiment.spack.SpackDiffExperiment

Bases: SpackExperiment

The main spack experiment does a diff.

diff(specA, specB)

Prepare setup to do a diff.

specA: the main package specB: the to splice

run(*args, **kwargs)

Perform an install with spack of some spec A and some spec B, across versions.

Arguments: package (specA_name): the name of package A replace (specB_name): the name of package B

For many cases, specA and specB might be the same, but not always.

class spliced.experiment.spack.SpackExperiment

Bases: Experiment

concretize(spec_name=None, error_message='spec-concretization-failed', spec=None, different_libs=False)

A shared function to attempt concretization.

do_install(spec, error_message, name=None, different_libs=False)

Helper function to do an install.

get_sorted_versions(spec)

Helper function to get sorted versions (for consistency)

get_spack_ld_library_paths(original)

Get all of spack’s changes to the LD_LIBRARY_PATH for elfcall

populate_elfcall(splice, original, spliced_spec)

Derive library metadata for each of the original and splice spec

run_elfcall(lib, ld_library_paths=None)

A wrapper to run elfcall and ensure we add LD_LIBRARY_PATH additions (usually from spack)

class spliced.experiment.spack.SpackSpliceExperiment

Bases: SpackExperiment

do_splice(splice_name, spec_main, transitive=True)

do the splice, the spliced spec goes into the main spec

mock_splice(splice_name, replace_name, spec_main)

A mock splice is not possible with spack (it usually means replacing one dependency with another that isn’t an actual dependency) but we can still install the needed specs and then add their libs / binaries for other predictors. A “mock” of different libs means different_libs is set to True on the splice.

run(*args, **kwargs)

Perform a splice with a SpecA (a specific spec with a binary), and SpecB (the high level spec that is a dependency that we can test across versions).

Arguments: package (specA_name): the name of the main package we are splicing up splice (specB_name): the spec we are removing / switching out replace (specC_name): the spec we are splicing in (replacing with)

For many cases, specB and specC might be the same, but not always.

spliced.experiment.spack.add_contenders(spec, loc='lib', match=None)

Given a spec, find contender binaries and/or libs

spliced.experiment.spack.add_libraries(spec, library_name=None)

Given a spliced spec, get a list of its libraries matching a name (e.g., a library that has been spliced in). E.g., if the spec is curl, we might look for zlib.

spliced.experiment.spack.get_linked_deps(spec)

A helper function to only return a list of linked deps

spliced.predict module

spliced.predict.get_predictors(names=None)

Get a lookup of predictors for an experiment to run.

spliced.predict.base module

class spliced.predict.base.Prediction

Bases: object

A prediction is a base for assessing a Splice and making predictions.

create_elfcall_deps_lookup(splice, libs)

This is subbing in a library with a version of itself, and requires binaries

find_elfcall_deps_for(splice, lib)

Given a library (that is assumed to be in metadata, from elfcall) find libraries that the linker is assumed to hit and find symbols.

predict(splice, predict_type=None)

spliced.predict.base.get_prefix(lib)

spliced.predict.base.match_by_prefix(meta, spliced_meta)

Given an iterable of two things, match based on library prefix. E.g., basename -> split at . -> match first piece

spliced.predict.base.time_run_decorator(func)

Decorator to add run seconds to result

spliced.predict.base.timed_run(*args, **kwargs)

spliced.predict.symbolator module

spliced.predict.libabigail module

class spliced.predict.libabigail.LibabigailPrediction

Bases: Prediction

abicompat = None

abidiff = None

diff_prediction(splice)

Only do pairwise diffs between libs

find_tool(name)

Find a specific named tool (abidiff or abicompat)

find_tooling()

Find abicompat and abidiff and add to class

full_prediction(splice)

A full prediction is run with spliced splice, includes libs and binaries.

predict(splice, predict_type=None)

Run libabigail to add to the predictions

run_abicompat(binary, original, lib)

Run abicompat against two libraries

run_abidiff(original_lib, replace_lib)

Run abi diff with an original and comparison library

debuginfo files:

By default, libabigail will look in /usr/lib/debug for separate debug files (.debug) generated by gcc. Alternate locations can be specified by using the environment variables LIBABIGAIL_DEBUGINFO_DIR{1,2} for the original and comparison libraries, respectively.

splice_different_libs(splice)

In the case of splicing “the same lib” into itself (a different version) we can do matching based on names.

splice_equivalent_libs(splice)

In the case of splicing “the same lib” into itself (a different version) we can do matching based on names. We can use abicomat with binaries, and abidiff for just between the libs.

spliced.predict.libabigail.add_to_path(path)

spliced.utils module