trustutils.jupyter

trustutils.jupyter.plot

Victor Banon Garcia CEA Saclay - DM2S/STMF/LGLS Mars 2021 - Stage 6 mois

We provide here a python package that can be used to graph and plot TRUSt data within jupyterlab. This work is based on the files package (a TRUST package that reads the son files).

class trustutils.jupyter.plot.Graph(title=None, subtitle=None, nX=1, nY=1, size=[12, 10], label_size=14, title_size=24, legend_size=14)

Bases: object

Graph is a class to plot .son file

add(x, y, marker='-', label=None, title=None, xIndice=None, yIndice=None, **kwargs)

Method to add a curve to the plot from a point probe.

Parameters:

  • x (float array) – x coordinates.

  • y (float array) – data to plot.

  • marker (str) – symbol of the ploted line (default=”-“)

  • label (str) – label of the curve.

  • title (str) – title of the curve.

  • xIndice (int) – first coordinates of the subplot list

  • yIndice (int) – second coordinates of the subplot list

  • kwargs (dictionary) – additional properties available in matplotlib.pyplot.plot

addPlot(coordonee, title=None)

Method to add a plot/subplot.

Parameters:

  • coordonee (str) – Adress of the file.

  • title (str) – Title of the plot.

addPoint(data, marker='-', compo=0, label='', func=None, **kwargs)

Method to add a curve to the plot from a point probe.

Parameters:

  • data (str) – Adress of the file.

  • marker (str) – symbol of the ploted line (default=”-“)

  • compo (str) – component we want to plot (for vector fields) (default=0)

  • label (str) – title of the curve .

  • func (function) – function to apply to data before plot (ex: computation of error)

  • kwargs (dictionary) – additional properties available in matplotlib.pyplot.plot

addResidu(data, marker='-', var='residu=max|Ri|', start=0, end=1000000000000, label='', **kwargs)

Method to add a curve to the plot from a dt_ev file.

Parameters:

  • data (str) – Adress of the file.

  • marker (str) – symbol of the ploted line.

  • var (str) – coordinate we want to plot (default=”residu=max|Ri|”)

  • start (float) – Time start.

  • end (float) – Time end.

  • label (str) – title of the curve .

  • kwargs (dictionary) – additional properties available in matplotlib.pyplot.plot

addSegment(data, time=None, marker='-', label=None, compo=0, func=None, funcX=None, transpose=False, **kwargs)

Method to add a curve to the plot from a segment of probes.

Parameters:

  • data (str) – Adress of the file.

  • time (double) – time of the parameter. If None, it equals to zero.

  • marker (str) – symbol of the ploted line (default=”-“)

  • label (str) – title of the curve.

  • compo (str) – component we want to plot (for vector fields) (default=0)

  • func (function Y = func(X,Y)) – function to apply to data before plot (ex: computation of error)

  • funcX (function X = funcX(X)) – function to apply to xcoordinates before plot

  • transpose (bool) – if True, transpose axis

  • kwargs (dictionary) – additional properties available in matplotlib.pyplot.plot

coordonee()

Method to add a data into the board.

Parameters:

None

Return type:

The coordinates of the subplot.

label(x_label, y_label, **kwargs)

Method to change labels.

Parameters:

  • x_label (str) – Label of x.

  • y_label (str) – Label of y.

  • kwargs (dictionary) – additional properties available in matplotlib.pyplot.subplot

legend(**kwargs)

Method to change labels.

Parameters:

**kwargs – option of matplotlib.pyplot.legend

scale(xscale='linear', yscale='linear')
xscalestr

The axis scale type to apply to the x-axis (default=”linear”)

yscalestr

The axis scale type to apply to the y-axis (default=”linear”)

visu(xmin=None, xmax=None, ymin=None, ymax=None)

Method to visualize all the data.

Parameters:

  • xmin (float) – Minimun of the ploted interval of x.

  • xmax (float) – Maximum of the ploted interval of x.

  • ymin (float) – Minimun of the ploted interval of y.

  • ymax (float) – Maximum of the ploted interval of y.

  • coordonee – coordinates in the subplotlist.

class trustutils.jupyter.plot.Table(columns)

Bases: object

Class to plot a Table.

addLine(ligne, name)

Method to add a row to the board.

Parameters:

  • ligne (list) – Row of the board.

  • name (str) – Name of the board.

load(data, name)

Method to add a data into the board.

Parameters:

  • data (str) – Adress of the file

  • name (str) – Name of the row.

loadPoint(data, name)

Method to add a data into the board.

Parameters:

  • data (str) – Adress of the file

  • name (str) – Name of the row.

setTitle(title)

Method to add title to the board.

Parameters:

title (str) – caption to ass.

sum(name)

Method to sum data in a column.

Parameters:

name (str) – Name of the column.

trustutils.jupyter.plot.loadText(data, index_column=0, nb_column=-1, transpose=True, dtype='float', skiprows=0, **kwargs)

Method for loading and saving files.

Parameters:

  • data (str) – name of the file we want to save.

  • index_column (int) – Index of the first column (default=0)

  • nb_column (int) – Number of columns loaded (default=None)

  • transpose (bool) – if we transpose the matrix (default=True)

  • dtype (str) – type of the data (default=”float”)

  • skiprows (int) – initial lines skip when reading (default=0)

  • kwargs (dictionary) – additional properties available in numpy.loadtxt() options

Returns:

matrix – matrix.

Return type:

array

trustutils.jupyter.plot.read_csv(data, **kwargs)

Method for loading files (wrap pandas.read_csv() function).

Parameters:

  • data (str) – name of the file we want to save.

  • comment (char, default = "#") – Character indicating that the remainder of line should not be parsed.

  • delim_whitespace (bool, default = True. Deprecated since version 2.2.0 -> Use sep="s+" instead.) – Specifies whether or not whitespace (e.g. ‘ ‘ or ‘ ‘) will be used as the sep delimiter.

  • kwargs (dictionary) – additional properties available in pandas.read_csv() options

Return type:

pandas Dataframe

trustutils.jupyter.run

Victor Banon Garcia / Adrien Bruneton CEA Saclay - DM2S/STMF/LGLS 03/2021

We provide here a Python package that can be used to run validation files from jupyterlab.

IMPORTANT: we assume that the methods of this module are invoked from the root directory of a validation form (i.e. a directory containing at least a ‘src’ subdirectory).

class trustutils.jupyter.run.TRUSTCase(directory, datasetName, nbProcs=1, execOptions='', excluNR=False)

Bases: object

Class which allows the user to load and execute a validation case

copy(targetName, targetDirectory=None, nbProcs=1, execOptions='', excluNR=False)

Copy a TRUST Case

Parameters:

  • targetName (str) – New name

  • targetDirectory (str) – New directory

  • nbProcs (int) – number of procs

  • execOptions (str) – TRUST Options to add at the execution of the test case

  • excluNR (bool) – to remove of the non regression test cases

dumpDataset(user_keywords=[])

Print out the .data file. TODO handle upper / lower case

Parameters:

list_Trust_user_words (list(str)) – List of word the user wants to color in red.

dumpDatasetMD()

Print out the .data file in a basic MarkDown format. This will not highlight TRUST keywords.

partition(verbose=False, overwritePartition=True)

Apply partitioning of specified test case with trust -partition if nbProcs>1 only This avoids to modify trust -partition to be able to call it with 1 cpu

Parameters:

  • verbose (bool)

  • overwritePartition (bool) –

    Whether to overwrite the partition from the dataset when running trust -partition If True, uses

    trust -partition xxx.data nbProcs

    which will overwrite (or create) the partition in the dataset, but will cut the domain in a single direction. This is the old and default behavior. If False, uses the partition from the dataset by using

    trust -partition xxx.data

    without specifying nbProcs Warning: this will not crash if there is no commented partition block in the dataset

runCase(verbose=False)

Move to the case directory and execute the current test case: - calls pre_run if any - run the case - calls post_run if any

Parameters:

  • verbose (bool)

  • self.last_run_err (The results of the run are stored in members self.last_run_ok and)

substitute(find, replace)

Substitute (in place and in the build directory) a part of the dataset

Parameters:

  • find (str) – Text we want to substitute.

  • replace (str) – Text to insert in replacement.

substitute_template(subs_dict)

Substitute (in place and in the build directory) a part of the dataset

Parameters:

subs_dict (dict) – Text we want to substitute with python Template formalism (character identified with $ in datafile)

class trustutils.jupyter.run.TRUSTSuite(runPrepare=True)

Bases: object

A set of TRUST cases to be run for the validation form.

addCase(case)
copySrc()

Copy content of src directory into build directory. WARNING: this is where we expect to be at the root of the validation form.

detectSserver()

Detect whether the Sserver is running on the machine

extractNRCases()

Prints out the list of cases in a suitable format for processing by validation and lance_test tools.

WARNING: do not modify this without looking at scripts get_list_cas_nr and get_nb_cas_nr in Validation/Outils/Genere_Courbe/scripts

getCases()
printCases()

display testCases

runCases(verbose=False, preventConcurrent=False)

Launch all cases for the current suite.

Parameters:

  • verbose (bool) – whether to print the console output of the run.

  • preventConcurrent (bool) – run the cases in the order they were provided, even if the Sserver is up and running, and the -parallel_sjob option was passed.

tablePerf()

Prints the table of performance

trustutils.jupyter.run.TRUST_parameters(version='', param=[])

Function that creates a cell Mardown giving TRUST parameters (version, binary)

Parameters:

  • version (str) – version of TRUST - if void TRUST_VERSION read

  • param (list(str)) – List of Parameter used in this test case

trustutils.jupyter.run.addCase(directoryOrTRUSTCase, datasetName='', nbProcs=1, execOptions='', excluNR=False)

Add a case to run to the list of globally recorded cases.

Parameters:

  • directoryOrTRUSTCase (str) – directory where the case is stored (relative to build/)

  • datasetName (str) – Name of the case we want to run.

  • nbProcs (int) – Number of processors

  • execOptions (str) – TRUST Options to add at the execution of the test case

  • excluNR (bool) – to remove of the non regression test cases

Return type:

TRUSTcase instance we want to launch.

trustutils.jupyter.run.addCaseFromTemplate(templateData, targetDirectory, dic, nbProcs=1, targetData=None, execOptions='', excluNR=False)

Add a case to run to the list of globally recorded cases.

Parameters:

  • targetDirectory (str) – targetDirectory where the case is stored (relative to build/)

  • templateData (str) – Name of the template datafile

  • dic (dictionary) – substitution of term (key identified with $ in datafile) by value in a new data file

  • targetData (str) – if provided, templateData will be copied as targetData + dictionary applied + added to TRUSTSuite()

  • nbProcs (int) – Number of processors

  • excluNR (bool) – to remove of the non regression test cases

Return type:

a new TRUSTcase instance we want to launch.

trustutils.jupyter.run.displayMD(txt)

Display text as markdown in Jupyter notebook.

trustutils.jupyter.run.dumpDataset(fiche, list_Trust_user_words=[])

Print out the .data file.

Parameters:

  • fiche (str) – Path of the file

  • list_Trust_user_words (list(str)) – List of word the user wants to color in red.

trustutils.jupyter.run.dumpDatasetMD(data)

Print the .data file in MD format.

Parameters:

datafile (str) – relative path (to build directory) + name of datafile

trustutils.jupyter.run.dumpText(fiche, list_keywords=[])

Print out the file.

Parameters:

fiche (str) – Path of the file

trustutils.jupyter.run.executeCommand(cmd, verbose=False, nonRegression=False)

Execute a bash command in the BUILD_DIRECTORY :param cmd: :type cmd: command to execute :param verbose: :type verbose: bool :param nonRegression: :type nonRegression: bool

trustutils.jupyter.run.executeScript(scriptName, verbose=False, nonRegression=False)

Execute a script shell in the BUILD_DIRECTORY

Parameters:

  • scriptName (str) – Name of the exec script

  • verbose (bool)

  • nonRegression (bool) – by default executeScript is inactive when option -not_run is applied (for non regression test)

trustutils.jupyter.run.extractNRCases()

Prints out the list of cases in a suitable format for processing by validation and lance_test tools.

WARNING: do not modify this without looking at scripts get_list_cas_nr and get_nb_cas_nr in Validation/Outils/Genere_Courbe/scripts

trustutils.jupyter.run.getCases()
trustutils.jupyter.run.getLastLines_(err_pth)

Get last 20 lines of an error file …

trustutils.jupyter.run.initBuildDirectory()

triggers build directory creation and copy src stuff into it

trustutils.jupyter.run.initCaseSuite()

Instantiate a default suite of cases

trustutils.jupyter.run.introduction(auteur, creationDate=None)

Function that creates an introduction cell Mardown

Parameters:

  • author (str) – Name of the author of the test case.

  • creationDate (date) – format dd/mm/YYYY

trustutils.jupyter.run.isExtractingNR()

Check environnement variable IS_EXTRACTING_NR to know if extracting_nr is active

trustutils.jupyter.run.printCases()

display testCases

trustutils.jupyter.run.reset()

Wipe out build directory completly and reset default suite.

trustutils.jupyter.run.runCases(verbose=False, preventConcurrent=False)

Launch all TRUST cases for the current validation form.

Parameters:

  • verbose (bool) – whether to print the console output of the run.

  • preventConcurrent (bool) – run the cases in the order they were provided, even if the Sserver is up and running, and the -parallel_sjob option was passed.

trustutils.jupyter.run.saveFileAccumulator(data)

Method for saving files for the Non Regression test

Parameters:

data (str) – name of the file we want to save.

trustutils.jupyter.run.saveFormOutput()

Dummy method to indicate that the output of the notebook should be saved. This method does nothing here, but its invokation is detected by the pre_save hook registered in the TRUST Jupyter configuration. By default, outputs of the validation forms are not saved.

trustutils.jupyter.run.tablePerf()

Prints the table of performance

trustutils.jupyter.run.useLataTools()

Load LataTools environment in the PYTHONPATH so that ‘import LataLoader’ can work.

trustutils.jupyter.run.useMEDCoupling()

Load MEDCoupling environment in the PYTHONPATH so that ‘import medcoupling’ can work.