qal.common package

Submodules

qal.common.diff module

Helper library for QAL diff operations.

copyright:Copyright 2010-2014 by Nicklas Boerjesson
license:BSD, see LICENSE for details.
qal.common.diff.cmp_key_columns(_left, _right, _key_columns)[source]

This functions compares the columns in the specified key fields only and returns data usable in a </=/>-comparer

qal.common.diff.compare(_left, _right, _key_columns, _full)[source]

The compare function takes two structurally identical 2-dimensional matrices, _left and _right, matches them using the columns in _key_colums, and returns a tuple of the results.

Parameters:
  • _left (2d-array) – The left matrix
  • _right (2d-array) – The right matrix
  • _key_columns (array) – An array with the key columns
  • _full (bool) – If the _full parameter is True, also the values in the rows are compared, and the third result, _different is populated with a list of rows where the values differ.
Return array _missing_left:
 

rows present in _right, but not in _left

Return array _missing_right:
 

rows present in _left, but not in _right

Return array _difference:
 

rows that by keys are present in _left and _right, but differ with in rows

Return 2d-array _right_s:
 

The _right matrix, but sorted by keys, often useful when one wants to continue massaging the data.

Note

The results are not in original order, but sorted by their keys.

qal.common.diff.diff_files(_file_a, _file_b)[source]
qal.common.diff.diff_strings(_a, _b)[source]
qal.common.diff.diff_to_text(_missing_left, _missing_right, _different)[source]

Creates a textual representation of the differences

Note

Not implemented.

qal.common.diff.match_all_columns(_left, _right)[source]

Match all columns in two arrays, return false if they differ anywhere.

Parameters:
  • _left (array) – The left array
  • _right (array) – The right array

Note

This function is a little bit tailored to QAL needs. The left column has to be used for column iteration, since _right might have extra columns for references to underlying structures that should not be compared.

qal.common.listhelper module

Functions for handling lists.

copyright:Copyright 2010-2014 by Nicklas Boerjesson
license:BSD, see LICENSE for details.
qal.common.listhelper.CI_index(_List, _value)[source]

Case-insensitively finds an item in a list

qal.common.listhelper.find_next_match(_list, _start_idx, _match)[source]

Finds next _match from _start_idx

qal.common.listhelper.find_previous_match(_list, _start_idx, _match)[source]

Finds previous _match from _start_idx

qal.common.listhelper.pretty_list(_array)[source]

Returns proper array data representation syntax, but with each row on a new text row, to make it more readable and usable.

qal.common.listhelper.unenumerate(value, _Type)[source]

Returns the value of a specific type

qal.common.mapping module

Created on Oct 20, 2014

@author: Nicklas Boerjesson

class qal.common.mapping.Mapping(_xml_node=None, _substitution=None)[source]

Bases: builtins.object

as_xml_node()[source]
dest_reference = None

A reference to the destination data location within its dataset. Can be a field name, XPath or similar.

is_key = None

If true, the mapping is a key field

load_from_xml_node(_xml_node)[source]
src_datatype = None

The source data data type

src_reference = None

A reference to the source data location within its dataset. Can be a field name, XPath or similar.

substitution = None

An instance of the substitution class. Kept for maintaining things lite incrementors and similar.

qal.common.resources module

Access functionality for resources.

copyright:Copyright 2010-2014 by Nicklas Boerjesson
license:BSD, see LICENSE for details.
class qal.common.resources.Resource[source]

Bases: builtins.object

The resource class represents a QAL resource. Could be any entity like a database server, flat file or a web page. Resources have a globally unique resource uuid

as_xml_node()[source]

This function encode an XML structure into resource objects. Uses lxml as opposed to parse_xml().

base_path = None

If this resource was read from an XML, base path is the directory where the XML was located. Makes it possible to deduce the absolute path from relative paths in the XML files, making them slightly more portable.

caption = None

The “friendly” name of the resource. For example “Database BDD04”.

data = None

A dict of all the custom data that belongs to the resource. Access this way: _filename = data[‘file_name’].’

type = None

The type of the resources. Possible current values: “CUSTOM”, “FLATFILE”, “MATRIX”, “XPATH”, “RDBMS”.

uuid = None

The unique identifier of the resources. Used to access the resource, wherever its data is stored.

class qal.common.resources.Resources(_resources_node=None, _resources_xml=None, _external_resources_callback=None, _base_path=None)[source]

Bases: qal.common.xml_utils.XML_Translation

The resource class provides access to resources available through either XML-definitions or callback functions.

as_xml_node()[source]

This function encode resources structure into an XML structure.

base_path = None

If resource was loaded from a file, its path is stores here, useful when translating relative paths.

external_resources_callback = None

Callback method for external lookup

get_resource(_uuid)[source]

Returns the resource with the corresponding uuid

local_resources = None

Local list of resources

parse_xml(_resources_node=None, _resources_xml=None)[source]

Parses an XML structure into resource objects. Uses lxml.

qal.common.resources.add_xml_subitem(_parent, _nodename, _nodetext)[source]
qal.common.resources.resource_types()[source]

Returns a list of the supported resource types

qal.common.settings module

Contains functionality for reading settings from ini-files

copyright:Copyright 2010-2014 by Nicklas Boerjesson
license:BSD, see LICENSE for details.
class qal.common.settings.UBPMSettings(filename)[source]

Bases: builtins.object

This class is responsible for reading settings from ini-files and holding them in memory.

get(_section, _option, _default=None)[source]

Get a certain option

reload(filename)[source]

Reload all information

qal.common.strings module

Contains miscellaneous string functions.

copyright:Copyright 2010-2014 by Nicklas Boerjesson
license:BSD, see LICENSE for details.
qal.common.strings.binary_int_to_bool(_value)[source]

Converts 1 to True and 1 and None to False.

qal.common.strings.bool_to_binary_int(_value)[source]

Converts True to 1 and False and None to 0.

qal.common.strings.empty_if_none(_string, _source=None)[source]

If _source if None, return an empty string, otherwise return string. This is useful when you build strings and want it to be empty if you don’t have any data. For example when building a comma-separated string with a dynamic number of parameters: .. code-block:: python

command = (_id +”;” + + empty_if_none(”;”+str(_comment), _comment))

qal.common.strings.empty_when_none(_string=None)[source]

If _string if None, return an empty string, otherwise return string.

qal.common.strings.make_path_absolute(_path, _base_path)[source]

Makes a path defined as relative in the XML resource definition absolute using _base_path.

qal.common.strings.parse_balanced_delimiters(_input, _d_left, _d_right, _text_qualifier)[source]

Removes all balanced delimiters. Handles multiple levels and ignores those contained within text delimiters.

qal.common.strings.string_to_bool(_value)[source]

Converts “True” to True None to False. Case-insensitive.

qal.common.substitution module

Variable substitution functionality

copyright:Copyright 2010-2014 by Nicklas Boerjesson
license:BSD, see LICENSE for details.
class qal.common.substitution.Substitution[source]

Bases: builtins.object

The Substitution class is responsible for replacing and generating variables for a substitution session. Using a session one can, for example, keep track of an identity increment.

builtin_substitutions = None
set_identity(_value)[source]
substitute(_input)[source]

The substitute method scans _input for known substitution variables. It tries to keep the value within the same data type.

Parameters:_input – The substitution string, potentially holding substitution variables
Returns:Returns a value where all substitutions are made

qal.common.transform module

Created on Nov 3, 2013

@author: Nicklas Boerjesson @todo: * Add input type checks for better error messages * Use format string for destination formatting as well (casting from any other primitive type to string)

class qal.common.transform.Cast(_xml_node=None, _substitution=None)[source]

Bases: qal.common.transform.CustomTransformation

ICasts a string to the specified type. The timestamp date format defaults to the ISO format if format_string is not set.

Possible format string directives at : http://docs.python.org/3.2/library/datetime.html#strftime-strptime-behavior

For example, 2013-11-06 22:05:42 is “%Y-%m-%d %H:%M:%S”.

as_xml_node()[source]
dest_type = None

The destination type

format_string = None

A format string where applicable

load_from_xml_node(_xml_node)[source]
class qal.common.transform.CustomTransformation(_xml_node=None, _substitution=None)[source]

Bases: builtins.object

This is the base class for all transformations

as_xml_node()[source]
do_on_done(_value=None, _error=None)[source]
init_base_to_node(_name)[source]
load_from_xml_node(_xml_node)[source]
on_done = None

On done is an event, triggered when the transformation has been run. Conveys the resulting value or error message.

order = None

Order dictates when the transformation is run.

substitution = None

An optional instance of the substitution class. Usually shared by several transformations.

transform(_value)[source]
class qal.common.transform.IfEmpty(_xml_node=None, _substitution=None)[source]

Bases: qal.common.transform.CustomTransformation

IfEmpty returns a specified value if the input value is NULL.

as_xml_node()[source]
load_from_xml_node(_xml_node)[source]
value = None
class qal.common.transform.Replace(_xml_node=None, _substitution=None)[source]

Bases: qal.common.transform.CustomTransformation

Replace returns a copy of the string in which the occurrences of old have been replaced with new, optionally restricting the number of replacements to max.

as_xml_node()[source]
load_from_xml_node(_xml_node)[source]
max = None

The max number of times to replace

new = None

The new value

old = None

The old value

class qal.common.transform.ReplaceRegex(_xml_node=None, _substitution=None)[source]

Bases: qal.common.transform.CustomTransformation

ReplaceRegex returns a copy of the string in which the occurrences of old have been replaced with new, optionally restricting the number of replacements to max.

as_xml_node()[source]
compiled_regex = None

The compiled regular expression

load_from_xml_node(_xml_node)[source]
max = None

The max number of times to replace

new = None

The new value

pattern = None

The old value

class qal.common.transform.Trim(_xml_node=None, _substitution=None)[source]

Bases: qal.common.transform.CustomTransformation

Trim returns a copy of the string in which all chars have been trimmed from the beginning and the end of the string (default whitespace characters). If the value parameter is set to either “beginning” or “end”, only the left or right end of the string is trimmed, respectively.

as_xml_node()[source]
load_from_xml_node(_xml_node)[source]
value = None
qal.common.transform.make_transformation_array_from_xml_node(_xml_node, _substitution=None)[source]
qal.common.transform.make_transformations_xml_node(_transformations)[source]
qal.common.transform.perform_transformations(_input, _transformations)[source]

qal.common.xml_utils module

Helper library for XML operations and their debugging.

Note

Will probably be deprecated or at least rewritten when QAL switches completely to lxml.

copyright:Copyright 2010-2014 by Nicklas Boerjesson
license:BSD, see LICENSE for details.
class qal.common.xml_utils.XML_Translation[source]

Bases: builtins.object

This is a base class for translations between XML structures and object structures.

debuglevel = 2
encoding = 'utf-8'
get_root_node(_nodename, _xml='', _node=None)[source]
namespace = None
nestinglevel = 0
prefix_own = None
prefix_schema = 'xsd'
prefix_xml = 'xsi'
schema_uri = None
qal.common.xml_utils.find_child_node(_node, _nodename)[source]

Find all child nodes of an XML tree

qal.common.xml_utils.xml_base_type_value(_node, _typename)[source]

Using the base type name, get appropriate data

qal.common.xml_utils.xml_find_non_text_child(_node)[source]

Finds the first child that is not of the Text node type.

qal.common.xml_utils.xml_get_allowed_value(_node, _type)[source]

Check if a value is allowed in a certain XML node

qal.common.xml_utils.xml_get_boolean(_node)[source]

Helper function to get a boolean value from XSD:booleans defaults

qal.common.xml_utils.xml_get_numeric(_node, _type='')[source]

Helper function to read a numeric value from the XML

qal.common.xml_utils.xml_get_text(_node)[source]

Helper function to get character data from an XML tree

qal.common.xml_utils.xml_isnone(_node)[source]
qal.common.xml_utils.xml_set_cdata(_node, _value, _lowercase=False)[source]

Helper to set character data in an XML tree

Module contents

qal.common - Commonly used functionality

This package gathers miscellaneous functionality that is used around QAL and the users of QAL.

copyright:Copyright 2010-2014 by Nicklas Boerjesson
license:BSD, see LICENSE for details.