Utility functions for handling bit masks and mask arrays

A module that provides functions for manipulating bitmasks and data quality (DQ) arrays.

stsci.tools.bitmask.interpret_bit_flags(bit_flags, flip_bits=None)

Converts input bit flags to a single integer value (bitmask) or None.

When input is a list of flags (either a Python list of integer flags or a sting of comma- or ‘+’-separated list of flags), the returned bitmask is obtained by summing input flags.

Note

In order to flip the bits of the returned bitmask, for input of str type, prepend ‘~’ to the input string. ‘~’ must be prepended to the entire string and not to each bit flag! For input that is already a bitmask or a Python list of bit flags, set flip_bits for True in order to flip the bits of the returned bitmask.

Parameters:
  • bit_flags (int, str, list, None) – An integer bitmask or flag, None, a string of comma- or ‘+’-separated list of integer bit flags, or a Python list of integer bit flags. If bit_flags is a str and if it is prepended with ‘~’, then the output bitmask will have its bits flipped (compared to simple sum of input flags). For input bit_flags that is already a bitmask or a Python list of bit flags, bit-flipping can be controlled through flip_bits parameter.
  • flip_bits (bool, None) – Indicates whether or not to flip the bits of the returned bitmask obtained from input bit flags. This parameter must be set to None when input bit_flags is either None or a Python list of flags.
Returns:

bitmask – Returns and integer bit mask formed from the input bit value or None if input bit_flags parameter is None or an empty string. If input string value was prepended with ‘~’ (or flip_bits was set to True), then returned value will have its bits flipped (inverse mask).
Return type:

int or None

Examples

>>> from stsci.tools.bitmask import interpret_bit_flags
>>> "{0:016b}".format(0xFFFF & interpret_bit_flags(28))
'0000000000011100'
>>> "{0:016b}".format(0xFFFF & interpret_bit_flags('4,8,16'))
'0000000000011100'
>>> "{0:016b}".format(0xFFFF & interpret_bit_flags('~4,8,16'))
'1111111111100011'
>>> "{0:016b}".format(0xFFFF & interpret_bit_flags('~(4+8+16)'))
'1111111111100011'
>>> "{0:016b}".format(0xFFFF & interpret_bit_flags([4, 8, 16]))
'0000000000011100'
>>> "{0:016b}".format(0xFFFF & interpret_bit_flags([4, 8, 16], flip_bits=True))
'1111111111100011'
stsci.tools.bitmask.bitfield_to_boolean_mask(bitfield, ignore_flags=0, flip_bits=None, good_mask_value=True, dtype=<class 'numpy.bool_'>)

Converts an array of bit fields to a boolean (or integer) mask array according to a bitmask constructed from the supplied bit flags (see ignore_flags parameter).

This function is particularly useful to convert data quality arrays to boolean masks with selective filtering of DQ flags.

Parameters:
  • bitfield (numpy.ndarray) – An array of bit flags. By default, values different from zero are interpreted as “bad” values and values equal to zero are considered as “good” values. However, see ignore_flags parameter on how to selectively ignore some bits in the bitfield array data.
  • ignore_flags (int, str, list, None (Default = 0)) –

    An integer bitmask, a Python list of bit flags, a comma- or ‘+’-separated string list of integer bit flags that indicate what bits in the input bitfield should be ignored (i.e., zeroed), or None.

    Setting ignore_flags to None effectively will make bitfield_to_boolean_mask interpret all bitfield elements as “good” regardless of their value.

    When ignore_flags argument is an integer bitmask, it will be combined using bitwise-NOT and bitwise-AND with each element of the input bitfield array (~ignore_flags & bitfield). If the resultant bitfield element is non-zero, that element will be interpreted as a “bad” in the output boolean mask and it will be interpreted as “good” otherwise. flip_bits parameter may be used to flip the bits (bitwise-NOT) of the bitmask thus effectively changing the meaning of the ignore_flags parameter from “ignore” to “use only” these flags.

    Note

    Setting ignore_flags to 0 effectively will assume that all non-zero elements in the input bitfield array are to be interpreted as “bad”.

    When ignore_flags argument is an Python list of integer bit flags, these flags are added together to create an integer bitmask. Each item in the list must be a flag, i.e., an integer that is an integer power of 2. In order to flip the bits of the resultant bitmask, use flip_bits parameter.

    Alternatively, ignore_flags may be a string of comma- or ‘+’-separated list of integer bit flags that should be added together to create an integer bitmask. For example, both '4,8' and '4+8' are equivalent and indicate that bit flags 4 and 8 in the input bitfield array should be ignored when generating boolean mask.

    Note

    'None', 'INDEF', and empty (or all white space) strings are special values of string ignore_flags that are interpreted as None.

    Note

    Each item in the list must be a flag, i.e., an integer that is an integer power of 2. In addition, for convenience, an arbitrary single integer is allowed and it will be interpretted as an integer bitmask. For example, instead of '4,8' one could simply provide string '12'.

    Note

    When ignore_flags is a str and when it is prepended with ‘~’, then the meaning of ignore_flags parameters will be reversed: now it will be interpreted as a list of bit flags to be used (or not ignored) when deciding which elements of the input bitfield array are “bad”. Following this convention, an ignore_flags string value of '~0' would be equivalent to setting ignore_flags=None.

    Warning

    Because prepending ‘~’ to a string ignore_flags is equivalent to setting flip_bits to True, flip_bits cannot be used with string ignore_flags and it must be set to None.

  • flip_bits (bool, None (Default = None)) –

    Specifies whether or not to invert the bits of the bitmask either supplied directly through ignore_flags parameter or built from the bit flags passed through ignore_flags (only when bit flags are passed as Python lists of integer bit flags). Occasionally, it may be useful to consider only specific bit flags in the bitfield array when creating a boolean mask as opposite to ignoring specific bit flags as ignore_flags behaves by default. This can be achieved by inverting/flipping the bits of the bitmask created from ignore_flags flags which effectively changes the meaning of the ignore_flags parameter from “ignore” to “use only” these flags. Setting flip_bits to None means that no bit flipping will be performed. Bit flipping for string lists of bit flags must be specified by prepending ‘~’ to string bit flag lists (see documentation for ignore_flags for more details).

    Warning

    This parameter can be set to either True or False ONLY when ignore_flags is either an integer bitmask or a Python list of integer bit flags. When ignore_flags is either None or a string list of flags, flip_bits MUST be set to None.

  • good_mask_value (int, bool (Default = True)) – This parameter is used to derive the values that will be assigned to the elements in the output boolean mask array that correspond to the “good” bit fields (that are 0 after zeroing bits specified by ignore_flags) in the input bitfield array. When good_mask_value is non-zero or True then values in the output boolean mask array corresponding to “good” bit fields in bitfield will be True (if dtype is numpy.bool_) or 1 (if dtype is of numerical type) and values of corresponding to “bad” flags will be False (or 0). When good_mask_value is zero or False then the values in the output boolean mask array corresponding to “good” bit fields in bitfield will be False (if dtype is numpy.bool_) or 0 (if dtype is of numerical type) and values of corresponding to “bad” flags will be True (or 1).
  • dtype (data-type) – The desired data-type for the output binary mask array.
Returns:

mask – Returns an array of the same dimensionality as the input bitfield array whose elements can have two possible values, e.g., True or False (or 1 or 0 for integer dtype) according to values of to the input bitfield elements, ignore_flags parameter, and the good_mask_value parameter.
Return type:

numpy.ndarray

Examples

>>> from stsci.tools import bitmask
>>> import numpy as np
>>> dqbits = np.asarray([[0,0,1,2,0,8,12,0],[10,4,0,0,0,16,6,0]])
>>> bitmask.bitfield_to_boolean_mask(dqbits, ignore_flags=0, dtype=int)
array([[1, 1, 0, 0, 1, 0, 0, 1],
       [0, 0, 1, 1, 1, 0, 0, 1]])
>>> bitmask.bitfield_to_boolean_mask(dqbits, ignore_flags=0, dtype=bool)
array([[ True,  True, False, False,  True, False, False,  True],
       [False, False,  True,  True,  True, False, False,  True]])
>>> bitmask.bitfield_to_boolean_mask(dqbits, ignore_flags=6, good_mask_value=0, dtype=int)
array([[0, 0, 1, 0, 0, 1, 1, 0],
       [1, 0, 0, 0, 0, 1, 0, 0]])
>>> bitmask.bitfield_to_boolean_mask(dqbits, ignore_flags=~6, good_mask_value=0, dtype=int)
array([[0, 0, 0, 1, 0, 0, 1, 0],
       [1, 1, 0, 0, 0, 0, 1, 0]])
>>> bitmask.bitfield_to_boolean_mask(dqbits, ignore_flags=6, flip_bits=True, good_mask_value=0, dtype=int)
array([[0, 0, 0, 1, 0, 0, 1, 0],
       [1, 1, 0, 0, 0, 0, 1, 0]])
>>> bitmask.bitfield_to_boolean_mask(dqbits, ignore_flags='~(2+4)', good_mask_value=0, dtype=int)
array([[0, 0, 0, 1, 0, 0, 1, 0],
       [1, 1, 0, 0, 0, 0, 1, 0]])
>>> bitmask.bitfield_to_boolean_mask(dqbits, ignore_flags=[2, 4], flip_bits=True, good_mask_value=0, dtype=int)
array([[0, 0, 0, 1, 0, 0, 1, 0],
       [1, 1, 0, 0, 0, 0, 1, 0]])
stsci.tools.bitmask.is_bit_flag(n)

Verifies if the input number is a bit flag (i.e., an integer number that is an integer power of 2).

Parameters:n (int) – A positive integer number. Non-positive integers are considered not to be “flags”.
Returns:result – True if input n is a bit flag and False if it is not.
Return type:bool