Utility functions for handling bit masks and mask arrays¶
A module that provides functions for manipulating bitmasks and data quality (DQ) arrays.
- stsci.tools.bitmask.bitfield_to_boolean_mask(bitfield, ignore_flags=0, flip_bits=None, good_mask_value=True, dtype=<class 'numpy.bool_'>)¶
Converts an array of bit fields to a boolean (or integer) mask array according to a bitmask constructed from the supplied bit flags (see
ignore_flags
parameter).This function is particularly useful to convert data quality arrays to boolean masks with selective filtering of DQ flags.
- Parameters:
- bitfieldnumpy.ndarray
An array of bit flags. By default, values different from zero are interpreted as “bad” values and values equal to zero are considered as “good” values. However, see
ignore_flags
parameter on how to selectively ignore some bits in thebitfield
array data.- ignore_flagsint, str, list, None (Default = 0)
An integer bitmask, a Python list of bit flags, a comma- or ‘+’-separated string list of integer bit flags that indicate what bits in the input
bitfield
should be ignored (i.e., zeroed), or None.Setting
ignore_flags
to None effectively will make bitfield_to_boolean_mask interpret allbitfield
elements as “good” regardless of their value.When
ignore_flags
argument is an integer bitmask, it will be combined using bitwise-NOT and bitwise-AND with each element of the inputbitfield
array (~ignore_flags & bitfield
). If the resultant bitfield element is non-zero, that element will be interpreted as a “bad” in the output boolean mask and it will be interpreted as “good” otherwise.flip_bits
parameter may be used to flip the bits (bitwise-NOT
) of the bitmask thus effectively changing the meaning of theignore_flags
parameter from “ignore” to “use only” these flags.Note
Setting
ignore_flags
to 0 effectively will assume that all non-zero elements in the inputbitfield
array are to be interpreted as “bad”.When
ignore_flags
argument is an Python list of integer bit flags, these flags are added together to create an integer bitmask. Each item in the list must be a flag, i.e., an integer that is an integer power of 2. In order to flip the bits of the resultant bitmask, useflip_bits
parameter.Alternatively,
ignore_flags
may be a string of comma- or ‘+’-separated list of integer bit flags that should be added together to create an integer bitmask. For example, both'4,8'
and'4+8'
are equivalent and indicate that bit flags 4 and 8 in the inputbitfield
array should be ignored when generating boolean mask.Note
'None'
,'INDEF'
, and empty (or all white space) strings are special values of stringignore_flags
that are interpreted as None.Note
Each item in the list must be a flag, i.e., an integer that is an integer power of 2. In addition, for convenience, an arbitrary single integer is allowed and it will be interpretted as an integer bitmask. For example, instead of
'4,8'
one could simply provide string'12'
.Note
When
ignore_flags
is a str and when it is prepended with ‘~’, then the meaning ofignore_flags
parameters will be reversed: now it will be interpreted as a list of bit flags to be used (or not ignored) when deciding which elements of the inputbitfield
array are “bad”. Following this convention, anignore_flags
string value of'~0'
would be equivalent to settingignore_flags=None
.Warning
Because prepending ‘~’ to a string
ignore_flags
is equivalent to settingflip_bits
to True,flip_bits
cannot be used with stringignore_flags
and it must be set to None.- flip_bitsbool, None (Default = None)
Specifies whether or not to invert the bits of the bitmask either supplied directly through
ignore_flags
parameter or built from the bit flags passed throughignore_flags
(only when bit flags are passed as Python lists of integer bit flags). Occasionally, it may be useful to consider only specific bit flags in thebitfield
array when creating a boolean mask as opposite to ignoring specific bit flags asignore_flags
behaves by default. This can be achieved by inverting/flipping the bits of the bitmask created fromignore_flags
flags which effectively changes the meaning of theignore_flags
parameter from “ignore” to “use only” these flags. Settingflip_bits
to None means that no bit flipping will be performed. Bit flipping for string lists of bit flags must be specified by prepending ‘~’ to string bit flag lists (see documentation forignore_flags
for more details).Warning
This parameter can be set to either True or False ONLY when
ignore_flags
is either an integer bitmask or a Python list of integer bit flags. Whenignore_flags
is either None or a string list of flags,flip_bits
MUST be set to None.- good_mask_valueint, bool (Default = True)
This parameter is used to derive the values that will be assigned to the elements in the output boolean mask array that correspond to the “good” bit fields (that are 0 after zeroing bits specified by
ignore_flags
) in the inputbitfield
array. Whengood_mask_value
is non-zero or True then values in the output boolean mask array corresponding to “good” bit fields inbitfield
will be True (ifdtype
is numpy.bool_) or 1 (ifdtype
is of numerical type) and values of corresponding to “bad” flags will be False (or 0). Whengood_mask_value
is zero or False then the values in the output boolean mask array corresponding to “good” bit fields inbitfield
will be False (ifdtype
is numpy.bool_) or 0 (ifdtype
is of numerical type) and values of corresponding to “bad” flags will be True (or 1).- dtypedata-type
The desired data-type for the output binary mask array.
- Returns:
- masknumpy.ndarray
Returns an array of the same dimensionality as the input
bitfield
array whose elements can have two possible values, e.g., True or False (or 1 or 0 for integerdtype
) according to values of to the inputbitfield
elements,ignore_flags
parameter, and thegood_mask_value
parameter.
Examples
>>> from stsci.tools import bitmask >>> import numpy as np >>> dqbits = np.asarray([[0,0,1,2,0,8,12,0],[10,4,0,0,0,16,6,0]]) >>> bitmask.bitfield_to_boolean_mask(dqbits, ignore_flags=0, dtype=int) array([[1, 1, 0, 0, 1, 0, 0, 1], [0, 0, 1, 1, 1, 0, 0, 1]]) >>> bitmask.bitfield_to_boolean_mask(dqbits, ignore_flags=0, dtype=bool) array([[ True, True, False, False, True, False, False, True], [False, False, True, True, True, False, False, True]]) >>> bitmask.bitfield_to_boolean_mask(dqbits, ignore_flags=6, good_mask_value=0, dtype=int) array([[0, 0, 1, 0, 0, 1, 1, 0], [1, 0, 0, 0, 0, 1, 0, 0]]) >>> bitmask.bitfield_to_boolean_mask(dqbits, ignore_flags=~6, good_mask_value=0, dtype=int) array([[0, 0, 0, 1, 0, 0, 1, 0], [1, 1, 0, 0, 0, 0, 1, 0]]) >>> bitmask.bitfield_to_boolean_mask(dqbits, ignore_flags=6, flip_bits=True, good_mask_value=0, dtype=int) array([[0, 0, 0, 1, 0, 0, 1, 0], [1, 1, 0, 0, 0, 0, 1, 0]]) >>> bitmask.bitfield_to_boolean_mask(dqbits, ignore_flags='~(2+4)', good_mask_value=0, dtype=int) array([[0, 0, 0, 1, 0, 0, 1, 0], [1, 1, 0, 0, 0, 0, 1, 0]]) >>> bitmask.bitfield_to_boolean_mask(dqbits, ignore_flags=[2, 4], flip_bits=True, good_mask_value=0, dtype=int) array([[0, 0, 0, 1, 0, 0, 1, 0], [1, 1, 0, 0, 0, 0, 1, 0]])
- stsci.tools.bitmask.interpret_bit_flags(bit_flags, flip_bits=None)¶
Converts input bit flags to a single integer value (bitmask) or None.
When input is a list of flags (either a Python list of integer flags or a sting of comma- or ‘+’-separated list of flags), the returned bitmask is obtained by summing input flags.
Note
In order to flip the bits of the returned bitmask, for input of str type, prepend ‘~’ to the input string. ‘~’ must be prepended to the entire string and not to each bit flag! For input that is already a bitmask or a Python list of bit flags, set flip_bits for True in order to flip the bits of the returned bitmask.
- Parameters:
- bit_flagsint, str, list, None
An integer bitmask or flag, None, a string of comma- or ‘+’-separated list of integer bit flags, or a Python list of integer bit flags. If bit_flags is a str and if it is prepended with ‘~’, then the output bitmask will have its bits flipped (compared to simple sum of input flags). For input bit_flags that is already a bitmask or a Python list of bit flags, bit-flipping can be controlled through flip_bits parameter.
- flip_bitsbool, None
Indicates whether or not to flip the bits of the returned bitmask obtained from input bit flags. This parameter must be set to None when input bit_flags is either None or a Python list of flags.
- Returns:
- bitmaskint or None
Returns and integer bit mask formed from the input bit value or None if input bit_flags parameter is None or an empty string. If input string value was prepended with ‘~’ (or flip_bits was set to True), then returned value will have its bits flipped (inverse mask).
Examples
>>> from stsci.tools.bitmask import interpret_bit_flags >>> "{0:016b}".format(0xFFFF & interpret_bit_flags(28)) '0000000000011100' >>> "{0:016b}".format(0xFFFF & interpret_bit_flags('4,8,16')) '0000000000011100' >>> "{0:016b}".format(0xFFFF & interpret_bit_flags('~4,8,16')) '1111111111100011' >>> "{0:016b}".format(0xFFFF & interpret_bit_flags('~(4+8+16)')) '1111111111100011' >>> "{0:016b}".format(0xFFFF & interpret_bit_flags([4, 8, 16])) '0000000000011100' >>> "{0:016b}".format(0xFFFF & interpret_bit_flags([4, 8, 16], flip_bits=True)) '1111111111100011'
- stsci.tools.bitmask.is_bit_flag(n)¶
Verifies if the input number is a bit flag (i.e., an integer number that is an integer power of 2).
- Parameters:
- nint
A positive integer number. Non-positive integers are considered not to be “flags”.
- Returns:
- resultbool
True
if inputn
is a bit flag andFalse
if it is not.