General Python Utilities

The modules and functions described here provide support for many of the most general operations used through out STScI_Python.

fileutil.py – General file functions

These were initially designed for use with PyDrizzle. These functions only rely on booleans ‘yes’ and ‘no’, PyFITS and readgeis.

This file contains both IRAF-compatibility and general file access functions. General functions included are:

DEGTORAD(deg), RADTODEG(rad)

DIVMOD(num,val)

convertDate(date)
    Converts the DATE date string into a decimal year.

decimal_date(date-obs,time-obs=None)
    Converts the DATE-OBS (with optional TIME-OBS) string into a decimal year

buildRootname(filename, extn=None, extlist=None)

buildNewRootname(filename, ext=None)

parseFilename(filename)
    Splits a input name into a tuple containing (filename, group/extension)

getKeyword(filename, keyword, default=None, handle=None)

getHeader(filename,handle=None)
     Return a copy of the PRIMARY header, along with any group/extension
     header, for this filename specification.

getExtn(fimg,extn=None)
    Returns a copy of the specified extension with data from PyFITS object
    'fimg' for desired file.

updateKeyword(filename, key, value)

openImage(filename,mode='readonly',memmap=False,fitsname=None)
     Opens file and returns PyFITS object.
     It will work on both FITS and GEIS formatted images.

findFile(input)

checkFileExists(filename,directory=None)

removeFile(inlist):
    Utility function for deleting a list of files or a single file.

rAsciiLine(ifile)
    Returns the next non-blank line in an ASCII file.

readAsnTable(input,output=None,prodonly=yes)
    Reads an association (ASN) table and interprets inputs and output.
    The 'prodonly' parameter specifies whether to use products as inputs
        or not; where 'prodonly=no' specifies to only use EXP as inputs.

isFits(input) - returns (True|False, fitstype), fitstype is one of
                ('simple', 'mef', 'waiver')

IRAF compatibility functions (abbreviated list):

osfn(filename)
    Convert IRAF virtual path name to OS pathname

show(*args, **kw)
    Print value of IRAF or OS environment variables

time()
    Print current time and date

access(filename)
    Returns true if file exists, where filename can include IRAF variables
stsci.tools.fileutil.DEGTORAD(deg)
stsci.tools.fileutil.DIVMOD(num, val)
stsci.tools.fileutil.Expand(instring, noerror=0)

Expand a string with embedded IRAF variables (IRAF virtual filename).

Allows comma-separated lists. Also uses os.path.expanduser to replace ‘~’ symbols.

Set the noerror flag to silently replace undefined variables with just the variable name or null (so Expand(‘abc$def’) = ‘abcdef’ and Expand(‘(abc)def’) = ‘def’). This is the IRAF behavior, though it is confusing and hides errors.

stsci.tools.fileutil.RADTODEG(rad)
stsci.tools.fileutil.access(filename)

Returns true if file exists.

stsci.tools.fileutil.buildFITSName(geisname)

Build a new FITS filename for a GEIS input image.

stsci.tools.fileutil.buildNewRootname(filename, extn=None, extlist=None)

Build rootname for a new file.

Use ‘extn’ for new filename if given, does NOT append a suffix/extension at all.

Does NOT check to see if it exists already. Will ALWAYS return a new filename.

stsci.tools.fileutil.buildRootname(filename, ext=None)

Build a new rootname for an existing file and given extension.

Any user supplied extensions to use for searching for file need to be provided as a list of extensions.

Examples

>>> rootname = buildRootname(filename, ext=['_dth.fits'])  
stsci.tools.fileutil.buildRotMatrix(theta)
stsci.tools.fileutil.checkFileExists(filename, directory=None)

Checks to see if file specified exists in current or specified directory.

Default is current directory. Returns 1 if it exists, 0 if not found.

stsci.tools.fileutil.convertDate(date)

Convert DATE string into a decimal year.

stsci.tools.fileutil.copyFile(input, output, replace=None)

Copy a file whole from input to output.

stsci.tools.fileutil.countExtn(fimg, extname='SCI')

Return the number of ‘extname’ extensions, defaulting to counting the number of SCI extensions.

stsci.tools.fileutil.decimal_date(dateobs, timeobs=None)

Convert DATE-OBS (and optional TIME-OBS) into a decimal year.

stsci.tools.fileutil.defvar(varname)

Returns true if CL variable is defined.

stsci.tools.fileutil.envget(var, default=None)

Get value of IRAF or OS environment variable.

stsci.tools.fileutil.findExtname(fimg, extname, extver=None)

Returns the list number of the extension corresponding to EXTNAME given.

stsci.tools.fileutil.findFile(input)

Search a directory for full filename with optional path.

stsci.tools.fileutil.findKeywordExtn(ft, keyword, value=None)

This function will return the index of the extension in a multi-extension FITS file which contains the desired keyword with the given value.

stsci.tools.fileutil.getDate()

Returns a formatted string with the current date.

stsci.tools.fileutil.getExtn(fimg, extn=None)

Returns the PyFITS extension corresponding to extension specified in filename.

Defaults to returning the first extension with data or the primary extension, if none have data. If a non-existent extension has been specified, it raises a KeyError exception.

stsci.tools.fileutil.getFilterNames(header, filternames=None)

Returns a comma-separated string of filter names extracted from the input header (PyFITS header object). This function has been hard-coded to support the following instruments:

ACS, WFPC2, STIS

This function relies on the ‘INSTRUME’ keyword to define what instrument has been used to generate the observation/header.

The ‘filternames’ parameter allows the user to provide a list of keyword names for their instrument, in the case their instrument is not supported.

stsci.tools.fileutil.getHeader(filename, handle=None)

Return a copy of the PRIMARY header, along with any group/extension header for this filename specification.

stsci.tools.fileutil.getKeyword(filename, keyword, default=None, handle=None)

General, write-safe method for returning a keyword value from the header of a IRAF recognized image.

Returns the value as a string.

stsci.tools.fileutil.getLTime()

Returns a formatted string with the current local time.

stsci.tools.fileutil.getVarDict()

Returns dictionary all IRAF variables.

stsci.tools.fileutil.getVarList()

Returns list of names of all IRAF variables.

stsci.tools.fileutil.help()
stsci.tools.fileutil.interpretDQvalue(input)

Converts an integer ‘input’ into its component bit values as a list of power of 2 integers.

For example, the bit value 1027 would return [1, 2, 1024]

stsci.tools.fileutil.isFits(input)
Returns:isFits – An (isfits, fitstype) tuple. The values of isfits and fitstype are specified as:
  • isfits: True|False
  • fitstype: if True, one of ‘waiver’, ‘mef’, ‘simple’; if False, None
Return type:tuple

Notes

Input images which do not have a valid FITS filename will automatically result in a return of (False, None).

In the case that the input has a valid FITS filename but runs into some error upon opening, this routine will raise that exception for the calling routine/user to handle.

stsci.tools.fileutil.listVars(prefix='', equals='\t= ', **kw)

List IRAF variables.

stsci.tools.fileutil.openImage(filename, mode='readonly', memmap=False, writefits=True, clobber=True, fitsname=None)

Opens file and returns PyFITS object. Works on both FITS and GEIS formatted images.

Notes

If a GEIS or waivered FITS image is used as input, it will convert it to a MEF object and only if writefits = True will write it out to a file. If fitsname = None, the name used to write out the new MEF file will be created using buildFITSName.

Parameters:
  • filename (str) – name of input file
  • mode (str) – mode for opening file based on PyFITS mode parameter values
  • memmap (bool) – switch for using memory mapping, False for no, True for yes
  • writefits (bool) – if True, will write out GEIS as multi-extension FITS and return handle to that opened GEIS-derived MEF file
  • clobber (bool) – overwrite previously written out GEIS-derived MEF file
  • fitsname (str) – name to use for GEIS-derived MEF file, if None and writefits==`True`, will use ‘buildFITSName()’ to generate one
stsci.tools.fileutil.osfn(filename)

Convert IRAF virtual path name to OS pathname.

stsci.tools.fileutil.parseExtn(extn=None)

Parse a string representing a qualified fits extension name as in the output of parseFilename and return a tuple (str(extname), int(extver)), which can be passed to astropy.io.fits functions using the ‘ext’ kw.

Default return is the first extension in a fits file.

Examples

>>> parseExtn('sci, 2')
('sci', 2)
>>> parseExtn('2')
('', 2)
>>> parseExtn('sci')
('sci', 1)
stsci.tools.fileutil.parseFilename(filename)

Parse out filename from any specified extensions.

Returns rootname and string version of extension name.

stsci.tools.fileutil.rAsciiLine(ifile)

Returns the next non-blank line in an ASCII file.

stsci.tools.fileutil.removeFile(inlist)

Utility function for deleting a list of files or a single file.

This function will automatically delete both files of a GEIS image, just like ‘iraf.imdelete’.

stsci.tools.fileutil.reset(*args, **kw)

Set IRAF environment variables.

stsci.tools.fileutil.set(*args, **kw)

Set IRAF environment variables.

stsci.tools.fileutil.show(*args, **kw)

Print value of IRAF or OS environment variables.

stsci.tools.fileutil.time(**kw)

Print current time and date.

stsci.tools.fileutil.unset(*args, **kw)

Unset IRAF environment variables.

This is not a standard IRAF task, but it is obviously useful. It makes the resulting variables undefined. It silently ignores variables that are not defined. It does not change the os environment variables.

stsci.tools.fileutil.untranslateName(s)

Undo Python conversion of CL parameter or variable name.

stsci.tools.fileutil.updateKeyword(filename, key, value, show=True)

Add/update keyword to header with given value.

stsci.tools.fileutil.verifyWriteMode(files)

Checks whether files are writable. It is up to the calling routine to raise an Exception, if desired.

This function returns True, if all files are writable and False, if any are not writable. In addition, for all files found to not be writable, it will print out the list of names of affected files.

stsci.tools.parseinput.checkASN(filename)

Determine if the filename provided to the function belongs to an association.

Parameters:filename (string) –
Returns:validASN
Return type:boolean value
stsci.tools.parseinput.countinputs(inputlist)

Determine the number of inputfiles provided by the user and the number of those files that are association tables

Parameters:inputlist (string) – the user input
Returns:
  • numInputs (int) – number of inputs provided by the user
  • numASNfiles (int) – number of association files provided as input
stsci.tools.parseinput.isValidAssocExtn(extname)

Determine if the extension name given as input could represent a valid association file.

Parameters:extname (string) –
Returns:isValid
Return type:boolean value
stsci.tools.parseinput.parseinput(inputlist, outputname=None, atfile=None)

Recursively parse user input based upon the irafglob program and construct a list of files that need to be processed. This program addresses the following deficiencies of the irafglob program:

parseinput can extract filenames from association tables
Returns:

  • This program will return a list of input files that will need to
  • be processed in addition to the name of any outfiles specified in
  • an association table.

Parameters:
  • - string (outputname) – specification of input files using either wild-cards, @-file or comma-separated list of filenames
  • - string – desired name for output product to be created from the input files
  • - object (atfile) – function to use in interpreting the @-file columns that gets passed to irafglob
Returns:

  • files - list of strings – names of output files to be processed
  • newoutputname - string – name of output file to be created.

stsci.tools.irafglob.irafglob(inlist, atfile=None)

Returns a list of filenames based on the type of IRAF input.

Handles lists, wild-card characters, and at-files. For special at-files, use the atfile keyword to process them.

This function is recursive, so IRAF lists can also contain at-files and wild-card characters, e.g. a.fits, @file.lst, *flt.fits.

STScI_Python Help Support

The versioninfo module reports the version information for a defined set of packages installed as part of STScI_Python which can be sent in as part of a help call. This information can then be used to help identify what software has been installed so that the source of the reported problem can be more easily identified.

stsci.tools.versioninfo.printVersionInfo()