FindJumpAnalyzer

Finds jumps or steps in the autocorrelations.

Searches the autocorrelation of each input at some (user specified) subset of frequencies for jumps or step-like features. Calculates the continuous wavelet transform of the autocorrelations using an asymmetric wavelet and then finds times that are local maxima of the modulus of the wavelet transform over a range of time scales. See “Singularity Detection and Processing with Wavelets” by Stephane Mallat and Wen Liang Hwang for more information.

DocLib 788 describes this analyzer and the associated theremin graph.

Metrics

dias_task_<task_name>_run_time_seconds

Time to process single run.

dias_task_<task_name>_files_total

Number of files processed.

dias_task_<task_name>_file_time_seconds

Time to process single file.

dias_data_<task_name>_freq_processed_total

Number of frequencies processed.

dias_data_<task_name>_input_processed_total

Number of correlator inputs processed.

dias_task_<task_name>_jumps_total

Number of jumps detected for each frequency and input.

Labels
freq : float
Centre frequency in MHz.
input : int
Feed number in cylinder order (also known as chan_id).

Output Data

File naming

<YYYYMMDD>T<HHMMSS>Z_chimecal_jumps/<SSSSSSSS>.h5
YYYYMMDD and HHMMSS are the date and time (in UTC) of the start of the underlying chimecal data acquisition from which the jump list was derived. SSSSSSSS is the number of seconds elapsed between the start of the file and the start of the acquisition.

Indexes

jump
1D array of type int that uniquely labels each jump.
window
1D array of type int that labels the time samples in the window around each jump.
weather_station_time
1D array of type float that contains the unix timestamp for datasets obtained from the weather station.

Searched

files
1D array of str containing the absolute path of the files that were searched for jumps.
freq
1D structured array that contains the centre and width of the frequency channels that were searched for jumps.
input
1D structured array that contains the chan_id and correlator_input of the inputs that were searched for jumps.
time
1D array of type float that contains the unix timestamps that were searched for jumps.

Datasets

accumulated_rain_fall
1D array with axis [weather_station_time] that contains the accumulated rain fall in millimeters between that time and rain_window hours before that time.
freq
1D structured array with axis [jump] that contains the centre and width of the frequency channels where each jump occured.
input
1D structured array with axis [jump] that contains the chan_id and correlator_input of the input where each jump occured.
time
1D array of type float with axis [jump] that contains the unix timestamp where each jump occured.
time_index
1D array of type int with axis [jump] that contains the index into the time axis where the jump occured. Must use the files in the searched group for this index to be valid.
jump_size
1D array of type float with axis [jump] that contains the size of the jump in fractional units.
jump_time
2D array of type float with axes [jump, window] that contains the unix timestamp for a window around each jump.
jump_auto
2D array of type float with axes [jump, window] that contains the autocorrelations for a window around each jump.
jump_flag
2D array of type bool with axes [jump, window] that indicates valid data in the jump_time and jump_auto datasets.

Config Variables

dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.offset

Process data this timedelta before current time.

Type:str
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.instrument

Search archive for corr acquisitions from this instrument.

Type:str
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.max_num_file

Maximum number of files to load into memory at once.

Type:int
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.output_suffix

Name for the output acquisition type.

Type:str
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.max_num_freq

Maximum number of frequencies to load into memory at once.

Type:int
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.freq_collapse

Average the normalized autocorrelations over all frequencies before applying the jump finding algorithm. If True, then the freq_low, freq_high, and freq_step config variables are used to determine the frequency selection. If False, then the freq_physical config variable is used.

Type:bool
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.freq_low

Search all frequencies above this threshold for jumps. Unit is MHz.

Type:float
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.freq_high

Search all frequencies below this threshold for jumps. Unit is MHz.

Type:float
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.freq_step

Downsample frequencies between freq_low and freq_high by a factor freq_step. Unit is number of frequency channels.

Type:int
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.freq_physical

Apply the jump finding algorithm to these frequency channels. Units are MHz.

Type:list
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.ignore_sun

Ignore times near solar transit.

Type:bool
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.ignore_point_sources

Ignore times near the transit of the two brightest point sources.

Type:bool
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.transit_window

Window in seconds around solar and point source transits to ignore if the ignore_sun or ignore_point_sources config variables are set.

Type:float
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.use_input_flag

Do not apply algorithm to inputs that are already flagged as bad.

Type:bool
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.input_threshold

Input is considered good if it was flagged as good for more than this fraction of the time. Only used if input_flag is True.

Type:float
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.rain_window

Query the prometheus server for the rain fall accumulated over this amount of time in hours.

Type:int
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.wavelet_name

Name of pywt wavelet. Must be asymmetric for the algorithm to work.

Type:str
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.threshold

Find jumps that are greater than this threshold in fractional units. For example, setting this value to 0.25 will find jumps in the autocorrelations that are more than 25 percent.

Type:float
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.search_span

A time sample is considered a local maxima if the modulus of the wavelet transform is greater than all neighboring time samples within search_span * scale (note this search is done separately at each scale).

Type:float
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.psigma_max

Find jumps that occur at the same time across scales to within this value. Units are number of time samples.

Type:float
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.min_rise

Find jumps that exceed threshold in less than this number of time samples.

Type:int
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.log_scale

Perform wavelet transform for scales that are logarithmically spaced.

Type:bool
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.min_scale

Minimum scale to evaluate wavelet transform. Should be less than or equal to min_rise. Currently there is no reason to set this to anything other than the value of min_rise.

Type:int
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.max_scale

Find jumps that exceed threshold for more than this number of time samples. The wavelet transform will be evaluated for scales up to max_scale + 1.

Type:int
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.num_scale

Number of scales to evaluate the wavelet transform. This is only used if log_scale is True.

Type:int
dias.analyzers.find_jump_analyzer.FindJumpAnalyzer.edge_buffer

Number of time samples at the edge that are considered invalid is set to edge_buffer * max_scale.

Type:int

Additionally, the config parameters of the base class are expected in the task file

dias.analyzer.Analyzer(name, write_dir, …) Base class for all dias analyzers.