Adds livetime correction to get data by jajmitchell · Pull Request #192 · TCDSolar/stixpy

jajmitchell · 2025-09-17T15:03:40Z

Adds livetime correction to get data.

Adds livetime correction to get_data

Adds detector and time masks

samaloney

Looks good - needs docs, change log and test/s

samaloney · 2025-10-03T14:40:34Z

+
+        # fmt: off
+        # For the moment copied from idl
+        trigger_to_detector = [0, 0, 7, 7, 2, 1, 1, 6, 6, 5, 2, 3, 3, 4, 4, 5, 13, 12,
+                            12, 11, 11, 10, 13, 14, 14, 9, 9, 10, 15, 15, 8, 8,]
+        # fmt: on
+
+        triggers = self.data["triggers"][:, trigger_to_detector].astype(float)[...]
+
+        _, livefrac, _ = get_livetime_fraction(triggers / self.data["timedel"].to("s").reshape(-1, 1))


So this is now in the instrument configuration namespace

stixpy/stixpy/config/instrument.py

Lines 78 to 92 in fb3004c

STIX_INSTRUMENT = SimpleNamespace(

__doc__="""This namespace contains the instrument name, visibility information, sub-collimator ADC mapping, sub-collimator parameters, and pixel configuration.

The visibility information is obtained from the _get_uv_points_data function,

the sub-collimator ADC mapping is read from the detector ADC mapping file,

the sub-collimator parameters are read from the sub-collimator parameters file,

and the pixel configuration is read from the pixel parameters file.

The namespace is used to encapsulate all the configuration data related to the STIX instrument.

This allows for easy access and management of the instrument's configuration data.""",

name="STIX",

vis_info=_get_uv_points_data(),

subcol_adc_mapping=np.array(read_det_adc_mapping()["Adc #"]),

subcol_params=_SUBCOL_PARAMS,

pixel_config=read_pixel_params(),

)

Implements corrections and get_spectrum function

get_spectrum and bkg_subtract now work

samaloney · 2025-11-25T11:31:44Z

Add properties for livetime and livefrac or deadfrac

SRM now functional with wedge grid transmission

SRM fully functional

samaloney

Overall this looks great 🚀

Leaving all my comment but I know it still a WIP so some of them I'm sure you'd address anyway.

Few high level things are:

docstrings
test
dockets where it makes sense
needs a rebase/merge of current main

To me the key aspect is understanding want someone reading the code to know what's happening and why can link to paper / ref where applicable.

IDListis just because it done one way in IDL doesn't mean we have to or even should do the same - we need to support it for comparisons but they only to a point.

Also need to maintain some level of backwards compatibly with old transmission correction etc and also ideally support incremental correction e.g new transmission but no spectral slope correction.

As much of this as possible should be done it get_data we can create return a dictionary, dataclass etc. If we wanted to go hole hog get_data would return Spectrum, SpectrumCollection, or SpectrumSequences objects but that could be a later PR.

Also I think the u, v coords may need to be updated too with the new transmission?

samaloney · 2026-04-22T12:03:29Z

 import astropy.units as u
 import numpy as np
 from astropy.table import Table
+import xraydb


So we already use roentgen should either migrate to xraydb or keep using

samaloney · 2026-04-22T12:05:10Z

+    ds = 5e-3
+    dh = 5e-2


Magic numbers need at least comment and ideally units

samaloney · 2026-04-22T12:05:29Z

+    return subc_transm
+
+
+def stx_grid_transmission(pitch, slit, thickness, L):


Needs a docstring and test

samaloney · 2026-04-22T12:05:50Z

+
+    # ;; Transmission for a wedge shape model for grid imperfections
+    g0 = slit_rep / pitch_rep + (pitch_rep - slit_rep) / pitch_rep * np.exp( - H_rep / L_rep )
+    ttt = L_rep / dh * ( 1. - np.exp(- dh / L_rep ) )


samaloney · 2026-04-22T12:07:25Z



-def get_livetime_fraction(trigger_rate, *, eta=1.10 * u.us, tau=10.1 * u.us):
+def get_livetime_fraction(trigger_rate,*, eta=1.1e-6 *u.s, tau=10.1e-6 * u.s):


Just wonder what motivated this change?

change back

samaloney · 2026-04-22T12:10:56Z

+    try:
+        elut.offset = elut_table["Offset"].reshape(32, 12)
+        elut.gain = elut_table["Gain keV/ADC"].reshape(32, 12)
+    except KeyError:
+        try:
+            elut.offset = elut_table["Offset (ADC)"].reshape(32, 12)
+            elut.gain = elut_table["Gain (keV/ADC)"].reshape(32, 12)
+
+        except KeyError:
+            elut.offset = elut_table["Offset (ADC)"].reshape(32, 12)
+            elut.gain = elut_table["Gain (ADC/keV)"].reshape(32, 12)
+
+


This is already fix on main

stixpy/stixpy/io/readers.py

Lines 136 to 140 in 4f4e0c7

elut = type("ELUT", (object,), dict())

elut.file = elut_file.name

offset_col = [col for col in elut_table.colnames if "offset" in col.casefold()][0]

gain_col = [col for col in elut_table.colnames if "gain" in col.casefold()][0]

elut.offset = elut_table[offset_col].reshape(32, 12)

samaloney · 2026-04-22T12:13:35Z

+        return counts_corr, counts_err, times, t_norm, livefrac, energies, elut_cor_fac
+
+
+    def get_spectrum(self, bkg_prod=None):


So ideally this should be handled by get_data e.g.

get_data(detector_indices=[[0,31]], pixel_indices=[[0,8]], elut=True, live=True, ...)

samaloney · 2026-04-22T12:19:54Z

+
+# NEED TO ADD IN THE DIST AND ANGLE READING AND BKG_SUBTRACT, GETTING THERE THOUGH!!!
+
+    def bkg_subtract(self, bkg_data):


I think this might be better as a plain function, e.g. not a method on the object.

samaloney · 2026-04-22T12:21:02Z

+
+        return spec_sub
+
+    def get_masked_srm(self,flare_location):


Need to support flat spectral assumption and slope septette the arm sliceing and other stuff

codecov · 2026-04-29T08:03:08Z

Codecov Report

❌ Patch coverage is 3.58423% with 538 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.19%. Comparing base (9778770) to head (adda454).

Files with missing lines	Patch %	Lines
stixpy/product/sources/science.py	0.55%	358 Missing ⚠️
stixpy/coordinates/flare_location.py	2.46%	79 Missing ⚠️
stixpy/calibration/grid.py	2.08%	47 Missing ⚠️
stixpy/calibration/elut.py	15.90%	37 Missing ⚠️
stixpy/coordinates/flare_angle.py	33.33%	16 Missing ⚠️
stixpy/calibration/detector.py	0.00%	1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (9778770) and HEAD (adda454). Click for more details.

HEAD has 19 uploads less than BASE

Flag BASE (9778770) HEAD (adda454)

20 1

Additional details and impacted files

@@             Coverage Diff             @@
##             main     #192       +/-   ##
===========================================
- Coverage   85.29%   50.19%   -35.11%     
===========================================
  Files          50       53        +3     
  Lines        3175     3676      +501     
===========================================
- Hits         2708     1845      -863     
- Misses        467     1831     +1364

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Everything has been moved onto get_data()

rh2082 · 2026-05-12T11:03:42Z

Hi, is there a particular reason why 6-10keV is chosen as the energy range in estimate_flare_location?

Flare location added from Laura's flarelist and slightly modified, flare angle calculator also added, background subration done for each pixel and detector without collapsing

jajmitchell added 3 commits September 17, 2025 15:21

Adds livetime correction to get_data

5fc327e

Adds livetime correction to get_data

Adds detector and time masks

d739718

Adds detector and time masks

Merge branch 'main' into adds_livetime_correction_to_get_data

4645db0

samaloney requested changes Oct 3, 2025

View reviewed changes

jajmitchell added 2 commits October 24, 2025 16:56

Implements corrections and get_spectrum function

677220e

Implements corrections and get_spectrum function

get_spectrum and bkg_subtract now work

bb73b37

get_spectrum and bkg_subtract now work

samaloney added this to the v0.4.0 milestone Nov 25, 2025

samaloney modified the milestones: v0.4.0, v0.3.0 Nov 25, 2025

jajmitchell added 12 commits December 11, 2025 17:58

SRM now functional with wedge grid transmission

f8a8890

SRM now functional with wedge grid transmission

SRM fully functional

10e8dd3

SRM fully functional

adds spec_obj

e11a2c4

Mostly working

8fb1a27

Livetime works!!!!

81c8544

bkg_sub works

dbb9edd

errors work

79ff5f7

works

150e10d

bkg sub finally works!

9d2f182

Adds systematic errors

e3ea8dd

Adds time range to meta in spec object

826c56f

removes print statements

18493f7

samaloney reviewed Apr 22, 2026

View reviewed changes

Merge branch 'main' into adds_livetime_correction_to_get_data

c20582c

New structure, counts match old version

f8e263b

Everything has been moved onto get_data()

jajmitchell added 3 commits May 13, 2026 18:34

Multiple features added

c1d410a

Flare location added from Laura's flarelist and slightly modified, flare angle calculator also added, background subration done for each pixel and detector without collapsing

Functions with data expanded

3500c7c

Uncertainties working

7643b08

remove print

adda454

	STIX_INSTRUMENT = SimpleNamespace(
	__doc__="""This namespace contains the instrument name, visibility information, sub-collimator ADC mapping, sub-collimator parameters, and pixel configuration.

	The visibility information is obtained from the _get_uv_points_data function,
	the sub-collimator ADC mapping is read from the detector ADC mapping file,
	the sub-collimator parameters are read from the sub-collimator parameters file,
	and the pixel configuration is read from the pixel parameters file.
	The namespace is used to encapsulate all the configuration data related to the STIX instrument.
	This allows for easy access and management of the instrument's configuration data.""",
	name="STIX",
	vis_info=_get_uv_points_data(),
	subcol_adc_mapping=np.array(read_det_adc_mapping()["Adc #"]),
	subcol_params=_SUBCOL_PARAMS,
	pixel_config=read_pixel_params(),
	)

		return subc_transm


		def stx_grid_transmission(pitch, slit, thickness, L):



		def get_livetime_fraction(trigger_rate, , eta=1.10 u.us, tau=10.1 * u.us):
		def get_livetime_fraction(trigger_rate,, eta=1.1e-6 u.s, tau=10.1e-6 * u.s):

	elut = type("ELUT", (object,), dict())
	elut.file = elut_file.name
	offset_col = [col for col in elut_table.colnames if "offset" in col.casefold()][0]
	gain_col = [col for col in elut_table.colnames if "gain" in col.casefold()][0]
	elut.offset = elut_table[offset_col].reshape(32, 12)

		return counts_corr, counts_err, times, t_norm, livefrac, energies, elut_cor_fac


		def get_spectrum(self, bkg_prod=None):


		# NEED TO ADD IN THE DIST AND ANGLE READING AND BKG_SUBTRACT, GETTING THERE THOUGH!!!

		def bkg_subtract(self, bkg_data):

Conversation

jajmitchell commented Sep 17, 2025

Uh oh!

samaloney left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

samaloney commented Nov 25, 2025

Uh oh!

samaloney left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Apr 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

rh2082 commented May 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

samaloney left a comment •

edited

Loading

codecov Bot commented Apr 29, 2026 •

edited

Loading