XAS somewhat slow
Our XAS routine might be quite slow. It was not originally written to work with xarray and was adapted to it while we had no experience with xarray. I ran some profiling and found out:
-
line 160 of XAS.py takes 93% of the computation time. The line reads:
stacked = nrun.stack(dummy_=['trainId', 'sa3_pId'])
-
next is line 201 of XAS.py which takes the remaining 7%. The line reads:
dummy, nosample = binning(rundata['nrj'], rundata, whichIo, bins)
so the actual binning takes only 7% of the total time. The stacking of dimension is necessary as we usually want to bin fast data (with pulseId and trainId dimension) like DSSC data and slow data (only trainId dimension) like the monochromator photon energy for example.
The reason why .stack
is slow is apparently related to https://github.com/pydata/xarray/issues/5202
Essentially, .stack
creates a new index such that the data could be unstacked. This index creation is both time consuming, memory expansive and unnecessary since we are not interested in unstacking the data.
To be followed.