"README.md" did not exist on "6f517f805b0544a48ebbaf3210e67da8aaeaa901"
Fix: 2.8.0 hp1
- Add validation of the data file using extra_data_validate
- Fix summary section of the AGIPD correction notebook
- Fix check gain_stats
- Fix path to the pdf in the serve_overview
Fix of the AGIPD pulse is still missing.
Edited by Mikhail Karnevskiy
Merge request reports
Activity
Filter activity
24 24 "source": [ 25 25 "in_folder = \"/gpfs/exfel/exp/MID/201931/p900107/raw\" # the folder to read data from, required\n", 26 26 "run = 11 # runs to process, required\n", 27 "out_folder = \"/gpfs/exfel/data/scratch/ahmedk/test/AGIPD_Corr\" # the folder to output to, required\n", 27 "out_folder = \"/gpfs/exfel/data/scratch/karnem/test/AGIPD_Corr\" # the folder to output to, required\n", 28 28 "calfile = \"/gpfs/exfel/data/scratch/haufs/agipd_on_demand/agipd_store_mid.h5\" # path to calibration file. Leave empty if all data should come from DB\n", 29 "sequences = [-1] # sequences to correct, set to -1 for all, range allowed\n", 29 "sequences = [0] # sequences to correct, set to -1 for all, range allowed\n", changed this line in version 4 of the diff
86 "\n", 87 " for i in range(1, counts.shape[0]):\n", 87 " for i in range(0, counts.shape[0]):\n", 88 88 " line = [ const_times[:,:,pos][idx==i][0] ]\n", 89 89 " mods = ''\n", 90 90 " for i_s, s in enumerate(seq):\n", 91 91 " if( const_times[i_s,:,0][idx[i_s]==i].shape[0]>0 ):\n", 92 " table[0][1] = 'Most of the modules'\n", 93 92 " mods = mods+ 'S{}: {}, '.format(seq[i_s], const_times[i_s,:,0][idx[i_s]==i])\n", 94 93 " line.append(mods)\n", 95 94 " table.append(line)\n", 96 95 "\n", 96 " if counts.shape[0] == 1:\n", 97 " table[0][1] = 'All modules'\n", 98 " else:\n", 99 " table[np.argmax(counts)][1] = 'Most of the modules'\n", changed this line in version 3 of the diff
Add validation of the data file using extra_data_validate
@karnem can you point me to where this is implemented, I could not find it
This was removed. We didn't decide if this should be implemented. See here: https://git.xfel.eu/gitlab/detectors/pycalibration/blob/20f77f30693177bf00df1c257aafecbd50c9141c/notebooks/AGIPD/AGIPD_Correct_and_Verify.ipynb
function
def validate_file(filename)Edited by Mikhail Karnevskiy-
OK, thanks.
I see that you spawn a new process to run the command line version. It is actually possible to use directly the python API, see as example:
from extra_data import RunDirectory, FileAccess from extra_data.validation import FileValidator, RunValidator, ValidationError # on a single file try: FileValidator(FileAccess(filepath)).validate() except ValidationError as verr: # verr contains all validation errors for this file # on a run directory try: RunValidator(runpath).validate() except ValidationError as verr: # verr contains all validation errors for this run # Alternative validator = FileValidator(FileAccess(filepath)) problems = validator.run_checks() # problems (list) contains all validation errors for this file validator = RunValidator(runpath) problems = validator.run_checks() # problems contains all validation errors for this run # run_check returns a list of dict containing problems associated files and additional infos # for nice print: print(str(ValidationError(problems)))
added 2 commits
Please register or sign in to reply