Draft: fix[Gotthard2][Dark]: Avoid trains with only 0. data
Description
@mramilli reported that the bad pixels looked a bit weird with all pixels assigned WRONG_GAIN_VALUE for medium or low gain stages.
This was a reason for the presence of some trains in the corresponding runs for these gains which had no data. A reason for this seems to be missing packets.
The fix is to exclude these trains from the dark constants processing.
I have also combined both functions into one functions that process all dark constants.
How Has This Been Tested?
Relevant Documents (optional)
Types of changes
- Bug fix (non-breaking change which fixes an issue)
Checklist:
Reviewers
Merge request reports
Activity
added 1 commit
- bfdfd542 - fix[Gotthard2][Dark]: Avoid trains with only 0. data
added 1 commit
- 728c8dc2 - fix: complete change. last MR was for testing only
added Test CAL Data Affected label
there are occasional frames which are full of zeros both in the ADC and gain bit data, which, for some reason, still hold a valid train ID. My suspect is that these are actually missing packets. Whatever the origin of these occasional frames, they mess with the wrong gain bit evaluation: apparently one of these "empty frames" is currently enough trigger the
WRONG_GAIN_BITlabel for all the strips.
378 378 " context.map(convert_train, dc)\n", 379 379 " step_timer.done_step(\"convert to 10bit\")\n", 380 380 "\n", 381 " def dark_constant_cell_processing(wid, index, gain_and_data):\n", 382 " g, d = gain_and_data\n", 383 "\n", 384 " # Filter out trains with no data. (e.g. missing packets)\n", 385 " nonzero_mask = np.all(d != 0, axis=(1, 2))\n", changed this line in version 4 of the diff
404 415 " data_gain = dc[src, \"data.gain\"].ndarray()\n", 405 416 " even_gain = data_gain[:, 20::2, :]\n", 406 417 " odd_gain = data_gain[:, 21::2, :]\n", 407 " raw_g = 3 if gain == 2 else gain\n", 408 "\n", 409 " def badpixels_cell(wid, index, g):\n", 410 " \"\"\"Check if there are wrong bad gain values.\n", 411 " Indicate pixels with wrong gain value across all trains for each cell.\"\"\"\n", 412 " badpixels_map[mod][\n", 413 " np.mean(g, axis=(0, 1)) != raw_g, index, gain\n", 414 " ] |= BadPixels.WRONG_GAIN_VALUE.value\n", 415 "\n", 416 " context.map(badpixels_cell, (even_gain, odd_gain))\n", 417 418 "\n", 419 " # Calculate Offset, Noise Maps, and Bad Pixels in one go\n", 420 " context.map(dark_constant_cell_processing, ((even_gain, even_data), (odd_gain, odd_data)))\n", changed this line in version 4 of the diff
- Resolved by Karim Ahmed
Sorry for not responding to the comments yet. I need to discuss something later in the cal standup related to this.
Edited by Karim Ahmed - Last reply by Karim Ahmed
added 1 commit
- 6ff5f1d2 - fix: add some fixes after confirming the source of issue
If this is really connected to the connectivity issue in https://redmine.xfel.eu/issues/178311, I'm no longer sure this is worth it.
This is an extra step in processing which would of course be better (performance wise) if there is no need for such validation.
Shall I close it or we leave it hanging there until we are 100% sure we don't need it?
Edited by Karim Ahmed
