Fix/get dir creation date
3 unresolved threads
3 unresolved threads
Add transparency to get_dir_creation_date()
nested methods and unify returned creation date to UTC
Description
This MR started by updating the returned creation_time from myMDC to UTC
and ended with splitting the functions for get_dir_creation_date
to different functions and adding transparent prints on which method is used at the end to derive the creation date for a run directory.
- Unify creation date to UTC
- add a separate function to get creation date from myMDC
- add a separate function to get creation date from METADATA/creationDate using Extra-data
- add tests for both functions and fix old test after unify datetime timezone.
- add a fixture to avoid running tests with myMDC connection. as it seems to be the current case at the moment for the CI.
How Has This Been Tested?
By running pytest tests/test_cal_tools locally.
Relevant Documents (optional)
Types of changes
- Bug fix (non-breaking change which fixes an issue)
- Refactor (refactoring code with no functionality changes)
- Test (additional or refactored tests)
Checklist:
- My code follows the code style of this project.
- I added tests where appropriate.
Reviewers
Edited by Karim Ahmed
Merge request reports
Activity
Filter activity
added 1 commit
- 5c8787b8 - split functions used by get_dir_creation_date and add tests for them
- Resolved by Karim Ahmed
338 raise FileNotFoundError( 339 "- Failed to read creation time, wrong input folder", 340 directory) from e 295 341 296 342 try: 297 run_info = get_run_info(proposal, run) 298 return dateutil.parser.parse(run_info['begin_at']) 343 return creation_date_metadata_client(proposal, run) 299 344 except Exception as e: 300 345 if verbosity > 0: 301 346 print(e) 302 347 303 directory = directory / 'r{:04d}'.format(run) 304 305 # Loop a number of times to catch stale file handle errors, due to 306 # migration or gpfs sync. I removed this, as this is handled through the webservice.
Edited by Karim Ahmed
88 @pytest.mark.requires_mdc 89 def test_creation_date_metadata_client(): 90 91 date = creation_date_metadata_client(900113, 9983) 92 assert isinstance(date, datetime) 93 assert str(date) == '2020-09-23 13:30:00+00:00' 94 95 96 @pytest.mark.requires_gpfs 97 def test_creation_date_file_metadata(): 98 99 folder = '/gpfs/exfel/exp/CALLAB/202031/p900113/raw/r9983' 100 101 date = creation_date_file_metadata(RunDirectory(folder)) 72 102 assert isinstance(date, datetime) 73 assert str(date) == '2019-12-16 08:52:25.196603' added 14 commits
-
d1dd4c3a...0d6324d8 - 11 commits from branch
master
- f9e750dc - first draft for fixing get_dir_creation_date
- ecdca549 - split functions used by get_dir_creation_date and add tests for them
- 8f064ffb - flake8
Toggle commit list-
d1dd4c3a...0d6324d8 - 11 commits from branch
- Resolved by Karim Ahmed
- Resolved by Karim Ahmed
- Resolved by Karim Ahmed
changed milestone to %3.5.0
removed milestone %3.5.0
changed milestone to %3.5.1
added Waiting for review label
- Resolved by Karim Ahmed
- Resolved by Karim Ahmed
removed Waiting for review label
added 1 commit
- d7fb02a1 - add a comment on parsing file not found error.
added 59 commits
-
d7fb02a1...94444be8 - 54 commits from branch
master
- 8ef805e1 - first draft for fixing get_dir_creation_date
- 5f4dc8ec - split functions used by get_dir_creation_date and add tests for them
- 99df6d18 - flake8
- c6b4a499 - avoid duplication in test_cal_tools
- bf21d241 - add a comment on parsing file not found error.
Toggle commit list-
d7fb02a1...94444be8 - 54 commits from branch
mentioned in commit ea1b0f06
Please register or sign in to reply