read/write support for csv, eventtxt and csz #3285
Conversation
trichter
added
the
build_docs
|
Doc link. Some functions are mising its doc string. |
|
Do we need an |
trichter
added
the
ready for review
Do we have another 'test_csv.py' anywhere? If so, that explains it. Pytest's test importer is a bit wonky. Maybe try giving your test file a more unique name? |
Doesn't look like it. |
|
Ah and needs to be added here: misc/docs/source/packages/index.rst |
| try: | ||
| import zlib # noqa: F401 | ||
| except ImportError: | ||
| pass |
There was a problem hiding this comment.
consider suppress from context lib here to make it more concise and clear.
There was a problem hiding this comment.
I did not change anything here for now
| with _open(fname) as f: | ||
| for _ in range(skipheader): | ||
| f.readline() | ||
| if names is not None: | ||
| kwargs.setdefault('fieldnames', _names_sequence(names)) | ||
| reader = csv.DictReader(f, **kwargs) | ||
| for row in reader: | ||
| if 'time' in row: | ||
| time = UTC(row['time']) | ||
| else: | ||
| time = UTC( | ||
| '{year}-{mon}-{day} {hour}:{minu}:{sec}'.format(**row)) | ||
| try: | ||
| if 'depm' in row: | ||
| dep = float(row['depm']) | ||
| else: | ||
| dep = float(row['dep']) * 1000 | ||
| if math.isnan(dep): | ||
| raise | ||
| except Exception: | ||
| dep = None | ||
| author = _string(row, 'author') | ||
| contrib = _string(row, 'contrib') | ||
| if author is not None or contrib is not None: | ||
| info = evmod.CreationInfo(author=author, agency_id=contrib) | ||
| else: | ||
| info = None | ||
| origin = evmod.Origin( | ||
| time=time, | ||
| latitude=row['lat'], | ||
| longitude=row['lon'], | ||
| depth=dep, | ||
| creation_info=info | ||
| ) | ||
| try: | ||
| # add zero to eliminate negative zeros in magnitudes | ||
| mag = float(row['mag']) + 0 | ||
| if math.isnan(mag): | ||
| raise | ||
| except Exception: | ||
| magnitudes = [] | ||
| else: | ||
| try: | ||
| magtype = row['magtype'] | ||
| if magtype.lower() in ('', 'none', 'null', 'nan'): | ||
| raise | ||
| except Exception: | ||
| magtype = default.get('magtype') | ||
| magauthor = _string(row, 'magauthor') | ||
| info = (evmod.CreationInfo(author=magauthor) if magauthor | ||
| else None) | ||
| magnitudes = [evmod.Magnitude( | ||
| mag=mag, magnitude_type=magtype, creation_info=info)] | ||
| try: | ||
| id_ = evmod.ResourceIdentifier(row['id'].strip()) | ||
| except Exception: | ||
| id_ = None | ||
| region = _string(row, 'region') | ||
| descs = ([evmod.EventDescription(region, 'region name')] | ||
| if region else []) | ||
| event = evmod.Event( | ||
| magnitudes=magnitudes, | ||
| origins=[origin], | ||
| resource_id=id_, | ||
| event_descriptions=descs | ||
| ) | ||
| events.append(event) | ||
| if format_check: | ||
| return True |
There was a problem hiding this comment.
Certainly optional, but this is a bit long. Maybe can be refactored into separate helper functions for reading each part?
| try: | ||
| author = origin.creation_info.author | ||
| except AttributeError: | ||
| author = '' | ||
| try: | ||
| contrib = origin.creation_info.agency_id | ||
| except AttributeError: | ||
| contrib = '' |
There was a problem hiding this comment.
These cant be re-phrased as getattr with default values because both creation_info and creation_info's sub attribute might be missing right?
|
Thanks for the thorough review @d-chambers, definitely some very valid points there 😃 |
megies
dismissed
their stale review
better review showing need for some changes
|
👍 looks good to me. |
What does this PR do?
Id adds read and write support for csv, eventtxt and csz. csz is a custom format and basically a collection of csv files bundled in a zip file (similar to numpys npz format) for io of a catalog with picks.
Why was it initiated? Any relevant Issues?
Fixes #1467
PR Checklist
masterfor new features,maintenance_...for bug fixesno_cilabel can be added to skip CI buildsJust add the
build_docstag to this PR.Docs will be served at docs.obspy.org/pr/{branch_name} (do not use master branch).
Please post a link to the relevant piece of documentation.
clients.fdsn) should be tested for the PR,just add the
test_networktag to this PR.CHANGELOG.txt.CONTRIBUTORS.txt.from all the CI builds look correct. Add the "upload_plots" tag so that plotting
outputs are attached as artifacts.
CODEOWNERSwith your github handleready for reviewlabel when you are ready for the PR to be reviewed.