Repository navigation
Address various warnings - #391
Conversation
| warnings.filterwarnings( | ||
| 'ignore', | ||
| message='Importing of PHRED 64 data is slow.*', | ||
| category=UserWarning | ||
| ) |
There was a problem hiding this comment.
Should we update the data format being used by the test instead?
There was a problem hiding this comment.
The transformers are testing transformations between the two data formats, so if we update the format, we wouldn't be testing the transformers that raise that warning.
| # casting of columns adapted from SO post: | ||
| # https://stackoverflow.com/a/36814203/3424666 | ||
| cols = df.columns | ||
| df[cols] = df[cols].apply(pd.to_numeric, errors='ignore') | ||
| for col in cols: | ||
| try: | ||
| df[col] = df[col].apply(pd.to_numeric) | ||
| except ValueError: | ||
| pass | ||
| return df |
There was a problem hiding this comment.
Lovely. Why would they do this to us...
I don't really understand why we were pulling the columns off the DataFrame in the first place. The SO post shows doing this when you want to drop specific columns and not apply to them, but we aren't doing that.
It probably should've just looked like this before:
df = df.apply(pd.to_numeric, errors='ignore')
Now that we can't ignore errors for god knows what reason, probably something like this:
def _to_numeric(col):
try:
return pd.to_numeric(col)
except ValueError:
return col
df = df.apply(_to_numeric)
The way it works currently in this PR, we're iterating over the columns then calling apply on each column. If we are going to use a for loop we don't need the apply:
for col in cols:
try:
df[col] = pd.to_numeric(df[col])
except ValueError:
pass
it is generally considered preferable to use .apply instead of your own for loop (and it's generally considered better to do vectorized operations over .apply, but I don't think we can do that here).
There was a problem hiding this comment.
GitHub is being VERY annoying about letting me actually suggest the diff I want here, but basically, delete lines 25-32 and replace them with:
def _to_numeric(col):
try:
return pd.to_numeric(col)
except ValueError:
return col
df = df.apply(_to_numeric)
Removed warnings regarding pandas deprecations and Phred 33-encoded data.
AI disclaimer
No AI used.