In a recent news update, I noticed that Contra Costa county had an entry where the title started with a zero-width space (\u200b) (sfbrigade/stop-covid19-sfbayarea#392 (comment)).
We already strip whitespace around titles and summaries in most news feeds, but it turns out string.strip() doesn’t handle some of the more complex whitespace like \u200b. We should probably have a better text stripping function that does this for some of the more complex whitespace characters (e.g. \u2000 - \u200b).
In a recent news update, I noticed that Contra Costa county had an entry where the title started with a zero-width space (
\u200b) (sfbrigade/stop-covid19-sfbayarea#392 (comment)).We already strip whitespace around titles and summaries in most news feeds, but it turns out
string.strip()doesn’t handle some of the more complex whitespace like\u200b. We should probably have a better text stripping function that does this for some of the more complex whitespace characters (e.g.\u2000-\u200b).