Fix idr0106 to download from ftp instead of github

[will-moore]

For all "github" studies, where the companion.ome file or .pattern file is on github and the rest of the files are available on ftp, we have made sure that the companion.ome file or .pattern file has the correct download url, but the tiff files do NOT get the correct URL.
This is probably the wrong way to go, as most users probably care more about downloading the tiff files that the pattern/companion file.

For some studies (idr0075, idr0079, idr0106) the client_paths in the Filesets are consistent with respect to download URLs on ftp. So we can get a download URL via simple replace.

To test, we can open the tsv from this PR and check those rows with:
https://proxy-idr-ap1.dundee.ac.uk/download_urls/?tsv=https://raw.githubusercontent.com/will-moore/idr.openmicroscopy.org/refs/heads/download_urls_github_ftp/_data/download_urls.tsv

We can also use the tsv for individual images. E.g. https://proxy-idr-ap1.dundee.ac.uk/webclient/img_detail/12689244/?check_links=true&tsv=https://raw.githubusercontent.com/will-moore/idr.openmicroscopy.org/refs/heads/download_urls_github_ftp/_data/download_urls.tsv - all the tif links are now "green", but companion.ome is not.

Other studies are more complex:
Known issues (GitHub studies not fixed by this PR):

• idr0163 - All data is on BIA, only csvs are at https://ftp.ebi.ac.uk/pub/databases/IDR/idr0163-greenwald-tnbc/
• idr0100 - Not possible to resolve with simple replace. e.g. https://github.com/IDR/idr0100-capar-myelin/blob/master/pattern/Sample1_C1.tif points to /uod/idr/filesets/idr0100-capar-myelin/20201022-ftp/experimentA/Sample_Myelin_Images1/Sample1_Axon.tif
• idr0099 - data/idr-metadata/idr0099-jain-beetlelightsheet/experimentA/20210208-patterns/9-3-15_Histone-eGFP/TP_T0_Chgreen_Ill0_Ang0,1,2.tif to https://ftp.ebi.ac.uk/pub/databases/IDR/idr0099-jain-beetlelightsheet/20201001-ftp/Akanksha_Jain_9-3-15_Histone-eGFP/TP0_Chgreen_Ill0_Ang0,1,2.tif BUT also data/idr-metadata/idr0099-jain-beetlelightsheet/experimentA/20210128-companions/22-06-16_Tc-Squash-eGFP/TP0_Ch0_Ill0_Ang1,2,3,4,5.tif to https://ftp.ebi.ac.uk/pub/databases/IDR/idr0099-jain-beetlelightsheet/20210128-ftp/TP0_Ch0_Ill0_Ang1,2,3,4,5.tif - NO simple replace, - same image: data/idr-metadata/idr0099-jain-beetlelightsheet/experimentA/20210128-companions/22-06-16_Tc-Squash-eGFP/TP212_Ch0_Ill0_Ang1,2,3,4,5.tif to https://ftp.ebi.ac.uk/pub/databases/IDR/idr0099-jain-beetlelightsheet/20201001-ftp/Akanksha_Jain_22-06-16_Tc-Squash-eGFP/TP212_Ch0_Ill0_Ang1,2,3,4,5.tif NB: https://ftp.ebi.ac.uk/pub/databases/IDR/idr0099-jain-beetlelightsheet/20210128-ftp/ only has 11 tiffs, all the rest are under https://ftp.ebi.ac.uk/pub/databases/IDR/idr0099-jain-beetlelightsheet/20201001-ftp/
• idr0065 (GitHub) uod/idr/metadata/idr0065-camsund-crispri/experimentA/companions/subpool-1_run-1_EXP-19-BQ3550/pheno/Pos101/phase/img_000000000.tiff - Too tricky to work this out!! https://ftp.ebi.ac.uk/pub/databases/IDR/idr0065-camsund-crispri/ has 3 ftp uploads (not in client_paths)
• idr0052 uod/idr/metadata/idr0052-walther-condensinmap/experimentA/companions/160719_NCAPD2gfpc272c78_MitoSys2/conc/cell0005_R0001/TR1_2_W0001_P0001_T0001.tif doesn't match https://ftp.ebi.ac.uk/pub/databases/IDR/idr0052-walther-condensinmap/20181113-ftp/MitoSys/160719_NCAPD2gfpc272c78_MitoSys2/cell0005_R0001/conctif/TR1_2_W0001_P0001_T0001.tif

[will-moore]

Test:

• idr0052 - failed! uod/idr/metadata/idr0052-walther-condensinmap/experimentA/companions/160719_NCAPD2gfpc272c78_MitoSys2/conc/cell0005_R0001/TR1_2_W0001_P0001_T0001.tif doesn't match https://ftp.ebi.ac.uk/pub/databases/IDR/idr0052-walther-condensinmap/20181113-ftp/MitoSys/160719_NCAPD2gfpc272c78_MitoSys2/cell0005_R0001/conctif/TR1_2_W0001_P0001_T0001.tif
• idr0075 ok
• idr0079 ok
• idr0106 ok

Will revert idr0052 change...

[pwalczysko]

What is needed here please ? Testing ? How ?

[will-moore]

Testing, yes please, using ?tsv=https://raw.githubusercontent.com/will-moore/idr.openmicroscopy.org/refs/heads/download_urls_github_ftp/_data/download_urls.tsv as in the first 2 links in the description to modify the download_urls page and individual pages to use the tsv from this PR.

On the download_urls page, check rows for idr0075, idr0079, idr0106.
Then try opening various images from those studies in iviewer, also appending the ?tsv=... to the URL once you've opened the viewer / download page.
E.g. idr106 image at https://proxy-idr-ap1.dundee.ac.uk/webclient/img_detail/12689244/?check_links=true
This is with the &check_links=true to get auto checking of the links.
That page looks like all the links are valid (green) but if you try clicking on a tif link, you'll just get github symlink.

By using the tsv from this PR: https://proxy-idr-ap1.dundee.ac.uk/webclient/img_detail/12689244/?check_links=true&tsv=https://raw.githubusercontent.com/will-moore/idr.openmicroscopy.org/refs/heads/download_urls_github_ftp/_data/download_urls.tsv all the tiff links are correct and will download the tif for you.

Unfortunately, only those 3 studies were amenable to this approach. Others are too unpredictable.