Skip to content

Add a retry on failed media uploads #35

Description

@joecorall

For large uploads it's possible a media upload fails. The log would look like

28-Dec-24 03:45:55 - ERROR - File not created for "/mnt/islandora_staging/Medieval/Antiphon_24/Antiphon_24_180.tif", POST request to "/file/upload/media/file/field_media_file?_format=json" returned an HTTP status code of "504" and a response body of b'<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n'.
28-Dec-24 03:45:56 - ERROR - Media not created, POST request to "/entity/media" returned an HTTP status code of "422" and a response body of b'{"message":"Unprocessable Entity: validation failed.\\nfield_media_file.0.target_id: This value should be of the correct primitive type.\\n"}'.
28-Dec-24 03:45:56 - ERROR - JSON request body used in previous POST to "/entity/media" was {'bundle': [{'target_id': 'file', 'target_type': 'media_type'}], 'name': [{'value': 'Antiphon_24_180.tif'}], 'field_media_file': [{'target_id': False, 'target_type': 'file'}], 'field_media_of': [{'target_id': 436648, 'target_type': 'node'}], 'field_media_use': [{'target_id': 16, 'target_type': 'taxonomy_term'}], 'status': {'value': '1'}}.
28-Dec-24 03:45:56 - ERROR - Media for /mnt/islandora_staging/Medieval/Antiphon_24/Antiphon_24_180.tif not created (HTTP respone code 422).
28-Dec-24 05:29:41 - ERROR - File not created for "/mnt/islandora_staging/Medieval/Antiphon_24/Antiphon_24_292.tif", POST request to "/file/upload/media/file/field_media_file?_format=json" returned an HTTP status code of "504" and a response body of b'<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n'.
28-Dec-24 05:29:43 - ERROR - Media not created, POST request to "/entity/media" returned an HTTP status code of "422" and a response body of b'{"message":"Unprocessable Entity: validation failed.\\nfield_media_file.0.target_id: This value should be of the correct primitive type.\\n"}'.
28-Dec-24 05:29:43 - ERROR - JSON request body used in previous POST to "/entity/media" was {'bundle': [{'target_id': 'file', 'target_type': 'media_type'}], 'name': [{'value': 'Antiphon_24_292.tif'}], 'field_media_file': [{'target_id': False, 'target_type': 'file'}], 'field_media_of': [{'target_id': 436760, 'target_type': 'node'}], 'field_media_use': [{'target_id': 16, 'target_type': 'taxonomy_term'}], 'status': {'value': '1'}}.
28-Dec-24 05:29:43 - ERROR - Media for /mnt/islandora_staging/Medieval/Antiphon_24/Antiphon_24_292.tif not created (HTTP respone code 422).
28-Dec-24 07:58:30 - ERROR - File not created for "/mnt/islandora_staging/Medieval/Antiphon_25/Antiphon_25_068.tif", POST request to "/file/upload/media/file/field_media_file?_format=json" returned an HTTP status code of "504" and a response body of b'<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n'.
28-Dec-24 07:58:31 - ERROR - Media not created, POST request to "/entity/media" returned an HTTP status code of "422" and a response body of b'{"message":"Unprocessable Entity: validation failed.\\nfield_media_file.0.target_id: This value should be of the correct primitive type.\\n"}'.
28-Dec-24 07:58:31 - ERROR - JSON request body used in previous POST to "/entity/media" was {'bundle': [{'target_id': 'file', 'target_type': 'media_type'}], 'name': [{'value': 'Antiphon_25_068.tif'}], 'field_media_file': [{'target_id': False, 'target_type': 'file'}], 'field_media_of': [{'target_id': 436993, 'target_type': 'node'}], 'field_media_use': [{'target_id': 16, 'target_type': 'taxonomy_term'}], 'status': {'value': '1'}}.
28-Dec-24 07:58:31 - ERROR - Media for /mnt/islandora_staging/Medieval/Antiphon_25/Antiphon_25_068.tif not created (HTTP respone code 422).

We should be able to parse that log and retry using a add_media workbench task. We'll just need to ensure:

  1. The only failures in the original job were media upload timeouts
  2. The add_media retry succeeds

If 1 and 2 pass, the job should be marked completed successfully.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions