Corrections diverses. by Daniel-KM · Pull Request #11 · smachefert/Omeka-S-module-ExtractOcr

Daniel-KM · 2026-02-26T09:07:56Z

No description provided.

… forme valeur1-valeur2

Copilot

Pull request overview

This PR updates the ExtractOcr Omeka S module with multiple extraction/packaging improvements, including new TSV output variants, better job behavior, and an upgrade path for existing installs.

Changes:

Add a second TSV export mode (full TSV vs grouped-by-word) and update extraction/storage logic accordingly.
Introduce an upgrade script to migrate filenames/settings and bump module metadata (version + constraints).
Add composer metadata, update translations/docs, and improve operational characteristics (memory clearing, CLI tool checks).

Reviewed changes

Copilot reviewed 10 out of 13 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
`src/Job/ExtractOcr.php`	Adds new TSV format handling, content-only mode, CLI checks, and memory management improvements.
`src/Form/ConfigForm.php`	Updates admin config options to expose the new TSV formats and documentation link.
`Module.php`	Updates install checks/messages, adds deferred job dispatch and EasyAdmin integration, extends format handling.
`data/scripts/upgrade.php`	New upgrade script to migrate settings and rename derivative files/DB sources.
`config/module.ini`	Bumps module version and updates Omeka version constraint.
`config/module.config.php`	Adjusts translator loader config and defaults (notably content property default).
`README.md`	Expands documentation to explain formats and recommended usage.
`language/template.pot`	Regenerates POT with new/updated strings.
`language/fr.po`	Updates French translations for new/changed strings.
`language/fr.mo`	Adds compiled French translation binary.
`composer.json` / `composer.lock`	Adds Composer package metadata for the module.
`.gitignore`	Expands ignored paths (vendor/node/phpunit/etc.).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-21T09:58:50Z

        $extensions = [
            \ExtractOcr\Job\ExtractOcr::FORMAT_ALTO => 'alto.xml',
            \ExtractOcr\Job\ExtractOcr::FORMAT_PDF2XML => 'xml',
            \ExtractOcr\Job\ExtractOcr::FORMAT_TSV => 'tsv',
+            \ExtractOcr\Job\ExtractOcr::FORMAT_TSV_BY_WORD => 'tsv',
        ];
        $settings = $services->get('Omeka\Settings');
        $targetTypesFiles = $settings->get('extractocr_types_files') ?: [];
        $targetTypesFiles = array_intersect($targetTypesFiles, array_flip($extensions));
        $targetTypesMedia = $settings->get('extractocr_types_media') ?: [];
        $targetTypesMedia = array_intersect($targetTypesMedia, array_flip($extensions));
-        if (!$targetTypesFiles && !$targetTypesMedia ) {
+        $targetContentStore = $settings->get('extractocr_content_store') ?: [];
+        $targetContentStore = array_intersect($targetContentStore, ['item', 'media_pdf', 'media_extracted']);
+        if (!$targetTypesFiles && !$targetTypesMedia && !$targetContentStore) {
            return;
        }


In extractOcr(), the filtering of configured formats is broken: array_intersect($targetTypesFiles, array_flip($extensions)) compares selected mime types against extension strings, so $targetTypesFiles/$targetTypesMedia will always end up empty and automatic extraction on item update won’t run as expected. This method also still uses legacy extension/filename conventions (xml, tsv) and uses the selected format string as mediaType when checking existing OCR media; that won’t match the actual media types produced by the job (e.g. the new TSV-by-word format maps to text/tab-separated-values and file extensions like by-word.tsv, full.tsv, pdf2xml.xml). Align the format filtering + extension/mediaType mapping here with src/Job/ExtractOcr::perform() so existing files/medias are detected correctly and jobs aren’t skipped or re-dispatched unnecessarily.

Copilot · 2026-04-21T09:58:51Z

+                ],
+                'attributes' => [
+                    'id' => 'item_ids',
+                    'placeholder' => '2-6 8 38-52 80-', // @ translate


The translator extraction marker is written as // @ translate (with a space). If your translation tooling relies on the exact @translate token, this placeholder string may be missed during POT generation; change it to // @translate for consistency with the rest of the module.

Suggested change

'placeholder' => '2-6 8 38-52 80-', // @ translate

'placeholder' => '2-6 8 38-52 80-', // @translate

Copilot · 2026-04-21T09:58:51Z

-#: src/Job/ExtractOcr.php:130
+#: Module.php:487
+msgid "Extract OCR: Extract ocr from files"
+msgstr "Extract OCR : extraire l’ocr des fichierse"


There’s a typo in the French translation: “fichierse” should be “fichiers”.

Suggested change

msgstr "Extract OCR : extraire l’ocr des fichierse"

msgstr "Extract OCR : extraire l’ocr des fichiers"

Copilot · 2026-04-21T09:58:51Z

+        $newFilepath = substr_replace($filepath, '.by-word.tsv', -4);
+        $result = @rename($filepath, $newFilepath);
+        if (!$result) {
+            $message = new Message(
+                $translator->translate('The file "%s" cannot be renamed.'), // @translate
+                $filename
+            );
+        }
+    }


On rename failure, a Message is created but never logged, shown to the user, or used to fail the upgrade. As a result, file renames can silently fail and leave the installation in a partially-upgraded state. Consider reporting these failures via $messenger->addError() / $logger->err() and/or throwing ModuleCannotInstallException when renames are required for correctness.

Copilot · 2026-04-21T09:58:52Z

+allows smaller files and a slightly faster search, but a slighly less accurate
+display. For alto, it is a valid but basic alto that does not contains metadata
+about fonts or structure.
+
+There are two tsv formats: the words can be stored globally, so the index is
+smaller and the search is quicker, or one by one, so an exact search can be done
+when there are multiple words and the search wrapped with `"`. The tsv format
+does not allows to display the transcription in IIIF viewers, but it allows to


Minor wording/spelling issues in this updated documentation section make it harder to read: e.g. “slighly” → “slightly”, “does not contains” → “does not contain”, and “does not allows” → “does not allow”. Consider proofreading these lines.

Suggested change

allows smaller files and a slightly faster search, but a slighly less accurate

display. For alto, it is a valid but basic alto that does not contains metadata

about fonts or structure.

There are two tsv formats: the words can be stored globally, so the index is

smaller and the search is quicker, or one by one, so an exact search can be done

when there are multiple words and the search wrapped with `"`. The tsv format

does not allows to display the transcription in IIIF viewers, but it allows to

allows smaller files and a slightly faster search, but a slightly less accurate

display. For alto, it is a valid but basic alto that does not contain metadata

about fonts or structure.

There are two tsv formats: the words can be stored globally, so the index is

smaller and the search is quicker, or one by one, so an exact search can be done

when there are multiple words and the search wrapped with `"`. The tsv format

does not allow to display the transcription in IIIF viewers, but it allows to

Copilot · 2026-04-21T09:58:52Z

+                'The command-line utility pdftotext is not available. Install the package poppler-utils.' //@translate
+            );
+        }
+
+        if ((int) shell_exec('hash pdftohtml 2>&- || echo 1')) {
+            throw new ModuleCannotInstallException(
+                'The command-line utility pdftohtml is not available. Install the package poppler-utils.' //@translate


These install-time exception messages are now hardcoded strings (not passed through $t->translate()), so they won’t be localized even though they’re marked //@translate and appear in the POT. Consider translating them before throwing ModuleCannotInstallException (consistent with the other install checks below).

Suggested change

'The command-line utility pdftotext is not available. Install the package poppler-utils.' //@translate

);

}

if ((int) shell_exec('hash pdftohtml 2>&- || echo 1')) {

throw new ModuleCannotInstallException(

'The command-line utility pdftohtml is not available. Install the package poppler-utils.' //@translate

$t->translate('The command-line utility pdftotext is not available. Install the package poppler-utils.') // @translate

);

}

if ((int) shell_exec('hash pdftohtml 2>&- || echo 1')) {

throw new ModuleCannotInstallException(

$t->translate('The command-line utility pdftohtml is not available. Install the package poppler-utils.') // @translate

Copilot · 2026-04-21T09:58:52Z

                'options' => [
                    'label' => 'Create file for formats', // @translate
+                    'info' => 'For more info on formats, see readme.', // @translate
+                    'documentation' => 'https://gitlab.com/Daniel-KM/Omeka-S-module-ExtractOcr#extract-ocr-modue-for-omeka-s',


The documentation URL has a typo (extract-ocr-modue-for-omeka-s), which makes the help link in the admin config form incorrect. Update it to the correct anchor in the README so admins can reach the referenced documentation.

Suggested change

'documentation' => 'https://gitlab.com/Daniel-KM/Omeka-S-module-ExtractOcr#extract-ocr-modue-for-omeka-s',

'documentation' => 'https://gitlab.com/Daniel-KM/Omeka-S-module-ExtractOcr#extract-ocr-module-for-omeka-s',

Copilot · 2026-04-21T09:58:52Z

+#: data/scripts/upgrade.php:152 data/scripts/upgrade.php:184
+#, php-format
+msgid "The file \"%s\" cannot be renamed."
+msgstr "Le dossier « %s » ne peut pas être renommé."


The French translation for “The file "%s" cannot be renamed.” currently says “Le dossier …” (folder) instead of “Le fichier …” (file), which changes the meaning and is confusing in upgrade error reporting. Update the msgstr accordingly.

Suggested change

msgstr "Le dossier « %s » ne peut pas être renommé."

msgstr "Le fichier « %s » ne peut pas être renommé."

Copilot · 2026-04-21T09:58:53Z

+                $translator->translate('The file "%s" cannot be renamed.'), // @translate
+                $filename
+            );


On rename failure in the /pdf2xml loop, a Message is created but never emitted. This can silently leave old filenames on disk while the DB/settings are updated, leading to mismatches. Report/handle the failure (messenger/logger and/or abort the upgrade) so admins know remediation is needed.

Suggested change

$translator->translate('The file "%s" cannot be renamed.'), // @translate

$filename

);

$translator->translate('The file "%1$s" cannot be renamed to "%2$s".'), // @translate

$filepath,

$newFilepath

);

$logger->err((string) $message);

throw new ModuleCannotInstallException((string) $message);

Daniel Berthereau and others added 30 commits November 25, 2024 00:00

Cleaned code.

cf86d79

Fixed creation of files.

232825a

Added a task for module Easy Admin.

ed691ec

Fixed strict type issue.

a0d77c0

Updated readme.

1445806

Released version 3.4.8.

39451f5

Cleaned code.

d04465c

Used extension ".pdf2xml.xml" for format pdf2xml.

e0d2c78

Added a tsv format with all words separately.

793d1b0

Improved sql to list only first pdf of item.

c8ed742

Removed storage in a property by default.

28e8f9e

Added a check for poppler utils.

736bccc

Updated languages.

587727c

Petite correction pour le traitement avec identifiants listés sous la…

7238438

… forme valeur1-valeur2

Edit ExtractOcr.php, correction placement de parenthèse

ba4fb7a

Allowed to store extracted text without storing files or media.

a39dea1

Defined option "tsv by word" to be available on install.

32d339c

Cached Transliterator to avoid to init it everytime.

4a16b1b

Cleared entity manager and improved the batch loop.

589a6d1

Replaced string concatenation by array for tsv.

de24f96

Skipped useless double extraction for tsv.

cfa4e18

Fixed running multiple sub-jobs on batch.

5e2b875

Updated composer type "omeka-s-module".

9402786

Improved security on xml.

5cadbb0

Updated .gitignore.

8e69029

Cleaned code.

21d13e8

Updated readme.

ad0e1be

Released version 3.4.9.

da68e94

Fixed check of image size with exif orientation.

9c717a5

Added reference id to logs for all jobs.

3330feb

Daniel Berthereau added 2 commits March 2, 2026 00:00

Fixed composer name.

8e1ef6f

Released version 3.4.10.

095ef30

Daniel-KM force-pushed the master branch from 853add8 to 095ef30 Compare March 9, 2026 11:40

Daniel Berthereau added 4 commits April 6, 2026 00:00

Used a deferred job to group all jobs on save.

029a12b

Made module compatible with Omeka S v3.1.

0228a34

Fixed upgrade.

b722890

Released version 3.4.11.

1f7697b

Copilot AI review requested due to automatic review settings April 21, 2026 09:53

Copilot started reviewing on behalf of Daniel-KM April 21, 2026 09:53 View session

Copilot AI reviewed Apr 21, 2026

View reviewed changes

Navigation Menu

Corrections diverses. - #11

Corrections diverses.#11
Daniel-KM wants to merge 36 commits into
smachefert:masterfrom
Daniel-KM:master

Daniel-KM commented Feb 26, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	'placeholder' => '2-6 8 38-52 80-', // @ translate
	'placeholder' => '2-6 8 38-52 80-', // @translate

	msgstr "Extract OCR : extraire l’ocr des fichierse"
	msgstr "Extract OCR : extraire l’ocr des fichiers"

	'documentation' => 'https://gitlab.com/Daniel-KM/Omeka-S-module-ExtractOcr#extract-ocr-modue-for-omeka-s',
	'documentation' => 'https://gitlab.com/Daniel-KM/Omeka-S-module-ExtractOcr#extract-ocr-module-for-omeka-s',

	msgstr "Le dossier « %s » ne peut pas être renommé."
	msgstr "Le fichier « %s » ne peut pas être renommé."

Conversation

Daniel-KM commented Feb 26, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants