Add indexes for queries made by CritiqueBrainz

[mwiencek]

These indexes will allow improving the performance of the get_authors_for_artist and get_literary_works_for_work queries in CritiqueBrainz [1], which are very frequently hitting the database and take close to 200 ms currently.

Note that these indexes alone won't do anything without also modifying the queries in CB to stop selecting from the author and work views. Those views add many additional (unneeded) joins which exceed PostgreSQL's join_collapse_limit and prevent the planner from optimizing things correctly.

That was primarily debugged using an LLM, and the code changes here were assisted by an LLM.

[1] See critiquebrainz/frontend/external/bookbrainz_db/common_entity.py in the CritiqueBrainz repository.

[mwiencek]

ESLint failed to run for some reason:

2026-09-10T16:13:28.3136793Z ##[group]Run yarn run lint --format rdjson --output-file eslint_report.rdjson
2026-09-10T16:13:28.3137393Z �[36;1myarn run lint --format rdjson --output-file eslint_report.rdjson�[0m
2026-09-10T16:13:28.3173024Z shell: /usr/bin/bash -e {0}
2026-09-10T16:13:28.3173290Z ##[endgroup]
2026-09-10T16:13:28.4477616Z yarn run v1.22.22
2026-09-10T16:13:28.4737892Z $ eslint . --format rdjson --output-file eslint_report.rdjson
2026-09-10T16:13:46.0223772Z error Command failed with exit code 1.
2026-09-10T16:13:46.0224824Z info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
2026-09-10T16:13:46.0336335Z ##[error]Process completed with exit code 1.

The same command works for me on this branch locally, so perhaps another transient failure.

[MonkeyDo]

Thanks for these improvements!
I've reviewed last week and am testing the performance on a few cases.

ESLint failed to run for some reason:
The same command works for me on this branch locally, so perhaps another transient failure.

Fixing that in #1372, will rebase afterwards

[MonkeyDo]

Can confirm the massive speed boosts, thanks a lot for looking into those.

I'm looking at some other DB/index improvements which I would like your review on when the time comes, but it seems quite clear that the use of the views as defined is detrimental at best.

[MonkeyDo]

For my own education, is the index on (identifier_id) rather than (identifier_id, set_id) because of the fact that there will be few relations between an identifier and its containing set (identifier can belong to many sets, but typically a set only contains few identifiers)?
I'm still wrapping my head around the tradeoff between index size and table scans; how do you benchmark that?

[MonkeyDo]

Ah, never mind, it's because there is already a identifier_set__identifier_pkey on (set_id, identifier_id) ?
Should it also have one in the other direction, i.e. (identifier_id, set_id) for use with the CB query?

[mwiencek]

The idens join in the new CB query is just JOIN identifier_set__identifier idens ON idens.identifier_id = iden.id (only on the identifier_id) so it doesn't actually need the set_id in the index. Although set_id is used in the subsequent join:

                  JOIN author_data ad ON ad.identifier_set_id = idens.set_id

So using (identifier_id, set_id) for the index does all allow for an index-only scan, meaning it doesn't have to retrieve set_id from the table because it's already in the index. I compared the plans with & without set_id in the index and it does change it to index-only, but doesn't affect performance from what I can tell.

These indexes are tiny though, at least compared to ones in MB. With no set_id it's 3992 kB and with it's 4336 kB, so it's probably worth adding still.

[mwiencek]

I'm still wrapping my head around the tradeoff between index size and table scans; how do you benchmark that?

I think the index size is mainly justified by how frequent the queries are, since our projects are significantly more read-heavy than write-heavy. In MB we have gigabytes of materialized tables to speed up artist pages because those are hit a lot. In this case the indexes are only a few megabytes:

bookbrainz=> SELECT pg_size_pretty(pg_relation_size('bookbrainz.identifier_set__identifier_identifier_id_idx'));
 pg_size_pretty 
----------------
 3992 kB
(1 row)

[MonkeyDo]

Thank you very much!