Skip to content

Commit 12194b5

Browse files
committed
prosopopee: allow selecting number of threads used for thumbnail generation
Generating thumbnails is done in parallel threads via multiprocessing.Pool. By default, Pool schedules tasks on as many threads as there are cpu threads on the host machine. Let's allow users to select the number of threads Pool can use. Signed-off-by: Quentin Schulz <foss@0leil.net>
1 parent 75b93aa commit 12194b5

1 file changed

Lines changed: 15 additions & 2 deletions

File tree

prosopopee/prosopopee.py

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,15 @@ def loglevel(string):
4747
help="Configure the logging level",
4848
)
4949
subparser = parser.add_subparsers(dest="cmd")
50-
subparser.add_parser("build", help="Generate static site")
50+
parser_build = subparser.add_parser("build", help="Generate static site")
51+
parser_build.add_argument(
52+
"-j",
53+
"--jobs",
54+
default=None,
55+
type=int,
56+
help="Specifies number of jobs (thumbnail generations) to run simultaneously. Default: number "
57+
"of threads available on the system",
58+
)
5159
subparser.add_parser("test", help="Verify all your yaml data")
5260
subparser.add_parser("preview", help="Start preview webserver on port 9000")
5361
subparser.add_parser("deploy", help="Deploy your website")
@@ -923,7 +931,12 @@ def main():
923931
logger.info("Success: HTML file building without error")
924932
sys.exit(0)
925933

926-
with Pool() as pool:
934+
# If prosopopee is started without any argument, 'build' is assumed but the jobs parameter
935+
# is not part of the namespace, so set its default to None (or 'number of available CPU
936+
# treads')
937+
jobs = args.jobs if args.cmd else None
938+
939+
with Pool(jobs) as pool:
927940
# Pool splits the iterable into pre-defined chunks which are then assigned to processes.
928941
# There is no other scheduling in play after that. This is an issue when chunks are
929942
# outrageously unbalanced in terms of CPU time which happens when most galleries are

0 commit comments

Comments
 (0)