Remove -U UNLIMITED memory option?

[pete4abw]

As posted in this issue thread by @bleedchf , there appears to be a bug or anomaly that makes the impact of -U strange. I reviewed the code and am not sure of the resolution or if a resolution is even needed. After all, systems are larger, faster, and ram is cheaper than in 2007 when I began contributing. Using -p1 -L9 should allow for max memory to be used with a heuristically computed compression window. the -w option also may be used to expand it.

[lukypko]

I don't understand.
If option -U is not used, then it will use 2/3 of available (or forced memory, using a -m parameter)
If option -U is used, then it will use a whole RAM

	/* Set maximum chunk size to 2/3 of ram if not unlimited or specified
	 * by a control window. When it's smaller than the file size, round it
	 * to page size for efficiency. */
	if (UNLIMITED)
		control->max_chunk = control->st_size;
	else if (control->window)
		control->max_chunk = control->window * CHUNK_MULTIPLE;
	else
		control->max_chunk = control->ramsize / 3 * 2;
	control->max_mmap = MIN(control->max_mmap, control->max_chunk);
	if (control->max_chunk < control->st_size)
		round_to_page(&control->max_chunk);

	if (!STDIN)
		st->chunk_size = MIN(control->max_chunk, len);
	else
		st->chunk_size = control->max_mmap;
	if (st->chunk_size < len)
		round_to_page(&st->chunk_size);

So would you like remove -U and enable to use all possible RAM?
I'm using option -U everytime.

[pete4abw]

When the file size is smaller than memory, -U may not have any negative impact. However, for files larger than ram, swapping can cause very slow performance. More analysis is needed. Leaving alone for now.

[lukypko]

But, there is no swapping. You do not need to have huge ram to compress file of any size.

• input file(s) are mapped into a virtual memory, this memory is not modified (you are not writing to source files, you are just reading them)
• as more free memory you have, operating system start to cache already read pages of input files in a real cache ram, as rzip algorithm re-reads already read data again and again (as it is searching for the same parts)
• performance of rzip process is independent of configured memory, so it is not important if you use -m 1 or -m 100 (or do not specify an -m option and use a whole ram)
• if there is no free memory for other processes, real cache ram is freed and used for something more important, then rzip algorithm's mmap operation have to read missing page from disk again
• additional compression method depends on configured ram size, it is trying to use a largest possible dictionary (I did not investigate that so much)

[pete4abw]

There is and there will be and for large files, very slow. The compression window will be the same as the file size. This code has not changed since at least 2007.