Post-1.0 Roadmap

[thekevinscott]

As we gear up to release the 1.0 release, I'd like to start a discussion into what comes after. What are some of the things you'd like to see next in UpscalerJS?

Here's a non-exhaustive list of ideas of mine, in no particular order:

New Models

We could continue adding new models to the UpscalerJS ecosystem. Some ones in particular I'd like to add:

• DDNM
• Mirnet

New Input Mechanisms

We could explore additional input mechanisms in UpscalerJS, including:

• Video
• 4D Tensors (e.g., support for multiple images)

We could also investigate support for text and other supplementary parameters such as masks for something like inpainting with DDN.

Performance Improvements

We could continue exploring performance improvements.

There's two general buckets:

• Improvements to the models, such as pruning
• Improvements to the core algorithms, like optimizing the patch size calculations
• First class support for transmitting memory to web workers (maybe leveraging the shared memory model)

Platform Support

Better support for other platforms, like bun or deno.

Is Node ESM support something worth exploring?

DX Improvements

We could explore things that make developers' lives easier, like:

• Adding the ability to load models in the browser from just a string, no installation necessary
• Adding the ability to load models from arbitrary endpoints, such as hugging face or GitHub.

---

If there are particular things you'd like to see in the next version of UpscalerJS, leave a comment below!

[alenatih]

Hello,

Could you please tell me if it is possible to integrate several different models in the same app, so that you can have a dropdown menu and a user can choose which model to choose? Like ersgan super resolution, deblurring, dehazing, denoising, deraining, low light enhancement, and retouching, all in one.
(For an app that works in a browser)

Thank you so much!

[alenatih]

Thank you so much!

[GgFred]

Do you mean something like this?

let upscaler = new Upscaler();

async function upscale(model: string) {
  // Dispose of the previous instance
  await upscaler.dispose();

  switch (model) {
    case "x2":
      upscaler = new Upscaler({ model: x2Model });
      break;
    case "deblurringModel":
      upscaler = new Upscaler({ model: deblurringModel });
      break;
    case "denoisingModel":
      upscaler = new Upscaler({ model: denoisingModel });
      break;
    default:
      console.warn(`Unknown model: ${model}`);
  }
}

This would create a new Upscaler instance each time you switch models, and dispose of the previous one to help manage memory. Is that roughly what you were suggesting?

[thekevinscott]

Yes, exactly @GgFred

[GgFred]

Maybe something like this:

import Upscaler from 'upscaler';

let upscaler = new Upscaler();
let currentModelName: string | null = null;

const modelMap: Record<string, () => Promise<any>> = {
  x2: () => import('@upscalerjs/esrgan-thick/2x'),
  deblurringModel: () => import('@upscalerjs/maxim-deblurring'),
  denoisingModel: () => import('@upscalerjs/maxim-denoising'),
};

async function upscale(model: string) {
  if (model === currentModelName) return;

  await upscaler.dispose();

  const loader = modelMap[model];
  if (!loader) return console.warn(`Unknown model: ${model}`);

  const { default: mod } = await loader();

  upscaler = new Upscaler({ model: mod });
  currentModelName = model;
}

I haven’t tested this code yet, but the idea is to not keep multiple models in memory and import each one only when needed, which is useful if you have many models in modelMap.

[thekevinscott]

The imported code is some small JSON configuration and a URL to the model. Instantiating Upscaler / disposing it takes care of loading and unloading the model from memory. So I don't think this will help too much, although it also won't hurt if you want to go this route.