Skip to main content

Using custom models

Introduction

Sometimes the best model for your files isn't in any predefined list. That's why we added full custom model support. You're free to plug in the exact model you want, from lightweight text encoders to large instruction-tuned transformers, and seamlessly use it inside your workflow. This gives you the freedom to experiment, optimize for speed or accuracy, and adapt your pipeline to any future model without waiting for updates.

Steps

LumiFind has two simple requirements for any custom model you want to use:

  1. The model must be publicly available on Hugging Face, since private repositories are not supported at the moment.
  2. The model must match the analysis type you assign it to. A document model should only be used for text based analysis and an image model should only be used for image workflows. Because automatic type validation is not available yet, we trust you to pick the correct model for each category.

Adding your own model is straightforward. Go to the model page on Hugging Face, locate the identifier in the format organization/model-name or username/model-name, and copy it. The copy icon on the page (highlighted in blue in the image below) gives you the exact string you need.

hugging-face-custom-models.png

You will often also find the information below what this model can be used for. This may give you a hint on which type of files you can use with this model. Below you can click to find the example models for a specific type:

Then, open the Settings window in LumiFind, click the "Custom" button next to the model selectors you want to replace, paste the identifier, and click "OK".

settings-custom-model-window-0.9.0-windows.png

The entered model will now be seen in the dropdown. Once you save your settings, it will be used in the next restart of the app.

Warning (and a long one): Changing a model means all data that depends on that model must be reanalyzed. For instance, switching to a custom document model will clear all document related analysis. If you have the Pro Edition, this includes voice and video entries, since they both rely on document embeddings. Likewise, setting a custom voice model resets video analysis because video also uses voice embeddings. This keeps your results consistent and ensures your new model is applied everywhere it matters.

Another warning: We are trying to integrate as many models as possible, but some models may not be supported. Since we cannot support all available models, please try and let us know why an unsupported model must be integrated in the next versions.