Using custom models

Introduction

Sometimes the best model for your files isn't in any predefined list. That's why we added full custom model support. You're free to plug in the exact model you want, from lightweight text encoders to large instruction-tuned transformers, and seamlessly use it inside your workflow. This gives you the freedom to experiment, optimize for speed or accuracy, and adapt your pipeline to any future model without waiting for updates.

Steps

LumiFind has a simple requirement for any custom model you want to use

The model must match the analysis type you assign it to. A document model should only be used for text based analysis and an image model should only be used for image workflows. Because automatic type validation is not available yet, we trust you to pick the correct model for each category.

Adding your own model is straightforward. Go to the model page on Hugging Face, locate the identifier in the format organization/model-name or username/model-name, and copy it. The copy icon on the page (highlighted in blue in the image below) gives you the exact string you need.

You will often also find the information below what this model can be used for. This may give you a hint on which type of files you can use with this model. Below you can click to find the example models for a specific type:

• Document Models
• Image Models
• Voice Models

Then, open the Settings window in LumiFind, click the "Custom" button next to the model selectors you want to replace, paste the identifier, and click "OK".

The entered model will now be seen in the dropdown. Once you save your settings, it will be used in the next restart of the app.

If you enter a private repository, a hugging face token must be entered via Settings and enabling Show Advanced Settings.

Note: Currently, we cannot estimate how big the model is. If you press "Files and versions" tab on the model page, the total size will be shown. See an example below:

Important: It is recommended to load models only from organizations/developers you trust. LumiFind cannot verify whether a model contains malicious code or if it is fully safe. In the "Files and versions" tab, some models include a "Safe" label. Selecting this label will display the results of the available virus scan reports.

Warning (and a long one): Changing a model means all data that depends on that model must be reanalyzed. For instance, switching to a custom document model will clear all document related analysis. If you have the Pro Edition, this includes voice and video entries, since they both rely on document embeddings. Likewise, setting a custom voice model resets video analysis because video also uses voice embeddings. This keeps your results consistent and ensures your new model is applied everywhere it matters.

Another warning: We are trying to integrate as many models as possible, but some models may not be supported. Since we cannot support all available models, please try and let us know why an unsupported model must be integrated in the next versions.