Offline Huggingface Set-up
To use Huggingface offline you will need to first download the models on a machine connected to the internet and then copy them across. The easiest way to do this is to get this service running on a machine connected to the internet and let it download all of the models for you. Note that you do not need to the rest of Aiimi Insight Engine running to do this, just the Python REST Service.
Information about how the Huggingface cache works can be found here:
See the Cache set up section
Steps:
Get the Python REST Service set up on an internet connected machine.
Enable the huggingfacener step in the config/endpoints.json file.
Run the service with run.bat – you should see log messages that indicate it downloading models from the internet – these are large and may take a few minutes.
Find the models – by default on windows the models can be found here:
Zip them up and move them to the same place on the target server. Make sure you place them into the right users .cache folder. You are probably running the service as a different user on the target system.
On the target system set an environment variable:
TRANSFORMERS_OFFLINE=1
Activate the venv for Python REST Service on the target server
Navigate to the endpoints subfolder
Run: python huggingfacener.py
You should see the following, which shows that the model was loaded offline and uses successfully:
