This guide assumes you are familiar with Python and JSON config files. It also assumes that you have worked through the prerequisite sections.
This section shows you how to run the service from the command line, which is useful for testing. It also shows you how to run it as a service, which is best suited to a production environment.
The AI Model Service should be found in the InsightMaker.Python folder that you created in the prerequisites step.
We recommend you get the service running from the command line first so you can easily spot and resolve issues.
Activate your virtual environment by navigating to the AIModelService/venv/Scripts folder and running: activate.bat
You should now see (venv) at the start of your command line.
On the command line navigate back to InsightMaker.Python\AIModelService
For Windows - Visual Studio runtime must be installed. https://visualstudio.microsoft.com/visual-cpp-build-tools/
Other Operating Systems - Check requirements https://pypi.org/project/torch/
Use ‘pip’ to install all the requirements in the latest requirements file:
Online: pip install -r requirements_3.12.2.txt
Offline (From an internet connected machine):
Create a temporary folder.
Navigate to the temporary folder.
pip download -r requirements_3.12.2.txt
Copy the resulting contents of the temporary folder to the target server.
Run the following command on the target server: pip install --no-index --find-links C:/tmp/libs -r requirements_3.12.2.txt
Replace the path to the temporary folder as appropriate.
If you plan to use a GPU to accelerate providers such as sentence transformers:
Uninstall torch with the following command: pip uninstall torch
Reinstall torch using the pip command generated by this site: https://pytorch.org/get-started/locally/
For example pip install torch --index-url https://download.pytorch.org/whl/cu118
In the InsightMaker.Pyton\AIModelService edit run.bat
Change the location to match your virtual environment.
Edit the main configuration to the service can register itself:
Open the config/config.json in an editor.
Scheme: This should remain http.
If you plan to use proxy requests through an HTTPS reverse proxy see our guide to using SSL.
Address: This should be the IP address or host name of the server. Do not use localhost.
Port, Threads , maxRequestHeaderSize, and maxRequestBodySize: should not need changing at this stage
System secret: This should contain the Aiimi Insight Engine system secret.
Admin API: This should point at the Admin API URL.
Trusted IPs: This should contain the IP Address of all your user interface servers.
Start the service by running: run.bat.
The first time the service starts, no providers will load as they are disabled by default. Continue to the enabling providers section to change the provider default.
