Installation and Setup

This guide assumes you are familiar with Python and JSON config files. It also assumes that you have worked through the prerequisite sections.

This section shows you how to run the service from the command line, which is useful for testing. It also shows you how to run it as a service, which is best suited to a production environment.

The AI Model Service should be found in the InsightMaker.Python folder that you created in the prerequisites step.

AI Model Service

From The Command Line

We recommend you get the service running from the command line first so you can easily spot and resolve issues.

Activate Python Virtual Machine

1. Activate your virtual environment by navigating to the AIModelService/venv/Scripts folder and running: activate.bat

   • You should now see (venv) at the start of your command line.

2. On the command line navigate back to InsightMaker.Python\AIModelService

Python Torch run-time prerequisites.

For Windows - Visual Studio runtime must be installed. https://visualstudio.microsoft.com/visual-cpp-build-tools/

Other Operating Systems - Check requirements https://pypi.org/project/torch/

Install Requirements

1. Use ‘pip’ to install all the requirements in the latest requirements file:

   • Online: pip install -r requirements_3.12.2.txt

   • Offline (From an internet connected machine):

     1. Create a temporary folder.

     2. Navigate to the temporary folder.

     3. pip download -r requirements_3.12.2.txt

     4. Copy the resulting contents of the temporary folder to the target server.

     5. Run the following command on the target server: pip install --no-index --find-links C:/tmp/libs -r requirements_3.12.2.txt

        • Replace the path to the temporary folder as appropriate.

2. If you plan to use a GPU to accelerate providers such as sentence transformers:

   1. Uninstall torch with the following command: pip uninstall torch

   2. Reinstall torch using the pip command generated by this site: https://pytorch.org/get-started/locally/

      • For example pip install torch --index-url https://download.pytorch.org/whl/cu118

3. In the InsightMaker.Pyton\AIModelService edit run.bat

4. Change the location to match your virtual environment.

Register the AI Model Service

Edit the main configuration to the service can register itself:

1. Open the config/config.json in an editor.

2. Scheme: This should remain http.

   • If you plan to use proxy requests through an HTTPS reverse proxy see our guide to using SSL.

3. Address: This should be the IP address or host name of the server. Do not use localhost.

4. System secret: This should contain the Aiimi Insight Engine system secret.

5. Admin API: This should point at the Admin API URL.

6. Trusted IPs: This should contain the IP Address of all your user interface servers.

config.json

7. Start the service by running: run.bat.

The first time the service starts, no providers will load as they are disabled by default. Continue to the enabling providers section to change the provider default.