[Feature] Ability to disable certain tasks in machine learning containers

[sithmein]

I have searched the existing feature requests, both open and closed, to make sure this is not a duplicate request.

• Yes

The feature

Follow up on #17029. It would be nice if the different machine learnings tasks - face recognition, image processing, smart search - had dedicated URLs. Then you can configure different container/hosts, e.g. face recognition and image processing on powerful on/off machines and smart search on always-on low performance systems.

Platform

• Server
• Web
• Mobile

[sithmein]

First attempt in splitting URLs into three different ones: https://github.com/sithmein/immich/tree/feat/machine-learning-dedicated-paths

[mertalev]

Before you spend more time on this, I don't think this is a direction we should go in. Imagine that we add OCR, image quality assessment, pet recognition, etc. Juggling URLs for all of this is messy and adds complexity for what I imagine is a quite niche feature.

[mertalev]

If having separate endpoints for image and text is enough, then that alone would be the simplest solution. The current endpoint already only expects one or the other, so it's cleaner to separate them. There should be a single endpoint for images and a single endpoint for text, though, not separate endpoints for facial recognition, clip, etc.

[sithmein]

I deliberately created more endpoints because

• facial recognition needs the least resources, both in terms of memory and CPU
• clip needs both much memory and CPU resource
• text search need much memory but not many CPU resources

Being able to route these three types of requests to different targets is worthwile in my opinion (this is how I do it).

[mertalev]

A single endpoint is more powerful because it means several ML tasks can use the same image sent in the same request, which is faster, reduces IO and lowers memory usage. It's also more composable since a single model output can be shared between multiple ML models that rely on it instead of reprocessing the shared output. I'm not interested in changing that at the moment.

[sithmein]

Sorry, but I don't understand your argument. Since clip, textual, and facial recognition are three separate tasks that are triggered independently it doesn't make a difference whether its individual or combined endpoints. There are no combined requests as far as I can tell from the source code.

[mertalev]

There are none right now, but that can change in the future with later tasks. For example, pet recognition could share the same request as face recognition, along with image quality assessment run on each detected face. Using the same API, each of these tasks could be requested together or independently with options for each task's model and parameters. It's ultimately a more flexible design.

[sithmein]

https://github.com/sithmein/immich/tree/feat/machine-learning-disable-tasks contains an implementation of this feature. The machine learning container accepts a new environment variable MACHINE_LEARNING_ENABLED_TASKS which by default is face,image,text. By changing the value you can exclude certain tasks from getting accepted.
I decided to return 501 instead of 422 because it's not a client error if the server-side does not support some functionality.
In addition I added path suffixes for the different tasks which makes it easier for the new backoff functionality (which is buggy by the way, see #17185; the above branch contains fixes for the bug).
Let me know what you think and if there are suggestions for further improvements.

[aviv926]

The problem still exists for me, here is more information:

docker compose file:

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/aviv926/immich-server:pr-machine-learning-disable-tasks

    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - ${MYLIB}:/usr/src/mnt/mylib

    env_file:
      - .env
    ports:
      - "2283:2283"
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false
  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:eaba718fecd1196d88533de7ba49bf903ad33664a92debb24660a922ecd9cac8
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: >-
        pg_isready --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" || exit 1; Chksum="$$(psql --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: >-
      postgres -c shared_preload_libraries=vectors.so -c 'search_path="$$user", public, vectors' -c logging_collector=on -c max_wal_size=2GB -c shared_buffers=512MB -c wal_compression=on
    restart: always
    ports:
      - "5432:5432"
volumes:
  model-cache:

docker compose file immich remote ML:

name: immich_remote_ml

services:
  immich-machine-learning_remote:
    container_name: immich-machine-learning_remote
    # For hardware acceleration, add one of -[armnn, cuda, rocm, openvino, rknn] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/aviv926/immich-machine-learning:pr-machine-learning-disable-tasks-cuda
    extends:
      file: hwaccel.ml.yml
      service: cuda # set to one of [armnn, cuda, rocm, openvino, openvino-wsl, rknn] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    restart: always
    env_file:
      - .env
    ports:
      - 3099:3003

volumes:
  model-cache:

ENV used for immich remote ML

IMMICH_LOG_LEVEL=verbose
MACHINE_LEARNING_ENABLED_TASKS=image, text

Logs immich remote ML:

immich-machine-learning_remote  | [04/26/25 21:12:29] INFO     Starting gunicorn 23.0.0
immich-machine-learning_remote  | [04/26/25 21:12:29] INFO     Listening at: http://[::]:3003 (8)
immich-machine-learning_remote  | [04/26/25 21:12:29] INFO     Using worker: immich_ml.config.CustomUvicornWorker
immich-machine-learning_remote  | [04/26/25 21:12:29] INFO     Booting worker with pid: 9
immich-machine-learning_remote  | [04/26/25 21:12:30] DEBUG    Could not load ANN shared libraries, using ONNX:
immich-machine-learning_remote  |                              libmali.so: cannot open shared object file: No such
immich-machine-learning_remote  |                              file or directory
immich-machine-learning_remote  | [04/26/25 21:12:30] DEBUG    RKNN is not available
immich-machine-learning_remote  | [04/26/25 21:12:31] INFO     Started server process [9]
immich-machine-learning_remote  | [04/26/25 21:12:31] INFO     Waiting for application startup.
immich-machine-learning_remote  | [04/26/25 21:12:31] INFO     Created in-memory cache with unloading after 300s
immich-machine-learning_remote  |                              of inactivity.
immich-machine-learning_remote  | [04/26/25 21:12:31] INFO     Enabled tasks: image,  face.
immich-machine-learning_remote  | [04/26/25 21:12:31] INFO     Initialized request thread pool with 16 threads.
immich-machine-learning_remote  | [04/26/25 21:12:31] DEBUG    Checking for inactivity...
immich-machine-learning_remote  | [04/26/25 21:12:31] INFO     Application startup complete.
immich-machine-learning_remote  | [04/26/25 21:12:41] DEBUG    Checking for inactivity...
immich-machine-learning_remote  | [04/26/25 21:12:51] DEBUG    Checking for inactivity...
immich-machine-learning_remote  | [04/26/25 21:13:01] DEBUG    Checking for inactivity...
immich-machine-learning_remote  | [04/26/25 21:13:11] DEBUG    Checking for inactivity...
immich-machine-learning_remote  | [04/26/25 21:13:21] DEBUG    Checking for inactivity...

Logs immich server:

immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RouterExplorer] Mapped {/api/users, GET} route
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RouterExplorer] Mapped {/api/users/me, GET} route
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RouterExplorer] Mapped {/api/users/me, PUT} route
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RouterExplorer] Mapped {/api/users/me/preferences, GET} route
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RouterExplorer] Mapped {/api/users/me/preferences, PUT} route
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RouterExplorer] Mapped {/api/users/me/license, GET} route
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RouterExplorer] Mapped {/api/users/me/license, PUT} route
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RouterExplorer] Mapped {/api/users/me/license, DELETE} route
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RouterExplorer] Mapped {/api/users/:id, GET} route
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RouterExplorer] Mapped {/api/users/profile-image, POST} route
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RouterExplorer] Mapped {/api/users/profile-image, DELETE} route
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RouterExplorer] Mapped {/api/users/:id/profile-image, GET} route
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RoutesResolver] ViewController {/api/view}:
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RouterExplorer] Mapped {/api/view/folder/unique-paths, GET} route
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:RouterExplorer] Mapped {/api/view/folder, GET} route
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:NestApplication] Nest application successfully started
immich_server    | [Nest] 19  - 04/26/2025, 9:10:49 PM     LOG [Api:Bootstrap] Immich Server is listening on http://[::1]:2283 [v1.131.3] [production] 
immich_server    | [Nest] 19  - 04/26/2025, 9:11:23 PM     LOG [Api:EventRepository] Websocket Connect:    YfoiN9lDuuyPcNUbAAAB
immich_server    | [Nest] 7  - 04/26/2025, 9:11:29 PM    WARN [Microservices:MachineLearningRepository] Machine learning request to "http://localhost:3099" failed: fetch failed
immich_server    | [Nest] 7  - 04/26/2025, 9:11:29 PM   ERROR [Microservices:{"id":"b87c77c9-fc9c-41da-ab87-65f2a9c7434d"}] Unable to run job handler (faceDetection/face-detection): Error: Machine learning request '{"facial-recognition":{"detection":{"modelName":"buffalo_l","options":{"minScore":0.6}},"recognition":{"modelName":"buffalo_l"}}}' failed for all URLs
immich_server    | Error: Machine learning request '{"facial-recognition":{"detection":{"modelName":"buffalo_l","options":{"minScore":0.6}},"recognition":{"modelName":"buffalo_l"}}}' failed for all URLs
immich_server    |     at MachineLearningRepository.predict (/usr/src/app/dist/repositories/machine-learning.repository.js:114:15)
immich_server    |     at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
immich_server    |     at async MachineLearningRepository.detectFaces (/usr/src/app/dist/repositories/machine-learning.repository.js:123:26)
immich_server    |     at async PersonService.handleDetectFaces (/usr/src/app/dist/services/person.service.js:239:52)
immich_server    |     at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:156:28)
immich_server    |     at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:126:13)
immich_server    |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_server    |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_server    | [Nest] 19  - 04/26/2025, 9:11:42 PM     LOG [Api:EventRepository] Websocket Disconnect: YfoiN9lDuuyPcNUbAAAB
immich_server    | [Nest] 19  - 04/26/2025, 9:11:44 PM     LOG [Api:EventRepository] Websocket Connect:    Ua7pnFgow3Qcmpo8AAAD
immich_server    | [Nest] 7  - 04/26/2025, 9:12:42 PM    WARN [Microservices:MachineLearningRepository] Machine learning request to "http://localhost:3099" failed: fetch failed
immich_server    | [Nest] 7  - 04/26/2025, 9:12:42 PM   ERROR [Microservices:{"id":"499ece43-25fa-49ba-9ca6-010c623b8ef1"}] Unable to run job handler (faceDetection/face-detection): Error: Machine learning request '{"facial-recognition":{"detection":{"modelName":"buffalo_l","options":{"minScore":0.6}},"recognition":{"modelName":"buffalo_l"}}}' failed for all URLs
immich_server    | Error: Machine learning request '{"facial-recognition":{"detection":{"modelName":"buffalo_l","options":{"minScore":0.6}},"recognition":{"modelName":"buffalo_l"}}}' failed for all URLs
immich_server    |     at MachineLearningRepository.predict (/usr/src/app/dist/repositories/machine-learning.repository.js:114:15)
immich_server    |     at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
immich_server    |     at async MachineLearningRepository.detectFaces (/usr/src/app/dist/repositories/machine-learning.repository.js:123:26)
immich_server    |     at async PersonService.handleDetectFaces (/usr/src/app/dist/services/person.service.js:239:52)
immich_server    |     at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:156:28)
immich_server    |     at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:126:13)
immich_server    |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_server    |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_redis     | 1:M 26 Apr 2025 21:14:59.092 * 100 changes in 300 seconds. Saving...
immich_redis     | 1:M 26 Apr 2025 21:14:59.093 * Background saving started by pid 84
immich_redis     | 84:C 26 Apr 2025 21:14:59.102 * DB saved on disk
immich_redis     | 84:C 26 Apr 2025 21:14:59.102 * RDB: 0 MB of memory used by copy-on-write
immich_redis     | 1:M 26 Apr 2025 21:14:59.193 * Background saving terminated with success
immich_redis     | 1:M 26 Apr 2025 21:20:45.195 * 100 changes in 300 seconds. Saving...
immich_redis     | 1:M 26 Apr 2025 21:20:45.195 * Background saving started by pid 169
immich_redis     | 169:C 26 Apr 2025 21:20:45.206 * DB saved on disk
immich_redis     | 169:C 26 Apr 2025 21:20:45.206 * RDB: 0 MB of memory used by copy-on-write
immich_redis     | 1:M 26 Apr 2025 21:20:45.296 * Background saving terminated with success
immich_redis     | 1:M 26 Apr 2025 21:26:45.172 * 100 changes in 300 seconds. Saving...
immich_redis     | 1:M 26 Apr 2025 21:26:45.172 * Background saving started by pid 254
immich_redis     | 254:C 26 Apr 2025 21:26:45.179 * DB saved on disk
immich_redis     | 254:C 26 Apr 2025 21:26:45.180 * RDB: 0 MB of memory used by copy-on-write
immich_redis     | 1:M 26 Apr 2025 21:26:45.273 * Background saving terminated with success
immich_server    | [Nest] 19  - 04/26/2025, 9:30:56 PM    WARN [Api:MachineLearningRepository~0y0hej0f] Machine learning request to "http://localhost:3099" failed: fetch failed
immich_server    | [Nest] 19  - 04/26/2025, 9:30:56 PM   ERROR [Api:ErrorInterceptor~0y0hej0f] Unknown error: Error: Machine learning request '{"clip":{"textual":{"modelName":"ViT-SO400M-16-SigLIP2-384__webli","options":{"language":"en-US"}}}}' failed for all URLs
immich_server    | Error: Machine learning request '{"clip":{"textual":{"modelName":"ViT-SO400M-16-SigLIP2-384__webli","options":{"language":"en-US"}}}}' failed for all URLs
immich_server    |     at MachineLearningRepository.predict (/usr/src/app/dist/repositories/machine-learning.repository.js:114:15)
immich_server    |     at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
immich_server    |     at async MachineLearningRepository.encodeText (/usr/src/app/dist/repositories/machine-learning.repository.js:137:26)
immich_server    |     at async SearchService.searchSmart (/usr/src/app/dist/services/search.service.js:62:27)

docker ps:

docker ps
CONTAINER ID   IMAGE                                                                            COMMAND                  CREATED          STATUS                             PORTS                    NAMES
cda23267da2c   ghcr.io/aviv926/immich-machine-learning:pr-machine-learning-disable-tasks-cuda   "tini -- python -m i…"   22 seconds ago   Up 22 seconds (health: starting)   0.0.0.0:3099->3003/tcp   immich-machine-learning_remote
17204c85710f   ghcr.io/aviv926/immich-server:pr-machine-learning-disable-tasks                  "tini -- /bin/bash s…"   26 seconds ago   Up 3 seconds (health: starting)    0.0.0.0:2283->2283/tcp   immich_server
8946dfb3461f   tensorchord/pgvecto-rs:pg14-v0.2.0                                               "docker-entrypoint.s…"   27 seconds ago   Up 26 seconds (health: starting)   0.0.0.0:5432->5432/tcp   immich_postgres
ca5de86e1cc6   redis:6.2-alpine                                                                 "docker-entrypoint.s…"   27 seconds ago   Up 26 seconds (health: starting)   6379/tcp                 immich_redis

[sithmein]

Remove the leading space in front of face in your definition MACHINE_LEARNING_ENABLED_TASKS.

[aviv926]

Remove the leading space in front of face in your definition MACHINE_LEARNING_ENABLED_TASKS.

The problem still exists.
Even when I use all the options:
MACHINE_LEARNING_ENABLED_TASKS=image,face,text
Can you share your .env& docker compose file?

[sithmein]

Sure:

name: immich

services:
  immich-machine-learning:
    image: dreg.meinl.ddnss.ch/immich/immich-machine-learning:${IMMICH_VERSION}
    environment:
      - MACHINE_LEARNING_REQUEST_THREADS=2
      - MACHINE_LEARNING_ENABLED_TASKS=face,image
      - IMMICH_LOG_LEVEL=debug
      - TZ=${TZ}
    volumes:
      - model-cache:/cache
      - /etc/localtime:/etc/localtime:ro
    restart: unless-stopped
    ports:
      - 3003:3003
    deploy:
      resources:
        limits:
          cpus: "4"
    healthcheck:
      disable: false
    profiles:
      - cpu

  immich-machine-learning-cuda:
    image: dreg.meinl.ddnss.ch/immich/immich-machine-learning:${IMMICH_VERSION}-cuda
    environment:
      - MACHINE_LEARNING_REQUEST_THREADS=2
      - MACHINE_LEARNING_ENABLED_TASKS=face,image
      - TZ=${TZ}
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
      file: hwaccel.ml.yml
      service: cuda # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
      - /etc/localtime:/etc/localtime:ro
    restart: unless-stopped
    ports:
      - 3003:3003
    deploy:
      resources:
        limits:
          cpus: "2"
    healthcheck:
      disable: false
    profiles:
      - cuda


volumes:
  model-cache:

TZ=Europe/Zurich
IMMICH_VERSION=v1.132.1-1

[aviv926]

Thanks, I did a docker system prune and it solved all the problems :)

[aviv926]

I just opened a feature request #17666 before I saw yours

With the new option of adding multiple endpoints, it would be nice to be able to choose which task each endpoint will work on, face recognition or CLIP.
This way, you can take advantage of the power of computers that are not powerful enough for CLIP models but are good enough for face recognition to offload other endpoints.

When none of the options are selected this will be used for a failover endpoint (Default).

You can choose to have an endpoint dedicated to both tasks (face recognition and CLIP) or just one of them.

This is the idea in terms of UI that could be considered if it gets implemented