Integrating OpenCV with FastAPI and Redis RQ Causes Task/Job Failures

[HuseynG]

I'm working on integrating a set of microservices into our application, focusing on a FastAPI server that interacts with Redis through RQ for task queueing. The setup involves three main files:

1. server.py: Defines FastAPI endpoints.
2. redis_processes.py: Manages Redis connections and queues tasks.
3. worker_tasks.py: Contains the tasks to be enqueued and executed by workers.

The flow is as follows: The FastAPI endpoint receives some inputs and queues jobs in Redis. These jobs are defined in 'worker_tasks.py' and are supposed to be processed by background workers.

However, I'm encountering a peculiar issue: whenever I import 'cv2' (OpenCV) into 'worker_tasks.py', and I receive the following error message: "Work-horse terminated unexpectedly; waitpid returned 6 (signal 6).", but not on terminal though. This problem seems to be specifically related to the inclusion of the OpenCV library in 'worker_tasks.py'.

Here's a simplified version of the code for each component:

'server.py' snippet:

from fastapi import FastAPI, UploadFile, File, Query, HTTPException, Security, Depends
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
from typing import List, Optional
from models import FeedbackTask
import redis_processes

app = FastAPI()
security = HTTPBearer()

# Endpoint and authentication details omitted for brevity
@app.post("/processfeedback/")
async def process_feedback(...): # parameters and processing
    return redis_processes.process_feedback(feedback_task)

'redis_processes.py' snippet:

import redis
from rq import Queue
from redis_settings import workers_settings

import threading

import time
from datetime import datetime

from models import FeedbackTask
import worker_tasks as worker_tasks



# Set up Redis connection
redis_conn = redis.Redis(host='localhost', port=6379, db=0)

ai_task_processor_q         = Queue(name= workers_settings[0]['q'],connection=redis_conn)
initial_db_writer_q         = Queue(name= workers_settings[1]['q'],connection=redis_conn)
ai_task_result_writer_q     = Queue(name= workers_settings[2]['q'],connection=redis_conn)


def process_feedback(feedback_process_task: FeedbackTask):
    # tasks.process_feedback(feedback_task)
    now = datetime.now()
    timestamp = int(time.mktime(now.timetuple()))
    feedback_process_task.collection_id = f"{feedback_process_task.user_id}_{timestamp}"

    job_db_w = initial_db_writer_q.enqueue(worker_tasks.add_raw_collection_to_db, feedback_process_task, result_ttl=-1)
    job_ai   = ai_task_processor_q.enqueue(worker_tasks.process_feedback, feedback_process_task, result_ttl=-1)

    # return {"job_id": job_ai.get_id()}
    return {"AI Feedback Process Job Status": job_ai.get_status(),
            "Raw Document Collection Write Status": job_db_w.get_status()}

'worker_tasks.py' snippet:

import cv2  # This import causes the issue
from models import FeedbackTask

def process_feedback(task: FeedbackTask) -> FeedbackTask:
    # Task processing logic here

The tasks fail only after adding 'import cv2' in 'worker_tasks.py'. I've verified that the tasks work as expected when this import is commented out. My initial guess was it might be a problem with worker environments or a conflict between libraries, but I haven't found conclusive evidence to pinpoint the issue.

Has anyone faced a similar issue or does anyone have insights on why importing OpenCV in a worker task could lead to job failures in this setup? Any suggestions on how to debug or resolve this issue would be greatly appreciated.

I am using python 3.9.18, and here is the link for the requirement.txt link

If you would like to replicate the issue, please following the following link.