You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ZenML's current implementation uses shared global state (like active stack and active pipeline), which prevents running multiple pipelines concurrently in the same process. If you attempt to trigger multiple pipelines simultaneously, subsequent calls will be blocked with the error:
755
+
756
+
```
757
+
Preventing execution of pipeline '<pipeline_name>'. If this is not intended behavior, make sure to unset the environment variable 'ZENML_PREVENT_PIPELINE_EXECUTION'.
758
+
```
759
+
{% endhint %}
760
+
761
+
The FastAPI example above uses threading, but due to ZenML's architecture, concurrent pipeline execution will fail. For production environments that need to handle concurrent pipeline requests, consider deploying your pipeline triggers through container orchestration platforms.
762
+
763
+
#### Recommended Solutions for Concurrent Execution
764
+
765
+
For production deployments, consider using:
766
+
767
+
1.**Kubernetes Jobs**: Deploy each pipeline execution as a separate Kubernetes Job for resource management and scaling
768
+
2.**Docker Containers**: Use a container orchestration platform like Docker Swarm or ECS to run separate container instances
769
+
3.**Cloud Container Services**: Leverage services like AWS ECS, Google Cloud Run, or Azure Container Instances
0 commit comments