MLEM now offers deployment to Kubernetes and Sagemaker with a single command.
Docker is a platform for shipping applications in an isolated environment known as a container.
Deploying to Docker essentially means running a model inside a Docker Container locally.
$ pip install mlem[docker] # or $ pip install docker
Deploying to a docker container involves 2 main steps:
docker buildunder the hood.
docker rununder the hood.
One can do this via a single line:
$ mlem deploy run docker_container deployment.mlem -m model 💾 Saving deployment to deployment.mlem ⏳️ Loading model from model.mlem 🛠 Creating docker image mlem-deploy-1666728279 🛠 Building MLEM wheel file... 💼 Adding model files... 🛠 Generating dockerfile... 💼 Adding sources... 💼 Generating requirements file... 🛠 Building docker image mlem-deploy-1666728279:latest... ✅ Built docker image mlem-deploy-1666728279:latest ✅ Container mlem-deploy-1666728455 is up
One can check the docker image built via
docker image ls which should give the
REPOSITORY TAG IMAGE ID CREATED SIZE mlem-deploy-1666728279 latest fad02f76dbed 19 seconds ago 734MB ...
and the running container with
docker container ls:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0aa976159580 mlem-deploy-1666728279:latest "/bin/sh -c 'sh run.…" 26 seconds ago Up 25 seconds mlem-deploy-1666728455 ...
A lot of parameters can be configured, the full list of which can be accessed
mlem deploy run docker_container -h.
$ mlem deploy run docker_container app.mlem \ --model model \ --server fastapi \ --ports.0 8080:8080
You can specify other servers to use. Note that
--ports.0 exposes port 8080 outside of the container. Each server typically
exposes some specific port. You can check the default port by running
mlem serve $SERVER --help.
For some server implementations (like FastAPI) you can also set change the
default listening port. The below command shows how to modify the listening port
5000 and than map it on the host network directly on the same port:
$ mlem deploy run docker_container app.mlem \ --model model \ --server fastapi \ --ports.0 5000:5000 \ --server.port 5000
Please note that when running in a docker container, you usually shouldn't care
about the server internal listening port and can use
--ports to expose the
server to any port to the host network, e.g.
Also, keep in mind that the param name in
--server.$PARAM that controls the
port can be different for different server implementations. Use
mlem serve $SERVER --help to find out the right one.