You can create deployments in the cloud from your models. This uses building and serving functions under the hood. For example, Heroku deployment combines Docker image building with FastAPI serving.
This functionality is experimental and is subject to change.
To deploy something somewhere, we need to define this “somewhere” first, or in
MLEM terms, declare a
target environment object. It will contain all the
information needed to access it. In the case of Heroku, all we need is an API
To declare a new target env, run
$ mlem declare env heroku staging -c api_key=<you api key> 💾 Saving env to .mlem/env/staging.mlem
MLEM will attempt to use the
HEROKU_API_KEY environment variable if no
api_key argument is provided.
Now, as we defined our target env, we can deploy our model there. Deployments are also MLEM objects, which means that they need to have their definition.
To create one for Heroku, we once again will use
declare command to configure
our deployment. We use
example-mlem-get-started for the app name, but you can
change it to something unique:
$ mlem declare deployment heroku myservice \ -c app_name=example-mlem-get-started \ -c model=rf \ -c env=staging 💾 Saving deployment to .mlem/deployment/service_name.mlem
Now we can actually run the deployment process (this can take a while):
$ mlem deployment run myservice ⏳️ Loading deployment from .mlem/deployment/myservice.mlem 🔗 Loading link to .mlem/env/staging.mlem 🔗 Loading link to .mlem/model/rf.mlem 💾 Updating deployment at .mlem/deployment/myservice.mlem 🏛 Creating Heroku App example-mlem-get-started 💾 Updating deployment at .mlem/deployment/myservice.mlem 🛠 Creating docker image for heroku 💼 Adding model files... 🛠 Generating dockerfile... 💼 Adding sources... 💼 Generating requirements file... 🛠 Building docker image registry.heroku.com/example-mlem-get-started/web... ✅ Built docker image registry.heroku.com/example-mlem-get-started/web 🔼 Pushed image registry.heroku.com/example-mlem-get-started/web to remote registry at host registry.heroku.com 💾 Updating deployment at .mlem/deployment/myservice.mlem 🛠 Releasing app my-mlem-service formation 💾 Updating deployment at .mlem/deployment/myservice.mlem ✅ Service example-mlem-get-started is up. You can check it out at https://my-mlem-service.herokuapp.com/
You can also define and run the deployment on-the-fly using
-c options for
mlem deployment run, e.g.:
$ mlem deployment run myservice \ -m model -t staging \ -c app_name=example-mlem-get-started
You can go here and see the same OpenAPI documentation. For details on it, refer to the Serving section. You can also try to do some requests:
from mlem.api import load from mlem.runtime.client.base import HTTPClient client = HTTPClient(host="http://example-mlem-get-started.herokuapp.com", port=80) res = client.predict(load("test_x.csv"))
Also, you can create a client using deployment meta object:
from mlem.api import load service = load("myservice") client = service.state.get_client() res = client.predict(load("test_x.csv"))
There is also the remote counterpart of
apply command. It will send requests
to your service instead of loading model into memory. There are two options to
achieve this in CLI: using the service address or the deploy meta.
$ mlem apply-remote http test_x.csv -c host=http://my-mlem-service.herokuapp.com -c port=80 --json [1, 0, 2, 1, 1, 0, 1, 2, 1, 1, 2, 0, 0, 0, 0, 1, 2, 1, 1, 2, 0, 2, 0, 2, 2, 2, 2, 2, 0, 0, 0, 0, 1, 0, 0, 2, 1, 0] $ mlem deployment apply myservice test_x.csv --json [1, 0, 2, 1, 1, 0, 1, 2, 1, 1, 2, 0, 0, 0, 0, 1, 2, 1, 1, 2, 0, 2, 0, 2, 2, 2, 2, 2, 0, 0, 0, 0, 1, 0, 0, 2, 1, 0]
You don’t even need to have the deployment metadata locally:
$ mlem deployment apply --json \ https://github.com/iterative/example-mlem-get-started/myservice \ https://github.com/iterative/example-mlem-get-started/test_x.csv
Finally, you can check the status of your service with:
$ mlem deployment status myservice running
And stop your service with
$ mlem deployment remove myservice ⏳️ Loading deployment from .mlem/deployment/myservice.mlem 🔗 Loading link to .mlem/env/staging.mlem 🔻 Deleting my-mlem-service heroku app 💾 Updating deployment at .mlem/deployment/myservice.mlem
Note, that it will not delete the deployment definition, just update its state.