MLEM now offers deployment to Kubernetes and Sagemaker with a single command.
Fly.io is a platform for deploying and running applications. If you don't have an experience with deploying to external platforms such as Sagemaker or Kubernetes, we recommend to start with Fly.io or Heroku.
$ pip install mlem[flyio] # or $ pip install fastapi uvicorn docker
To create applications on Fly.io platform all you need is to install their CLI tool and log in with it:
flyctlusing this instruction
flyctl auth login
As with other deployment targets, you can deploy your model to Fly.io in a single command, for example:
$ mlem deploy run flyio cv-app \ --model torch_resnet \ --app_name mlem-cv \ --scale_memory 1024 \ --server streamlit \ --server.request_serializer torch_image \ --server.ui_port 8080 \ --server.server_port 8081
Besides, you can pre-define deployment environment with
mlem declare env flyio
and deployments with
mlem declare deployment flyio app, to use them later like
mlem deployment run --load app --model=models/rf.
If you have multiple Fly.io organizations, you need to specify the
which you can get from running
flyctl orgs list. Fly.io by default uses a
personal organization if you don't have any.
For more information on declaring environments and deployments, see Deployments User Guide. For an example of creating and using those declarations, check out examples for Heroku.
The application is now live on Fly.io. You can go to the application and see the OpenAPI documentation. For details on it, refer to the Serving section. You can also try to do some requests:
from mlem.api import load from mlem.runtime.client import HTTPClient client = HTTPClient(host="https://mlem-cv.fly.dev", port=None) # note port=None res = client.predict("myimage.jpg")
Also, you can create a client using deployment meta object:
from mlem.api import load service = load("cv-app") client = service.get_client() res = client.predict("myimage.jpg")