Newer
Older
# Monitor Machine Learning Models Deployed on Torch Serve
This is a demo showing possible ways to monitor machine learning models deployed on Torch Serve using prometheus, Loki & Grafana.
## How to run it
After cloning the repo, on the command line type
`docker-compose up`
## Model Registering
There are different techniques with which you can register your trained model for deployment. For the purpose of simplicity, we're registering a pre-trained resnet-18 model for animal image classification. To do so, on the command terminal use:
`curl --write-out %{http_code} --silent --output /dev/null --retry 5 -X POST "http://10.5.0.11:8081/models?url=https://torchserve.pytorch.org/mar_files/resnet-18.mar&initial_workers=1&synchronous=true"`
To predict the class of a cat image by the model, follow the following steps
`curl -O https://raw.githubusercontent.com/pytorch/serve/master/docs/images/kitten_small.jpg`
`curl -F input=@kitten_small.jpg http://10.5.0.6/prediction`
The expected output should look something like this
`{'tabby': 0.7695330381393433,'tiger_cat': 0.1335214078426361,`
`'Egyptian_cat': 0.05591932684183121,`
`'lynx': 0.03647241368889809,`
`'tiger': 0.0013710658531636}`
## IPs & Ports to use for setup in Grafana
To setup Grafana, use the following url,
Grafana: http://localhost:3000 (username: admin, password: admin)
To setup Prometheus and Loki as data sources, use the following urls
Prometheus: http://10.5.0.7:9090
Loki: http://10.5.0.9:3100