Skip to content
Snippets Groups Projects
README.md 1.5 KiB
Newer Older
# Monitor Machine Learning Models Deployed on Torch Serve
This is a demo showing possible ways to monitor machine learning models deployed on Torch Serve using prometheus, Loki & Grafana. 
After cloning the repo, on the command line type
## Model Registering 

There are different techniques with which you can register your trained model for deployment. For the purpose of simplicity, we're registering a pre-trained resnet-18 model for animal image classification. To do so, on the command terminal use: 

`curl --write-out %{http_code} --silent --output /dev/null --retry 5 -X POST "http://10.5.0.11:8081/models?url=https://torchserve.pytorch.org/mar_files/resnet-18.mar&initial_workers=1&synchronous=true"`
## Prediction Example 
To predict the class of a cat image by the model, follow the following steps 
`curl -O https://raw.githubusercontent.com/pytorch/serve/master/docs/images/kitten_small.jpg`
Hammam Abdelwahab's avatar
Hammam Abdelwahab committed
`curl -F input=@kitten_small.jpg http://10.5.0.6/prediction`
The expected output should look something like this 

`{'tabby': 0.7695330381393433,'tiger_cat': 0.1335214078426361,`
 `'Egyptian_cat': 0.05591932684183121,`
 `'lynx': 0.03647241368889809,`
 `'tiger': 0.0013710658531636}`


## IPs & Ports to use for setup in Grafana  

To setup Grafana, use the following url, 

Grafana: http://localhost:3000  (username: admin, password: admin)

To setup Prometheus and Loki as data sources, use the following urls

Prometheus: http://10.5.0.7:9090

Loki: http://10.5.0.9:3100