Moving a service across servers

It is often the case that a machine learning service is used in some application, while it is being refined on the side through many witty manipulations and trials. DeepDetect allows to quickly move machine learning services across servers, typically from a development to a production server.

Below we assume we are moving a service from development to production server:

  • production server is already running dede
  • target service is service1 on production server, will be replaced by service2
  • we avoid any downtime

Steps for moving a service from development to production server:

  • Copy the repository model dataset to the production server, in what follows this repository has value /path/to/production/service2/model
  • Assuming service1 is the running target, create a new service2 service with a PUT /services/service2 call. Of importance:
    • do not use template parameters in the mllib object as this would erase your existing neural net model architecture
    • as an example a typical image classification service creation call would look like:

  curl -X PUT "http://localhost:8080/services/service2" -d '{
         "mllib":"caffe",
         "description":"image classification service",
         "type":"supervised",
         "parameters":{
           "input":{
             "connector":"image"
           },
           "mllib":{
             "nclasses":1000
           }
         },
         "model":{
           "repository":"/path/to/production/service2/model"
         }
       }'
  • service2 is now ready for action, a few tips:
    • run one or more test POST /predict calls: the first call may load/reload the full model and thus be slightly less quick than subsequent calls
    • redirect your calls to service2
    • you can leave service1 running or use it to compare results in real-time
  • phase service1 out with a DELETE /services/service1 call, see API

Notes:

  • it is not possible to rename a live service
  • if you are calling development and production servers from the same endpoint, beware that /path/to/production/service2/model may differ from /path/to/development/service2/model and thus you need to change the PUT call accordingly