Multi Model Server Documentation

Basic Features

• Serving Quick Start - Basic server usage tutorial
• Model Archive Quick Start - Tutorial that shows you how to package a model archive file.
• Installation - Installation procedures and troubleshooting
• Serving Models - Explains how to use multi-model-server.
  • REST API - Specification on the API endpoint for MMS
  • Model Zoo - A collection of MMS model archive (.mar) files that you can use with MMS.
• Packaging Model Archive - Explains how to package model archive file, use model-archiver.
• Docker - How to use MMS with Docker and cloud services
• Logging - How to configure logging
• Metrics - How to configure metrics

Advanced Features

• Advanced settings - Describes advanced MMS configurations.
• Custom Model Service - Describes how to develop custom inference services.
• Unit Tests - Housekeeping unit tests for MMS.
• Benchmark - Use JMeter to run MMS through the paces and collect benchmark data.
• Model Serving with Amazon Elastic Inference - Run Model server on Elastic Inference enabled EC2 instances.

Example Projects

• MMS on Fargate, Serverless Inference - The project which illustrates the step-by-step process to launch MMS as a managed inference production service, on ECS Fargate.
• MXNet Vision Service - An example MMS project for a MXNet Image Classification model. The project takes JPEG image as input for inference.
• LSTM - An example MMS project for a recurrent neural network (RNN) using long short-term memory (LSTM). The project takes JSON inputs for inference against a model trained with a specific vocabulary.
• Object Detection - An example MMS project that uses a pretrained Single Shot Multi Object Detection (SSD) model that takes image inputs and infers the types and locations of several classes of objects.