4 – Diagram of the Deployment pattern
Praedictio is developed as a
cloud native application. It’s Container based approach helps in scaling
the system easily and implementing a cloud neutral deployment pattern which can
be deployed in Kubernetes.
Event engine is designed to
provide actionable insights from the predictions. The events engine provide a
simple interface to define actionable insight alerts.
3 – Architecture diagram of the praedictio services engine
Following is the Design diagram
of Prediction Services Engine. Python Django framework will be used to
implement the component
API request serving
Prediction Services Engine
handles following functions,
To achieve process isolation,
each model is managed in a separate Docker container.. Using this mechanism it
is expected that performance variabilities and instability of novel and immature
machine learning frameworks has no interference with the overall availability
of Clipper. The state of a model such as it’s model parameters would be
provided to the container at the time it is being initialized and the container
itself would be stateless afterwords. Hence machine learning frameworks which are identified
as being resource intensive can be replicated over multiple machines or can be
given GPU access.
Model Containers enable encapsulation of range
of Machine Learning frameworks and Models within a single API. To add a new
type of model to Clipper, model builders only need to implement the standard
batch prediction interface. Clipper supports language specific container
bindings for Python Java and c++. Building a model container is super easy and
it is done by inheriting the base container and adding the required
dependencies and encapsulating the prediction invocation with the common
It also supports several most
widely used machine learning frameworks: Apache Spark MLLib , Scikit-Learn,
Caffe , TensorFlow, and HTK. While these frameworks span multiple application
domains, programming languages, and system requirements, each was added using
fewer than 25 lines of code.
Consequently, models can be
modified or swapped transparently to the application. To achieve low latency,
high throughput predictions, Clipper implements a range of optimizations. In
the model abstraction layer, Clipper caches predictions on a permodel basis and
implements adaptive batching to maximize throughput given a query latency
target. In the model selection layer, Clipper implements techniques to improve
prediction accuracy and latency.
2 – Architecture diagram of prediction serving system
The prediction pipeline utilizes
the clipper prediction serving system as the core technology. It has a model
abstraction layer responsible for providing a common prediction interface,
ensuring resource isolation, and optimizing the query workload for batch
oriented machine learning frameworks. The first layer exposes a common API that
abstracts away the heterogeneity of existing ML frameworks and models.
The training pipeline composes
data through the ETL engine and the models will trained. The trained models are
updated frequently. A model repository is maintained to provide versioning of
the models. The model is serialized and the hyper parameters and accuracies are
also logged for analyzing experiments. We will discuss the training pipeline
extensively in the Alpha Release.
1- High level component overview
Figure 1 shows a high-level
component overview and architecture of the machine learning platform and
highlights the components discussed in the following sections:
Beta – Action Engine
Alpha – Training pipeline , Admin panel
MVP – Prediction Serving System and API gateway
The Praedictio platforms road map has been envisioned to
deliver the core components in an iterative manner.
Product Road Map
Only a small fraction of a
machine learning platform is the actual code implementing the training
algorithm. If the platform handles and encapsulates the complexity of machine
learning deployment, engineers and scientists have more time to focus on the modeling
Production-level reliability and scalability.
Providing an admin and
configuration framework is only possible if components also share utilities
that allow them to communicate and share assets. A Praedictio user is only
exposed to one admin panel to manage all components.
Easy-to-use configuration and tools.
Most machine learning pipelines
execute the components in a sequential manner leading to all the components to
be re-executed with the growth of data
to be fed. This becomes a bottleneck
since most of the real world use cases require continuous training. Preadictio
supports several continuation strategies that result from the interaction
between data visitation and warm-starting options.
We chose to use TensorFlow and
Scikit Learn as the trainer but the platform design is not limited to these
One factor in choosing (or dismissing) a machine learning platform is its
coverage of existing algorithms. Scikit holds a wide variety of pre implemented
ML algorithms and TensorFlow provides full flexibility for implementing any
type of model architecture.
There is a large and growing
number of machine learning frameworks. Each framework has strengths and
weaknesses and many are optimized for specific models or application domains
(e.g., computer vision). Thus, there is no dominant framework and often
multiple frameworks may be used for a single application. In a situation where training data grows
requirement arises for a framework with distributed training leading to change
of frameworks once selected as the best available in Machine Learning. Even though common model exchange formats had
been introduced in the past due to the rapid technological advancements and
fact that additional errors arising from parallel implementations for training
and serving these common message formats didn’t gain popularity.
One machine learning platform for many learning tasks.
The Praedictio design adopts the
Platform Design and
Praedictio introduces a modular
architecture to simplify model development and deployment across frameworks and
applications. Furthermore, by introducing caching, batching, and adaptive model
selection techniques, Praedictio reduces prediction latency and improves
prediction throughput, accuracy, and robustness without modifying the
underlying machine learning frameworks. The platform also can be Integrated
with enterprise systems, while satisfying stringent data security, privacy, or
Machine Learning is growing it’s
popularity in a wide spectrum of business domains to cater the need of
providing customer focused, accurate and robust business insights. One of the
biggest challenges in creating and maintaining a Machine Learning based
prediction system is orchestrating the Model Creation, Learning, Model
Validation and Deployment and Infrastructure Maintenance in Production
environment. With the high volatility of data and improved
learning models deploying fresh models become trickier. Most machine learning frameworks and systems
only address model training or
deployment and connectivity between different components is done ad hoc
via glue code or custom scripts. Praedictio integrates the aforementioned compoents
into one platform simplifying the platform configuration and reducing time to
production while increasing scalability.
Praedictio can run on-premise or
on any cloud platform and serve highly accurate business predictions that will
enable the business owners and decision makers to make timely decisions on
Praedictio is a business
predictions framework that provides powerful predictive analytics by analysing
the business data scattered across different repositories in an organization.
Praedictio framework will enable developers and data scientists to integrate
data driven ML models to business applications quickly and easily with powerful
tools to aggregate data, do data modeling, training and deploying.
Praedictio : Machine Learning