Using Kubernetes as a common orchestration layer for all containerized apps has sev-
eral benefits:
• Better resource utilization through centralized scheduling of data science and
other containerized applications
• Workload portability
• A single scheduling solution for multiple environments, on-premises or in multi-
ple clouds
• The ability for IT to create self-service environments for data scientists and other
data users
Several strategic product introductions in recent years have accelerated the use of con-
tainers in data science applications. The 2.3 release of Apache Spark with native Kuber-
netes support made Kubernetes much more accessible to data scientists, enterprise
companies and startups trying to make sense of data. Mesosphere, another orchestra-
tor, announced its support for Kubernetes at the end of 2017.
The two most influential developments were the advancement of the Kubeflow project
and the introduction of Kubernetes on NVIDIA GPUs. Both of these changed the whole
outlook of learning models.
WINTER 2019 | THE DOPPLER | 49