The two most influential developments were the
advancement of the Kubeflow project and the
introduction of Kubernetes on NVIDIA GPUs. Both
changed the whole outlook of learning models.
In the past, data scientists developed their algorithms in complete isolation, using eso-
teric, proprietary systems and languages. Kubeflow technology enables the data scien-
tist and software engineer to share roles, creating a new system of DataOps
collaboration.
NVIDIA
Kubernetes on NVIDIA GPUs extends the orchestration platform with GPU acceleration
capabilities across multicloud environments. A GPU is a specialized processor that can
be used to accelerate highly parallelized, computationally intensive workloads. Because
of their processing power, GPUs have been found to be particularly well-suited to deep
learning workloads. Using Kubernetes on NVIDIA CUDA drivers, teams can automate
the deployment, maintenance, scheduling and operation of multiple GPU-accelerated
application containers across clusters of nodes.
Conclusion
For the most part, data applications still live in the old world of IT, on Hadoop or Spark
platforms in on-premises environments. Companies have too much invested to rip and
replace everything overnight. But hybrid data environments are coming – quickly – and
early adopters are benefiting from the change. They are confident that containers run-
ning in Kubernetes clusters will accelerate big data development by enabling system
and app code to be reused. The data paradigm is evolving and the Kubernetes commu-
nity is driving the change.
WINTER 2019 | THE DOPPLER | 51