The Great Migration to Second Generation Big Data Platforms
Joey Jablonski
Due to operational challenges within data centers , many customers now want to leverage native services and capabilities provided by the major public cloud providers .
First generation big data platforms are characterized as on-premise deployments , on physical servers , often running a software stack from Cloudera , Hortonworks , DataStax or similar vendors . These old world platforms enable many companies to begin analyzing data in new ways by providing a unique set of capabilities to analysts . These game-changing big data platforms allow companies to compete in markets in new ways .
While these systems provide new capabilities , they often come at a high cost . Costs include both capital expenses to purchase unproven capabilities , and operational expenses to keep these complex platforms functioning . Compared to the operational requirements of public cloud services , many on-premise deployments are held back by a three-fold increase in the number of operational resources needed to maintain the platform , manage capacity , respond to incidents , manage upgrades and educate platform users . Every resource that is being consumed by platform maintenance ( Figure 1 ) is less support for application developers and data scientists looking to extract value out of the data stored in these big data platforms .
Because of these operational challenges , many customers are now actively moving their big data workloads to second generation platforms in order to leverage native services and capabilities provided by the major public cloud providers . AWS , Google and Azure all have rapidly maturing and highly competitive managed services and PaaS capabilities . These enable organizations to focus their resources on core business needs and advanced capabilities such as data integration , data analysis and reporting .
Even with second generation big data platforms , the challenges around data quality , model accuracy and data integration will continue . But now the staff
56 | THE DOPPLER | WINTER 2017