FPT Software's Technology Community
  • Contact Us

Intersection of the Cloud and Big data

Ting Hsuan Lin | September 27, 2016

The two biggest trends in the data center today are cloud computing and big data. This writing examines the intersection of these. For consumers, big data is about using large datasets from new or diverse sources to provide meaningful and actionable information about how everything in the world works. For example, Netflix can use customer data to produce that shows tailored to their audiences. For producers, however, big data is about the technology necessary to handle these large, diverse datasets. Producers characterize big data in terms of volume, variety, and velocity. How much data is there, of what types, and how quickly can you derive value from it?

The macro trends behind big data

Just as adopting a service-oriented approach is the macro trend behind the cloud, there are several macro trends behind big data.

  • The first trend is Consumption– We consume data as a part of the everyday activities in our personal and working lives. From booking a fight, to finding a partner, to diagnosing disease, data is driving many more decisions today than it has in the past. We live in a relatively new social context where people increasingly want to make data-driven decisions.
  • The second trend is Instrumentation. We collect data at each step in many of our activities, and a couple of these are now produced by machines instead of people. From supply chains to Fitbits, we collect information about all our activities with the intent to measure and analyze them.
  • The third trend is Exploration.The relatively easy access to this abundance of data means we can use it to construct, test, and consume experiments that were previously not feasible.Ultimately, big data is about the change in relationship between our data and us and, in the context of this column, the implications of this change on cloud technology.

    The relationship between Big data and Cloud

    So what is the relationship between big data and the cloud? Big data has its origins in the cloud. Apache Hadoop, one of the most widely used big data technologies today, was built on research from Google and initially deployed at Yahoo. Google invented this technology because indexing the Web was infeasible with existing systems. Now companies adopting Hadoop are bringing cloud architecture into their data centers.

    The simultaneous rise of cloud and big data technologies isn’t coincidental. They’re mutually reinforcing. The expansion of the cloud continues to drive both the creation of new big data technologies and big data adoption by making it easier and cheaper to access storage and computing resources. Big data infrastructures also play a role in this trend. For instant, recent advances in the Apache Hadoop ecosystem enable more types of workloads and more tenants to share a cluster. Users will be able to run more types of infrastructures on the same Hadoop cluster, which itself may be running on a cloud infrastructure. As big data infrastructures become more generic, the cloud infrastructure will add more
    specialized services for data storage, processing, and analysis.

    The combination of big data, cloud computing, and new algorithms and techniques for visualizing information enables converged analytics—performing analytics on data from many different sources. These new techniques for data delivery and data management also enable cloud-based Analytics as a service (AaaS).

    From security and privacy to pricing models, the combination of big data and cloud computing is having a substantial impact on the nontechnical aspects of our lives as well. There is a tension between our desire for converged analytics and cloud computing, which is about sharing more computing resources and data with increasingly diverse tenants, and our desire for better privacy controls and data protection. Usage-based pricing models are forcing us to rethink how we produce and consume technology. Future columns will look at how policies and economics are being shaped by these technological advances.

    Future columns will examine how people are using these trends together, big data developments from cloud builders, and how people are making the cloud better through data.

    Source: http://doi.ieeecomputersociety.org/10.1109/MCC.2014.12

    – ChauNH2 – 

Leave a Reply

Your email address will not be published. Required fields are marked *

… [Trackback]

[…] Read More Infos here: tech-community.fpt-software.com/intersection-of-the-cloud-and-big-data/ […]