Businesses are constantly looking for more cost effective, salable and flexible ways to gain insights from the huge volume and variety of data that they are collecting. With unprecedented amounts of information being collected about customers, systems, operations, web interactions, product usage,etc., there is huge opportunity for businesses to make better decisions, perform more efficiently, and gain competitive advantage. However traditional analytics systems are not ready to deal with this data deluge.Enterprise data warehouses tend to be expensive and inflexible, and traditional OLAP tools are limited by performance and scalability issues when dealing with Big Data.
These are just some of the reasons enterprises are moving to Hadoop as a complementary solution to their data warehouses and datamarts. Hadoop provides the flexibility and cost effective scalability required to process and store huge amounts of structured and unstructured data using commodity hardware, and this open source framework scales seemingly without limits. But, when it comes to analyzing all that data, Hadoop has some major limitations.
Hadoop does provide scalability and flexibility, but makes it difficult for business users to access and manipulate data. It is difficult to use, and connecting traditional BI and analytics tools directly to Hadoop typically results in significant performance issues. Even with the advances in Hive and Impala and the advent of Spark, Hadoop is inherently a batch-processing environment,not designed for interactive analytics.
And while the cost of compute power and storage has reduced dramatically over the years, the cost of human time is more expensive than ever. It is not acceptable to submit a query and wait for several minutes for a response, even when dealing with huge amounts of data. Business users are demanding self-service and interactive analytics, along with the ability to deal with data at any scale and granularity, with instantaneous response times.
Taming Big Data
Kyvos Insights, headquartered in Los Gatos, CA, has taken direct aim at these problems with an emphasis on Business Intelligence for Big Data. According to Ajay Anand, the company’s vice president of products, “We address the need for self-service, interactive analytics on Big Data by delivering an OLAP solution that works directly on Hadoop. Our ‘Cubes on Hadoop’ technology eliminates the performance and scalability limitations of traditional solutions, and enables business users to do interactive, multi-dimensional analytics on Hadoop at unprecedented scale.”
The Kyvos solution has been specifically designed for Big Data applications and works directly on data that has been loaded into the Hadoop Distributed File System (HDFS). Data can be in both structured and semi-structured formats from a wide variety of sources. There is no limitation on volume.
Kyvos provides a simple, visual interface that allows users to transform data and design and build multidimensional OLAP cubes in a distributed environment on Hadoop.
These cubes can be built on data with hundreds of billions of rows and dimensions with cardinality in the hundreds of millions. And even with data of this size, users can interact with the cubes in real time, getting a response to even complex queries within seconds.
The Kyvos solution scales linearly as the volume of data increases, allowing cubes with trillions of cells to be built and stored in a distributed manner. Incremental cube builds allow new data to be ingested into the system efficiently. Cubes can be structured to include data at the lowest levels of granularity, as well as aggregates for different levels of the cube’s hierarchal dimensions. A business user can explore and drill down to the lowest levels of granularity in the data interactively.
“By structuring data within the cube in this fashion, Kyvos has created a unique solution that transforms Hadoop from a batch processing environment into an interactive user experience,” says Anand. “Given these near instantaneous response times, business users can browse, slice and dice, drill up and drill down, and visualize their data at any scale using Kyvos.”
The Kyvos solution empowers business users to access, visualize and interact directly with their data on Hadoop, at any scale, without depending on programmers or IT resources.
Just a few of the many benefits of this unique approach include:
- It removes the complexity of Hadoop, enabling business users to directly access Big Data without needing to write any code
- Users are able to visually interact with their data, and explore, slice and dice and examine all aspects of their data with instantaneous response times
- Users can simply and visually transform their data, define data relationships and business logic, with a simple, drag and drop user interface
- Cubes for for multidimensional analytics are built directly on Hadoop, with linear scalability, eliminating the data size limitations of traditional OLAP solutions
- Incremental cube builds enable new data to be quickly and efficiently added to the cubes
- Because analytic processing is done on Hadoop, data does not have to be moved to an external data mart or analytics platform
- Query responses are in real time, even when dealing with huge amounts of data
Concludes Anand, “Kyvos allows IT to provide its business users with an indispensable tool that lets them become productive immediately, without requiring any programming or Hadoop experience. This is a comprehensive and unique approach to OLAP specifically designed for Big Data – one that combines exceptional scalability with self-service, interactive analytics.”