Efrat Nissimov Efrat Nissimov Director of Product Management - cVidya

Featured Blogs

Hadoop - A Big Data Breakthrough

  • Today, most of the business risks faced by communications service providers (CSPs) revolve around the exponential growth in data traffic on telecoms networks. One of the biggest challenges CSPs face in containing these risks, which include fraud and revenue leakage, is the enormous variety of applications and use cases generating all this data traffic.

    As they surf the web, check Facebook, use apps, exchange messages, watch videos and swap files, a CSP’s customers can quickly generate billions of data points. To prevent fraud, detect revenue leakages, enable up-selling and open up new revenue streams, CSPs need to store and analyze all these data points. But trying to do that using conventional data warehouses simply isn’t financially viable.

    Massive parallel processing

    The only feasible way to analyze very big data is to use distributed computing via Hadoop. What exactly is Hadoop? It is a standards-based, open-source software framework that harnesses inexpensive commodity servers to provide massive parallel processing.

    Crucially, Hadoop has made it cost-effective to use deep packet inspection (DPI) to detect fraud and revenue leaks, as well as identify new revenue opportunities. DPI solutions can show a CSP which web sites a customer is visiting, what apps they are using, who is in their social network and the kind of traffic flowing between two different points on its network. As a result, DPI generates vast amounts of data: For example, one of our customers in North America is generating almost 32 million records per second, which simply wouldn't be possible to collect or analyze without Hadoop.

    Together, Hadoop and DPI can have a profound impact on a CSP's Fraud Management, Revenue Assurance and Marketing Analytics capabilities. As most fraud today targets metered data services, rather than unmetered voice and messaging services, DPI is a key tool for a fraud manager. Many CSPs' data plans now come with specific add-ons, such as unlimited access to YouTube for an extra $5 a month. While these add-ons can increase ARPU, they are also vulnerable to service abuse and fraudulent activities. For example, a customer might try to disguise an application so it appears to be something else in order to avoid payment– a CSP can detect this kind of fraud only by using DPI. 

    With all that data, CSPs can also identify the interest areas of particular customers with a very high degree of accuracy. They can use that info to better sell their own products and services as well as those of third parties, such as media companies, retailers and app developers.

    More comprehensive, fewer compromises

    Hadoop can also help a CSP's Revenue Assurance team deal with much greater scale and complexity, enabling it to detect and plug leaks much faster.  As the number of connected SIMs rises and the Internet of Things expands, CSPs’ Revenue Assurance solutions will have to analyze a much greater volume of data from many more sources.

    Without Hadoop, CSPs will have to be highly selective. One of our customers in APAC, (which we believe to have deployed the largest centralized Revenue Assurance solution in the world) for example, is only able to store 30% of the data it collects for Revenue Assurance purposes. And this data can only be stored for between 14 and 30 days. With Hadoop, it could store 100% of the data for 120 days, enabling it to detect more fraud and prevent more revenue leakage, while still achieving a cost reduction of 70%.

    To enable all of our customers to harness the power of Hadoop to enhance Fraud Management, Revenue Assurance and Marketing Analytics, cVidya is offering different implementation models, each addressing different needs. We can supply a complete solution, encompassing all the hardware and software needed for Hadoop, or we can integrate our solutions into the CSP’s existing Hadoop deployment. Whichever implementation model they choose, a CSP will quickly see major benefits. Although small operators (with approximately 1 million subs) won't necessarily realise significant cost savings, they will be able to store far more information without incurring additional expenses. For medium and large operators, Hadoop is even more attractive - they can expect to see a major increase in their capabilities, while still saving 70% on the cost of hardware and software.

    But these immediate benefits may just be the tip of the iceberg. Ultimately, the business benefits of being able to properly analyse customer data could be enormous. As our understanding of the potential of big data increases, many further applications and insights are likely to emerge.

    In other words, we believe there is still a huge amount of value to be discovered. Hadoop will help us find that value, to the benefit of the communications industry and its billions of customers.


    Efrat Nissimov
    About Efrat Nissimov Efrat Nissimov works as Director of Product Management at cVidya
    More information : www.cvidya.com