Some of the most challenging problems in big data today are in online advertising. BlueKai, the largest online data exchange processes 2 Trillion transactions per month, Liverail processes 25% of all video ads on the internet and adMarketplace stores over 100 million user profiles and data on 10 million distinct traffic sources. These platforms demand ultra-low latency, but each request results in very complicated aggregations of analysis and data. I recently talked to Alex, Andrei and Mike from BlueKai, Liverail and adMarketplace during dataweek 2012:
Liverail CTO Andrei Dunca: LiveRail is an online advertising technology company. We focus exclusively on video advertising. We provide technology solutions for the entire video advertising industry from the sell side to the buy side. We service publishers, vertical networks, ad networks, trading desks, DSP’s and even agencies.
adMarketplace CTO Mike Yudin: adMarketplace does what’s called search syndication. So, you’ve probably heard at sessions here that it’s all about display marketing mostly, and the gold standard that everybody’s trying to get to and can’t quite get to is search, right? Google search, AdWords, that’s the gold standard and nobody talks about this, right? Why? Because Google figured it out, so what’s there to talk about? It’s been figured out.
So, it turns out, not quite – comScore just published a study that 40% of all searches online happen outside of search engines. So, you may not even realize this, but Google kind of conditioned all of us to use search boxes, and search boxes are all over the place. You go to Amazon, what do you do? You search. You misspell a URL and you get to a parked domain, and there’s going to be a search box. There are tool bars in our browsers and we search in these tool bars.
So, there are all these different searches that happen and the search is the way to express user intent. So, what our company does is aggregate all this non-major search engine search traffic from all these disparate traffic searches. There are lots of them and so it’s a very difficult and big data problem.
We like to talk about this in three dimensions of targeting:
- So, you’ve got the user. That’s dimension one.
- This user is coming to you from a certain traffic source, that’s two.
- The keyword itself or the search term, that’s three.
So, that’s a very, very big data problem and it’s hard technologically. We’re here to talk about how the rubber hits the road, how this all happens, so that, when you see a webpage and there’s an ad on this page, it shows up instantaneously and correctly.
Alex Hooshmand, Co-Founder, Chief Strategy Officer & SVP Operations, BlueKai: BlueKai started out as the first and now largest online data exchange. What that means is, we go to websites where useful behavior by users is occurring, where we capture information about what users are doing. We get that data out to places where that data can be used for ad targeting.
About two years ago, we also started looking at providing a platform for advertisers and publishers to capture, organize, and also get their data out to places where ads could be targeted.
Most recently, we’ve been expanding not only to getting data out to places where ads are targeted, but also getting data to wherever it could be used, so e-commerce systems, site optimization, video – trafficking and targeting systems. So, think of us as a data activation system, a data management platform, and on both fronts I think we’re probably the largest out there.
Watch the video and see what Alex, Andrei and Michael have to say about the challenges of architecting and operating Real-time Big Data platforms.