Cloud Discussion-Use cases of Cloud in terms of Media, Distribution and Analytics
Use cases of Cloud in terms of Media, Distribution and Analytics.
You take any cloud you have media sharing and media distribution services.
Azure has CDN, AWS has CDN, Azure has Streaming Service, AWS has streaming services.
AWS will give you the bit torrent option also. It will give you the bit torrent download urls for storage as well. NEXTFlix is the most popular implementation of it.
Academic computing, uses the CDN and storage for their Lab data etc.
Search Engine:
Why are we talking about search engine here in media and distribution services, the reason is today the way we use search engine is totally different, if you look at google, yahoo these are the example of web content search engines.
Those days are gone, today's search are not my content as are form of web content. so normal search engine can't search these content (unstructured). If you will look today at your contents- are in form of no SQL data, example you look at any social n/w ing site, your data is in no sql, look at your Facebook, look at your Twitter, or look at blogs or Linked-in, back end is no SQL.
When I say data is no SQL then point is how can I search the data? the most important thing is how can you get inside out of these data? Inside means nothing but how can I perform the analytics on these data?.
Example- you will take example of normal application, you will have OLTP data store, day to day data will get accumulated into OLTP, then I 'll create the OLAP data and I'll run simple ETL operation, which extracts the data from OLTP and put into OLAP then feed this OLAP data to Data Warehouse Engine then its generates the Quebe and Quebe get used by reporting tools to generates the reports, these are called analytical report.
So this the the structured way of performing analytics. So what will happen if data is in form of NO-SQL, so how can you analyse those data?
That's where we are using tools like Search Engine, that means let me feed whole data to the search engine, lets create the indexes of particular data, these search engines giving you such APIs which are enriched with Analytics.
Example..
Tell me where you call search and where you call Analytics. may be as a end user you will see it is search but actual at back end it is happening analytics, example..you take Zomato, you stand one place and you say that - show me the restaurants within the range of 5 KM, so it will give you the result like..
1-2KM=> number of restaurants,
2=4=> number of restaurants
...
so and so on. then you will click on any one of the link and it will display all the restaurants from that range.
Do you call it a search? Yes, from end user perspective it is a search but from backend you have very good aggregation that has performed then you have something called like group by, so they have performed at the back end called analytics.
If you look at hotel booking site's data there is nothing in form of structured data, whole hotel data is in form of no-sql. this the way we are using the search engine now a days, thats call big data analytics based search engines.
Lets take an example of Hadoop, it is big data management system, so now whole data is in Hadoop the how can I derive the analytics out of those data? ye we can... let me feed my data to search engine and perform analytics on it, then use the apis to get and query the data to generate the report. This it the way we are using search engine now a days.
There is the one popular search engine available in the market from Apache is called Lucien, but Lucien is raw, it set of jar files nothing else, so many organization took this Lucien and wrote their wrappers on top of Lucien.
One of the popular wrapper is SOLR, it is very popular one. but it was popular initially but it was lacking on what you call is scalability, because it was only one node deployment at that time. Now we can say 1 or 2 ear back SOLR came with multi node deployment that is called SOLR CLOUD.
But at the same time there was one popular company who wrote is own wrapper which was excellent product that is nothing but Elastic Search, again its wrapper over Lucien and it is known for multi server deployment. and today n number of places we are using Elastic Search, Ebay, PayTM are using Elastic search, because all these places data is no-sql data so how can I analysed it?.
In E bay, Paytm they using search on ECOM to give the recommendation, they are not actually the search but they are more on Analytics side. If you look at Amazon, Amazon has native support for Elastic Search. Elastic search is service in Amazon.
Then come to social n/w, what is most important is huge data, where to store huge data, so if you look at any cloud they have no-sql database as a service, Azure comes with what we called as a Table (native storage) and Document Storage (now they have cosmos db), Amazon comes with Dynamo DB, lie on prime MONGO, CASENDRA,
You take any cloud you have media sharing and media distribution services.
Azure has CDN, AWS has CDN, Azure has Streaming Service, AWS has streaming services.
AWS will give you the bit torrent option also. It will give you the bit torrent download urls for storage as well. NEXTFlix is the most popular implementation of it.
Academic computing, uses the CDN and storage for their Lab data etc.
Search Engine:
Why are we talking about search engine here in media and distribution services, the reason is today the way we use search engine is totally different, if you look at google, yahoo these are the example of web content search engines.
Those days are gone, today's search are not my content as are form of web content. so normal search engine can't search these content (unstructured). If you will look today at your contents- are in form of no SQL data, example you look at any social n/w ing site, your data is in no sql, look at your Facebook, look at your Twitter, or look at blogs or Linked-in, back end is no SQL.
When I say data is no SQL then point is how can I search the data? the most important thing is how can you get inside out of these data? Inside means nothing but how can I perform the analytics on these data?.
Example- you will take example of normal application, you will have OLTP data store, day to day data will get accumulated into OLTP, then I 'll create the OLAP data and I'll run simple ETL operation, which extracts the data from OLTP and put into OLAP then feed this OLAP data to Data Warehouse Engine then its generates the Quebe and Quebe get used by reporting tools to generates the reports, these are called analytical report.
So this the the structured way of performing analytics. So what will happen if data is in form of NO-SQL, so how can you analyse those data?
That's where we are using tools like Search Engine, that means let me feed whole data to the search engine, lets create the indexes of particular data, these search engines giving you such APIs which are enriched with Analytics.
Example..
Tell me where you call search and where you call Analytics. may be as a end user you will see it is search but actual at back end it is happening analytics, example..you take Zomato, you stand one place and you say that - show me the restaurants within the range of 5 KM, so it will give you the result like..
1-2KM=> number of restaurants,
2=4=> number of restaurants
...
so and so on. then you will click on any one of the link and it will display all the restaurants from that range.
Do you call it a search? Yes, from end user perspective it is a search but from backend you have very good aggregation that has performed then you have something called like group by, so they have performed at the back end called analytics.
If you look at hotel booking site's data there is nothing in form of structured data, whole hotel data is in form of no-sql. this the way we are using the search engine now a days, thats call big data analytics based search engines.
Lets take an example of Hadoop, it is big data management system, so now whole data is in Hadoop the how can I derive the analytics out of those data? ye we can... let me feed my data to search engine and perform analytics on it, then use the apis to get and query the data to generate the report. This it the way we are using search engine now a days.
There is the one popular search engine available in the market from Apache is called Lucien, but Lucien is raw, it set of jar files nothing else, so many organization took this Lucien and wrote their wrappers on top of Lucien.
One of the popular wrapper is SOLR, it is very popular one. but it was popular initially but it was lacking on what you call is scalability, because it was only one node deployment at that time. Now we can say 1 or 2 ear back SOLR came with multi node deployment that is called SOLR CLOUD.
But at the same time there was one popular company who wrote is own wrapper which was excellent product that is nothing but Elastic Search, again its wrapper over Lucien and it is known for multi server deployment. and today n number of places we are using Elastic Search, Ebay, PayTM are using Elastic search, because all these places data is no-sql data so how can I analysed it?.
In E bay, Paytm they using search on ECOM to give the recommendation, they are not actually the search but they are more on Analytics side. If you look at Amazon, Amazon has native support for Elastic Search. Elastic search is service in Amazon.
Then come to social n/w, what is most important is huge data, where to store huge data, so if you look at any cloud they have no-sql database as a service, Azure comes with what we called as a Table (native storage) and Document Storage (now they have cosmos db), Amazon comes with Dynamo DB, lie on prime MONGO, CASENDRA,
Comments
Post a Comment