Architecture for enduring knowledge-extraction from online social networks

Abstract

Nowadays social networks and media play significant role in daily life. All our life in the real world is recorded in the digital space as well. Scientists have enormous potential in researching issues such as social influence on top news and top news influence on society. Its impact on daily life spans such diverse areas as digital marketing, public opinion analysis, political monitoring and disaster notification. Any task of processing such a large data stream needs a coherent architecture that will fit the analyzed resource. In the presented work, we set ourselves the task of creating a highly loaded, fault-tolerant, scalable system for extracting and processing data from various social networks and analyzing data in real time. The solution is architecture in the form of a set of modules. Modules have their own characteristics depending on the work performed, from collecting textual data to direct processing and extraction of knowledge.