SocialScope: A Real-Time Semantic Event Retrieval Framework using Deep Sentence Embeddings and Sentiment-Aware Ranking

In the fast-paced digital world we live in, it has become increasingly important to track and know what is happening today, in real-time, on social media platforms. This article presents the development of a Semantic Event Search Engine that retrieves and organizes realtime data from Twitter, Reddit, YouTube, and Google News. The purpose of this system is to allow users to search for any ongoing or past event and provide them with the most salient and contemporary content, regardless of whether the topic is trending worldwide or not. This system uses a hybrid approach for data fetching - backfilling scheduled hourly scraping on predefined trending queries with on-demand, real-time data retrieval on new user queries. We used a MongoDB backend as a way to store, query, and update significant volumes of event data in a scalable way with minimal overhead. The architecture of the system is modular, with separate fetcher modules for each platform. Because of this modularity, we can easily add future extensions such as semantic ranking, sentiment analysis, and event clustering. Overall, the demonstration project shows how integrating real-time data and a better semantic understanding of events can enhance information discovery across social platforms. The work seeks, as Godber says, to bridge the gap between event detection and user intent. This makes it especially useful for journalists, researchers and analysts and in general users that seek information on events in a timely and relevant manner.