Code Documentation

This section provides an overview of the key classes and functions implemented in the WordStream-Extension project. The documentation covers the primary files essential for the extension, including their roles and functionalities.

sentiment.html

Libraries & Frameworks

D3.js (v7): For dynamic visualizations like the wordstream.
D3 Cloud: Used to generate word clouds.
jQuery: For DOM manipulation and UI management.
noUiSlider: Controls sentiment threshold sliders.
Custom CSS: Provides styling for the layout, wordstream, sliders, and UI.

HTML Structure

.title: Contains page navigation links.
#datasetSelector: Dropdown for selecting the sentiment dataset.
#positiveThreshold & #negativeThreshold: Sliders for adjusting sentiment thresholds.
#wordstream: Visualization container for the wordstream.
#modalBackdrop & #detailModal: Modal elements for detailed view.

JavaScript Functionality

D3.js: Dynamically generates the wordstream visualization.
noUiSlider: Manages sentiment threshold adjustments.
smain.js: Initializes and updates visualizations based on user input.
jQuery: Handles UI interactions (e.g., opening modals, updating UI elements).

Key Features

Dataset Selection: Dropdown to select sentiment datasets.
Threshold Sliders: Adjusts positive and negative sentiment filters.
Wordstream Visualization: Displays sentiment analysis in word cloud form.
Detailed Modal View: Provides in-depth visualizations in a modal.
Loading State: Displays a spinner while processing data.

File Dependencies

jquery-ui.css, dd.css, d3.slider.css, main.css, wstyles.css, sstyles.css: Styles for UI components and visualizations.
jquery-1.9.0.min.js, jquery-ui.min.js, nouislider.min.js: Libraries for UI controls and sliders.
smain.js: Custom JS for sentiment analysis logic.

smain.js

Variables

selectedCategories: Array of selected categories for word filtering (e.g., 'location', 'person').
margin, width, height: Layout settings for chart visualization.
words: Array of words to display in the visualization.

Functions

loadData(dataset): Loads and processes the sentiment dataset.
showDetail(wordText): Displays a modal with detailed sentiment analysis for a word.
updateVisualization(): Updates the word cloud based on selected categories and thresholds.
createTimeline(years): Visualizes word trends over time.

Classes & IDs

#detailModal, #modalBackdrop, .loading-spinner, #wordstream, #detailChart: UI elements for modals, loading states, and visualization containers.
.tooltip, .word, .year-marker: For visual interactivity and markers.

Interactive Elements

positiveThreshold: Slider for adjusting the positive sentiment threshold.
negativeThreshold: Slider for adjusting the negative sentiment threshold.
#categorySelector: Dropdown for selecting categories to filter the word cloud.
#datasetSelector: Dropdown for selecting the dataset to load.

Event Listeners

noUiSlider on positiveThreshold: Updates the visualization based on the selected positive sentiment threshold.
noUiSlider on negativeThreshold: Updates the visualization based on the selected negative sentiment threshold.
#categorySelector: Triggers the update of the word cloud when categories are changed.
#datasetSelector: Triggers the loading of a new dataset when the selection changes.

sstyles.css

noUi-handle: Customizes slider handle appearance.
title: Styles the page title.
container, sidebar: Layout and positioning of main content and sidebar.
visualization: Styles for the wordstream and visualization area.
controls: Controls styling for sticky top bar.
modal-related: Styles for modals (e.g., backdrop, close button, content).
loading-spinner: Appearance and positioning of the loading spinner.
wordstream, timeline: Adjusts layout and visual effects for the word cloud and timeline.

Wordcloud sentiment.html

The Wordcloud sentiment.html file provides the structure and layout for the Sentiment Cloud visualization within the WordStream-Extension project. It includes the necessary HTML elements, external libraries, and custom scripts to render and interact with the sentiment word cloud.

<div class="title">: Navigation bar with links to different sections of the WordStream application.
<div class="container">: Main container that holds the sidebar and visualization sections.
<div class="sidebar">: Sidebar containing control elements for dataset selection, category filtering, and threshold adjustments.
<div class="visualization">: Container for the word cloud visualization and legend.
<div id="wordcloud" class="wordcloud">: Element where the word cloud is rendered.
<div id="yearSlider" class="slider">: Slider for selecting the year range.
<div id="positiveThreshold" class="slider">: Slider for adjusting the positive sentiment threshold.
<div id="negativeThreshold" class="slider">: Slider for adjusting the negative sentiment threshold.

d3.v7.min.js: D3.js library for creating data-driven visualizations.
d3-scale-chromatic.v1.min.js: D3.js module for color scales.
d3.layout.cloud.min.js: D3.js layout for word clouds.
jquery-1.9.0.min.js: jQuery library for DOM manipulation.
jquery-ui.min.js: jQuery UI for interactive elements.
nouislider.min.js: noUiSlider library for creating sliders.

wmain.js: Custom JavaScript for handling the functionality and interactivity of the Sentiment Cloud visualization.
wstyles.css: Custom CSS for styling the Sentiment Cloud visualization and its components.

Wmain.js

The Wmain.js script manages the functionality and interactivity of the Sentiment Cloud visualization within the WordStream-Extension project. It handles data loading, filtering, and rendering of the word cloud based on sentiment analysis.

words: Array to store processed word data.
width and height: Dimensions of the visualization canvas.
minFontSize and maxFontSize: Font size range for words in the word cloud.
yearRange: Object to store the minimum and maximum years in the dataset.
selectedCategories: Array of categories selected for filtering words.

colorScale: D3 sequential color scale for mapping sentiment scores to colors.

layoutPositive: D3 layout for the positive sentiment word cloud.
layoutNegative: D3 layout for the negative sentiment word cloud.

drawPositive(words): Renders the positive sentiment word cloud.
drawNegative(words): Renders the negative sentiment word cloud.

initializeSliders(): Initializes the year, positive threshold, and negative threshold sliders using noUiSlider.

updateFilters(): Filters words based on selected year and sentiment thresholds, then updates the word clouds.

loadData(dataset): Loads and processes the selected dataset, updating the word cloud and category selector.

updateCategorySelector(categories): Updates the category selector dropdown based on available categories in the dataset.

init(): Initializes the visualization, sets up event listeners, and loads the initial dataset.

wstyles.css

The wstyles.css file provides the styling for the Sentiment Cloud visualization within the WordStream-Extension project. It ensures a cohesive and visually appealing user interface by defining styles for various elements and components.

.container: Styles the main container, using flexbox for layout and spacing.
.sidebar: Styles the sidebar, including padding, background color, border, and height.
.control-group: Styles control groups within the sidebar, including layout, margin, and padding.
#datasetSelector, #categorySelector: Styles the dataset and category dropdown selectors.
.slider: Styles the sliders for year and sentiment thresholds.
.noUi-target, .noUi-horizontal, .noUi-connect, .noUi-handle, .noUi-tooltip: Styles for the noUiSlider components, including track, handle, and tooltips.
.visualization: Styles the visualization container, ensuring it flexes to fill available space.
.wordcloud: Styles the word cloud container, including dimensions, border, and position.
.word: Styles individual words in the word cloud, including cursor and hover effects.
.title: Styles the title bar, including padding.
.button: Styles navigation buttons, including padding, margin, border, and active state.
.legend: Styles the legend container, including layout, background, border, and shadow.
.legend div: Styles individual legend items, including dimensions, text alignment, and background colors for positive and negative sentiments.
.middle-line: Styles the middle line in the word cloud, indicating separation between positive and negative sentiments.

Data Folder

The /data folder contains all datasets used for the WordStream-Extension, along with scripts for preprocessing the data.

The /data/preprocess directory includes Python scripts responsible for cleaning, processing, and preparing raw data for visualization. Each script contains the name of the dataset it preprocesses, the original datasets can be found in /data/raw.

We integrated three new datasets into WordStream: Rotten Tomatoes movie reviews, CNN news articles, and Reddit posts from the /datasets subreddit. The preprocessing pipeline for these datasets included:

Data Collection: Acquired datasets from trusted sources, ensuring a diverse range of text corpora.
Data Cleaning: Performed text normalization, including lowercasing, removal of special characters, and handling of missing values to ensure consistency across datasets. For this we used Pandas.
Keyword Extraction: Utilized the SpaCy library in Python to extract relevant keywords from each dataset, enhancing the quality of the visualization.
Sentiment Analysis: Applied the VADER Sentiment Analyzer to compute sentiment scores for each text entry, categorizing sentiments as positive, neutral, or negative.
Additional data Transformation: Necessary to visualize the data in the new visualization we built. The data was aggregated data by year, calculating the average sentiment per text and the frequency of each keyword annually to facilitate temporal analysis.

Other datasets containing data such as social media posts and fact-check articles were added. However, due to reasons that we were not able to understand, these datasets proved to be difficult to visualize in the WordStream. In order not to waste the effort invested in these datasets, they are only visualizible in the SentimentCloud and SentimentStream tabs.