The hospitality industry is a service sector fundamentally shaped by customer perception and satisfaction. In the current digital era, online reviews have become one of the most influential sources of information for prospective guests prior to making accommodation decisions (Caryn et al., 2025). Google Maps, through its integrated review feature, has emerged as one of the most widely used platforms for hotel reviews in Indonesia, given its accessibility and high user adoption rate (Haq & Rachmat, 2020). Reviews posted on this platform directly influence a hotel's digital reputation-positive reviews strengthen public trust and attract new guests, while unaddressed negative reviews can progressively erode the hotel's image and reduce booking intent (Syafrimansyah, 2023). This dynamic makes timely and systematic review monitoring a critical operational necessity for hotel management.

The volume of reviews on Google Maps continues to grow at a pace that renders manual monitoring increasingly impractical. Without an automated mechanism, negative reviews may go undetected for extended periods, preventing hotel management from responding in a timely manner (Litvin et al., 2008). This challenge is directly observed at Aveta Hotel Malioboro, which had accumulated more than 3,430 reviews on Google Maps as of January 2026. The current monitoring process at the hotel relies on manual review checks, with notifications concentrated within a single account, resulting in slow information dissemination and delayed responses-particularly during peak operational periods. The absence of a structured, automated system creates a significant operational gap that can affect both service quality and the hotel's competitive standing.

Sentiment analysis offers a viable solution to this problem by enabling the automatic classification of customer opinions into predefined polarity categories, such as positive and negative (Liu, 2012). Among the available machine learning approaches, Naive Bayes and Support Vector Machine (SVM) have been widely adopted for text-based sentiment classification due to their complementary strengths: Naive Bayes performs efficiently on short, informal texts, while SVM demonstrates robust generalization on high-dimensional feature spaces commonly found in natural language data (Dwitama & Paputungan, 2025). When combined with modern data retrieval techniques and notification infrastructure, these algorithms can serve as the backbone of an operational monitoring system that responds to new reviews in near real-time. Prior studies on sentiment analysis using Naive Bayes and SVM have consistently demonstrated the effectiveness of these algorithms across various review-based datasets. Handayanto et al. (2021) evaluated both methods on Google Reviews of Summarecon Mall Bekasi and found that SVM achieved higher accuracy than Naive Bayes, supporting its suitability for practical deployment. Alfiyanti & Indra (2024) further explored this comparison on Google Maps reviews of Taman Mini Indonesia Indah (TMII), finding that SVM excels in precision critical for correctly flagging negative reviews while Naive Bayes exhibits stronger recall. (Ipmawati et al. (2024) confirmed SVM's practicality in a tourism review context on Google Maps with an accuracy of 87%, while Sarimole & Kuadrat (2024) reported similar trends on Twitter data, where SVM reached 87.95% accuracy compared to Naive Bayes at 81.65%. Despite these well-established findings, a consistent limitation across all prior studies is their exclusive focus on model evaluation without developing any functional monitoring system. None of the reviewed works integrated the trained model into an end-to-end system capable of automatically retrieving new reviews, classifying sentiment, storing results, and notifying stakeholders in real time.

This gap motivates the present study, which bridges algorithmic evaluation and system implementation by proposing a web-based hotel review monitoring system that integrates Naive Bayes and SVM sentiment classification with SerpAPI-based review retrieval and Telegram Bot notifications. The novelty of this research lies not only in comparing the two algorithms under both original and resampled class distribution scenarios, but also in embedding the best- performing model into a fully operational monitoring system tailored to the hospitality context. Accordingly, this study aims to first to evaluate and compare the performance of Naive Bayes and SVM in classifying sentiment from Google Maps hotel reviews under varying class distribution conditions and secondly to design and implement a web-based monitoring system that automates review retrieval, sentiment classification, and real-time notification delivery to support data-driven hotel review management.

Sentiment Analysis

Sentiment analysis, also referred to as opinion mining, is a field within natural language processing, computational linguistics, and text mining that systematically analyzes opinions, sentiments, evaluations, and emotions expressed by a speaker or writer toward a particular topic, product, service, or organization (Liu, 2012). This study applies binary document-level sentiment analysis to Indonesian-language Google Maps reviews of Aveta Hotel Malioboro, where the primary objective is to automatically distinguish negative feedback from positive feedback to support timely managerial response.

Google Maps Review

Google Maps Review is a user-generated content feature embedded within Google Maps that allows individuals to submit written reviews and star ratings for places, businesses, and services they have visited (Haq & Rachmat, 2020). As one of the most widely accessed location-based platforms globally, Google Maps serves as a high-visibility channel where customer opinions are publicly visible and directly influence a business's perceived reputation. For hotels, reviews on this platform are particularly impactful because they are indexed by Google Search and accessible without requiring an account, making them a primary reference point for prospective guests during accommodation decision-making (Syafrimansyah, 2023). In this study, Google Maps reviews are selected as the primary data source due to their public accessibility, structured metadata, and direct relevance to hotel reputation management in the Indonesian hospitality context.

Text Preprocessing

Text Preprocessing is a foundational step in natural language processing that transforms raw, unstructured text into a clean and structured format suitable for computational analysis (Khofifah et al., 2022). Raw user-generated text, such as hotel reviews, typically contains informal language, typographical errors, slang, punctuation, and irrelevant characters that introduce noise into machine learning models. To address this, preprocessing in this study consists of six sequential stages: (1) Case Folding—converting all text to lowercase to ensure uniformity; (2) Cleaning—removing punctuation, symbols, numbers, and non- alphabetic characters; (3) Tokenization—splitting sentences into individual word tokens; (4) Normalization—standardizing informal expressions and slang words using a custom domain-specific dictionary; (5) Stopword Removal—eliminating high-frequency words that carry no sentiment value; and (6) Stemming— reducing words to their root form using the Sastrawi library for Indonesian text. Each stage contributes to improving feature quality and, consequently, model classification accuracy (Gabriella et al., 2023).

Web Scraping

Web scraping is an automated data extraction technique that retrieves structured information from web pages by parsing their underlying HTML structure (Dwicahyo & Ratnasari, 2023;Khder, 2021). While web scraping broadly refers to the programmatic collection of data from websites, DOM (Document Object Model) scraping specifically targets the rendered DOM tree of a web page including dynamically loaded content generated by JavaScript making it particularly suitable for platforms such as Google Maps, where review data is not embedded in the static HTML source but rendered dynamically at runtime (Khder, 2021a). DOM scraping was selected in this study specifically for the initial historical data collection phase because Google Maps renders review content dynamically through JavaScript, making static HTML parsing insufficient for comprehensive data retrieval.

Figure 1. Research Conceptual Framework

Based on the theoretical studies and previous research reviewed, it can be concluded that the growing volume of online customer reviews on platforms such as Google Maps presents both an opportunity and a challenge for hotel management. While user-generated reviews contain valuable insights into customer satisfaction, the manual monitoring of such reviews is time-consuming, inconsistent, and prone to delayed responses. Most existing studies have addressed sentiment analysis and review monitoring as separate concerns, without integrating automated data collection, real-time classification, and instant notification into a single unified workflow. Therefore, this study combines automated data retrieval via SerpAPI, sentiment classification using Naive Bayes and Support Vector Machine (SVM) with TF-IDF feature extraction and oversampling for class imbalance, and real-time alert delivery via Telegram Bot into a single integrated monitoring system, aimed at producing a more responsive and data-driven solution for hotel reputation management.

Data Collection

A total of 2,449 historical customer reviews of Aveta Hotel Malioboro were collected from Google Maps spanning February 2020 to May 2025, using DOM (Document Object Model) scraping via browser-based automation. This technique was selected because Google Maps renders review content dynamically through JavaScript, making it inaccessible via static HTML parsing (Dwicahyo & Ratnasari, 2023). The collected data includes user names, timestamps, star ratings, and review text.

Figure 1. Historical Data Scraping Flow

Text Preprocessing

Text preprocessing was applied to clean and standardize raw review text prior to feature extraction. The pipeline consisted of six sequential stages: (1) Case Folding converting all text to lowercase; (2) Cleaning removing punctuation, numbers, and non-alphabetic characters; (3) Tokenization splitting sentences into individual tokens; (4) Normalization—standardizing informal language using a custom hotel-domain slang dictionary; (5) Stopword Removal eliminating non- informative words while retaining negation terms such as tidak, bukan, and jangan that carry significant sentiment value; and (6) Stemming reducing words to their root form using the Sastrawi library (Khofifah et al., 2022).

Model Training and Evaluation

Two classification models were trained: Multinomial Naive Bayes and Linear Support Vector Machine (LinearSVC). Hyperparameter optimization was conducted using GridSearchCV with 5-fold Stratified K-Fold Cross Validation and macro F1-score as the optimization metric, ensuring class balance was maintained across each fold (Alfiyanti & Indra, 2024). Two experimental scenarios were evaluated: (1) original class distribution, and (2) oversampled training data. Optimal parameters were alpha=1.0 and C=0.05 for the original scenario, and alpha=0.01 and C=2 for the oversampling scenario. Model performance was assessed using accuracy, precision, recall, and F1-score.

Figure 2. System Flowchart Diagram

Figure 3. Use Case Diagram

Figure 4. Activity Diagram

Figure 5. Entity Relationship Diagram

Figure 6. Database Table Relationship Design

System Implementation

The best-performing model was integrated into a web-based monitoring system developed using Flask, MySQL, APScheduler, SerpAPI, and Telegram Bot API. SerpAPI retrieves the latest Google Maps reviews at periodic intervals, the trained model classifies their sentiment, MySQL stores review data and classification results, and the Telegram Bot delivers automated notifications to all registered subscribers (Egamberdiev et al., 2024; Maududy & Nursamsi, 2023). Finally, black-box functional testing was conducted to verify that all system components operated as designed.

Data Collection and Labeling Results

The DOM scraping process successfully collected 2,449 historical customer reviews of Aveta Hotel Malioboro from Google Maps, spanning from February 20, 2020 to May 25, 2025. The dataset includes user names, review timestamps, star ratings, and review text.

Figure 7. Collected Historical Review Data

Figure 8. Sentiment Labeling Results Distribution

he results demonstrate that each preprocessing stage contributes to progressively reducing noise and improving feature quality. Notably, the normalization stage effectively standardized informal language common in Google Maps reviews, while the retention of negation words during stopword removal preserved critical sentiment-bearing tokens such as 'tidak' (not) and 'kurang' (less/insufficient).

Figure 9. Class Distribution After Preprocessing

Figure 10. Training and Test Data

Figure 11. Training and Test Data

Figure 12. Dashboard Page Implementation

Figure 13. Analytics Page

Figure 14. Subscriber Management

The Notification Log page Figure 15 displays the complete history of automated Telegram notifications, including review_id, subscriber chat_id, delivery status, and timestamps and Review Histories page Figure 16 presents a searchable, filterable table of all processed reviews with classification results from both models, enabling direct comparison of Naive Bayes and SVM outputs.

Figure 15. Notification Log Page

Figure 16. Review Histories Page

The evaluation results indicate that neither model is absolutely superior across all experimental scenarios. In the original class distribution scenario, Naive Bayes achieved a higher accuracy (0.9494) than SVM (0.9425). However, SVM demonstrated higher precision (0.9534) and F1 Score (0.9463), reflecting its greater ability to minimize false positives in negative review detection. This pattern aligns with Alfiyanti & Indra (2024), who found that in their TMII review study, Naive Bayes excelled in recall while SVM was superior in precision—a consistent pattern suggesting that algorithm selection should be guided by the specific operational priority of the monitoring use case.

In the resampling scenario, contrasting effects were observed. Naive Bayes experienced a decline in accuracy from 0.9494 to 0.9402 and in F1 Score from 0.9449 to 0.9422, indicating that synthetic augmentation of training data disrupted its feature independence assumption. This outcome is consistent with the nature of Multinomial Naive Bayes, which is sensitive to changes in term frequency distributions introduced by oversampling. In contrast, SVM in the resampling scenario achieved the highest overall performance with accuracy 0.9540, precision 0.9514, recall 0.9540, and F1 Score 0.9522. This improvement aligns with findings by Handayanto et al. (2021) and Maulana et al. (2024), both of whom concluded that SVM generally outperforms Naive Bayes in Google review sentiment tasks. The improvement can be attributed to SVM's margin-maximizing structure, which benefits from a more balanced class distribution in training data, allowing it to better learn the decision boundary separating positive and negative classes. Comparing with related studies, Sarimole & Kuadrat (2024) reported SVM accuracy of 87.95% and Naive Bayes of 81.65% on Twitter data, while Ipmawati et al. (2024) reported SVM accuracy of 87% on Google Maps tourism reviews. This study achieves notably higher accuracy (95.40% for SVM with resampling) on hotel review data. The performance improvement is attributable to the domain- specific preprocessing pipeline, including the custom normalization dictionary tailored for hotel-related informal language, retention of negation words during stopword removal, and the use of StratifiedKFold hyperparameter optimization with F1-macro scoring.

From a system implementation perspective, the monitoring system successfully operationalizes the trained model in a practical hospitality context. The integration of SerpAPI with APScheduler enables near real-time monitoring without continuous manual intervention. The Telegram Bot integration distributes monitoring alerts to multiple subscribers simultaneously, directly addressing the centralized-notification limitation identified in the existing manual system. The role-based access control and hotel-based data isolation ensure both data security and operational relevance for multi-property management scenarios. All twelve functional features passed black-box testing, confirming end-to-end system reliability from authentication to automated monitoring.

This study draws two main conclusions. First, the performance comparison of Naive Bayes and SVM demonstrates that both algorithms are viable sentiment classifiers for Google Maps hotel reviews. The best overall performance was achieved by SVM in the resampled training data scenario, with accuracy 0.9540, precision 0.9514, recall 0.9540, and F1 Score 0.9522. The resampling approach proved more beneficial for SVM than for Naive Bayes, making SVM with minority-class oversampling the recommended model for this classification task. The superiority of SVM is consistent with prior studies and can be attributed to its structural advantage in handling high-dimensional and imbalanced text classification problems.

Second, this research successfully designed and implemented a web- based hotel review monitoring system integrating sentiment analysis and a Telegram Bot. The system automatically retrieves the latest reviews via SerpAPI, performs sentiment classification using both trained models, stores results in a MySQL database, displays information through an interactive dashboard and analytics pages, manages Telegram subscribers, and delivers automated notifications. Operating in a near real-time manner through periodic scheduler- based scraping, the system provides practical value to hotel management by enabling faster, more structured, and data-driven review monitoring that transforms raw customer feedback into actionable operational intelligence.