GitHub (deployment repo):multisource-deployment, Docker Compose orchestration of the FastAPI backend and React frontend as separate submodules. Worth confirming with yourself which of these two repos is the current source of truth, since they may have diverged.
This entry replaces the pre-publication draft that was on the site before. The system described below matches what actually shipped in the published paper, not the original school-project framing.
🔍 Introduction
Data-driven requirement elicitation is increasingly used in software engineering because of how much user-generated text is now available online, but most existing approaches only look at a single source (just app reviews, or just social media). A single source only captures one slice of what users actually need. This project proposes and evaluates a framework that pulls from three heterogeneous online sources at once, app reviews, X (Twitter), and online news, and turns them into structured, usable requirement artifacts.
🎯 Project Overview
The framework runs as a five-stage pipeline, applied across six real case studies spanning different application domains (examples include ExaMove, a fitness app; Ez-Parky, an IoT smart-parking system; and LOGS, an internal iOS mall-equipment management app):
Data Acquisition: LLM-driven query generation, then parallel scraping across app stores, news, and X
AI-Based Extraction: LLM-driven extraction with a strict output schema, benchmarked across 5 different models
Semantic Clustering: groups near-duplicate requirements and picks one representative story per group
Use Case Diagram Generation: automatically renders the clustered requirements as a UML diagram
🔑 Key Features
LLM-driven dynamic query generation
Instead of relying on manually chosen keywords (which introduce researcher bias and inconsistency), a case study description is fed to an LLM, which decomposes it semantically and generates a set of targeted search queries. A fixed prompt template is used across all six case studies, so the only thing that varies between them is the project description itself, not the query logic.
Multi-source parallel acquisition
App stores: parallel calls to the Google Play Store and Apple App Store search APIs, normalized into a unified schema
News: a read-through cache checks the local database before hitting the News API, cutting redundant API calls and latency
X (Twitter): retrieves top (not just most recent) tweets to prioritize high-engagement content, capturing engagement metrics and author verification status alongside the text
Source-specific preprocessing
Each source gets its own cleaning pipeline, because the noise profile is different for each one:
Tweets: strips RT markers, extracts @mentions separately (to avoid confusing the NER step), converts #hashtags and $cashtags to plain text rather than deleting them outright
Two extraction approaches, benchmarked head-to-head
Rule-based: spaCy (en_core_web_lg) dependency parsing plus WordNet lexical validation. WHO is found via subject/object dependencies validated against NER or WordNet person/group categories; WHAT via verb-phrase pattern matching filtered by WordNet software-relevant verb categories (cognition, communication, creation, motion, possession, contact, perception); WHY via advcl/xcomp/ccomp dependency clauses.
AI-based: an LLM is given a strict JSON schema prompt (who / what / why / evidence / sentiment / confidence / NFRs / business impact / pain point) and required to output structured user stories with a confidence score. Stories below a 0.5 confidence threshold are dropped. If the first pass doesn't produce deeper insight fields (NFRs, business impact, pain points), a second fallback inference call fills them in.
One commercial model (Gemini 2.5 Flash) was benchmarked against four open-source models (GPT-OSS:20b, Gemma:31b, Qwen3:14b, Mistral:7b) and the rule-based baseline, across all three data sources.
Semantic clustering and diagram generation
Requirements are embedded with Sentence-BERT (all-MiniLM-L6-v2, 384-dim), then grouped with Agglomerative Hierarchical Clustering (cosine distance, average linkage, 0.5 distance threshold), chosen over k-means specifically because the number of natural requirement groups isn't known in advance. For each cluster, the story closest to the cluster's centroid is picked as the representative. The top 10 clusters by size are kept and automatically rendered as a UML Use Case Diagram via PlantUML (WHO maps to Actor, WHAT maps to Use Case, WHY is kept in the underlying story but not shown on the diagram).
Ground truth: 176 expert-annotated instances (135 app reviews, 35 tweets, 6 news articles), scored with Precision, Recall, and F1 using partial span matching.
Source data characteristics (average length grows a lot from reviews to news):
Metric
App Reviews (N=135)
Twitter (N=35)
Online News (N=6)
Avg. character length
185.39
417.63
3,584.28
Avg. token count
34.99
65.74
565.44
Extraction performance (F1 per aspect), App Reviews:
Method
WHO F1
WHAT F1
WHY F1
Rule-based
0.4005
0.4062
0.3185
Gemini 2.5 Flash
0.6402
0.6139
0.9667
GPT-OSS:20b
0.5496
0.6945
0.0807
Gemma:e4b
0.4442
0.6789
0.0795
Qwen3:14b
0.4904
0.6498
0.0630
Mistral:7b
0.6183
0.6507
0.0617
Extraction performance (F1 per aspect), Twitter:
Method
WHO F1
WHAT F1
WHY F1
Rule-based
0.2257
0.3712
0.3381
Gemini 2.5 Flash
0.8636
0.5530
0.9034
GPT-OSS:20b
0.3714
0.5834
0.1938
Gemma:31b
0.2281
0.6039
0.3248
Qwen3:14b
0.2606
0.6321
0.2267
Mistral:7b
0.3239
0.6362
0.2354
Extraction performance (F1 per aspect), Online News:
Method
WHO F1
WHAT F1
WHY F1
Rule-based
0.2220
0.2292
0.1514
Gemini 2.5 Flash
0.6882
0.5730
0.8146
GPT-OSS:20b
0.0000
0.0000
0.0000
Gemma:31b
0.1111
0.1037
0.1799
Qwen3:14b
0.2897
0.3533
0.4543
Mistral:7b
0.3175
0.2818
0.4041
(Full precision and recall breakdowns for every method are in the published paper, linked above.)
Headline findings:
Gemini 2.5 Flash was the strongest model across all three sources and all three aspects, with its biggest edge on WHY (implicit motivation), the hardest aspect for every other method.
Open-source models were often competitive on WHAT (explicit, functional asks) but fell sharply on WHY, suggesting implicit reasoning is still where smaller open models lag behind.
News was consistently the hardest source for every method. Requirements in news are usually indirect, third-person, and spread across multiple stakeholders in a single article, unlike reviews and tweets where users state what they want directly.
Expert evaluation (six project owners, one per case study, 1–5 Likert scale):
Metric
Mean score
Range
Semantic Coherence
4.33
3.0 – 5.0
Topic Distinctiveness
3.83
2.0 – 5.0
Cluster Representativeness
4.17
3.0 – 5.0
System Interpretability
4.67
4.0 – 5.0
System Interpretability scored highest: project owners agreed that reviewing the generated artifacts sped up requirements engineering. Topic Distinctiveness scored lowest, the system tends to over-split topics into redundant clusters that mean roughly the same thing.
⚡ Challenges & Lessons Learned
News is a hard source, structurally, not just noisily. Even the best model dropped noticeably on news versus reviews or tweets, and one open-source model (GPT-OSS:20b) scored a flat 0.0000 across every aspect on news, a genuine failure worth stating plainly rather than averaging away.
Implicit reasoning (WHY) separates commercial from open-source models far more than explicit extraction (WHAT) does. This has a real practical implication: if a project can tolerate weaker WHY extraction, an open-source model may be a perfectly reasonable, cheaper choice.
Clustering over-splits topics. The 3.83 mean score on Topic Distinctiveness, the lowest of the four expert metrics, is an honest limitation: the system creates more clusters than it should, and cluster-merging logic is a known area for improvement, not something the paper claims to have solved.
Representative-story selection carries hallucination risk, flagged directly by the evaluating experts, since the "most central" story in a cluster is still LLM-influenced content, not a guaranteed ground truth.
No data-balancing was applied on purpose. The pipeline preserves the natural, uneven distribution of how much data comes back per topic, and relies on clustering downstream to manage that imbalance rather than forcing an artificial balance upstream.
🎉 Conclusion
Multisource integration is more complete than any single source alone: app reviews surface direct functional asks, X surfaces emerging opinions and motivations, and news surfaces the strategic, third-person context that neither of the other two sources captures. Published as first author in JISEBI (Scopus Q2), government-funded, with the framework evaluated against expert-annotated ground truth on real, live-scraped data across six case studies rather than a single static benchmark.
Requirement Elicitation from Heterogeneous Online Sources