Requirement Ellicitation from Multi Source Data
Last Updated February 2, 2025natural language processing, python
Requirement Elicitation from Multi-Source Data
🔍 Introduction
Requirement elicitation is a crucial phase in software development to ensure applications align with user needs. This project introduces an innovative Requirement Elicitation from Multi-Source Data Application, designed to gather user requirements from various sources—news platforms, Google Play Store, and Apple App Store.
🎯 Project Overview
The primary objective of this system is to allow users to input queries and retrieve relevant software requirements automatically. The application leverages scraping techniques and APIs, then transforms extracted data into structured user stories and use case diagrams, making the process efficient and data-driven.
🔑 Key Features
Data Collection
The system collects data from:
- 📱Google Play Store & Apple App Store – Extracting user reviews to identify software requirements.
- 📰 News Sources – Gathering industry insights to understand broader market trends.
Data Processing
To transform raw text into structured requirements, the following NLP techniques are employed:
- Named Entity Recognition (NER) – Identifies key entities (users, features, services).
- Part-of-Speech (POS) Tagging – Analyzes sentence structure to extract actionable phrases.
- Semantic Role Labeling (SRL) – Determines relationships between words to contextualize user needs.
- Redundancy Elimination – Filters out duplicate or irrelevant data.
Requirement Generation
Once processed, the extracted insights are converted into User Stories following this structure:
- As a [user], I want [feature], so that [benefit].
These user stories are then visualized through Use Case Diagrams, providing a structured representation of software functionalities.
🏗 Tech Stack
- Frontend: Built using Next.js for a dynamic and responsive UI.
- Backend: Developed using Django with Django Ninja for API management.
- Database: MongoDB to store collected data and processed requirements.
- Scraping & APIs: Custom scrapers for Play Store & App Store, and integration with News Catcher API.
- NLP Processing: Implemented using spaCy for text analysis and requirement extraction.
⚡ Challenges & Lessons Learned
- User Reviews vs. News Articles: App store reviews provide detailed user feedback, while news sources offer macro-level insights into market trends.
- Efficiency Gains: Automating the requirement elicitation process reduces manual effort and enhances scalability.
- Challenges: Handling large datasets required optimization techniques, and filtering non-software-related content remains a challenge.
🎉 Conclusion
Developing the Requirement Elicitation from Multi-Source Data Application was an insightful experience, showcasing the power of NLP, data scraping, and software requirement engineering. The SRS serves as a robust blueprint, ensuring a structured and efficient development process.
This project underscores how AI-driven requirement elicitation can transform software development workflows, automating manual efforts while improving accuracy. If you're passionate about AI-driven software analysis, let's connect and explore more innovations! 🚀