🧠 MarketMind

AI-Powered Competitive Pricing Intelligence for E-Commerce Sellers

MarketMind is an intelligent pricing assistant that helps e-commerce sellers maximize profits through data-driven competitive analysis. By automatically scraping competitor prices from Amazon and eBay, the system uses machine learning to recommend optimal pricing strategies tailored to your costs and inventory.

---

✨ Features

🔍 Intelligent Market Research

• Automated Competitor Discovery: Finds up to 15 relevant competitors on Amazon & eBay using advanced search algorithms
• Smart Product Matching: Uses image hashing and fuzzy text matching to identify similar products
• Multi-Currency Support: Handles 40+ global currencies with automatic USD conversion

🤖 AI-Powered Price Optimization

• XGBoost ML Model: Predicts demand at different price points
• Cost-Based Strategies: Recommends prices starting from your actual costs (e.g., Cost × 1.5 for 50% profit)
• Market-Aware Capping: Ensures recommendations stay competitive with market averages
• Multiple Pricing Strategies:
  • 🎯 Target (50% profit margin)
  • 🏆 AI Optimal (demand-optimized)
  • ⚡ Market Match (competitive positioning)
  • 💨 Quick Sale (volume-focused)

🛡️ Robust Web Scraping

• Anti-Detection Technology: Selenium with stealth mode to bypass bot detection
• Retry Mechanisms: Automatic retry on failures for reliability
• Amazon Interstitial Handler: Bypasses "Continue Shopping" blocking pages
• 20+ Price Selectors: Adapts to different Amazon/eBay layouts (2024-2025)

📊 Professional Dashboard

• Clean UI: Modern dark theme with intuitive navigation
• Real-Time Progress: Live scraping status with progress indicators
• Top 10 Competitors: Focused view of best matches with similarity scores
• Detailed Logging: Full transparency with downloadable scraping logs
• Viability Alerts: Warns about unprofitable products or tight margins

---

🚀 Quick Start

Prerequisites

• Python 3.9 or higher
• Google Chrome browser (required for Selenium)

Installation

1. Clone the repository

   git clone https://github.com/MD-Code404/MarketMind.git
   cd MarketMind

2. Create virtual environment

   python -m venv venv
   
   # Windows
   .\venv\Scripts\activate
   
   # macOS/Linux
   source venv/bin/activate

3. Install dependencies

   pip install -r requirements.txt

Usage

1. Launch the application

   streamlit run app.py

2. Open your browser

   • Navigate to http://localhost:8501

3. Analyze a product

   • Enter your product URL (Amazon or eBay)
   • Input your cost per unit
   • Specify available inventory
   • Click "Analyze Market & Recommend Price"

4. Review strategies

   • View 2-4 pricing strategies based on market conditions
   • Check competitor analysis and market snapshot
   • Download detailed logs if needed

---

🛠️ Tech Stack

Category	Technology
Frontend	Streamlit 1.52+
ML Model	XGBoost, scikit-learn
Web Scraping	Selenium, selenium-stealth
Search	DuckDuckGo Search API
Image Matching	ImageHash, Pillow
Text Matching	thefuzz (Levenshtein distance)
Data Processing	pandas, numpy

---

📁 Project Structure

MarketMind/
├── .streamlit/
│   └── config.toml          # Streamlit theme configuration
├── src/
│   ├── scraper.py           # Web scraping with Selenium
│   ├── search.py            # Competitor URL discovery
│   ├── matcher.py           # Product similarity matching
│   └── optimizer.py         # ML-based price optimization
├── app.py                   # Main Streamlit application
├── demand_model.joblib      # Pre-trained XGBoost model
├── model_features.joblib    # Model feature list
├── requirements.txt         # Python dependencies
├── .gitignore              # Git ignore rules
└── README.md               # This file

---

🔬 How It Works

1. Product Scraping

Your Product URL → Selenium → Extract (Title, Price, Image, Sales Volume)

• Launches headless Chrome with anti-detection
• Handles Amazon "Continue Shopping" interstitials
• Extracts data using 20+ CSS/XPath selectors

2. Competitor Discovery

Product Name → DuckDuckGo Search → Filter URLs → Top 15 Competitors

• Searches Amazon & eBay (10 results per site)
• Filters for product pages (/dp/ or /itm/)
• De-duplicates and limits to 15 URLs

3. Similarity Matching

Image Hash + Fuzzy Title Match → Combined Score → Filter (≥65%)

• Image Matching: Perceptual hashing (60% threshold)
• Title Matching: Token set ratio (75% threshold for text-only)
• Fallback: Title-only if images unavailable

4. Data Cleaning

Raw Competitors → Price Validation → Outlier Removal → Valid Set

• Removes prices below dynamic threshold (50% of your product price)
• Applies statistical outlier detection (±2 standard deviations)
• Filters currency conversion failures

5. Price Optimization

Cost × 1.5 → Check vs Market → AI Model → Recommend Price

• Calculates target: Cost × 1.5 (50% profit)
• If target > market average, caps at market price
• Uses XGBoost to predict demand at various prices
• Recommends optimal price maximizing profit

---

💡 Key Algorithms

Cost-Based Pricing Logic

ideal_price = cost × 1.5  # 50% profit margin

if ideal_price <= market_average:
    ✅ Recommend ideal_price
    Status: "Great News! Achieve 50% profit"

elif ideal_price > market_average:
    ⚠️ Cap at market_average
    Status: "Tight Margins - Consider lower profit"

elif cost >= market_average:
    ❌ ERROR: Unprofitable product
    Status: "Cannot sell profitably"

Similarity Scoring

if has_images:
    score = (title_similarity × 0.6) + (image_similarity × 0.4)
    threshold = 65%
else:
    score = title_similarity
    threshold = 75%

---

⚙️ Configuration

Currency Support

Supports 40+ currencies including:

• Major: USD, EUR, GBP, JPY, CHF, CAD, AUD, NZD
• Asian: CNY, INR, KRW, SGD, HKD, THB, MYR, PHP, VND
• European: SEK, NOK, DKK, PLN, CZK, HUF, RON
• Middle East/Africa: AED, SAR, ILS, ZAR, EGP, TRY
• Latin America: MXN, BRL, ARS, CLP, COP

See app.py:DEFAULT_RATES_TO_USD for complete list.

Customization

Adjust profit margin:

# In app.py line 514
ideal_price = user_cost_per_unit * 1.5  # Change 1.5 to desired multiplier

Adjust competitor limit:

# In app.py line 281
max_results_per_site=10  # Change to desired number per site

Adjust similarity threshold:

# In app.py line 322
if combined_similarity_score >= 65:  # Change 65 to stricter/looser value

---

📊 Performance

• Scraping Speed: ~30 seconds per URL (industry standard)
• Competitor Analysis: ~7-10 minutes for 15 competitors
• Success Rate: 70-85% (depends on Amazon rate limiting)
• Accuracy: Similarity matching >90% for exact products

---

🚨 Known Limitations

1. Amazon Rate Limiting: Amazon may temporarily block after multiple requests. Wait 5-10 minutes and retry.
2. Scraping Speed: ~30s per product is necessary to avoid bot detection.
3. Regional Variations: Prices may vary by Amazon region (.com vs .co.uk, etc.).
4. Model Accuracy: ML predictions are estimates based on synthetic training data.

---

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Development Setup

# Fork the repo and clone
git clone https://github.com/MD-Code404/MarketMind.git

# Create a feature branch
git checkout -b feature/amazing-feature

# Make changes and test
streamlit run app.py

# Commit and push
git add .
git commit -m "Add amazing feature"
git push origin feature/amazing-feature

---

📜 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

🙏 Acknowledgments

• Streamlit - Beautiful web framework
• Selenium - Robust web automation
• XGBoost - Powerful ML library
• DuckDuckGo - Privacy-focused search API

---

📧 Contact

For questions or support, please open an issue on GitHub.

---

⭐ Star History

If you find MarketMind useful, please consider giving it a star! ⭐

---

Made with ❤️ for e-commerce sellers worldwide