# AI News Digest Julia Skill

A comprehensive Julia-based skill for aggregating, filtering, and compiling AI/tech news from multiple RSS sources into daily digest formats.

## Features

🤖 **AI-Focused Filtering** - Advanced content scoring using keywords, company mentions, and relevance algorithms

📡 **Multi-Source Aggregation** - 15+ premium tech news sources including TechCrunch, The Verge, Ars Technica, and AI-specific blogs

🔄 **Smart Deduplication** - Eliminates duplicate stories using text similarity analysis

📊 **Multiple Output Formats** - JSON, Markdown, and HTML email formats

⚡ **Fast & Efficient** - Concurrent RSS fetching with intelligent caching

## Quick Start

```bash
# Install dependencies
julia install.jl

# Generate daily AI digest (default: top 5 stories, markdown format)
./ai-news-digest.sh

# Tech news only, 10 stories
./ai-news-digest.sh --sources tech --limit 10

# AI-focused sources, JSON output
./ai-news-digest.sh --sources ai --output json --verbose

# Email newsletter format
./ai-news-digest.sh --output email --limit 3
```

## OpenClaw Integration

```bash
# Add to skills registry
/ai-news-digest --sources ai --limit 5 --output markdown
```

## Configuration

The `config.json` file controls:

- **RSS Feed Sources** - URLs for tech and AI news feeds
- **AI Filter Settings** - Keywords, companies, and scoring weights  
- **Output Templates** - Formatting for different output types
- **Deduplication Rules** - Similarity thresholds and weights

### Adding New Sources

Edit `config.json` to add RSS feeds:

```json
{
  "feeds": {
    "tech": [
      {
        "name": "New Tech Blog",
        "url": "https://example.com/feed.xml",
        "category": "tech"
      }
    ]
  }
}
```

### Tuning AI Relevance

Adjust the AI filtering algorithm:

```json
{
  "ai_filter": {
    "threshold": 60,           // Minimum score for inclusion (0-100)
    "title_weight": 3.0,       // Weight for title matches
    "content_weight": 1.0,     // Weight for content matches  
    "company_bonus": 20        // Bonus points for company mentions
  }
}
```

## Output Examples

### Markdown Format
```markdown
# AI News Digest - March 6, 2026

## 1. OpenAI Releases GPT-5 with Multimodal Capabilities
**Source:** TechCrunch | **AI Score:** 95/100 | **Published:** 2 hours ago

OpenAI announced GPT-5 with enhanced multimodal capabilities...
[Read more →](https://techcrunch.com/...)
```

### JSON Format
```json
{
  "digest_date": "2026-03-06",
  "metadata": {
    "total_stories": 5,
    "average_ai_score": 78.4,
    "sources_count": 8
  },
  "stories": [
    {
      "title": "OpenAI Releases GPT-5",
      "url": "https://techcrunch.com/...",
      "ai_score": 95,
      "rank": 1,
      "matched_keywords": ["OpenAI", "GPT", "AI"],
      "matched_companies": ["OpenAI"]
    }
  ]
}
```

## Sources Included

### Tech News Sources
- **TechCrunch** - Leading tech startup news
- **The Verge** - Technology, science, art, and culture
- **Ars Technica** - In-depth tech analysis
- **Wired** - Technology and its impact on culture
- **VentureBeat** - Enterprise technology news
- **Fast Company Tech** - Innovation and technology trends
- **MIT Technology Review** - Emerging technologies
- **IEEE Spectrum** - Engineering and technology

### AI-Focused Sources
- **OpenAI Blog** - Official OpenAI announcements  
- **Google AI Blog** - Google's AI research updates
- **Anthropic News** - Anthropic's AI safety research
- **Towards Data Science** - Medium's data science publication
- **Machine Learning Mastery** - Practical ML tutorials
- **MIT AI News** - MIT's artificial intelligence research
- **AI News** - Dedicated AI industry news

## Dependencies

- **Julia 1.6+** - Core runtime
- **HTTP.jl** - RSS feed fetching
- **EzXML.jl** - XML/RSS parsing
- **JSON.jl** - JSON output formatting
- **ArgParse.jl** - Command-line argument parsing

## Testing

```bash
# Run test suite
julia test.jl

# Test specific functionality
julia -e "include(\\"test.jl\\"); test_ai_filtering()"
```

## Automation

Set up daily digest generation:

```bash
# Add to crontab for daily 9 AM digest
0 9 * * * /path/to/ai-news-digest.sh --output email > /tmp/daily-digest.html

# OpenClaw heartbeat integration
/ai-news-digest --sources ai --limit 5 | tee daily-ai-digest.md
```

## Customization

### Custom Keywords

Add domain-specific keywords to improve filtering:

```json
{
  "ai_filter": {
    "keywords": [
      "artificial intelligence", "machine learning", "neural network",
      "computer vision", "natural language processing",
      "your-custom-keyword", "domain-specific-term"
    ]
  }
}
```

### Custom Output Templates

Modify output formatting in the `output` section of `config.json`:

```json
{
  "output": {
    "templates": {
      "slack": {
        "header": ":robot_face: *AI News Digest*\\n",
        "story": "• *{title}* ({source}) - {time_ago}\\n  {summary}\\n  <{url}|Read more>\\n\\n"
      }
    }
  }
}
```

## Performance

- **Feed Fetching**: ~2-3 seconds for all 15 sources
- **AI Filtering**: ~1-2 seconds for 100+ stories  
- **Memory Usage**: <50MB for typical daily processing
- **Concurrent Processing**: Parallel RSS fetching for speed

## Troubleshooting

### Common Issues

**Julia not found**: Install Julia from https://julialang.org/downloads/

**RSS feed errors**: Check network connectivity and feed URLs in config

**Low AI scores**: Adjust `threshold` in `ai_filter` configuration

**Missing dependencies**: Run `julia install.jl` to reinstall packages

### Debug Mode

```bash
# Verbose output for debugging
./ai-news-digest.sh --verbose

# Test single feed
julia -e "include(\\"src/rss_parser.jl\\"); stories = fetch_rss_stories(Dict(\\"name\\" => \\"TechCrunch\\", \\"url\\" => \\"https://techcrunch.com/feed/\\"), 24); println(length(stories))"
```

## License

MIT License - Feel free to modify and redistribute.

---

*Built for OpenClaw by the AI News Digest team*