# Quick Reference: Multi-TG Account Management Implementation

## 📁 File Structure & Purpose

```
alwaydata-v3/
├── app.py                 # Flask dashboard & API endpoints
├── core.py                # Telethon client manager & forwarding logic
├── db.py                  # Database layer with connection pooling
├── config.py              # Configuration management (env + DB)
├── crypto_vault.py        # Fernet encryption/decryption utilities
├── ctrader_spot_client.py # cTrader Open API spot price client
├── signal_processor.py    # Signal processing & price augmentation
├── forwarding.py          # Message forwarding logic
├── signal_parser.py       # Signal text parsing (entry/SL/TP)
├── llm_classifier.py      # LLM-based signal classification
├── passenger_wsgi.py      # Passenger WSGI entry point
├── 
├── scripts/               # Deployment & maintenance scripts
│   ├── apply_schema.py    # Apply database schema/migrations
│   ├── cpanel_mysql.py    # cPanel MySQL management via UAPI
│   ├── ctrader_consumer.py # Fetch & decrypt cTrader tokens
│   ├── ctrader_refresh.py # Refresh cTrader OAuth tokens
│   ├── le_issue_quill.py  # Let's Encrypt certificate issuance
│   ├── remote_db.py       # Remote database inspection
│   ├── seed_config.py     # Seed .env to remote config_vars
│   └── selector_env.py    # CloudLinux selector env vars
│
├── templates/             # Flask HTML templates
│   ├── index.html         # Main dashboard
│   └── components/        # Dashboard UI components
│       ├── _accounts_panel.html
│       ├── _auth_modal.html
│       ├── _bots_panel.html
│       ├── _channel_options.html
│       ├── _ctrader_panel.html
│       ├── _llm_panel.html
│       ├── _pair_editor.html
│       ├── _pairs_panel.html
│       ├── _pairs_table.html
│       ├── _settings_panel.html
│       ├── _signals_panel.html
│       ├── _status_panel.html
│       └── _toast.html
│
├── schema.sql             # Database schema (CREATE IF NOT EXISTS)
├── migrate_existing.sql   # Idempotent migrations for existing DBs
├── requirements.txt       # Python dependencies
├── deploy.sh              # Full deployment script
├── .env.example           # Environment variable template
├── .env                  # Local environment (gitignored)
└── logs/                 # Application logs
    └── app.log
```

---

## 🔧 Key Components

### 1. **Session Management** (core.py)

**Architecture:**
- One `TelegramClient` per account
- `StringSession` for portable session storage
- Per-account isolation of auth state and QR state
- MySQL GET_LOCK for multi-process safety

**Key Classes:**
```python
ForwarderCore:
├── clients: Dict[int, TelegramClient]      # Account ID → Client
├── _auth_state: Dict[int, dict]            # Per-account auth state
├── _qr_state: Dict[int, dict]              # Per-account QR state
├── loop: asyncio.EventLoop                 # Async event loop
├── thread: Thread                          # Event loop thread
└── _executor: ThreadPoolExecutor           # For blocking DB calls
```

**Session Flow:**
```
Phone Auth:
  send_code() → send_code_request() → sign_in() → session.save() → DB persist

QR Auth:
  start_qr_login() → QRLogin.recreate() → wait for scan → 
  complete_qr_2fa() → sign_in(password) → session.save() → DB persist
```

### 2. **Database Layer** (db.py)

**Connection Pooling:**
- SQLAlchemy `QueuePool` over pymysql
- Pool size: 5, max overflow: 3
- Connection recycle: 120 seconds
- Read/write timeout: 10 seconds

**Encryption:**
- Fernet symmetric encryption (AES-128 in CBC mode)
- Key derived from Flask secret key via SHA-256
- Encrypted values prefixed with `enc:`

**Core Tables:**
```sql
-- Configuration
config_vars: Key-value store with encrypted secrets

-- Telegram Accounts
tg_accounts: Account metadata + session strings

-- Forwarding Rules  
channel_pairs: Source → destination mappings

-- Message Sync
message_map: Source message → destination message mapping

-- cTrader Integration
ctrader_accounts: Encrypted OAuth tokens + account metadata

-- Signal Processing
signal_log: Processed signals with price data
```

### 3. **Configuration** (config.py)

**Hierarchy:**
1. Environment variables (.env file)
2. DB-backed overrides (config_vars table)
3. Module-level defaults

**Security:**
- Sensitive values encrypted in DB with `enc:` prefix
- Fernet key derived from FLASK_SECRET_KEY
- Secrets never logged or exposed in errors

**Key Variables:**
```python
# Telegram
TG_API_ID, TG_API_HASH, TG_PHONE, TG_PASSWORD

# Web Dashboard
WEB_PIN, FLASK_SECRET_KEY, PORT

# Database
MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB

# cTrader
CTRADER_CLIENT_ID, CTRADER_CLIENT_SECRET, CTRADER_TOKEN_SECRET
CTRADER_ACCOUNT_ID, CTRADER_USE_LIVE, CTRADER_SYMBOL

# Signal Processing
SIGNAL_BOT_TOKEN, SIGNAL_ADMIN_CHAT_ID
SIGNAL_WEBHOOK_URL, SIGNAL_WEBHOOK_SECRET

# LLM Classification
LLM_ENABLED, LLM_PROVIDER, LLM_BASE_URL, LLM_MODEL
LLM_API_KEY, LLM_WEBHOOK_URL, LLM_WEBHOOK_SECRET
```

### 4. **Deployment** (deploy.sh)

**Pipeline:**
```bash
1. rsync code to server (excludes .env, .htaccess, etc.)
2. pip install requirements.txt in selector venv
3. Apply schema.sql + migrate_existing.sql to remote MySQL
4. Seed .env values into config_vars DB table
5. Set MYSQL_* + FLASK_SECRET_KEY in CloudLinux selector
6. Restart Passenger (touch tmp/restart.txt)
7. Health check (curl /api/health)
```

**Environment:**
- CloudLinux Python Selector
- Phusion Passenger behind LiteSpeed
- Python 3.13 virtualenv
- MariaDB (cPanel)

---

## 🚀 API Endpoints

### Authentication
```
POST /api/auth/login          # PIN authentication
POST /api/auth/logout         # Logout
GET  /api/auth/status         # Auth status
```

### Accounts
```
GET    /api/accounts                    # List accounts
POST   /api/accounts                    # Create account
PATCH  /api/accounts/<id>               # Update account
DELETE /api/accounts/<id>               # Delete account
POST   /api/accounts/<id>/reset         # Reset account auth
POST   /api/accounts/reset-stale        # Reset all stale accounts

# Phone Auth
POST   /api/accounts/<id>/send_code     # Send auth code
POST   /api/accounts/<id>/sign_in        # Complete sign-in

# QR Auth
POST   /api/accounts/<id>/qr            # Start QR login
POST   /api/accounts/<id>/qr/check       # Check QR status
POST   /api/accounts/<id>/qr/cancel      # Cancel QR login
POST   /api/accounts/<id>/qr/2fa         # Complete QR 2FA

# Dialogs
GET    /api/accounts/<id>/dialogs       # Get account dialogs
POST   /api/accounts/<id>/dialogs/refresh # Refresh dialogs
```

### Forwarding Pairs
```
GET    /api/pairs                      # List pairs
POST   /api/pairs                      # Create pair
PATCH  /api/pairs/<id>                 # Update pair
DELETE /api/pairs/<id>                 # Delete pair
POST   /api/pairs/<id>/toggle          # Toggle pair enabled/disabled
POST   /api/pairs/<id>/test_emit       # Test signal emission
```

### Bot Tokens
```
GET    /api/bots                       # List bot tokens
POST   /api/bots                       # Add bot token
DELETE /api/bots/<id>                  # Delete bot token
POST   /api/bots/<id>/verify            # Verify bot token
```

### Configuration
```
GET    /api/config                      # Get Telegram config
POST   /api/config/telegram/test        # Test Telegram API
GET    /api/config/signals              # Get signal config
POST   /api/config/signals              # Update signal config
GET    /api/config/llm                  # Get LLM config
POST   /api/config/llm                  # Update LLM config
POST   /api/config/llm/test             # Test LLM connection
```

### Status & Health
```
GET    /api/health                      # Health check
GET    /api/status                      # Full status
GET    /api/price                       # Current price (cached)
GET    /api/signals                     # Signal history
```

### Forwarder Control
```
POST   /api/forwarder/start              # Start forwarding
POST   /api/forwarder/stop               # Stop forwarding
```

### Backup
```
GET    /api/backup                      # Export backup
```

### cTrader OAuth
```
GET    /callback                        # OAuth callback handler
```

---

## 🔐 Security Features

### 1. **Authentication**
- **PIN Protection**: Dashboard requires 4+ digit PIN
- **CSRF Protection**: All mutating requests require CSRF token
- **Rate Limiting**: In-memory per-IP rate limiting (5 req/60s default)
- **Session Management**: Secure session handling with Flask

### 2. **Encryption**
- **Fernet Symmetric**: AES-128 in CBC mode
- **Key Derivation**: SHA-256 hash of Flask secret key
- **Database Storage**: Encrypted values prefixed with `enc:`
- **cTrader Tokens**: Encrypted with separate CTRADER_TOKEN_SECRET

### 3. **Network Security**
- **HTTPS Enforcement**: HTTP → HTTPS redirection
- **Security Headers**: CSP, X-Frame-Options, HSTS, etc.
- **Secure Cookies**: HttpOnly, Secure, SameSite=Lax
- **IP Validation**: Telegram IP ranges could be validated

### 4. **Input Validation**
- **Type Checking**: Basic type validation on inputs
- **Range Validation**: Chat ID range checking
- **Sanitization**: HTML escaping in templates
- **Error Handling**: No sensitive data in error messages

---

## 📊 Database Schema

### Core Tables

#### config_vars
```sql
CREATE TABLE IF NOT EXISTS config_vars (
    var_key VARCHAR(128) PRIMARY KEY,
    var_value TEXT,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);
```
- Stores all configuration values
- Encrypted values prefixed with `enc:`
- Updated automatically on config changes

#### tg_accounts
```sql
CREATE TABLE IF NOT EXISTS tg_accounts (
    id INT AUTO_INCREMENT PRIMARY KEY,
    name VARCHAR(128) NOT NULL,
    phone VARCHAR(32) DEFAULT '',
    session_string TEXT,
    is_primary BOOLEAN DEFAULT FALSE,
    is_authorized BOOLEAN DEFAULT FALSE,
    status VARCHAR(32) DEFAULT 'pending',
    qr_token BINARY(16) DEFAULT NULL,
    qr_expires_at TIMESTAMP NULL DEFAULT NULL,
    phone_code_hash VARCHAR(128) DEFAULT '',
    pending_session TEXT,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);
```
- Stores Telegram account metadata
- `session_string` contains the Telethon session
- `pending_session` used during auth flow
- `is_primary` indicates the default account

#### channel_pairs
```sql
CREATE TABLE IF NOT EXISTS channel_pairs (
    id INT AUTO_INCREMENT PRIMARY KEY,
    source_chat_id BIGINT NOT NULL,
    dest_chat_id BIGINT NOT NULL,
    source_title VARCHAR(512),
    dest_title VARCHAR(512),
    enabled BOOLEAN DEFAULT TRUE,
    include_text BOOLEAN DEFAULT TRUE,
    include_media BOOLEAN DEFAULT TRUE,
    skip_standalone_media BOOLEAN DEFAULT FALSE,
    forward_as_link BOOLEAN DEFAULT FALSE,
    price_augment BOOLEAN DEFAULT FALSE,
    forward_via_bot BOOLEAN DEFAULT FALSE,
    augment_symbol VARCHAR(32) DEFAULT NULL,
    filter_type ENUM('none','text_only','media_only') DEFAULT 'none',
    llm_enabled BOOLEAN DEFAULT FALSE,
    account_id INT DEFAULT NULL,
    bot_token_id INT DEFAULT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    UNIQUE KEY unique_pair (source_chat_id, dest_chat_id)
);
```
- Defines forwarding rules
- `account_id` specifies which Telegram account to use
- `bot_token_id` specifies which bot token to use for bot forwarding
- Various filtering and processing options

#### message_map
```sql
CREATE TABLE IF NOT EXISTS message_map (
    id INT AUTO_INCREMENT PRIMARY KEY,
    source_chat_id BIGINT NOT NULL,
    source_msg_id BIGINT NOT NULL,
    dest_chat_id BIGINT NOT NULL,
    dest_msg_id BIGINT NOT NULL,
    via_bot BOOLEAN DEFAULT FALSE,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    UNIQUE KEY unique_msg (source_chat_id, source_msg_id, dest_chat_id),
    KEY idx_lookup (source_chat_id, dest_chat_id, source_msg_id),
    KEY idx_created (created_at)
);
```
- Maps source messages to destination messages
- Enables edit/delete synchronization
- `via_bot` indicates if forwarded via bot API

#### ctrader_accounts
```sql
CREATE TABLE IF NOT EXISTS ctrader_accounts (
    id INT AUTO_INCREMENT PRIMARY KEY,
    account_id BIGINT NOT NULL,
    user_id BIGINT DEFAULT NULL,
    broker_name VARCHAR(128) DEFAULT '',
    broker_title VARCHAR(128) DEFAULT '',
    account_number BIGINT DEFAULT NULL,
    is_live BOOLEAN DEFAULT FALSE,
    deposit_currency VARCHAR(8) DEFAULT '',
    balance_cents BIGINT DEFAULT NULL,
    money_digits INT DEFAULT 2,
    leverage INT DEFAULT NULL,
    account_type VARCHAR(32) DEFAULT '',
    account_status VARCHAR(32) DEFAULT '',
    encrypted_payload TEXT NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    UNIQUE KEY unique_account (account_id)
);
```
- Stores cTrader account metadata
- `encrypted_payload` contains encrypted OAuth tokens
- Encrypted with CTRADER_TOKEN_SECRET

#### signal_log
```sql
CREATE TABLE IF NOT EXISTS signal_log (
    id INT AUTO_INCREMENT PRIMARY KEY,
    source_chat_id BIGINT NOT NULL,
    source_message_id BIGINT NOT NULL,
    signal_text TEXT,
    signal_date INT,
    reply_to_message_id BIGINT,
    received_at_ms BIGINT NOT NULL,
    price_fetched_at_ms BIGINT,
    symbol VARCHAR(32) NOT NULL,
    symbol_id BIGINT,
    bid DECIMAL(18, 5),
    ask DECIMAL(18, 5),
    spread DECIMAL(18, 5),
    tick_timestamp_ms BIGINT,
    price_available BOOLEAN DEFAULT FALSE,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    UNIQUE KEY unique_signal (source_chat_id, source_message_id),
    KEY idx_created (created_at),
    KEY idx_source_msg (source_chat_id, source_message_id)
);
```
- Logs all processed signals with price data
- Used for signal history and analysis

---

## 🛠️ Deployment Commands

### Local Development
```bash
# Setup
cp .env.example .env
python3 dev_generate_keys.py
mysql -h $MYSQL_HOST -u $MYSQL_USER -p$MYSQL_PASSWORD $MYSQL_DB < schema.sql

# Install dependencies
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Run
python app.py

# Access
open http://localhost:8301
```

### cPanel Deployment
```bash
# Full deploy with restart
./deploy.sh -r

# Dry run (preview changes)
./deploy.sh --dry-run

# Deploy without restart
./deploy.sh
```

### Database Management
```bash
# Apply schema manually
python3 scripts/apply_schema.py --env .env --ssh-host jnc

# Inspect remote database
python3 scripts/remote_db.py tables
python3 scripts/remote_db.py config
python3 scripts/remote_db.py tg
python3 scripts/remote_db.py pairs
python3 scripts/remote_db.py signal-log --limit 20
python3 scripts/remote_db.py query "SELECT COUNT(*) FROM message_map"
```

### cTrader Token Management
```bash
# Check which tokens need refresh
python3 scripts/ctrader_refresh.py --dry-run

# Refresh tokens
python3 scripts/ctrader_refresh.py

# Decrypt and inspect tokens
python3 scripts/ctrader_consumer.py
python3 scripts/ctrader_consumer.py --output tokens.json
```

### cPanel MySQL Management
```bash
# List databases
python3 scripts/cpanel_mysql.py list-dbs

# Create database
python3 scripts/cpanel_mysql.py create-db ssfx_v3

# Create user
python3 scripts/cpanel_mysql.py create-user ssfxuser

# Grant privileges
python3 scripts/cpanel_mysql.py grant --user ssfxuser --db wovewbge_ssfx_v3 --privileges ALL
```

---

## 🎯 Key Features

### 1. **Multi-Account Support**
- Multiple Telegram accounts with isolated sessions
- Phone code and QR code authentication
- Per-account forwarding rules
- Account-specific dialogs and channels

### 2. **Message Forwarding**
- 1-to-many forwarding (one source to multiple destinations)
- Per-rule filters (text-only, media-only, include/exclude)
- Edit and delete synchronization via message mapping
- Forward as link or via bot API

### 3. **Signal Processing**
- Price augmentation from cTrader Open API
- Multi-symbol support
- Signal parsing (entry, SL, TP extraction)
- Outbound webhook emission
- Optional LLM classification

### 4. **cTrader Integration**
- OAuth 2.0 flow with token encryption
- Spot price streaming
- Token vault with encrypted storage
- External token refresh capability

### 5. **Web Dashboard**
- PIN-protected access
- Real-time status monitoring
- Account management UI
- Forwarding rule configuration
- Signal history viewer
- cTrader account linking

---

## 🚨 Troubleshooting

### Common Issues

**App not responding:**
```bash
# Check health
curl -sk https://quill.nx.kg/f/api/health

# Check Passenger
ssh jnc "ps aux | grep passenger"

# Restart
ssh jnc "touch ~/quill.nx.kg/tmp/restart.txt"

# Check logs
ssh jnc "tail -50 ~/quill.nx.kg/logs/app.log"
```

**Database connection errors:**
```bash
# Verify credentials
python3 scripts/remote_db.py tables

# Check config
python3 scripts/remote_db.py config

# Verify selector env vars
ssh jnc "cloudlinux-selector get --json --interpreter python --user wovewbge"
```

**Telegram listener not running:**
```bash
# Check health with details
curl -sk https://quill.nx.kg/f/api/health | python3 -m json.tool

# Restart app
ssh jnc "touch ~/quill.nx.kg/tmp/restart.txt"
```

**Config not updating:**
```bash
# Re-seed config
python3 scripts/seed_config.py --env .env --ssh-host jnc

# Verify
python3 scripts/remote_db.py config

# Restart
ssh jnc "touch ~/quill.nx.kg/tmp/restart.txt"
```

---

## 📚 Research Documents

For comprehensive analysis and recommendations:

- **Full Research**: [`RESEARCH_MULTI_TG_ACCOUNT_MANAGEMENT.md`](RESEARCH_MULTI_TG_ACCOUNT_MANAGEMENT.md)
- **Executive Summary**: [`RESEARCH_SUMMARY.md`](RESEARCH_SUMMARY.md)

These documents contain:
- Detailed architecture comparison
- Security analysis and recommendations
- Implementation examples and code samples
- Technology stack evaluation
- Step-by-step improvement roadmap

---

*Last updated: August 6, 2026*