# cTrader Token Vault — External Integration Guide

How downstream systems (trading bots, copy-trading engines, dashboards) can
securely request and decrypt cTrader OAuth tokens stored in the forwarder's
vault, then use those tokens to call the cTrader Open API (Spotware).

---

## 1. Overview

```
┌─────────────────────┐         ┌──────────────────────┐
│   Your Bot / System │         │  cPanel Forwarder    │
│                     │         │                      │
│  1. GET /f/api/   │ ──────► │  Returns encrypted   │
│     ctrader/accounts│         │  payload from DB     │
│                     │         │                      │
│  2. Decrypt with    │         │                      │
│  CTRADER_TOKEN_     │         │                      │
│     SECRET          │         │                      │
│                     │         │                      │
│  3. Call cTrader    │ ──────► │  Spotware API        │
│     Open API        │         │  (orders, positions) │
│                     │         │                      │
│  4. Refresh when    │ ──────► │  Spotware token      │
│     expired         │         │  endpoint            │
│                     │         │                      │
│  5. Push back       │ ──────► │  POST /f/api/        │
│     encrypted       │         │  ctrader/accounts/   │
│                     │         │  <id>/refresh        │
└─────────────────────┘         └──────────────────────┘
```

**Design principle:** The forwarder stores tokens encrypted with
`CTRADER_TOKEN_SECRET` (Fernet). Any system that holds the same secret can
decrypt tokens. Keep the secret out of version control and rotate it if
compromised (note that rotation invalidates tokens already stored in the DB).

---

## 2. Prerequisites

- **`CTRADER_TOKEN_SECRET`** — generated once by `dev_generate_keys.py`, kept
  on the forwarder and on any consumer that needs to decrypt.
- **MySQL/MariaDB credentials** to the forwarder's database, OR access to the
  HTTP API.
- **cTrader Open API credentials** (client_id, client_secret) shared with the
  forwarder (same OAuth app).

---

## 3. Fetch Encrypted Tokens

### 3.1 Via HTTP API (recommended for remote systems)

```bash
curl -s 'https://quill.nx.kg/f/api/ctrader/accounts?encrypted=1' \
  -H 'Cookie: session=...'  # authenticate via the dashboard PIN session
```

Response:

```json
{
  "status": "ok",
  "accounts": [
    {
      "account_id": 46659639,
      "account_number": 5244306,
      "broker_name": "Pepperstone",
      "broker_title": "Pepperstone",
      "is_live": false,
      "deposit_currency": "USD",
      "encrypted_payload": "gAAAAABk...",
      "created_at": "2026-07-09T17:23:45"
    }
  ]
}
```

> **Note:** `encrypted=1` is required to receive the payload. Without it, the API
> returns metadata only.

### 3.2 Direct DB Query (for co-located systems)

```sql
SELECT account_id, encrypted_payload
FROM ctrader_accounts
WHERE account_id = 46659639;
```

---

## 4. Decrypt the Token

### 4.1 Python (official method)

```python
import os
from crypto_vault import decrypt_payload

secret = os.environ["CTRADER_TOKEN_SECRET"]
encrypted_payload = "gAAAAABk..."  # from API or DB

payload = decrypt_payload(secret, encrypted_payload)

print(payload)
# {
#   "access_token": "eyJhbGciOiJSUzI1NiIs...",
#   "refresh_token": "def50200...",
#   "expires_at": 1752086400,
#   "user_id": 12345678
# }

access_token = payload["access_token"]
refresh_token = payload["refresh_token"]
```

### 4.2 How the crypto works

- **Algorithm:** Fernet from the `cryptography` library (AES-128-CBC + HMAC-SHA256)
- **Key derivation:** a Fernet key is derived from `CTRADER_TOKEN_SECRET` via
  `SHA-256` so any reasonably long/high-entropy string works.
- **Output:** a single url-safe token string; no manual chunking is needed.

---

## 5. Call cTrader Open API

### 5.1 REST API (Spotware)

Use the decrypted `access_token` as a Bearer token:

```python
import urllib.request, json

access_token = "eyJhbGciOiJSUzI1NiIs..."

# Get account info
req = urllib.request.Request(
    f"https://api.spotware.com/connect/tradingaccounts?access_token={access_token}",
    headers={"Accept": "application/json"}
)
with urllib.request.urlopen(req, timeout=15) as resp:
    data = json.loads(resp.read().decode())
    accounts = data.get("data", [])

# Get open positions
req = urllib.request.Request(
    f"https://api.spotware.com/connect/tradingaccounts/{account_id}/positions?access_token={access_token}",
    headers={"Accept": "application/json"}
)
```

### 5.2 Streaming / gRPC (live prices & execution)

```python
from ctrader_open_api.messages.OpenApiMessages_pb2 import (
    ProtoOAApplicationAuthReq,
    ProtoOAAccountAuthReq,
    ProtoOANewOrderReq,
)

# After TCP+SSL connect to demo.ctraderapi.com:5035:
app_auth = ProtoOAApplicationAuthReq()
app_auth.clientId = client_id
app_auth.clientSecret = client_secret
# send...

acct_auth = ProtoOAAccountAuthReq()
acct_auth.ctidTraderAccountId = account_id
acct_auth.accessToken = access_token
# send...

# Now you can send orders, subscribe to spots, etc.
```

---

## 6. Token Refresh

Access tokens expire after **1 hour**. Use the `refresh_token` to get a new pair:

```python
import urllib.request, urllib.parse, json

def refresh_token(refresh_token: str, client_id: str, client_secret: str) -> dict:
    data = urllib.parse.urlencode({
        "grant_type": "refresh_token",
        "refresh_token": refresh_token,
        "client_id": client_id,
        "client_secret": client_secret,
    }).encode()

    req = urllib.request.Request(
        "https://openapi.ctrader.com/apps/token",
        data=data,
        method="POST",
        headers={"Content-Type": "application/x-www-form-urlencoded"}
    )
    with urllib.request.urlopen(req, timeout=15) as resp:
        return json.loads(resp.read().decode())

# Returns:
# {
#   "access_token": "new_access_token...",
#   "refresh_token": "new_refresh_token...",
#   "expires_in": 3600,
#   "token_type": "Bearer"
# }
```

> **Important:** Each refresh invalidates the old refresh_token. Store the new
> `refresh_token` and overwrite the old one.

---

## 7. Push Refreshed Tokens Back to Vault

After refreshing, re-encrypt and push back so the forwarder (and other consumers)
get the updated tokens:

### 7.1 Via HTTP API

```python
import os, urllib.request, json
from crypto_vault import encrypt_payload

new_payload = {
    "access_token": new_tokens["access_token"],
    "refresh_token": new_tokens["refresh_token"],
    "expires_at": int(time.time()) + new_tokens["expires_in"],
    "user_id": user_id,
}

secret = os.environ["CTRADER_TOKEN_SECRET"]
encrypted = encrypt_payload(secret, new_payload)

data = json.dumps({"encrypted_payload": encrypted}).encode()
req = urllib.request.Request(
    f"https://quill.nx.kg/f/api/ctrader/accounts/{account_id}/refresh",
    data=data,
    headers={
        "Content-Type": "application/json",
        "X-CSRF-Token": csrf_token,  # obtain from /api/csrf
    },
    method="POST",
)
with urllib.request.urlopen(req, timeout=15) as resp:
    print(resp.status)  # 200 = saved
```

### 7.2 Direct DB Update (for co-located systems)

```python
import pymysql

conn = pymysql.connect(host=..., user=..., password=..., database=...)
with conn.cursor() as cur:
    cur.execute(
        "UPDATE ctrader_accounts SET encrypted_payload=%s WHERE account_id=%s",
        (encrypted, account_id)
    )
conn.commit()
```

---

## 8. Automated Refresh Script

A ready-to-use script is included:

```bash
python3 scripts/ctrader_refresh.py
```

This script:
1. Fetches all `ctrader_accounts` via the HTTP API
2. Decrypts each with `CTRADER_TOKEN_SECRET`
3. Refreshes expired tokens via Spotware
4. Re-encrypts with the same secret
5. Writes back to the database

Run it on a cron schedule (every 30 minutes):

```bash
*/30 * * * * cd /Volumes/ExMac/code/ssfx/alwaydata-v3 && \
  /usr/local/bin/python3 scripts/ctrader_refresh.py >> /tmp/ctrader_refresh.log 2>&1
```

---

## 9. Token Payload Schema

The JSON inside the encrypted envelope:

```json
{
  "access_token": "eyJhbGciOiJSUzI1NiIs...",
  "refresh_token": "def50200...",
  "expires_at": 1752086400,
  "user_id": 12345678
}
```

| Field | Type | Description |
|-------|------|-------------|
| `access_token` | string | Spotware OAuth access token (valid ~1 hour) |
| `refresh_token` | string | Used to obtain new access_token without re-authenticating |
| `expires_at` | int | Unix timestamp when access_token expires |
| `user_id` | int | cTrader user id (from Spotware profile) |

---

## 10. Security Checklist

- [ ] **`CTRADER_TOKEN_SECRET` never committed to Git** — `.env` is ignored
- [ ] **Secret never synced inside the app code** — keep it only in server `.env`
- [ ] **Encrypted payloads transferred over HTTPS**
- [ ] **DB credentials rotated regularly**
- [ ] **Secret rotated on compromise** — delete all `ctrader_accounts` rows and
      re-authenticate via OAuth after rotation
- [ ] **Access tokens short-lived** (1 hour) — minimise blast radius if intercepted

---

## 11. Example: Complete Trade Execution Flow

```python
import os
from crypto_vault import decrypt_payload, encrypt_payload
from ctrader_open_api.messages.OpenApiMessages_pb2 import ProtoOANewOrderReq
import json, urllib.request, time

secret = os.environ["CTRADER_TOKEN_SECRET"]

# ── 1. Fetch and decrypt ──────────────────────────────────────────
encrypted = fetch_encrypted_payload_from_api(account_id=46659639)
payload = decrypt_payload(secret, encrypted)

# ── 2. Refresh if expired ─────────────────────────────────────────
if payload["expires_at"] < time.time() + 300:
    new_tokens = refresh_via_spotware(
        payload["refresh_token"],
        client_id="your_client_id",
        client_secret="your_client_secret",
    )
    payload["access_token"] = new_tokens["access_token"]
    payload["refresh_token"] = new_tokens["refresh_token"]
    payload["expires_at"] = int(time.time()) + new_tokens["expires_in"]
    # Push back to vault
    encrypted = encrypt_payload(secret, payload)
    push_back_to_api(account_id=46659639, encrypted_payload=encrypted)

# ── 3. Connect to cTrader and authenticate ────────────────────────
# (TCP+SSL to demo.ctraderapi.com:5035, send ProtoOAApplicationAuthReq,
#  then ProtoOAAccountAuthReq with payload["access_token"])

# ── 4. Send order ─────────────────────────────────────────────────
order = ProtoOANewOrderReq()
order.ctidTraderAccountId = 46659639
order.symbolId = 41  # XAUUSD on Pepperstone
order.orderType = ProtoOANewOrderReq.MARKET
order.tradeSide = ProtoOANewOrderReq.BUY
order.volume = 10000  # 0.1 lots in cTrader units
# ... send via protobuf framing
```

---

## 12. Troubleshooting

| Problem | Cause | Fix |
|---------|-------|-----|
| `CTRADER_TOKEN_SECRET is not configured` | Secret missing or empty in `.env` | Run `python dev_generate_keys.py` |
| `Invalid token or wrong CTRADER_TOKEN_SECRET` | Wrong secret or corrupted payload | Verify the secret matches the one used on the forwarder |
| `invalid_grant` on refresh | Refresh token expired or revoked | Re-authenticate via `/f/callback` (user must re-consent) |
| `401 Unauthorized` from Spotware | Access token expired | Refresh before calling API |
| `No tick cached yet` from `/api/price` | Spot client not started or symbol not subscribed | Check `CTRADER_SYMBOL` and that a price-augment pair is enabled |

---

*Generated for project: `/Volumes/ExMac/code/ssfx/alwaydata-v3`*
*Protocol: cTrader Open API (Spotware) v2*
*Encryption: Fernet (AES-128-CBC + HMAC-SHA256)*