|
| 1 | +<!-- |
| 2 | +--- |
| 3 | +weight: 802 |
| 4 | +title: "Data & SymInfo" |
| 5 | +description: "Loading and creating OHLCV data and symbol information for programmatic use" |
| 6 | +icon: "database" |
| 7 | +date: "2025-03-31" |
| 8 | +lastmod: "2026-03-17" |
| 9 | +draft: false |
| 10 | +toc: true |
| 11 | +categories: ["Programmatic", "Data"] |
| 12 | +tags: ["ohlcv", "syminfo", "data-converter", "csv", "custom-data"] |
| 13 | +--- |
| 14 | +--> |
| 15 | + |
| 16 | +# Data & SymInfo |
| 17 | + |
| 18 | +PyneCore needs two things to run a script: **OHLCV data** (candles) and **SymInfo** (symbol |
| 19 | +metadata). This page covers all the ways to provide them. |
| 20 | + |
| 21 | +## OHLCV Data |
| 22 | + |
| 23 | +### The OHLCV Type |
| 24 | + |
| 25 | +Every candle in PyneCore is an `OHLCV` namedtuple: |
| 26 | + |
| 27 | +```python |
| 28 | +from pynecore.types.ohlcv import OHLCV |
| 29 | + |
| 30 | +candle = OHLCV( |
| 31 | + timestamp=1704067200, # Unix epoch in SECONDS (not milliseconds!) |
| 32 | + open=42000.0, |
| 33 | + high=42500.0, |
| 34 | + low=41800.0, |
| 35 | + close=42300.0, |
| 36 | + volume=1000.0, |
| 37 | +) |
| 38 | +``` |
| 39 | + |
| 40 | +> **Important:** Timestamps are in **seconds**. Many exchange APIs (CCXT, Binance) return |
| 41 | +> milliseconds — divide by 1000. |
| 42 | +
|
| 43 | +### Option 1: From a CSV File |
| 44 | + |
| 45 | +Use `DataConverter` to convert CSV data to PyneCore's binary OHLCV format: |
| 46 | + |
| 47 | +```python |
| 48 | +from pathlib import Path |
| 49 | +from pynecore.core.data_converter import DataConverter |
| 50 | +from pynecore.core.ohlcv_file import OHLCVReader |
| 51 | + |
| 52 | +csv_path = Path("data/BTCUSD_1h.csv") |
| 53 | + |
| 54 | +# Convert CSV → .ohlcv binary + .toml metadata |
| 55 | +DataConverter().convert_to_ohlcv(csv_path) |
| 56 | + |
| 57 | +# Read the converted data |
| 58 | +ohlcv_path = csv_path.with_suffix(".ohlcv") |
| 59 | +with OHLCVReader(ohlcv_path) as reader: |
| 60 | + for candle in reader.read_from(reader.start_timestamp, reader.end_timestamp): |
| 61 | + print(candle.close) |
| 62 | +``` |
| 63 | + |
| 64 | +The converter automatically detects: |
| 65 | +- Column mapping (timestamp, open, high, low, close, volume) |
| 66 | +- Timezone from timestamps (DST-aware) |
| 67 | +- Tick size, trading hours, symbol type |
| 68 | + |
| 69 | +### Option 2: Create OHLCV Objects Directly |
| 70 | + |
| 71 | +For custom data sources (APIs, databases, websockets), create OHLCV objects directly: |
| 72 | + |
| 73 | +```python |
| 74 | +from pynecore.types.ohlcv import OHLCV |
| 75 | + |
| 76 | +# From a REST API |
| 77 | +def fetch_from_api(): |
| 78 | + response = requests.get("https://api.exchange.com/ohlcv/BTCUSD/1h") |
| 79 | + for bar in response.json(): |
| 80 | + yield OHLCV( |
| 81 | + timestamp=bar["time"], # must be seconds |
| 82 | + open=bar["o"], high=bar["h"], |
| 83 | + low=bar["l"], close=bar["c"], |
| 84 | + volume=bar["v"], |
| 85 | + ) |
| 86 | + |
| 87 | +# From a pandas DataFrame |
| 88 | +def from_dataframe(df): |
| 89 | + for row in df.itertuples(): |
| 90 | + yield OHLCV( |
| 91 | + timestamp=int(row.Index.timestamp()), |
| 92 | + open=row.open, high=row.high, |
| 93 | + low=row.low, close=row.close, |
| 94 | + volume=row.volume, |
| 95 | + ) |
| 96 | + |
| 97 | +# From a database |
| 98 | +def from_database(cursor): |
| 99 | + cursor.execute("SELECT ts, o, h, l, c, vol FROM candles ORDER BY ts") |
| 100 | + for row in cursor: |
| 101 | + yield OHLCV(timestamp=row[0], open=row[1], high=row[2], |
| 102 | + low=row[3], close=row[4], volume=row[5]) |
| 103 | +``` |
| 104 | + |
| 105 | +`ScriptRunner` accepts any `Iterable[OHLCV]` — lists, generators, and readers all work. |
| 106 | + |
| 107 | +### Option 3: From an Exchange (CCXT) |
| 108 | + |
| 109 | +```python |
| 110 | +import ccxt |
| 111 | +from pynecore.types.ohlcv import OHLCV |
| 112 | + |
| 113 | +exchange = ccxt.binance({"enableRateLimit": True}) |
| 114 | +raw = exchange.fetch_ohlcv("BTC/USDT", "1h", limit=200) |
| 115 | + |
| 116 | +candles = [ |
| 117 | + OHLCV( |
| 118 | + timestamp=bar[0] // 1000, # CCXT returns milliseconds! |
| 119 | + open=bar[1], high=bar[2], low=bar[3], close=bar[4], volume=bar[5], |
| 120 | + ) |
| 121 | + for bar in raw |
| 122 | +] |
| 123 | +``` |
| 124 | + |
| 125 | +## SymInfo (Symbol Information) |
| 126 | + |
| 127 | +SymInfo tells PyneCore about the financial instrument — currency, tick size, timezone, market type, |
| 128 | +etc. Scripts access this via `syminfo.*` (e.g., `syminfo.mintick`, `syminfo.currency`). |
| 129 | + |
| 130 | +### Option 1: Load from TOML |
| 131 | + |
| 132 | +When you convert a CSV file, a `.toml` file is automatically generated: |
| 133 | + |
| 134 | +```python |
| 135 | +from pynecore.core.syminfo import SymInfo |
| 136 | + |
| 137 | +syminfo = SymInfo.load_toml(Path("data/BTCUSD_1h.toml")) |
| 138 | +``` |
| 139 | + |
| 140 | +### Option 2: Create Manually |
| 141 | + |
| 142 | +For custom data sources, build SymInfo by hand: |
| 143 | + |
| 144 | +```python |
| 145 | +from pynecore.core.syminfo import SymInfo |
| 146 | + |
| 147 | +syminfo = SymInfo( |
| 148 | + prefix="BINANCE", # exchange/provider name |
| 149 | + description="Bitcoin / USD", # human-readable name |
| 150 | + ticker="BTCUSD", # symbol ticker |
| 151 | + currency="USD", # quote currency |
| 152 | + basecurrency="BTC", # base currency |
| 153 | + period="60", # timeframe: "1", "5", "15", "60", "D", "W", "M" |
| 154 | + type="crypto", # "stock", "forex", "crypto", "futures", "index" |
| 155 | + mintick=0.01, # smallest price increment |
| 156 | + pricescale=100, # 1 / mintick |
| 157 | + minmove=1, # minimum price movement in pricescale units |
| 158 | + pointvalue=1.0, # profit per 1 unit price move per 1 contract |
| 159 | + timezone="UTC", # IANA timezone (e.g., "America/New_York") |
| 160 | + volumetype="base", # "base", "quote", "tick", "n/a" |
| 161 | + opening_hours=[], # trading session hours (empty for 24/7 crypto) |
| 162 | + session_starts=[], # session start times |
| 163 | + session_ends=[], # session end times |
| 164 | +) |
| 165 | +``` |
| 166 | + |
| 167 | +### Common SymInfo Configurations |
| 168 | + |
| 169 | +**Crypto (24/7 trading):** |
| 170 | + |
| 171 | +```python |
| 172 | +SymInfo( |
| 173 | + prefix="BINANCE", description="BTC / USDT", ticker="BTCUSDT", |
| 174 | + currency="USDT", basecurrency="BTC", period="60", |
| 175 | + type="crypto", mintick=0.01, pricescale=100, minmove=1, pointvalue=1.0, |
| 176 | + timezone="UTC", volumetype="base", |
| 177 | + opening_hours=[], session_starts=[], session_ends=[], |
| 178 | +) |
| 179 | +``` |
| 180 | + |
| 181 | +**Forex:** |
| 182 | + |
| 183 | +```python |
| 184 | +SymInfo( |
| 185 | + prefix="FX", description="EUR / USD", ticker="EURUSD", |
| 186 | + currency="USD", basecurrency="EUR", period="60", |
| 187 | + type="forex", mintick=0.0001, pricescale=10000, minmove=1, pointvalue=1.0, |
| 188 | + timezone="America/New_York", volumetype="tick", |
| 189 | + opening_hours=[], session_starts=[], session_ends=[], |
| 190 | +) |
| 191 | +``` |
| 192 | + |
| 193 | +**US Stocks:** |
| 194 | + |
| 195 | +```python |
| 196 | +SymInfo( |
| 197 | + prefix="NASDAQ", description="Apple Inc.", ticker="AAPL", |
| 198 | + currency="USD", period="D", |
| 199 | + type="stock", mintick=0.01, pricescale=100, minmove=1, pointvalue=1.0, |
| 200 | + timezone="America/New_York", volumetype="base", |
| 201 | + opening_hours=[], session_starts=[], session_ends=[], |
| 202 | +) |
| 203 | +``` |
| 204 | + |
| 205 | +### Period Values |
| 206 | + |
| 207 | +The `period` field uses the same values as TradingView's Pine Script `timeframe.period`: |
| 208 | + |
| 209 | +| Timeframe | Period value | |
| 210 | +|-----------|-------------| |
| 211 | +| 1 minute | `"1"` | |
| 212 | +| 5 minutes | `"5"` | |
| 213 | +| 15 minutes | `"15"` | |
| 214 | +| 30 minutes | `"30"` | |
| 215 | +| 1 hour | `"60"` | |
| 216 | +| 4 hours | `"240"` | |
| 217 | +| Daily | `"D"` | |
| 218 | +| Weekly | `"W"` | |
| 219 | +| Monthly | `"M"` | |
0 commit comments