Start with free public datasets
Connected your DB but have nothing to put in it? Start with one of these three. All free, no API key, and the loader script plus the exact declaration values (query, columns) are copy-paste ready. The script is built around a parser map — adding your own source is one more entry.
Get the fetchers
Clone the fetcher repo onto the machine that runs your agent. The only dependency is pg. ★ DATABASE_URL in .env must be a write account. A read-only account is supposed to reject the INSERT; that one is for the SoriTrading integration.
git clone https://github.com/valueconnectinc/soritrading-fetchers ~/soritrading-fetchers cd ~/soritrading-fetchers && npm install cp .env.example .env # DATABASE_URL
https://github.com/valueconnectinc/soritrading-fetchers · Open source · one folder per fetcher · cron included
The three below are **starters**. The repo also carries VIX, crypto implied volatility (DVOL), the yield curve, the dollar index, stablecoin market cap and DeFi TVL — all free, no API key. Run `node run.mjs list` for the full list. Market series have no value on US holidays (missing, not zero), and DVOL starts in March 2021.
1. Fear & Greed Index
alternative.me0 (extreme fear) to 100 (extreme greed), daily, full history since Feb 2018. A sentiment axis price alone cannot show — the best first test.
node run.mjs fear-greed
query: SELECT ts, v FROM fear_greed WHERE ts BETWEEN $1 AND $2 tsCol: ts · valCol: v · dbAlias: main
2. Bitcoin hash rate
blockchain.comA chain-native metric (TH/s) on a different axis from price, useful as a regime filter. Values are large and slow-moving, so a ratio to its moving average works better than the raw level.
node run.mjs btc-hashrate
query: SELECT ts, v FROM btc_hashrate WHERE ts BETWEEN $1 AND $2 tsCol: ts · valCol: v · dbAlias: main
3. US federal funds rate (practice)
FRED (DFF)Daily effective federal funds rate — the classic macro regime filter. ★ Note: a similar axis already ships as ctx.macro. Feeding the same value through two paths makes results ambiguous about which one was used, so treat this as integration practice only.
node run.mjs fed-funds
query: SELECT ts, v FROM macro_fed_funds WHERE ts BETWEEN $1 AND $2 tsCol: ts · valCol: v · dbAlias: main
Scheduled collection (cron)
Installs each fetcher's own schedule into crontab. Re-running replaces the block instead of stacking lines, and your own entries are left alone. Logs go to a per-fetcher file under logs/.
node run.mjs --install-cron
- Use epoch seconds for the time column (ms and ISO are auto-detected, and the tsUnit field of the verification report tells you how it was read).
- Re-running the script is safe — it upserts, so no duplicates and only newer values change. Re-run periodically for fresh data.
- Backtest results carry a data fingerprint (row count, hash). If the DB content changes the fingerprint changes, and two results with different fingerprints were not run on the same data.
- Backtests that use external data run on the agent only (the data lives on your machine). The browser runner releases those jobs.
- Tables you expose become visible to the AI (the columns, sample and query levels send real data) — do not expose tables containing personal information.
All documentation — İstifadə qaydaları