Modules - technical documentation

downloader module

Different functions for downloading and saving data from exchange.

functions available for:

history of trades
history of orders
open orders
deposits
withdrawals
daily snapshots
klines

raises: SystemExit, in case config file has not been provided.
TODO: add technical indicators to kline download
TODO: re-work withdrawals equal to deposits (maybe merge them into one function ‘transfers’? and create once csv file)
TODO: standardize column names accross different files (e.g. insertTime vs. updateTime vs. updatetime or asset vs. coin)
TODO: provide option for addresslist-translation-file to translate cryptic address names into human readable names and use it for deposits & withdrawals
TODO: balances: add average entry price of open orders for locked assets + estimated exit value

binance_reporting.downloader.balances(account_name: str, account_type: str, PUBLIC: str, SECRET: str, balances_file: str = '', bal_fut_positions_file: str = '', bal_fut_assets_file: str = '', writetype: str = 'w')

download balances from exchange and write it into a csv file if provided

Procedure:

check if account is SPOT or FUTURES (there are different data models behind these two)
add current USDT price for every asset in the balance
in case there is no USDT price found: ‘0’ value will be filled in
save the data into the respective csv files (if provided), otherwise just return balance

Parameters

account_name (str) – required; added to csv file for easier tracking
account_type (SPOT or FUTURE) – required. The type of the account.
PUBLIC (str) – required; public part of API key to open connection to exchange
SECRET – required; secret part of API key to open connection to exchange
balances_file (str) – optional; name and location of the csv file to be filled with balances info (spot or futures)
bal_fut_positions_file (str) – optional; name and location of the csv file to be filled with future positions info
vbal_fut_assets_file – optional; name and location of the csv file to be filled with futures assets info
writetype (str) – optional; indicates if balances should be added (‘a’) or a new file should be written (‘w’)

Returns

writes csv file with balances of the account (if filenames have been provided)
the USD value of free & locked assets, cash available and the overall portfolio value

Return type

float64

binance_reporting.downloader.daily_account_snapshots(account_name, account_type, PUBLIC, SECRET, snapshots_balances_file, snapshots_positions_file, snapshots_assets_file)

download daily account snapshots from exchange and write it into a csv file

Procedure:

check if previous downloads exists, read them and determine the date of the last downloaded snapshot
walk through the assets of that day one-by-one to determine the close-price of that asset on the date of the snapshot
in case there is no price found: ‘0’ value will be filled in
save the data into the respective csv files

If you want to re-download all snapshots again, you just need to delete this file. Be cautious: Binance only holds max. 180 days of snapshots. In case you want to go back furhter, these days might be your only available information about older snapshots written by this procedure in case you have run it before. I recommend to save the previous version and add the missing dates manually into the csv-file.

adding some additional data to daily balances:

updatetime
USDT Price per asset from the date of the snapshot
PortVal = Value of all balances together from this day

Parameters

account_name (str) – required. The name of the account. Is used in csv exports.
account_type (SPOT or FUTURE) – required. The type of the account.
PUBLIC (str) – required. Public key of your exchange account
SECRET (str) – required. Secret key of your exchange account
snapshot_balances_file (str) – optional. filename (incl. absolute path), where the balances per day are exported to (in csv-format). For SPOT and FUTURE accounts.
snapshot_positions_file (str) – optional. filename (incl. absolute path), where the positions per day are exported to (in csv-format). For FUTURE accounts only.
snapshot_assets_file (str) – optional. filename (incl. absolute path), where the assets per day are exported to (in csv-format). For SPOT and FUTURE accounts.

Returns

portfolio value and written csv file(s) in case filename(s) have been provided

Return type

float64

TODO

make snapshot positions file an optional parameter

binance_reporting.downloader.deposits(account_name, account_type, PUBLIC, SECRET, deposits_file)

download account deposits from exchange and write them into a csv file

Procedure:

check if account is SPOT or FUTURES (there are different data models behind these two)
verify if file already exists and determine last recorded deposit
For every deposit, following data is being added to the downloaded data from the exchange:
- USDT price of the asset (close price from the day of transaction)
- In case of the price of the coin is not available anymore, ‘0’ value is being filled in.
- overall value of coins in USDT from the day of the transaction
- time of transaction in UTC format
save the downloaded deposits to csv file

Parameters

account_name (str) – required; added to csv file for easier tracking
account_type (SPOT or FUTURE) – required. The type of the account.
PUBLIC (str) – required; public part of API key to open connection to exchange
SECRET – required; secret part of API key to open connection to exchange
deposits_file (str) – required; name and location of the csv file to be filled with the deposits

Returns

writes csv file with deposits of the binance account
dataframe with all deposits

TODO

add deposits for Futures Account

binance_reporting.downloader.klines(dir, symbols, intervals, indicators, indicators_config)

downloading historic ohlc data from exchange

Procedure:

verify if kline data has been downloaded already previously
if so, determine the timestamp of the last read kline
check if new klines are available on the exchange
if so, add these to the existing klines if available
write ohlc data into a file per pair and kline interval
create new file for all data from 1d kline interval for use in excel

Parameters

dir (str) – required; name and location of the directory where the date should be written to
symbols (list) – required. list of trading pairs for which the klines should be downloaded for
intervals (list) – required; list of intervals (e.g. 1m, 5m, 1d) for which the klines should be downloaded for
indicators (str) – optional; indicators, which should be added to the csv file
indicators_config (str) – optional; parameters for the indicators, if required

Returns

writes csv files with downloaded klines and technical indicators (one file for each provided symbol)

Return type

csv file

This data can be used for backtesting (currently done in excel)

Note

Indicators are not yet implemented

Further information: description of headers for klines is documented here: https://python-binance.readthedocs.io/en/latest/binance.html?highlight=get_historical_klines_generator#module-binance.client

TODO: klines: avoid downloading klines for pairs which dont provide up-to-date data anymore (e.g. last entry is longer ago than 10 times the selected timeframe)
TODO: klines: cleanup files, which dont have up-to-date data anymore
TODO: adding technical indicators after downloading klines according to config file

binance_reporting.downloader.open_orders(account_name, account_type, PUBLIC, SECRET, open_orders_file)

get open orders and write them to csv file

Procedure:

check if account is SPOT or FUTURES (there are different data models behind these two)
connect to exchange and download all open orders
save the downloaded open orders to csv file

Parameters

account_name (str) – required; added to csv file for easier tracking
account_type (SPOT or FUTURE) – required. The type of the account.
PUBLIC (str) – required; public part of API key to open connection to exchange
SECRET – required; secret part of API key to open connection to exchange
open_orders_file (str) – required; name and location of the csv file to be filled with the open orders

Returns

writes csv file with open orders of the provided account

Return type

csv file

TODO

include open orders for FUTURES accounts as well

binance_reporting.downloader.orders(account_name, account_type, PUBLIC, SECRET, list_of_trading_pairs, orders_file)

get orders and write them to csv file

Procedure:

check if account is SPOT or FUTURES (there are different data models behind these two)
verify if file already exists and determine last recorded order
loop through provided trading pairs and download historic orders if available
save the downloaded orders to csv file

Parameters

account_name (str) – required; added to csv file for easier tracking
account_type (SPOT or FUTURE) – required. The type of the account.
PUBLIC (str) – required; public part of API key to open connection to exchange
SECRET – required; secret part of API key to open connection to exchange
list_of_trading_pairs (list) – required; list of trading pairs for which orders should be downloaded; if list is empty, every trading pair is being checked (there are over 2k trading pairs, so this can take a while)
orders_file (str) – required; name and location of the csv file to be filled with historic orders

Returns

writes csv file with historic orders of the provided account

Return type

csv file

TODO

include orders for FUTURES accounts as well

binance_reporting.downloader.prices(prices_file)

read prices for all trading pairs and write them to prices.csv file

Procedure:

connect to exchange and download all prices
save the downloaded prices to csv file

Parameters: prices_file (str) – required; name and location of the csv file to be filled with the prices
Returns: writes csv file with prices of all trading pairs on the exchange

binance_reporting.downloader.trades(account_name, account_type, PUBLIC, SECRET, list_of_trading_pairs, trades_file)

get trades and write them to csv file

Procedure:

check if account is SPOT or FUTURES (there are different data models behind these two)
verify if file already exists and determine last recorded trade
loop through provided trading pairs and download historic trades if available
save the downloaded trades to csv file

Parameters

account_name (str) – required; added to csv file for easier tracking
account_type (SPOT or FUTURE) – required. The type of the account.
PUBLIC (str) – required; public part of API key to open connection to exchange
SECRET – required; secret part of API key to open connection to exchange
list_of_trading_pairs (list) – required; list of trading pairs for which trades should be downloaded; if list is empty, every trading pair is being checked (there are over 2k trading pairs, so this can take a while)
trades_file (str) – required; name and location of the csv file to be filled with historic trades

Returns

writes csv file with historic trades of the provided account

Return type

csv file

TODO

include trades for FUTURES accounts as well

binance_reporting.downloader.withdrawals(account_name, account_type, PUBLIC, SECRET, withdrawals_file)

download account withdrawals from exchange and write them into a csv file

Procedure:

check if account is SPOT or FUTURES (there are different data models behind these two)
verify if file already exists and determine last recorded withdrawal
For every withdrawal, following data is being added to the downloaded data from the exchange:
- USDT price of the asset (close price from the day of transaction)
- In case of the price of the coin is not available anymore, ‘0’ value is being filled in.
- overall value of coins in USDT from the day of the transaction
- time of transaction in UTC format
save the downloaded withdrawals to csv file

Parameters

account_name (str) – required; added to csv file for easier tracking
account_type (SPOT or FUTURE) – required. The type of the account.
PUBLIC (str) – required; public part of API key to open connection to exchange
SECRET – required; secret part of API key to open connection to exchange
withdrawals_file (str) – required; name and location of the csv file to be filled with the withdrawals

Returns

writes csv file with withdrawals of the binance account
dataframe with all withdrawals

TODO

add withdrawals for Futures Account

ticker module

sending messages to telegram channels

functions available for:

sending balance and PnL for given accounts
send summary msg of balances for group of accounts (e.g. all SPOT accounts or all FUTURES accounts etc.)

TODO: add function to ask ad-hoc account status
TODO: add function to send msg after every trade on given accounts

binance_reporting.ticker.send_bal(accounts, account_groups, telegram_token)

sending short balance status msg to telegram channels

Goal

give regular updates about account balances and PnL via telegram

Procedure

collect balances from account given by utilizing downloader.balances function
calcuate cash, portfolio value and profit from given accounts
send short messages to telegram channels as provided in the config file
loop through account groups and send summary messages to telegram channel (if provided)

Note

Please make sure that the telegram token has access to the telegram channel and the telegram channel is public

Parameters

accounts (dict) – required; provided as in config yaml file
account_groups – required; provided as in config yaml file
telegram_token (string) – required; token of telegram account, which will send the messages

Returns

account status messages in telegram channels

helper module

helper modules for binance-reporting library

Modules available

API close connection
API weight check and cool down if overheated
removing blank lines in csv files
get symbols from exchange
merging all files in a directory into one file
merging multiple files into one file
read configuration file (yaml)

binance_reporting.helper.API_close_connection(client)

close API connection of a given client

Goal

Avoid having left-over connections to the API to keep the environment neat and clean.

Procedure

get listen key of client and close stream

Parameters: client (object) – required
Returns: None
TODO: add different connection types for futures etc.

binance_reporting.helper.API_weight_check(client)

verify current payload of Binance API and trigger cool-off if 75% of max payload is reached

Goal

Avoiding errors while downloading data from binance.

Procedure

check what the current payload is
if 75% of max payload has been reached, cool-off is initiated
send a keepalive signal for the api connection

Parameters: client (object) – required
Returns: the payload value
TODO: read current max value for Payload from Binance config
TODO: SAPI API seems to have threshold of 12000 => incorporate those (discovered during snapshot downloads)

binance_reporting.helper.file_remove_blanks(filename)

read csv file and remove blank lines

Goal

Sometimes there are blank lines added to csv files when writing them in Windows. This function removes them.

Procedure

load provided file into panda dataframe
dropping all empty rows from the dataframe
writing file back

Parameters: filename (str) – required; must include the complete absolute path to the file
Returns: written csv file without empty rows

binance_reporting.helper.get_symbols(patterns: list = [''])

get available trading pairs from exchange which contain a given string (e.g. USDT)

Goal

reduce the amount of trading pairs to walk through, e.g. when downloading historic trades

Procedure

get list of available trading pairs from exchange
filter the list according to pattern provided

Parameters

no parameter = get all trading pairs
a string = trading pairs, which contain this string
a list of strings = trading pairs, which contain any of the strings in the list

Parameters: pattern (str or list) – required (if empty, all trading pairs will be returned)
Returns: list of filtered trading pairs available on exchange

binance_reporting.helper.merge_files(files_src: list, file_trgt: str)

merging all given files into one file

Goal

creating one file with all records from different files for better usage in excel pivot tables
used to combine account information from different accounts into one file
being used for balances, withdrawals, deposits, snapshots

Procedure

get a list of source files
loop through all files and append to target file

Parameters

files_src (list) – required; list of files, which should be merged
file_trgt (str) – required; provides filename for merged csv file

Returns

csv file with all the merged info

binance_reporting.helper.merge_klines(klines_dir_src: str, klines_dir_trgt: str, filename_trgt: str)

merging all klines files of a given directory into one file

Goal

creating one file with all the klines of all trading-pairs for better usage in excel pivot tables
only being done for ‘1d’ timeframes

Procedure

get the source directory with all the kline csv files
loop through all files and append to target filename

Parameters

klines_dir_src (str) – required; provides complete path to source directory with all klines csv files
klines_dir_trgt (str) – required; provides complete path to target directory for the merged csv files
filename_trgt (str) – required; provides filename for merged csv file

Returns

csv file with all the merged klines

binance_reporting.helper.read_config(args)

read config from a given file and convert it into a dictionary

Procedure

read the default config hardcoded in the script
read config from given file (provided as argument)
update default config with config from file
give the dictionary with config infos back as a result

Parameters: args (list) – required; config file as provided when calling the script
Returns: dictionary with config info