Transaction Fee Estimations: How To Save On Gas? Part 1
Today, Upvest is excited to launch two new API endpoints - to estimate the transaction fee for Ethereum and transaction fee for Bitcoin. Transactions executed on the blockchain usually have fees associated with them. The transaction fee provided by the sender, determines how fast a given transaction gets processed, legitimated and confirmed. Therefore, it becomes important to know how much to pay, and understand what confirmation speeds (or times) the varying fee costs are going to get you. In this post, we will describe in detail why we built an API endpoint (currently in Beta) to estimate such transaction fees and how it helps you save on total fee costs.
Transaction Fee Basics
Users and developers of products built on blockchains frequently come across the terms Transaction Fees, Gas, Gas Price and Gas Limit. Feel free to jump ahead to the next section if you already know what this is.
For those who are new to this, let’s cover some of the fee basics for both blockchains. Ethereum and Bitcoin implement the Proof-of-Work algorithm in which “Miners”, i.e. blockchain validators make use of special algorithms that require significant amounts of computational effort to solve a cryptographic “problem”. Finding a successful solution to this problem validates a set of transactions and includes them into a cryptographically verifiable block. Essentially, Miners race to find the “golden Nonce,” a numerical value that validates a given block. The successful Miner gets compensated for the costs of electricity and computational hardware for solving this puzzle (the “Proof-Of-Work”). This reward for mining a successful block is called block reward. It is made up of two components a static block subsidy and a dynamic transaction fee, both usually paid for in the blockchain’s native currency, i.e., Ether or Bitcoin. Block subsidy is usually provided by newly minting a fixed amount of blockchain native currency that goes directly to the Miner and the transaction fee, provided by the sender.
In the process of building blocks of validated transactions, many competing “forks” form, and exist in parallel at any given time. So, the network continuously determines the main branch on the chain of blocks or the “canonical chain”. An attacker can secretly construct a longer branch sequence and publish it as the canonical chain, by using more than 50% of the network’s computing power (51% attack). This attack becomes more expensive, the more blocks the attacker wants to overwrite. A 51% attack could affect the network at any time, so transactions are only considered to be confirmed when the blockchain has grown by a certain number of blocks, called the Confirmation Level. On achieving a Confirmation Level of N (N is variable for different blockchains) the mined transaction is considered confirmed. Confirmed transactions can be found in a given block for which both the block subsidy and the transaction fee goes to the successful miner (or a mining pool). The longer the canonical chain is, after the block that the transaction is in, higher is the confidence in the transaction's immutability.
In addition, in the case of Ethereum, the Ethereum Virtual Machine (EVM) allows for execution of smart contract bytecode. Execution of this code could lead to a change of state of the blockchain. A change of state to Ethereum is also transmitted and recorded globally throughout the distributed network. During this entire process it cannot be known in advance if a computation would end in a fixed amount of time or may presumably, go on forever. This is a well known problem, aptly named the Turing halting problem. A way to confront this challenge, in Ethereum, is by introducing a cost per unit step of the computation. In general, more the bytecode to process, higher the transaction fee. Thus, the total transaction fee is the Gas consumed (in the computational steps) multiplied by the Gas Price.
Now, the end users (or developers on their behalf) submitting these transactions and/or calling smart contract functions to make a change of state on the blockchain, need to compensate the Miners in the form of some valuable currency. In the case of Ethereum (ETH) and Bitcoin (BTC), the smallest unit of cryptocurrencies for both chains - Wei (10^-18 ETH) and Satoshis (10^-8 BTC), is the currency used to cover the transaction fees. In the Bitcoin parlance, fees for transactions are calculated in Satoshis per virtual byte (vbyte) of transaction data. This resource provides a time-series plot of the transaction fee essentials for Bitcoin. In Ethereum things get a bit more nuanced, documented well in this guide. Here’s a brief summary of the fee essentials for Ethereum:
- Gas Price is the price a user is willing to pay per unit of Gas for the work done, in Gwei. (1 GWei is 10-⁹ ETH)
- Gas Limit is the maximum amount of Gas you’re willing to pay for the submitted transaction to be mined.
- Transaction fee = Gas consumed x Gas Price
- If Transaction fee <= Gas Limit, the transaction is executed and mined into a block, the state of the blockchain is updated.
- If Transaction fee > Gas Limit, the transaction is reverted, the Gas consumed from executing EVM bytecode still goes to the Miner for the computations performed, but the state of Ethereum is not modified.
- As transactions can have varying payloads, various transactions require varying amounts of Gas in order to be mined. Common transactions, such as an Ether transfer, have a fixed amount of Gas required to be processed (i.e. 21,000).
- Block Gas Limit is the maximum Gas allowed in a block (i.e. a maximum amount of computational effort required to process all of the transactions being included within that block). Hence each block has a theoretical maximum number of transactions that it can contain.
- Etherscan.io provides a time series plot of the different essentials.
Transaction Confirmation Times & Gas Fee Spikes
A Gas Fee market results from the demand created by transaction senders and the supply for mining them, determined by the Miners. Both Ethereum and Bitcoin impose a limit on the amount of computation that can be done per block with a Block Gas Limit. The amount of time each block approximately takes to be constructed and mined is also fixed, called the Block Time. Given that any transaction has a minimum Gas consumption and all blocks a Block Gas Limit, there is a theoretical limit on the total number of Ethereum / Bitcoin transactions that can be mined in a day. Conclusively, both blockchains have a transaction throughput or maximum number of transactions per second (approximately 31 transactions per second (tps) at the time of writing for Ethereum).
The proportion of supply and demand determines the “cost” of a transaction or the “cost” of Gas (Gas Price in Ethereum) at any given time. Therefore, if demand side chooses to get their transactions included in a block sooner, then they need to pay a higher price for their transactions per unit of Gas. In the case of an increase in network activity, the demand for transactions increases; this can lead to a spike in transaction fees. Businesses that often transact on the blockchain or employ smart contracts on a day-to-day basis to automate business logic suddenly find that the transactions they were paying a certain fee for would not get mined unless they were to pay up to ten times as much as before. If the transaction senders are not aware of the fee spike, it often leads to their transactions taking much longer than expected, to get mined. In certain circumstances, where the transaction fee remains high, these low-fee (relatively) pending transactions may even get completely dropped off by the network. If end-users start seeing failed transactions, they get discouraged to execute further transactions. This can lead to a drop in revenue generating activity on the blockchain for these businesses and also negatively affects marketplace liquidity.
For example - when a large number of users interact with smart contracts associated with a very popular decentralized application (DApp), Miners start choosing the transactions that offer higher Gas Price and Gas Limit and other parameters they consider important for their mining strategies. In October 2017, the Cryptokitties DApp caused a network congestion which resulted in a crazy spike, where the average Gas Price went from 5 GWei before the game was released on the mainnet, to a value of 50 GWei right afterwards. In certain cases, the transaction fees got even higher than the perceived value of the game’s collectible itself.
To overcome such issues, a model that provides fee estimates for various blockchains should take their network congestions into consideration. It should proportionally raise the recommended Gas Price to an optimal economical value to get the transaction mined in the required amount of time.
An Adaptive Fee API
Upvest is excited to release its Transaction Fee API endpoints that models the transaction fee spikes in real time while making sure that they do not result in unreasonably high estimations. Fee spikes, occurring as a result of circumstances similar to the one mentioned above, are normalized by a proprietary smoothing algorithm. The model calculates the appropriate fee by utilizing data from different public sources for both Bitcoin and Ethereum. It then computes a weighted average from the collected values. The weights on each source may differ depending on its capacity to represent the network’s congestion conditions. The endpoints provide the transaction fee estimates (in blockchain native currencies) in real-time for variable transaction confirmation times (or block times):
Transaction Fee for Ethereum,
- Fastest: below 30 seconds (next block)
- Fast: below 2 minutes (<10 blocks)
- Medium: around 5 minutes (<20 blocks)
- Slow: around 30 minutes a.k.a safe-low (<120 blocks)
Transaction Fee for Bitcoin,
- Fast: around or below 15 minutes (within 1-2 blocks)
- Medium: around 30 minutes (within 3-4 blocks)
- Slow: above 1 hour but below 2 hours (>5 blocks)
The fastest prediction for Ethereum should not be used in most cases, unless a transaction needs to be urgently included in the next block. The Gas Price is higher than the market average for fastest, to cover for this particular case. Try out the API for Ethereum Fees here. A sample of the returned values in Gwei is seen here:
Try out the API for Bitcoin Fees here. A sample of the returned values in Satoshis/VByte is seen here:
These endpoints can easily handle thousands of requests per second. They are also adaptive (self-healing) such that the endpoints provide accurate estimates even with the degradation of one or more sources. If a data source becomes unavailable, the estimation algorithm ignores the data source by expunging it from the database. In the case of major network spikes or if some of the sources fail, the Fees API algorithm will still provide dependable estimates. At the time of writing, there are few fee estimation endpoints with these production ready features. So, we're super excited to invite you to try them out (in Beta)! Here’s a time series plot depicting fee estimation data for both endpoints:
An Advanced Fee Prediction API
Upvest plans to adapt the current algorithm and use it as a source to develop a more intricate machine learning model to predict fees based on historical transaction data and pending transactions. As we develop the algorithm we can also extend it to other protocols, provided there are public data sources and historical data availability.
Do you have unique ideas on estimating transaction fees or want to know how a fee estimation API can help your business? DM our Engineering Lead at Upvest, Paulius Uza (upheaver#5049) on Discord and if you would like to know more, browse our documentation and start experimenting with the Upvest Fees API here!