I am trying to capture all the replaced transactions. I have complied Bitcon Core with tracing enabled and listen to the mempool:replaced as suggested in this question.
This method works as expected and I can get the replaced transactions. However, in some cases (about 20% of all the transactions I capture) the tracepoint alerts me of the transaction being replaced, but then I am not capturing the replacement of that transaction.
I can think of two reasons why this is happening:
- It could be that some of the replacements do not enter my mempool. But if that is the case, how does the tracepoint knows that the transactions is being replaced? Also, it is weird that this would happen with so many transactions (20% as explained earlier).
- The second option is that the script that process the transaction is not considering some cases, which ends up in losing transactions. The script is a modified version of this example. I have modified it so I can capture the children of the replaced transactions as well. The data I gather with this script is later saved in a pkl file, which is why it has a shared list.
I don’t know why I am losing so many transactions, and I was wondering if someone could help see what I am doing wrong.
Here are the modified functions:
def handle_replaced(_, data, size):
# Establish a new RPC connection for this event
rpc_connection_replaced = connection()
event = bpf["replaced_events"].event(data)
hash_replaced = bytes(event.replaced_hash)[::-1].hex()
hash_new = bytes(event.replacement_hash)[::-1].hex()
tx_time = get_timestamp()
hex_tx = rpc_connection_replaced.getrawtransaction(hash_new)
new_tx = rpc_connection_replaced.decoderawtransaction(hex_tx)
new_tx['hex'] = hex_tx
# Determine parent transactions that are still in the mempool and are not the transaction itself
parents = set([x['txid'] for x in new_tx['vin'] if x['txid'] in mempool.keys() and x['txid'] != new_tx['txid']])
# Retrieve the old transaction info from the mempool
old = mempool.get(hash_replaced, None)
if old is not None: # Some transactions may be missing initially
mempool.pop(hash_replaced, None)
# Share the replaced event details via list_shared queue
list_shared.put((old, [new_tx, tx_time, parents]))
# Find all child transactions that reference the new transaction as a parent
childs = [i[0]['txid'] for i in mempool.values() if new_tx['txid'] in i[2]]
# Recursive function to collect child transaction IDs
def child(ll):
if len(ll) == 0:
return []
new_childs = [i[0]['txid'] for i in mempool.values() if ll - i[2] != ll]
return list(ll) + new_childs + child(set(new_childs))
if len(childs) > 0:
childs = child(set(childs))
# Share the connection between the new transaction and the first child
list_shared.put(([new_tx, tx_time, parents], mempool[childs[0]]))
# Share connections between subsequent child transactions
for i in range(len(childs)-1):
list_shared.put((mempool[childs[i]], mempool[childs[i+1]]))
# Add the new transaction to the mempool with its timestamp and parent set
mempool[new_tx['txid']] = [new_tx, tx_time, parents]
logger.info('-----------')
logger.info('New RBF!')
def handle_added(_, data, size):
rpc_connection_added = connection()
event = bpf["added_events"].event(data)
hash_new = bytes(event.hash)[::-1].hex()
hex_tx = rpc_connection_added.getrawtransaction(hash_new)
tx_raw = rpc_connection_added.decoderawtransaction(hex_tx)
tx_raw['hex'] = hex_tx
parents = set([x['txid'] for x in tx_raw['vin'] if x['txid'] in mempool.keys()])
mempool[tx_raw['txid']] = [tx_raw, get_timestamp(), parents]
def handle_removed(_, data, size):
event = bpf["removed_events"].event(data)
if event.reason != b'replaced':
txid_rem = bytes(event.hash)[::-1].hex()
keys = mempool.keys()
if txid_rem in keys:
mempool.pop(txid_rem)
logger.info('-----------')
logger.info(f'Removed. Reason:{event.reason}')
