If you are using default suggested fee and the total fee of your transaction (TX) is very high, then the reason is per my understanding either that
A) the Bitcoin has currently too many transactions to confirm (so the fees are high). You may check it at https://bitcoinfees.net
B) your TX size (in bytes) is high thanks to for example high number of unspent transaction outputs (UTXO) that are going to be spend in your new TX. One can customize and decrease Satoshi/byte fee of the TX and wait longer time for the confirmation. If you can afford sending lower amount and send the rest later, then other option is to minimize number of mentioned UTXO that will be used in your TX (lowering total TX byte size which will decrease the fee that depends on it). In Electrum wallet app, it is Coins tab and Coin control option, selecting low number of bigger UTXO). Yet if you want to sweep all funds, you may need to spend all UTXO anyway soon or later – but maybe schedule it for the time when network will be utilized lower: https://jochen-hoenicke.de/queue/#BTC%20(default%20mempool),1y,count – it may help to estimate how often there are periods of low fees. Btw. some BTC servers has limit of minimum accepted sat/byte, though you may change used server in many wallet apps.
