Improving the speed of Deluge

We have been working on optimizing Deluge to use less resources and handle more torrents. Most issues with speed gets noticable when the number of torrents grow past a given threshold. Deluge 1.3 will not be getting any more optimizations or improvements, so any new code will go into the develop branch.

Tickets for performance issues that have not yet been fixed or applied to the develop branch are listed in the performance milestone.

Current state of Deluge (v1.3)

Deluge v1.3 handles hundreds of torrents fairly well. The GTKUI uses a lot of CPU to render the torrent list, so to keep CPU usage at a minimum it is a good idea to run in deamon/client mode and only connect with to the daemon when you need.

When the torrent list reaches 1000 torrents, the CPU usage gets more and more noticable, the more torrents the slower it gets.

The following patches have been applied to 1.3: * Cache tracker icons in gtkui ([#2248](https://dev.deluge-torrent.org/ticket/2248)) git commit (gtkui) * Small (but significant) speed optimizations to the torrentview ([#2259](https://dev.deluge-torrent.org/ticket/2259)) git commit (gtkui) (smaller version of #2184 for 1.3 branch) * Speed optimizations to the torrentview in GTKUI ([#2184](https://dev.deluge-torrent.org/ticket/2184)), git commit

State of git-develop (To be v2)

git-develop has been tested and works well with 2-3000 torrents. Currently the biggest problem is that to make libtorrent v0.16.X handle that many active torrents, the settings active_dht_limit, active_tracker_limit, active_lsd_limit must be set. This is currently not implemented in git-develop, but applying the patch proposed in #2257 should solve this. this can also be fixed by using the ltconfig plugin found in the list of plugins.

The following patches have already been applied which improves the speed of Deluge with many torrents greatly:

GTKUI patches==

• Speed optimizations to the torrentview in GTKUI ([#2184](https://dev.deluge-torrent.org/ticket/2184)), git commit
• Cache tracker icons in gtkui ([#2248](https://dev.deluge-torrent.org/ticket/2248)) (git commit)
• Speed optimizations to sessionproxy ([#2285](https://dev.deluge-torrent.org/ticket/2285)) git commit
• Add caching to the data-funcs in ui/gtkui/torrentview_data_funcs.py git commit
• Speedup removing multiple torrents from core ([#2250](https://dev.deluge-torrent.org/ticket/2250)) (gtkui)

Core patches

• Speed optimizations to the daemon git commit
• Speedup set_prioritize_first_last in torrent.py ([#2246](https://dev.deluge-torrent.org/ticket/2246)), git commit (mainly on startup)
• Move cleanup of prev_status from torrent.py to torrentmanager.py ([#2379](https://dev.deluge-torrent.org/ticket/2379)) git commit
• Application layer protocol for transfering RPC messages + utf8 decoding error ([#2116](https://dev.deluge-torrent.org/ticket/2116)), git commit (rpc communication between daemon and clients)
• Speed up adding multiple torrents ([#2406](https://dev.deluge-torrent.org/ticket/2406)) (core/gtkui)

Proposed patches: * Allow setting of active_dht_limit, active_tracker_limit and active_lsd_limit. Fix bugs. ([#2257](https://dev.deluge-torrent.org/ticket/2257)) (daemon)

Other issues that need investigation: * Pausing/resuming multiple torrents is very slow. This is probably because when a torrent changes state, a lot of operations are done (event listeners in GUI presumable), so with hundreds of torrents changing state the result is too much work needing to be done. * CPU usage on an idle daemon can still be around 5% ([#1490](https://dev.deluge-torrent.org/ticket/1490)). * Reducing the number of calls (executions) torrent.update_state during startup (loading torrents) * Investigate if libtorrent.file_progress is a bottleneck