VOOZH about

URL: https://codeberg.org/edestcroix/Recordbox/issues/243

โ‡ฑ #243 - Progress bar for full library rebuild doesn't update until completion - edestcroix/Recordbox - Codeberg.org


Progress bar for full library rebuild doesn't update until completion #243

Open
opened by manan-gup ยท 5 comments
๐Ÿ‘ manan-gup
Contributor
Copy link

The progress bar for the full library rebuild stays at zero throughout the rebuilding process and only moves once the task is complete.

smol::future::zip(self.handle_updates(receiver),self.spawn_sync(full,sender)).await;


I think it's probably because the spawn_sync method needs to run on a separate thread as it's doing a lot of work and that's likely blocking the UI from updating.

On a related but slightly separate note, why have you used deadpool instead of r2d2, which comes as a feature of diesel? Just curious.

The progress bar for the full library rebuild stays at zero throughout the rebuilding process and only moves once the task is complete. https://codeberg.org/edestcroix/Recordbox/src/commit/717d0675387d119b5b8442b26ecc204d162f0443/src/library.rs#L193 I think it's probably because the `spawn_sync` method needs to run on a separate thread as it's doing a lot of work and that's likely blocking the UI from updating. On a related but slightly separate note, why have you used `deadpool` instead of `r2d2`, which comes as a feature of `diesel`? Just curious.
๐Ÿ‘ Emmett de St. Croix
Owner
Copy link
Lines 200 to 203 in 717d067
conn.interact(move|conn|{
sync::sync_library(conn,&root,&sender,full);
})
.await

Closures provided to conn.interact(..) are run on a separate thread by deadpool, so spawn_sync isn't blocking the UI, it should just be idle awaiting for this interact call to complete. The UI should be updating when it gets messages from this thread.

It could be due to how the glib main loop prioritizes tasks. As far as I know the actual redrawing of the UI in GTK doesn't happen right away when something changes, but is instead queued on the main loop to run when it's idle. So then in the handle_updates function here:

Lines 207 to 211 in 717d067
asyncfn handle_updates(&self,receiver: Receiver<ProgressUpdate>){
whileletOk(update)=receiver.recv().await{
matchupdate{
ProgressUpdate::Total(n)=>self.emit_by_name::<()>("sync-started",&[&n]),
ProgressUpdate::Progress=>self.emit_by_name::<()>("sync-progress",&[]),

I'm guessing that the main loop only goes idle when that await on the receiver is pending waiting for a new message, so if a new progress update happens to arrive before the previous one is finished being handled the main loop never goes idle and never switches to processing all the UI updates that are getting queued up.

I've just tried changing the handle_updates function to run as a low-priority idle task that only checks for progress updates when it runs, instead of waking up and taking priority whenever there is a progress update, so theoretically the main loop should prioritize UI updates more this way. Not sure if it fixes it completely because I didn't have this issue happening on my devices to confirm a difference. (probably just due to how async timing happens to work out on different CPUs or something)

On a related but slightly separate note, why have you used deadpool instead of r2d2, which comes as a feature of diesel? Just curious.

I used deadpool over r2d2 because deadpool has better async integration. With r2d2 everything is blocking/non-async, and in general it seems more designed for multi-threading situations, while deadpool uses all async functions that felt more ergonomic to use since I was already using async code everywhere. Deadpool manages it's own threadpool internally for each connection and exposes an async interface rather than you having to spawn threads/async tasks manually.

https://codeberg.org/edestcroix/Recordbox/src/commit/717d0675387d119b5b8442b26ecc204d162f0443/src/library.rs#L200-L203 Closures provided to `conn.interact(..)` are run on a separate thread by deadpool, so `spawn_sync` isn't blocking the UI, it should just be idle awaiting for this `interact` call to complete. The UI should be updating when it gets messages from this thread. It could be due to how the glib main loop prioritizes tasks. As far as I know the actual redrawing of the UI in GTK doesn't happen right away when something changes, but is instead queued on the main loop to run when it's idle. So then in the `handle_updates` function here: https://codeberg.org/edestcroix/Recordbox/src/commit/717d0675387d119b5b8442b26ecc204d162f0443/src/library.rs#L207-L211 I'm guessing that the main loop only goes idle when that await on the receiver is pending waiting for a new message, so if a new progress update happens to arrive before the previous one is finished being handled the main loop never goes idle and never switches to processing all the UI updates that are getting queued up. I've just tried changing the `handle_updates` function to run as a low-priority idle task that only checks for progress updates when it runs, instead of waking up and taking priority whenever there is a progress update, so theoretically the main loop should prioritize UI updates more this way. Not sure if it fixes it completely because I didn't have this issue happening on my devices to confirm a difference. (probably just due to how async timing happens to work out on different CPUs or something) > On a related but slightly separate note, why have you used deadpool instead of r2d2, which comes as a feature of diesel? Just curious. I used deadpool over r2d2 because deadpool has better async integration. With r2d2 everything is blocking/non-async, and in general it seems more designed for multi-threading situations, while deadpool uses all async functions that felt more ergonomic to use since I was already using async code everywhere. Deadpool manages it's own threadpool internally for each connection and exposes an async interface rather than you having to spawn threads/async tasks manually.
๐Ÿ‘ manan-gup
Author
Contributor
Copy link

Oh, I didn't know that about conn.interact.... The deadpool docs are really sparse. Thanks for looking into this and explaining the issue. If you can commit the handle_updates change to a new branch then I'll be happy to test it out.

deadpool uses all async functions that felt more ergonomic to use since I was already using async code everywhere

Thanks for the explanation. That makes sense. I've slowly taught myself basics of GTK development but async & parallel code is still a little beyond me at the moment. ๐Ÿ˜…
EDIT: Ok, I just saw the changes in the devel branch. I'll test them and get back.

Oh, I didn't know that about `conn.interact...`. The deadpool docs are really sparse. Thanks for looking into this and explaining the issue. If you can commit the `handle_updates` change to a new branch then I'll be happy to test it out. >deadpool uses all async functions that felt more ergonomic to use since I was already using async code everywhere Thanks for the explanation. That makes sense. I've slowly taught myself basics of GTK development but async & parallel code is still a little beyond me at the moment. ๐Ÿ˜… EDIT: Ok, I just saw the changes in the devel branch. I'll test them and get back.
๐Ÿ‘ manan-gup
Author
Contributor
Copy link

I tested the dev-release and I'm getting the same behaviour as before. The only thing I can think of is that I'm on CachyOS and that probably has some kernel scheduler changes that interfere with this. If I can test it with the stock Arch kernel then I'll report back.

I tested the `dev-release` and I'm getting the same behaviour as before. The only thing I can think of is that I'm on CachyOS and that probably has some kernel scheduler changes that interfere with this. If I can test it with the stock Arch kernel then I'll report back.
๐Ÿ‘ manan-gup
Author
Contributor
Copy link

Ok, I tested it out with the stock Arch kernel. The old code doesn't work with any kernel on my machine. The new code works with the Arch kernel but not the CachyOS kernel. I guess you can close this issue once the new changes are merged.

Ok, I tested it out with the stock Arch kernel. The old code doesn't work with any kernel on my machine. The new code works with the Arch kernel but not the CachyOS kernel. I guess you can close this issue once the new changes are merged.
๐Ÿ‘ manan-gup
Author
Contributor
Copy link

I asked the CachyOS devs and they think it was the commit that was reverted here:
github.com/CachyOS/linux@799e792dfd
It will be resolved once kernel 7 (tested the dev-release with this and it works) is released.

I asked the CachyOS devs and they think it was the commit that was reverted here: https://github.com/CachyOS/linux/commit/799e792dfd75424f4a84b78fc5156dd6b19debed It will be resolved once kernel 7 (tested the `dev-release` with this and it works) is released.
Sign in to join this conversation.
No Branch/Tag specified
Clear milestone
No items
No milestone
Clear projects
No items
No project
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
edestcroix/Recordbox#243
Reference in a new issue
No description provided.
Delete branch "%!s(

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?