VOOZH about

URL: https://dev.to/hiyoyok/parallel-file-transfers-in-rust-how-i-made-android-sync-actually-fast-ao5

⇱ Parallel File Transfers in Rust — How I Made Android Sync Actually Fast - DEV Community


All tests run on an 8-year-old MacBook Air. All results from shipping 7 Mac apps as a solo developer. No sponsored opinion.

Sequential file transfer is slow. HiyokoAutoSync uses parallel transfers with a concurrency limit. Here's how I built it.


The problem with sequential transfer

Copy 100 photos from Android to Mac one at a time: each transfer waits for the previous to complete. ADB overhead per file adds up. On a large library, this takes minutes.

Parallel transfers use the available bandwidth more efficiently. Same 100 files, 6 concurrent transfers: significantly faster.


tokio::sync::Semaphore for concurrency control

Unlimited parallelism isn't better — it overwhelms the ADB connection and the device. A semaphore limits concurrent transfers to a useful number:

use tokio::sync::Semaphore;
use std::sync::Arc;

const MAX_CONCURRENT: usize = 6;

async fn transfer_files(files: Vec<FileEntry>) -> Result<(), AppError> {
 let semaphore = Arc::new(Semaphore::new(MAX_CONCURRENT));
 let mut handles = vec![];

 for file in files {
 let sem = Arc::clone(&semaphore);
 let handle = tokio::spawn(async move {
 let _permit = sem.acquire().await.unwrap();
 transfer_single_file(&file).await
 });
 handles.push(handle);
 }

 // Collect results
 for handle in handles {
 handle.await??;
 }

 Ok(())
}

6 concurrent transfers was the sweet spot in my testing — fast without overwhelming the connection.


Progress tracking across parallel transfers

Each transfer needs to report progress independently. Use an atomic counter:

use std::sync::atomic::{AtomicUsize, Ordering};

let completed = Arc::new(AtomicUsize::new(0));
let total = files.len();

for file in files {
 let completed = Arc::clone(&completed);
 let handle_clone = app_handle.clone();

 tokio::spawn(async move {
 let _permit = sem.acquire().await.unwrap();
 transfer_single_file(&file).await?;

 let done = completed.fetch_add(1, Ordering::Relaxed) + 1;
 handle_clone.emit("transfer-progress", Progress {
 completed: done,
 total,
 current_file: file.name.clone(),
 }).ok();

 Ok::<(), AppError>(())
 });
}

Error handling in parallel

One failed transfer shouldn't kill all others. Collect errors and report at the end:

let results: Vec<Result<(), AppError>> = futures::future::join_all(handles)
 .await
 .into_iter()
 .map(|r| r.unwrap_or_else(|e| Err(AppError::Task(e.to_string()))))
 .collect();

let errors: Vec<_> = results.into_iter().filter_map(|r| r.err()).collect();
if !errors.is_empty() {
 // Report partial failure — some files transferred, some didn't
}

Partial success is better than all-or-nothing for file transfers.


The result

6-lane parallel transfer on HiyokoAutoSync is noticeably faster than sequential for large photo libraries. The semaphore pattern is reusable for any parallel work with a concurrency limit.


TL;DR: Use tokio::sync::Semaphore with MAX_CONCURRENT = 6 to parallelize ADB file transfers without overwhelming the connection. Track progress with AtomicUsize, and use join_all with per-error collection so one failed transfer doesn't abort the rest.


If this was useful, a ❤️ helps more than you'd think — thanks!

HiyokoAutoSync | X → @hiyoyok