9 Jul 2019 18:34 | apple | macOS
I recently had the logic board replaced in my 2017 Macbook Pro. I use the
awesome Carbon Copy Cloner to keep an image of my system as a bootable backup
which can then be easily restored when the machine is returned to me.
This time however I had some issues with iCloud Drive. After restoring the
backup I found it wasn't syncing. I tried the usual troubleshooting steps: turn
it off, reboot the machine, turn it back on again. Still wasn't syncing, just
sitting there with all my files and folders seemingly stuck in the state of
"waiting to download". I scoured the internet, lots of people with the same
issue and no obvious solution. Curiously though some of the threads had people
reporting that their stuff would just suddenly start syncing out of nowhere
without them having done anything to prompt it.
Well I have AppleCare so with a due sense of exhaustion and dread I gave them a
call. We started a screen share session so they could see the problem. Their
first suggestion was to try creating a new user account and seeing if that would
sync. I took exception to this - I don't need this to work on another account, I
need it to work on *this* account. I hung up the call and decided to investigate
myself.
I knew from previous messing around with macOS that the main process involved in
iCloud Drive syncing was bird. I decided to see if it was logging anything:
$ log stream |egrep -i bird
2019-07-09 18:11:09.546067+0100 0x5afcd Activity 0x6c4a4 1609 0 bird: (CloudDocsDaemon) server-zone/did-sync-down
2019-07-09 18:11:10.672324+0100 0x5afcd Activity 0x6c4a5 1609 0 bird: (CloudDocsDaemon) server-zone/did-sync-down
2019-07-09 18:11:11.626342+0100 0x5afcd Activity 0x6c4a6 1609 0 bird: (CloudDocsDaemon) server-zone/did-sync-down
2019-07-09 18:11:12.629151+0100 0x5afcd Activity 0x6c4a7 1609 0 bird: (CloudDocsDaemon) server-zone/did-sync-down
2019-07-09 18:11:13.496400+0100 0x64538 Activity 0x6c4a8 1609 0 bird: (CloudDocsDaemon) server-zone/did-sync-down
2019-07-09 18:11:14.429404+0100 0x5afcd Activity 0x6c4a9 1609 0 bird: (CloudDocsDaemon) server-zone/did-sync-down
Interesting. These log messages seem to indicate that it's syncing something
down, but what? My files don't appear to be syncing at all. There's no obvious
activity in the Finder. Curious I had a look at what files the bird process had
open:
# lsof -p 1609
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
bird 1609 admin cwd DIR 1,4 832 2 /
bird 1609 admin txt REG 1,4 26656 1400386 /System/Library/PrivateFrameworks/CloudDocsDaemon.framework/Versions/A/Support/bird
bird 1609 admin txt REG 1,4 21024 3013734 /Library/Preferences/Logging/.plist-cache.hjohswfh
bird 1609 admin txt REG 1,4 27154336 2775567 /usr/share/icu/icudt62l.dat
bird 1609 admin txt REG 1,4 32768 3015531 /Users/admin/Library/Caches/com.apple.bird/unlink/211B4D60-AB4C-416B-B0B2-0951D64A4CB7/db/server.db-shm
bird 1609 admin txt REG 1,4 32768 3082860 /Users/admin/Library/Application Support/CloudDocs/session/db/server.db-shm
bird 1609 admin txt REG 1,4 32768 3015536 /Users/admin/Library/Caches/com.apple.bird/unlink/211B4D60-AB4C-416B-B0B2-0951D64A4CB7/db/client.db-shm
bird 1609 admin txt REG 1,4 32768 3082865 /Users/admin/Library/Application Support/CloudDocs/session/db/client.db-shm
bird 1609 admin txt REG 1,4 65536 3083096 /Users/admin/Library/Application Support/CloudDocs/session/s/com~apple~CloudDocs:0/db.db-shm
bird 1609 admin txt REG 1,4 288952 897141 /Library/Application Support/CrashReporter/SubmitDiagInfo.domains
bird 1609 admin txt REG 1,4 238448 2720071 /private/var/db/timezone/tz/2019a.1.0/icutz/icutz44l.dat
bird 1609 admin txt REG 1,4 6270976 3079606 /private/var/folders/3b/xf71n9xx2jz2spvzcy9wc4fh0000gn/0/com.apple.LaunchServices-231-v2.csstore
bird 1609 admin txt REG 1,4 973824 2745534 /usr/lib/dyld
bird 1609 admin 0r CHR 3,2 0t0 311 /dev/null
bird 1609 admin 1u CHR 3,2 0t0 311 /dev/null
bird 1609 admin 2u CHR 3,2 0t0 311 /dev/null
bird 1609 admin 3r DIR 1,4 192 3002924 /Users/admin/Library/Application Support/CloudDocs
bird 1609 admin 4u REG 1,4 4096 3015527 /Users/admin/Library/Caches/com.apple.bird/unlink/211B4D60-AB4C-416B-B0B2-0951D64A4CB7/db/server.db
bird 1609 admin 5u REG 1,4 4096 3082856 /Users/admin/Library/Application Support/CloudDocs/session/db/server.db
bird 1609 admin 6u REG 1,4 32768 3015531 /Users/admin/Library/Caches/com.apple.bird/unlink/211B4D60-AB4C-416B-B0B2-0951D64A4CB7/db/server.db-shm
bird 1609 admin 7u REG 1,4 487424 3015532 /Users/admin/Library/Caches/com.apple.bird/unlink/211B4D60-AB4C-416B-B0B2-0951D64A4CB7/db/client.db
bird 1609 admin 8u REG 1,4 4095312 3082859 /Users/admin/Library/Application Support/CloudDocs/session/db/server.db-wal
bird 1609 admin 9u REG 1,4 32768 3015536 /Users/admin/Library/Caches/com.apple.bird/unlink/211B4D60-AB4C-416B-B0B2-0951D64A4CB7/db/client.db-shm
bird 1609 admin 10u REG 1,4 4096 3015527 /Users/admin/Library/Caches/com.apple.bird/unlink/211B4D60-AB4C-416B-B0B2-0951D64A4CB7/db/server.db
bird 1609 admin 11u REG 1,4 32768 3082860 /Users/admin/Library/Application Support/CloudDocs/session/db/server.db-shm
bird 1609 admin 12u REG 1,4 4096 3082861 /Users/admin/Library/Application Support/CloudDocs/session/db/client.db
bird 1609 admin 13u REG 1,4 4853392 3082864 /Users/admin/Library/Application Support/CloudDocs/session/db/client.db-wal
bird 1609 admin 14u REG 1,4 32768 3082865 /Users/admin/Library/Application Support/CloudDocs/session/db/client.db-shm
bird 1609 admin 15u REG 1,4 4096 3082856 /Users/admin/Library/Application Support/CloudDocs/session/db/server.db
bird 1609 admin 16 NPOLICY
bird 1609 admin 17r REG 1,4 487424 3015532 /Users/admin/Library/Caches/com.apple.bird/unlink/211B4D60-AB4C-416B-B0B2-0951D64A4CB7/db/client.db
bird 1609 admin 18r REG 1,4 4096 3015527 /Users/admin/Library/Caches/com.apple.bird/unlink/211B4D60-AB4C-416B-B0B2-0951D64A4CB7/db/server.db
bird 1609 admin 19u REG 1,4 4095312 3082859 /Users/admin/Library/Application Support/CloudDocs/session/db/server.db-wal
bird 1609 admin 20r REG 1,4 4096 3082861 /Users/admin/Library/Application Support/CloudDocs/session/db/client.db
bird 1609 admin 21r REG 1,4 4096 3082861 /Users/admin/Library/Application Support/CloudDocs/session/db/client.db
bird 1609 admin 22u systm 0x455f0d0e6f39ca23 0t0 [ctl com.apple.netsrc id 8 unit 8]
bird 1609 admin 23r REG 1,4 487424 3015532 /Users/admin/Library/Caches/com.apple.bird/unlink/211B4D60-AB4C-416B-B0B2-0951D64A4CB7/db/client.db
bird 1609 admin 24r REG 1,4 4096 3015527 /Users/admin/Library/Caches/com.apple.bird/unlink/211B4D60-AB4C-416B-B0B2-0951D64A4CB7/db/server.db
bird 1609 admin 25r REG 1,4 535632 3015535 /Users/admin/Library/Caches/com.apple.bird/unlink/211B4D60-AB4C-416B-B0B2-0951D64A4CB7/db/client.db-wal
bird 1609 admin 26r REG 1,4 2488512 3015530 /Users/admin/Library/Caches/com.apple.bird/unlink/211B4D60-AB4C-416B-B0B2-0951D64A4CB7/db/server.db-wal
bird 1609 admin 27r REG 1,4 4096 3082856 /Users/admin/Library/Application Support/CloudDocs/session/db/server.db
bird 1609 admin 28r REG 1,4 4853392 3082864 /Users/admin/Library/Application Support/CloudDocs/session/db/client.db-wal
bird 1609 admin 29r DIR 1,4 832 2 /
bird 1609 admin 30r REG 1,4 4095312 3082859 /Users/admin/Library/Application Support/CloudDocs/session/db/server.db-wal
bird 1609 admin 31r REG 1,4 4096 3082861 /Users/admin/Library/Application Support/CloudDocs/session/db/client.db
bird 1609 admin 32r REG 1,4 4096 3082856 /Users/admin/Library/Application Support/CloudDocs/session/db/server.db
bird 1609 admin 33r REG 1,4 4853392 3082864 /Users/admin/Library/Application Support/CloudDocs/session/db/client.db-wal
bird 1609 admin 34r REG 1,4 4095312 3082859 /Users/admin/Library/Application Support/CloudDocs/session/db/server.db-wal
bird 1609 admin 35u REG 1,4 1658200064 3083092 /Users/admin/Library/Application Support/CloudDocs/session/s/com~apple~CloudDocs:0/db.db
bird 1609 admin 36u REG 1,4 4321912 3083095 /Users/admin/Library/Application Support/CloudDocs/session/s/com~apple~CloudDocs:0/db.db-wal
bird 1609 admin 37u REG 1,4 65536 3083096 /Users/admin/Library/Application Support/CloudDocs/session/s/com~apple~CloudDocs:0/db.db-shm
bird 1609 admin 38r REG 1,4 288952 897141 /Library/Application Support/CrashReporter/SubmitDiagInfo.domains
Interesting. Lots of files that end in .db - databases perhaps? The one most
obviously associated with cloud docs seemed to be:
/Users/admin/Library/Application Support/CloudDocs/session/s/com~apple~CloudDocs:0/db.db
So I went to that directory and had a look. Sure enough this file is a SQLite
database. What's more it was slowing growing in size. So the obvious inference
is that this is some kind of metadata database that is being actively synced by
the system. Presumably when this is done the files themselves could be synced.
And that turned out to be the case. Eventually it finished syncing its metadata
and the files themselves started appearing. So there wasn't actually any
problem, but within the first minute of my call with Apple Support they agreed
with me that there *was* a problem and would no doubt have wasted hours of my
time trying to fix something that wasn't actually broken.
I haven't dug too deeply into how the syncing process works but I have a theory
about why it stalled for so long in my case. I had some git repos in my iCloud
Drive that had a *lot* of files in them. One of them had over 77k files. The way
the SQLite database was growing and the corresponding log messages being
reported by the bird process lead me to think that it was syncing records in the
database one by one or in batches, rather than simply downloading a snapshot of
it. This is presumably slow because the number of API requests is a factor of
the number of files for which it has to hold metadata. Somebody with a huge
number of files in their iCloud Drive would presumably experience this taking
even longer than I did, especially if they didn't have a particularly fast
internet connection.
Why Apple chose to make this process invisible in the UI is a mystery. Their
discussion forums are full of people baffled by this. They must have had endless
support calls and genius bar visits in relation to it, and in many of those
cases there may have been nothing actually wrong. The only issue was that the UI
gives no indication of what it's doing so the user thinks it isn't doing
anything
This is bad design.
