HN disclosure: I’m the author of Photos Backup Anywhere, but this thread mirrors the exact issues that pushed me to write it.
One thing that surprised me when digging into Apple Photos is how much state isn’t represented by just files-on-disk. Albums, Live Photos (paired assets), bursts, slo-mo, edits, and even “simple” things like adjusted capture dates are all tracked separately, and most export/backup tools end up flattening or partially reconstructing that on restore.
The approach I took was to treat Photos as the source of truth and verify restored items against it, rather than assuming filesystem metadata is enough. As far as I know, this is the only tool that restores albums and correctly round-trips all Photos item types while preserving location data, creation dates, and modification dates when restoring back into Photos.
My current process for offloading photos off the iPhone is to copy them in subsequent batches of '0-9999' from the 'Image Capture' app.
This is because I usually have far more than 10K photos and apple starts renaming the files after 9999 as 00001(1) for the rest. This is pretty undesirable.
Is there a way for me to export unmodified raw/jpeg/live/videos off the iphone to an external drive without a macbook with a large enough ssd, and wanting to use icloud as an intermediate bottleneck?
Do you take into account the iPhone not holding the original images of every photo? It will offload originals and just keep thumbnails if the library is too large.
Mine is approaching 1.5TB, I’ve got no hope of keeping that all on an iPhone, and also no guarantee that any given photo is fully available locally.
Aren't there hooks on the filesystem layer that downloads them when you access them? E.g I can browse via terminal to my iCloud Drive somehow and cat etc works on files which aren't local (after locking to download them first).
Does the PhotoSync app permit that? I use it to copy files to my NAS but it has some USB-related options I never explored. I used to use Image Capture but heard of PhotoSync and have never looked back.
I have taken time to slowly extract photos from old androids and its such a nightmare, and if you cant get a meaningful interface to load you have to resort to tooling that scrapes the whole drive and hope it grabs everything.
Backing up “mercurial” Photos data is only half the problem. The tricky part is restoring it in a way Photos actually recognizes as equivalent to the original library state. Photos Backup Anywhere restore works by re-importing items while explicitly reapplying Photos-level attributes: paired assets for Live Photos, burst membership and picks, slo-mo metadata, edits, locations, adjusted capture dates, and then reconstructing albums after the items exist again in the library.
In other words, the filesystem copy isn’t treated as the source of truth. The restore verifies items against what was backed up and only then rebuilds higher-level structure like albums. That’s the piece I didn’t see addressed elsewhere, and what originally motivated me to build it.
This is awesome! This might be a great replacement to attempting to get the Windows app to work. Has anyone had luck with the iCloud app on windows?
Similar to some other folks in this thread I have ~2TB of iCloud data, a Macbook with far less than 2TB of space, an external hard drive somewhere with the external Photo Library that I need to plug in if I want to look at photos on the Macbook, and a Windows desktop with 10TB+ of rusty disks.
I was excited when they added the iCloud app + iCloud photos to Windows, but it never seems to catch up or finish what it is doing. It appears to be almost constantly download at 50MB/s, stressing both disk & internet, and yet navigating to the folder reveals that they are all 'available when online'.
It seems like there is not an option in Windows to actually grab everything in full quality (actually now that I look at it - its gotten to 944GB on disk / 1.91TB total, so it is getting there.)
I guess a real question - with these photos finally on a Windows desktop - is there a better photo browser than Microsoft photos that can show the HEIC and the Live Photo?
Automatic de-duplication of photos with the same name
I recently went through a year’s worth of photos from my wife’s phone, and found three distinct “img_0001.jpg”’s just in that single year. Apple’s naming convention is so short sighted that I’d be terrified letting a piece of software try to dedupe it “by name “
Surprisingly, there is no official way to download all (400 Gb) photos from iCloud. Here is an open-source command-line tool to download all your iCloud photos.
That’s not true. On any Mac or iPhone you can choose the iCloud Photo Library storage option to download all instead of letting the system optimize the storage. And if you turn off iCloud Photo Library, it will also try to download it all. I know this because I stopped using iCloud Photo Library and that was how I got all my photos downloaded.
+1 to this method. After optimise storage is disabled on the Mac, wait for all photos to download. Then, open the photos library bundle and you'll see every photo there, full res. Copy them wherever you like.
Also, if you leave optimise storage disabled and continue to use Photos, every photo will be cloned in any local or cloud backups of your machine. This strategy creates additional photo redundancy separate from iCloud while still benefiting from library syncing.
This was my strategy too, but with a disgusting script which quit photos.app, rsync the photo library to a network share, then reopened photos.app so that it kept downloading from iCloud.
Not sure if the open/close is required, but I didn’t want to find out.
I don’t fully trust iCloud Drive / Photos therefore I use FSViewer to download all photos from my iOS device du jour (making sure to keep the HEIF formats), this way I get the Edited (slo-mo, live, portrait, usw) and pristine versions as Jobs intended. All kidding aside, after the gray area gate of 2017-2021 I had to find a more reliable backup workflow. As of today I only use iCloud Drive / Photos to extract some RAW photos that for some reason some picky apps don’t save to the photo album (looking at you ProCam 8.0). I made several tests including hash comparisons and imagemagick diffs and I am quite pleased.
Someone gave me a new iPhone (120GB) and a new MacBook Pro and asked me to download all their photos from iCloud. Long story short, after 120GB of photos were synchronised to the iPhone, the MacBook Pro refused to copy them, and now there's no storage left on the iPhone.
Also, Photos on Mac doesn't have an option to download photos directly, so the only valid option Apple offers is to download them through the web interface (max 1,000 at a time).
There is no official way to download iCloud library that is over phone capacity. Period.
You keep asserting to the contrary, but I've been syncing my entire photos library to my Mac for years, since it was iPhoto even.
Obviously if you have a larger photos library than storage space on a particular device, you cannot synchronize the entire library to that specific device. e.g. my photos library vastly exceeds my iPhone 13 mini storage, so on my iPhone, I don't sync everything. But my Mac has 2 TB of storage, and Photos is setup to sync all my photos, and does so, reliably, and has been, again, for years now.
Additionally, unlike with this open source tool, I can keep advanced data protection enabled.
I have tried 3 different Macs with different versions of macOS prior to looking for a workaround, and everywhere the result is the same: old photos are not downloaded automatically from iCloud, and there is no button to start this process - for this exact reason.
Want to prove me wrong? Create a new macOS user and open Photos with your iCloud. It will be empty until you start copying photos from your phone. It will take much less time than arguing here.
You're arguing with a lot of people who have personally seen this work. You can listen to other people. You can also go to an Apple Store and let them show you what's going wrong here.
Perhaps no one here has tried to download an entire iCloud library at once, or perhaps size is an issue, but that doesn't change the fact that there is no download button for iCloud Photos and iCloud Photos Downloader simply solves this. That's what this post is about.
I can personally confirm I've downloaded an entire iCloud library at once, to a brand new Mac, using the 'Download Originals to this Mac' option. As have many others here, I would think.
That's literally what that option is for.
If it's not working for you, you might be dealing with a bug, or perhaps you haven't given it enough time to sync. If you go to Photos > Library and scroll down, it should show you the sync status.
Thanks, that was a relief because I realised I didn't see the sync status at the bottom. It appears that Monterey hides the status message at the bottom by default, and I had to pull the page down twice to see it.
Long story short, iCloud wasn't syncing photos "due to performance" and this message was hidden.
iCloud Photos Downloader is an option, yes, but it is incorrect to say that Apple does not provide an official way to do this on Mac. Again, I direct you to the Apple Store so someone can show you in person, since you won't listen to anyone on here.
Photos on MacOS indeed synchronise photos with iCloud.
After our conversation I had tried to understand why I indeed don't see any status and I found out, that to get one in Monterey iOS I must need to scroll down of the collection and after, at the bottom pul whole page for the second time. Status message appears and it was saying that syncing was disabled due to Mac performance (I didn't asked for this).
Apologies, for misleading, code543 and thank you for consistence.
However, I must admit that I'm happy that found iCloud Photos Downloader as a result, also I liked that it's downloading all photos in date/folder structure.
Let me be one more voice telling you that you are wrong. I just did this morning.
In settings, "download originals to this mac", select all photos, file -> "export unmodified originals" will trigger the Photos app to download every file from iCloud into your local library (as well as exporting them to wherever you want)
I guess "there is no download button" but dude...I don't need iCloud Photos Downloader.
Thanks for letting me know. May I ask what macOS version you use?
Unfortunately, I'm unable to locate any button, status bar, or option to refresh or pull everything from iCloud in macOS Photos. There aren't even any details showing what percentage of iCloud is currently synchronized with macOS Photos. With nothing to debug, I can only conclude that for some reason the sync isn't working in my case.
It's great if this works for you and you don't need iCloud Photos Downloader, but for some reason I don't have that luxury.
That doesn’t sound right. My photo library is larger than my iPhone’s storage yet downloads fine on my Mac. Just need to make sure “optimise storage” is enabled on the iPhone and disabled on the Mac.
Once everything’s downloaded on the Mac, you can either export through the Apple Photos menu or just copy the “originals” directly from the Photos bundle.
This works because you had synchronised your iPhone with your Mac previously. If you start with an empty Photos library and phone, it is impossible to put all the photos on the phone and thus transfer them to your Mac.
Thanks to Apple's exceptional software quality the app has plenty of bugs and good luck exporting a lot of files out of said library - you're in for an endless game of spinners (it does some network IO on the main thread), "not responding" and memory leaks.
> Thanks to Apple's exceptional software quality the app has plenty of bugs…
I use Photos for macOS daily and I've never run into a bug with my 50K+ photos library. (To be fair, Photos doesn't do that much, and I use it more as a master catalog with Aperture's spiritual successor Nitro.)
> …and good luck exporting a lot of files out of said library…
Not sure why you would need luck to copy the "Originals" folder from the library package.
Technically, there is: users of the European Union can get a full export of all data that Apple has about them, including all the stored photos. It can be requested from here: https://privacy.apple.com/
How does the archive they provide look like? Many zip files?
I would like to retrieve them and offload to another storage service but I don’t have local storage enough to hold all of it at the same time, unpack and then reupload. I would need to do it in stages.
Yes, many ZIP files. You can select the ZIP file sizes, from 1 to 25 GB, iirc. Although a few end up larger than 25 for some reason. And took 1-2 days for Apple to "prepare".
You can request a chunk size and then it prepares them. I specified max chunk size and it took almost a week to give me a list of file downloads from 45-60GB each. 31 zip files to download.
It sounds really weird that instead of making a separate utility, or allowing you to download iCloud Photos in the native Photos application on Mac, Apple requires you to go through a legal procedure.
I'm OK with clicking a button to download all photos to Mac, but there is no such button. Or maybe there was one previously, but it has now disappeared.
If your Mac doesn’t have enough space, export them to a USB hard drive or if you’re using the download originals option, first move your library location to the USB drive as also described on the link above.
You were absolutely right. Someone in the thread mentioned a status indicator, and it triggered me that I hadn't seen it at all.
It appears that on Monterey I need to pull the page down twice to show the sync status, and when I found it, the message said that photos weren't syncing due to "Performance."
That's exactly what I expected to work, but for some reason this approach failed for me on a new Mac with an empty Photos library. I enabled "Download Originals," but 10+ years of iCloud photos never appeared. There's no manual "fetch all from iCloud" button, no progress indicator, no way to diagnose what's wrong - the sync just silently fails. Luckily, iCloud Photos Downloader bypasses Photos entirely and pulls directly from iCloud.
> Users of Google and Apple’s photo cloud services can now transfer images between them. It was already possible to export photos and videos from iCloud to Google Photos, but now it can also be done the other way around: from Google Photos to iCloud.
As long as you are signed into the Mac with the same iCloud account used on the iPhone, this will download them all. No, you do not need to get them all downloaded to the iPhone ever for any reason for this to work. Period. You need to stop repeating that, because it is wrong. How many people have to say the same thing?
Yes, you will have to go into a hidden folder to access the Originals once they're downloaded if you want to copy them somewhere else, but it's like two clicks.
I've been using Mac since Mac OS X 10.4 (~2005) and was under the same impression.
However, in reality, when you use the same Apple account on both devices with the Photos app on macOS (yes, with the 'Download Originals' checkbox enabled), it only downloads photos that you upload from your phone.
And if you look at the iCloud tab in the Photos app, it says 'Automatically _upload_ and store all your photos and videos in iCloud', so it works from Mac to iCloud, and doesn't help to download full iCloud library.
No, you are not correct. How many people have to tell you this?
It absolutely works the way I said it does, because I have seen it work that way. Just because you accidentally turned off iCloud Photos in your Apple Account settings on that Mac (or some other similar issue) does not mean it does not work this way when properly signed in.
If you want something to try, go to System Settings -> Apple Account -> Photos and see if "Sync this mac" is turned off. It needs to be on. There could be other ways that this feature is disabled, but that is one of them.
Not seeing something work is not evidence that it does not work. You have not seen it work, but that is not proof it does not work.
Seeing it work is evidence that it works. I have seen it work.
Other people have seen it work that way, and their replies are all over this thread. Apple documents that it works this way.
Yes, it will upload photos to iCloud if enabled, but it also downloads them.
When you take a new photo, it synchronises with all your devices, and therefore you see it on your Mac, iPhone, etc. However, if you get a new Mac (I got one because my library was under capacity), Photos will not start synchronising your 10-year-old photos until you process them through the phone.
When you first turn on iCloud Photos, it can take a while to upload your photos to iCloud. You can continue to use Photos while your photos are being uploaded.
To have iCloud Photos sync photos to all your devices, sign in to the same Apple Account to turn on iCloud Photos on all your devices.
One of us is missing something.
In Photos.app I clicked download originals. The photos are there on my Mac. It’s bit gross to get at them though - right click on app > show package contents.
Are you wanting a way that doesn’t involve the photos app?
If you open the Photos app (macOS) connected to iCloud with an empty library, there will be no photos until you import them from your phone. Hope this is clear now.
iCloud via browser has a limit of 1k photos per download.
My bad — it appears that Monterey wasn't syncing with iCloud due to "Performance." Photos on Monterey hides the status message at the bottom of the page, and I had to pull the page down twice to finally see it.
While not free, and not for any other platform than macOS. The program Parachute[1] in the App Store is very nice in downloading both photos from your library as well as files from the various locations.
Another option for iOS at least is PhotoSync. It’s nice, you can pull from photos and push to basically any remote service or local server. I have it backing up to both my nas and b2.
It works well enough, but it's not without flaws either.
The desktop version works reliably, if you can get macOS to keep shares mounted for long enough, and mount them on request. The scheduler is also kinda wonky.
The iOS version has so far never finished an incremental backup overnight of our ~1TB individual libraries. It handles resume/suspend well, but for some reason, while it exports unmodified originals, it doesn't include AAE files, which the desktop version does.
PhotoSync does everything right, with the exception of trying to keep state of what has been exported, which makes little sense as it doesn't support restoring photos.
Anyone know if it works with ADP? I emailed them months ago but no one ever replied.
On a related question, is there a download solution that does work with ADP? I’m looking to mitigate any potential account lockout issues for family members (and, no, they will not switch out of the ecosystem).
It does. It uses PhotoKit to access photos, so it basically uses your Apple Photos app (iOS or Mac) to download the photos.
The only scripted solution I can think of that works with ADP is osxphotos[^1], but that also uses PhotoKit, and requires the user to be signed in.
Personally I use PhotoSync [^2] to backup our photos from phones to a NAS. It works reliably, and supports exporting unmodified originals as well as edited versions, and XMP/AAE metadata alongside it.
I’ve been using usbmuxd+ifuse to copy the photo files straight from the phone. No need to wait for an upload/download to some remote server, just a direct cable from the phone to my computer. I get the original files, and can even move (instead of copy) to clear up the phone.
I was just thinking about this today. Apples lack of any 3rd party integration for things like this and iMessage is really annoying sometimes. In addition to a secondary backup, I’d love to automatically sync some photos from a certain album to my parents photo frame. Or if I take a nice nature shot have it sync to a Samsung frame tv. I get the benefits of the walled garden but esp w photos and messaging it seems like opening up a little would allow for some innovation
Thanks for this project. Our family generates about 2TB of media a year, and it’s been like that for a while, so we’re sitting at roughly 12TB total. That’s very much the long tail of personal media.
I’m not ready to pay $60/month, but I do like iCloud’s memories and other photo features. My compromise is simple:
- I use docker-icloudpd to download our iCloud Photos to local storage over time. It’s been the most practical way I’ve found to back up multiple accounts into one place, though it does require occasional re-auth every so often.
- I keep only the last ~2 years of media in iCloud and delete older ones after they’re archived locally.
- For browsing and searching the older archive, I use Immich, which has been a great self-hosted personal photo cloud experience with a modern app feel.
For storage, I’ve found fast local disk matters a lot once you’re digging up photos from 5+ years ago. Something like an OWC 4M2 with M.2 drives keeps the experience snappy; a typical HDD-based NAS can feel sluggish when you just want to quickly pull up an old memory.
Encryption at rest. If someone breaks into my house and steals my server I'd rather they not be able to get data. I can do LUKS but I really want the data to only ever be decrypted client side.
Music and AI features are still lagging in Immich, and I can understand why. Immich machine learning is not flushed out yet. If Immich has plans for creating marketplace for extensibility like plugins, in the current era of Claude code, I am sure we will end up with many options or features.
Wish I’d seen this 3 days ago. Needed to backup our Shared Library and did the following (about 10K photos/videos at 300gb, had enough space so full downloaded to MacBook, not optimize. 1. Repair iPhoto library, (Wait 24 hrs to re-sync to iCloud) - initially looks like it moved all photos to personal. 2. Select small chunks, by year worked well enough - selecting All gave me the spinning ball. Then export unmodified to external hdd into folders organized by year. 3. Moved entire photo library file to another external hdd. 4. Open iPhoto and select external hdd library as primary library, let it re-sync to iCloud (Wait 24 hrs). iPhoto now running off external HDD library and I’ll backup to separate external HDD monthly. Repair function and wired Ethernet connection were biggest game changers to previous attempts.
Not in my experience, previously ran it setup externally with a shared library for at least a year or longer, https://support.apple.com/en-ca/108345. Typically I backup to external drive 2 ways, copy of library itself and unmodified export, only had to go through above after I needed to rebuild the laptop. OP app and or https://photosbackup.app/ , seems like they might enhance my setup, will have to take a closer look.
My concern with backing up iCloud Photos with anything but Apple Photos is that there are some proprietary formats like Live Photos and slow mo video for which exports are lossy. Also, Apple Photos stores all edits non-destructively, so 'flattening' the edits into a single file for export is also a lossy operation.
It seems like an obvious improvement for Time Machine to support full backups while using optimized storage on the primary system.
My library is large too (roughly a third larger). After years of far more complicated storage/backup solutions, I settled w/ a second Photos library on an external hdd w/ optimize storage disabled. I plug the drive in and open this library every so often to update and then duplicate the drive for an off-site copy. Day to day, I use a Photos library on my primary drive with optimize storage enabled.
I’ve found unreasonable value in being able to search through hundreds of thousands of photos from my phone, so I went all-in on Photos.app. Though one enabling factor is that my photography workflow has drastic simplified in recent years to doing very little post (except for astrophotography, which I try and keep wip out of Photos.app anyway).
I had tried this but found it a little bit weird - switching back and forth on the same device between the 'hard drive w/ full files' library and the 'primary drive optimize storage' didn't really seem easy.
IIRC Photos.app will not even open if the default library you are pointing at is not there (i.e drive was unplugged). Are you able to just open up the library file directly and it will work as expected?
I also recall when changing Photos.app back to the HDD Library it did a ~2h 'rebuild' session before it even started downloading the new photos, but maybe thats acceptable with the 'every so often' approach.
I’ve used this tool for years and it’s great. But it really saves just the raw data. You’d never get it back in to Apple Photos as nice as when you pulled it out. Metadata is missing. Live Photos come out as an image and a similarly named video. But I treat it as the emergency backup. If some Apple DC burns down or they ban my Apple ID for some reason, at least the photos still exist.
Could the first obvious improvement please be its speed?
My god. The local Time Machine backup is slower on a 10gb network than Backblaze over the Internet. It isn’t even close.
I reinstalled my system and attempted for weeks to get Time Machine to complete a first backup. Every time I started it, the progress bar would fill up about 60% and then stall, and eventually kernel panic if the system was left idle for hours. Never happened before I reinstalled, though I have had it randomly decide the backup is corrupt and it has to start over. macOS deserves a better first-party backup feature.
I backup ~3-4GB a day with Time Machine to my local NAS and it takes less 10 minutes. Albeit it should take 30 seconds if it was maxing out the network speed.
Asking for anything out of Time Machine is a lost cause. It’s essentially a completed and legacy product.
I migrated to Linux + Pika Backup. For photos I use Ente Photos with their managed cloud storage plus a continuous export to my NAS.
Ente is surprisingly well integrated with iOS, you really don’t need to use Apple’s solution. It automatically backs up photos I take in the background.
The most annoying thing for me is if you set the date for a photo, it gets stored externally rather than modifying the photo metadata. So when you switch platform, every photo which didn't originally have a captured at date ends up reset to the current day every time you move.
For edits, I don't care too much about just baking them in since it's unlikely I'm going back to old photos and want to undo the crop.
In my experience migrating to another provider from iCloud, this hasn’t been a significant issue. Live Photos in particular are not really proprietary in the sense that they’re implemented in an extremely simple way that basically every photo tool understands. ~~Slow motion videos are also not proprietary, they’re just a plain video file.~~ <<< edit: I think I’m wrong about slow motion
> Slow motion videos are also not proprietary, they’re just a plain video file.
I haven't looked into the implementation details, but Photos lets you adjust the section of the video that is played back in slow motion. I thought if you share a slow-mo video, it gets re-encoded to bake this in (i.e., one second at 240fps gets exported as four seconds at 60fps).
You can go to https://privacy.apple.com, log in with your Apple ID, select "Request to transfer a copy of your data" and then select "iCloud photos and videos to Google Photos".
Does anyone have any idea for why Apple makes it so difficult to keep photos downloaded?
For context, try tapping 'optimize photos' in iPhone storage settings and then figure out how to turn off the feature without using Google. Not only is the toggle nearly impossible to find, but it's also hidden from being searchable
> For context, try tapping 'optimize photos' in iPhone storage settings
Same place it’s always been. In Settings -> App -> Photos, toggle Download and Keep Originals. Same place it is for macOS as well. It’s not that magical. Search for “photos icloud” and you’ll be led to the setting for it.
I used to use the Photos desktop app to move my photos (“select from the app and drop into a folder somewhere” worked best) to a separate folder a lot (and regularly) until I started using ente. Now ente not only backs up to their e2ee cloud but its desktop app keeps those media synced to the OS of my choice on my laptop.
But I can still not escape Apple’s gonorrhoeic naming and organisation.
Pro: FOSS of course; it works, with limitations (that’s mostly Apple) and glitches (that’s entirely ente)
Cons: really subpar non-native apps (desktop app is quite a dumb app as well) :( (and barely and useful additional features that lets a user do some batch/organisational changes or so)
All my vertical videos in iCloud show up cropped horizontal for some reason. If I go to edit I see the whole video. I really do not want to trust any cloud provider to maintain my years of archives of family photos and videos. Glad things like this exist. I just need properly date-foldered files, without no duplciates. Is that so hard?
Related: does anyone know of a way to delete the original videos on trimmed ones? Apparently all edits to videos are non-destructive, so the 10-minute video that I trimmed to 2 seconds still takes up 8gb.
I’ve been using this for several years now on a little unraid box to download new photos nightly. There’s a few docker containers that wrap in support for notifying when 2FA is required etc. Always makes me nervous, the access it has, but I’d rather have my photos backed up somewhere I own.
I'm sure grandparent meant to modify it so they'd just have to click "Backup to cloud" on their iPhone and instead of the iPhone sending their files to Apple's servers, it sends them to a local backup server...
Incredible. I have such a hacked-together system to get my iCloud Photos backed up to an external drive while not filling my main laptop drive. This would be even better.
I use this to sync my wife's photos to Immich and it works great, however the auth process is a bit of a pain (not the fault of icloudpd) and have to reauth every few months.
I’ve wrapped it in some short scripts which notifies on auth failure and it’s an easy process to run the auth script. But there’s no way to avoid the bi-monthly inconvenience I don’t think.
If I'm not mistaken, the Photos app stores originals on disk and then uses a sqlite database to track metadata (and maybe edits to photos as well, given that it has the originals). Seems reasonable to me.
I have a script to scan files from my camera and add a compressed copy to a folder. This folder was supposed to work with the iCloud for windows (10) program, but one day it just stopped working.
Wow, I will definitely give this a try. I have tens of thousands of photos in iCloud and I literally can’t export them all at once. Photos app chokes and crashes and manually babysitting smaller batches is a pain. It’s pretty clear they want to make it as hard as possible
Fuck, my wife got a notice that she would have to increase her iCloud storage so last week began the process of ordering a backup of all her pictures so I could get them off iCloud and organized on some drives at home. We got 12 zips of the pictures along with csv's and some metadata, and I literally just finished iterating on the script to sort them into year-based folders and convert all the HEIC shit into JPG. It's running literally right now.
I mean can't you go to privacy.apple.com, ask for an archive of your data, and then they'll email you the link to a zip file in a week or two? I'm pretty sure this is what my girlfriend did when she transitioned from an iPhone to a Pixel. I think there's even a specific checkbox for photos/videos
I pay around 10 euros a month to apple just so I can sync my photos from iphone to mac and ipad. That’s the only reason I need the 2 TB for icloud service. With an app like this I could download and keep copies and get rid of iCloud subscription?
This will let you download all of your photos that already exist on iCloud Photos.
Going forward, you’d want to set up some other way to sync photos you take from your phone to your other devices. I can personally recommend Synology Photos for simplicity[1], or Immich[2] for an open-source (and in my opinion, slightly better) alternative you can run on any hardware, if you’d like to set up an always-on NAS. These are “Apple Photos” or “Google Photos” equivalents that you host yourself.
Alternatively, something like Syncthing[3] is a dead-simple way to sync your photos to various other devices as and when they are online, if you’d prefer to manage your photos in an ordinary file manager.
I’d be remiss not to mention that, for any solution where you move off the cloud to a central storage location of your own, you really must make backups to keep your photos safe. The 3-2-1 rule is a standard recommendation.
Thanks. I cannot get iCloud sync to work at all. It consumes CPU, asks for logins repeatedly, etc but fails to actually do its job. When I think of its bugs and all the issues with the latest iOS (bugs and performance on recent hardware), I am thinking of exiting the Apple ecosystem entirely.
Became more fascinated with the history of my small hometown (Paris, Texas) TLDR: Much of it was wiped out by a 1916 fire. I spent some time recently vibe-coding this interactive map to provide some kind of historic visualization ( which enabled me to see the impact better )
https://gorch.com/parisfiremap/
HN disclosure: I’m the author of Photos Backup Anywhere, but this thread mirrors the exact issues that pushed me to write it.
One thing that surprised me when digging into Apple Photos is how much state isn’t represented by just files-on-disk. Albums, Live Photos (paired assets), bursts, slo-mo, edits, and even “simple” things like adjusted capture dates are all tracked separately, and most export/backup tools end up flattening or partially reconstructing that on restore.
The approach I took was to treat Photos as the source of truth and verify restored items against it, rather than assuming filesystem metadata is enough. As far as I know, this is the only tool that restores albums and correctly round-trips all Photos item types while preserving location data, creation dates, and modification dates when restoring back into Photos.
Project page is here if it’s useful: https://photosbackup.app/
Happy to explain details if anyone’s curious — there are a lot of sharp edges in Photos once you go beyond “export originals”.
My current process for offloading photos off the iPhone is to copy them in subsequent batches of '0-9999' from the 'Image Capture' app.
This is because I usually have far more than 10K photos and apple starts renaming the files after 9999 as 00001(1) for the rest. This is pretty undesirable.
Is there a way for me to export unmodified raw/jpeg/live/videos off the iphone to an external drive without a macbook with a large enough ssd, and wanting to use icloud as an intermediate bottleneck?
I use libimobiledevice on linux
plug iphone into usb. lsusb should show it.
I backup my photos with:
Actually, I backup all of /mnt not just DCIM, but that answer is for you. I also backup the entire phone with: but in this form it either does the photos as data files, or doesn't back them up. I think it is a complete backup.Do you take into account the iPhone not holding the original images of every photo? It will offload originals and just keep thumbnails if the library is too large.
Mine is approaching 1.5TB, I’ve got no hope of keeping that all on an iPhone, and also no guarantee that any given photo is fully available locally.
Aren't there hooks on the filesystem layer that downloads them when you access them? E.g I can browse via terminal to my iCloud Drive somehow and cat etc works on files which aren't local (after locking to download them first).
Does the PhotoSync app permit that? I use it to copy files to my NAS but it has some USB-related options I never explored. I used to use Image Capture but heard of PhotoSync and have never looked back.
That looks like it might do the trick. I feel like this should be something possible only using first party apps but I'll take it! Thanks.
I did it on Linux once I extracted them all as-is in the strange storage way that iOS stores them but I dont recall steps to make it mount the drive.
That would be perfect, I might chase down this path again. It's been a while since I've tried to directly mount the iphone as a drive on linux.
I have taken time to slowly extract photos from old androids and its such a nightmare, and if you cant get a meaningful interface to load you have to resort to tooling that scrapes the whole drive and hope it grabs everything.
My current workflow for this is to install the 'simpleSSHD' app on the android phone and rsync the files off at full network speed.
The sshd running on the phone also supports key based auth , so it's pretty simple to automate.
Thanks for commenting, do you support S3 compatible targets? Backblaze B2, for example.
Not directly, no. If you can mount the S3 target as a drive, it can be used as a backup destination.
Love this app! Kudos
have you looked at parachute backup? they also boast ability to backup the more mercurial types of iPhotos data.
Backing up “mercurial” Photos data is only half the problem. The tricky part is restoring it in a way Photos actually recognizes as equivalent to the original library state. Photos Backup Anywhere restore works by re-importing items while explicitly reapplying Photos-level attributes: paired assets for Live Photos, burst membership and picks, slo-mo metadata, edits, locations, adjusted capture dates, and then reconstructing albums after the items exist again in the library.
In other words, the filesystem copy isn’t treated as the source of truth. The restore verifies items against what was backed up and only then rebuilds higher-level structure like albums. That’s the piece I didn’t see addressed elsewhere, and what originally motivated me to build it.
This is awesome! This might be a great replacement to attempting to get the Windows app to work. Has anyone had luck with the iCloud app on windows?
Similar to some other folks in this thread I have ~2TB of iCloud data, a Macbook with far less than 2TB of space, an external hard drive somewhere with the external Photo Library that I need to plug in if I want to look at photos on the Macbook, and a Windows desktop with 10TB+ of rusty disks.
I was excited when they added the iCloud app + iCloud photos to Windows, but it never seems to catch up or finish what it is doing. It appears to be almost constantly download at 50MB/s, stressing both disk & internet, and yet navigating to the folder reveals that they are all 'available when online'.
It seems like there is not an option in Windows to actually grab everything in full quality (actually now that I look at it - its gotten to 944GB on disk / 1.91TB total, so it is getting there.)
I guess a real question - with these photos finally on a Windows desktop - is there a better photo browser than Microsoft photos that can show the HEIC and the Live Photo?
I was there until I saw this feature:
Automatic de-duplication of photos with the same name
I recently went through a year’s worth of photos from my wife’s phone, and found three distinct “img_0001.jpg”’s just in that single year. Apple’s naming convention is so short sighted that I’d be terrified letting a piece of software try to dedupe it “by name “
> ... Apple’s naming convention.
No, it is not. It came from DCF standard that predates smart phones.
https://en.wikipedia.org/wiki/Design_rule_for_Camera_File_sy...
The 4 digits numbers are, of cause, came from the 8.3 file name length constraint. Gonna love CP/M.
Surprisingly, there is no official way to download all (400 Gb) photos from iCloud. Here is an open-source command-line tool to download all your iCloud photos.
That’s not true. On any Mac or iPhone you can choose the iCloud Photo Library storage option to download all instead of letting the system optimize the storage. And if you turn off iCloud Photo Library, it will also try to download it all. I know this because I stopped using iCloud Photo Library and that was how I got all my photos downloaded.
+1 to this method. After optimise storage is disabled on the Mac, wait for all photos to download. Then, open the photos library bundle and you'll see every photo there, full res. Copy them wherever you like.
Also, if you leave optimise storage disabled and continue to use Photos, every photo will be cloned in any local or cloud backups of your machine. This strategy creates additional photo redundancy separate from iCloud while still benefiting from library syncing.
Or use the great osxphotos tool that works with Apple Photo’s SQLite database to let you manage all the photos in your library.
https://github.com/RhetTbull/osxphotos
Demo .gif sold me
(Been meaning to make a software demo gif gallery, best way to understand many categories of apps)
This was my strategy too, but with a disgusting script which quit photos.app, rsync the photo library to a network share, then reopened photos.app so that it kept downloading from iCloud.
Not sure if the open/close is required, but I didn’t want to find out.
I don’t fully trust iCloud Drive / Photos therefore I use FSViewer to download all photos from my iOS device du jour (making sure to keep the HEIF formats), this way I get the Edited (slo-mo, live, portrait, usw) and pristine versions as Jobs intended. All kidding aside, after the gray area gate of 2017-2021 I had to find a more reliable backup workflow. As of today I only use iCloud Drive / Photos to extract some RAW photos that for some reason some picky apps don’t save to the photo album (looking at you ProCam 8.0). I made several tests including hash comparisons and imagemagick diffs and I am quite pleased.
Someone gave me a new iPhone (120GB) and a new MacBook Pro and asked me to download all their photos from iCloud. Long story short, after 120GB of photos were synchronised to the iPhone, the MacBook Pro refused to copy them, and now there's no storage left on the iPhone.
Also, Photos on Mac doesn't have an option to download photos directly, so the only valid option Apple offers is to download them through the web interface (max 1,000 at a time).
There is no official way to download iCloud library that is over phone capacity. Period.
> Photos on Mac doesn't have an option to download photos directly
Yes it does. It's called Download Originals to this Mac.
https://support.apple.com/guide/photos/use-icloud-photos-pht...
You keep asserting to the contrary, but I've been syncing my entire photos library to my Mac for years, since it was iPhoto even.
Obviously if you have a larger photos library than storage space on a particular device, you cannot synchronize the entire library to that specific device. e.g. my photos library vastly exceeds my iPhone 13 mini storage, so on my iPhone, I don't sync everything. But my Mac has 2 TB of storage, and Photos is setup to sync all my photos, and does so, reliably, and has been, again, for years now.
Additionally, unlike with this open source tool, I can keep advanced data protection enabled.
This is from the iCloud manual:
> Any new photos and videos you add to Photos appear on all your devices that have iCloud Photos turned on.
You have your photos because they are new. If they had been taken before, they would not have synchronised automatically with Photos on MacOS.
Please stop repeating your incorrect points that are contradicted by everyone else’s real experiences.
Yes, new ones will be uploaded. That doesn’t mean old ones won’t also be downloaded.
I have tried 3 different Macs with different versions of macOS prior to looking for a workaround, and everywhere the result is the same: old photos are not downloaded automatically from iCloud, and there is no button to start this process - for this exact reason.
Want to prove me wrong? Create a new macOS user and open Photos with your iCloud. It will be empty until you start copying photos from your phone. It will take much less time than arguing here.
You're arguing with a lot of people who have personally seen this work. You can listen to other people. You can also go to an Apple Store and let them show you what's going wrong here.
Perhaps no one here has tried to download an entire iCloud library at once, or perhaps size is an issue, but that doesn't change the fact that there is no download button for iCloud Photos and iCloud Photos Downloader simply solves this. That's what this post is about.
I can personally confirm I've downloaded an entire iCloud library at once, to a brand new Mac, using the 'Download Originals to this Mac' option. As have many others here, I would think.
That's literally what that option is for.
If it's not working for you, you might be dealing with a bug, or perhaps you haven't given it enough time to sync. If you go to Photos > Library and scroll down, it should show you the sync status.
Thanks, that was a relief because I realised I didn't see the sync status at the bottom. It appears that Monterey hides the status message at the bottom by default, and I had to pull the page down twice to see it.
Long story short, iCloud wasn't syncing photos "due to performance" and this message was hidden.
Thanks once again!
iCloud Photos Downloader is an option, yes, but it is incorrect to say that Apple does not provide an official way to do this on Mac. Again, I direct you to the Apple Store so someone can show you in person, since you won't listen to anyone on here.
I confirm that you was absolutely right!
Photos on MacOS indeed synchronise photos with iCloud.
After our conversation I had tried to understand why I indeed don't see any status and I found out, that to get one in Monterey iOS I must need to scroll down of the collection and after, at the bottom pul whole page for the second time. Status message appears and it was saying that syncing was disabled due to Mac performance (I didn't asked for this).
Apologies, for misleading, code543 and thank you for consistence.
However, I must admit that I'm happy that found iCloud Photos Downloader as a result, also I liked that it's downloading all photos in date/folder structure.
Let me be one more voice telling you that you are wrong. I just did this morning.
In settings, "download originals to this mac", select all photos, file -> "export unmodified originals" will trigger the Photos app to download every file from iCloud into your local library (as well as exporting them to wherever you want)
I guess "there is no download button" but dude...I don't need iCloud Photos Downloader.
Thank you! The status message was indeed hidden on Monterey, and syncing was blocked due to "Performance."
Thanks for letting me know. May I ask what macOS version you use?
Unfortunately, I'm unable to locate any button, status bar, or option to refresh or pull everything from iCloud in macOS Photos. There aren't even any details showing what percentage of iCloud is currently synchronized with macOS Photos. With nothing to debug, I can only conclude that for some reason the sync isn't working in my case.
It's great if this works for you and you don't need iCloud Photos Downloader, but for some reason I don't have that luxury.
That doesn’t sound right. My photo library is larger than my iPhone’s storage yet downloads fine on my Mac. Just need to make sure “optimise storage” is enabled on the iPhone and disabled on the Mac.
Once everything’s downloaded on the Mac, you can either export through the Apple Photos menu or just copy the “originals” directly from the Photos bundle.
This works because you had synchronised your iPhone with your Mac previously. If you start with an empty Photos library and phone, it is impossible to put all the photos on the phone and thus transfer them to your Mac.
No, I’ve downloaded the entire library to a new Mac. It worked fine.
Thank you. Yes, indeed. I found out that Monterey is not syncing iCloud due to "Performance".
And people say Linux is hard to work with....
Okay, and if there are 300 or 500 GB of photos, how do you synchronise them with your iPhone?
that's good to know. can I then download the photos from iPhone to a backup hard-drive or transfer to a folder in my computer?
Yes over USB 2.0 until recently.
Thanks to Apple's exceptional software quality the app has plenty of bugs and good luck exporting a lot of files out of said library - you're in for an endless game of spinners (it does some network IO on the main thread), "not responding" and memory leaks.
But hey at least we've got Liquid (gl)ass now.
Worked well for me for 70k photos but took a long time
> Thanks to Apple's exceptional software quality the app has plenty of bugs…
I use Photos for macOS daily and I've never run into a bug with my 50K+ photos library. (To be fair, Photos doesn't do that much, and I use it more as a master catalog with Aperture's spiritual successor Nitro.)
> …and good luck exporting a lot of files out of said library…
Not sure why you would need luck to copy the "Originals" folder from the library package.
Technically, there is: users of the European Union can get a full export of all data that Apple has about them, including all the stored photos. It can be requested from here: https://privacy.apple.com/
I was able to request a photo dump as a non-EU customer using this link.
How does the archive they provide look like? Many zip files? I would like to retrieve them and offload to another storage service but I don’t have local storage enough to hold all of it at the same time, unpack and then reupload. I would need to do it in stages.
Yes, many ZIP files. You can select the ZIP file sizes, from 1 to 25 GB, iirc. Although a few end up larger than 25 for some reason. And took 1-2 days for Apple to "prepare".
You can request a chunk size and then it prepares them. I specified max chunk size and it took almost a week to give me a list of file downloads from 45-60GB each. 31 zip files to download.
While that’s a pain for you, it’s also a pain if they have multiple files for those that have enough storage.
Photo management is a bit of a nightmare as it’s an awful lot of small(ish) files.
Doesn't work if you have Advanced Data Protection (end to end encryption) enabled.
It sounds really weird that instead of making a separate utility, or allowing you to download iCloud Photos in the native Photos application on Mac, Apple requires you to go through a legal procedure.
I'm OK with clicking a button to download all photos to Mac, but there is no such button. Or maybe there was one previously, but it has now disappeared.
> or allowing you to download iCloud Photos in the native Photos application on Mac
Here’s the official documentation page for exporting directly using Photos for Mac without syncing everything locally: https://support.apple.com/guide/photos/download-photos-to-yo...
You can also choose to sync all photos locally with Photos for Mac by setting “Download Originals to this Mac” as described on this page which is what I do to keep a local copy: https://support.apple.com/guide/photos/photos-settings-pht51...
If your Mac doesn’t have enough space, export them to a USB hard drive or if you’re using the download originals option, first move your library location to the USB drive as also described on the link above.
You were absolutely right. Someone in the thread mentioned a status indicator, and it triggered me that I hadn't seen it at all.
It appears that on Monterey I need to pull the page down twice to show the sync status, and when I found it, the message said that photos weren't syncing due to "Performance."
Thank you.
That's exactly what I expected to work, but for some reason this approach failed for me on a new Mac with an empty Photos library. I enabled "Download Originals," but 10+ years of iCloud photos never appeared. There's no manual "fetch all from iCloud" button, no progress indicator, no way to diagnose what's wrong - the sync just silently fails. Luckily, iCloud Photos Downloader bypasses Photos entirely and pulls directly from iCloud.
I think if you are on PC. But on Mac with the Photos App
Cmd+A > File > Export Unmodified Originals
There is no straight way to download photos from iCloud to Mac.
The comment you're replying to just gave one, I did it recently for 70k photos. select all > export. That's it. It just took 30 hours.
I'm not sure it's surprising. Apple doesn't want you to leave and making something as important as your photos difficult to move helps with that.
From 2024:
> Users of Google and Apple’s photo cloud services can now transfer images between them. It was already possible to export photos and videos from iCloud to Google Photos, but now it can also be done the other way around: from Google Photos to iCloud.
https://www.techzine.eu/news/applications/122196/google-and-... (2023 Data Transfer Initiative (DTI))
What’s missing?
The files are there on the Mac, they are there to download on the cloud (various mentions of method mentioned here).
There is NO button on Mac to download photos from iCloud. You can only do this from your phone and then synchronise them to your Mac.
Yes, there is a button on Mac: https://support.apple.com/guide/photos/use-icloud-photos-pht...
As long as you are signed into the Mac with the same iCloud account used on the iPhone, this will download them all. No, you do not need to get them all downloaded to the iPhone ever for any reason for this to work. Period. You need to stop repeating that, because it is wrong. How many people have to say the same thing?
Yes, you will have to go into a hidden folder to access the Originals once they're downloaded if you want to copy them somewhere else, but it's like two clicks.
I've been using Mac since Mac OS X 10.4 (~2005) and was under the same impression.
However, in reality, when you use the same Apple account on both devices with the Photos app on macOS (yes, with the 'Download Originals' checkbox enabled), it only downloads photos that you upload from your phone.
And if you look at the iCloud tab in the Photos app, it says 'Automatically _upload_ and store all your photos and videos in iCloud', so it works from Mac to iCloud, and doesn't help to download full iCloud library.
No, you are not correct. How many people have to tell you this?
It absolutely works the way I said it does, because I have seen it work that way. Just because you accidentally turned off iCloud Photos in your Apple Account settings on that Mac (or some other similar issue) does not mean it does not work this way when properly signed in.
If you want something to try, go to System Settings -> Apple Account -> Photos and see if "Sync this mac" is turned off. It needs to be on. There could be other ways that this feature is disabled, but that is one of them.
Not seeing something work is not evidence that it does not work. You have not seen it work, but that is not proof it does not work.
Seeing it work is evidence that it works. I have seen it work.
Other people have seen it work that way, and their replies are all over this thread. Apple documents that it works this way.
Yes, it will upload photos to iCloud if enabled, but it also downloads them.
When you take a new photo, it synchronises with all your devices, and therefore you see it on your Mac, iPhone, etc. However, if you get a new Mac (I got one because my library was under capacity), Photos will not start synchronising your 10-year-old photos until you process them through the phone.
I hope I've made it clear now.
Your point has been clear the whole time. It is still not correct.
You can use the Photos app on your Mac and download originals
There are two options in the file menu - you can export the originals, and there is an option to export them as .jpg.
From iPhotos (Photos) app to hard drive.
Subject is to download photos from iCloud.
When you first turn on iCloud Photos, it can take a while to upload your photos to iCloud. You can continue to use Photos while your photos are being uploaded. To have iCloud Photos sync photos to all your devices, sign in to the same Apple Account to turn on iCloud Photos on all your devices.
https://support.apple.com/en-gb/guide/photos/phtf5e48489c/ma...
Select all, Files > Export. It's simple to download iCloud Photos.
One of us is missing something. In Photos.app I clicked download originals. The photos are there on my Mac. It’s bit gross to get at them though - right click on app > show package contents.
Are you wanting a way that doesn’t involve the photos app?
You can do that from iCloud over a browser.
If you open the Photos app (macOS) connected to iCloud with an empty library, there will be no photos until you import them from your phone. Hope this is clear now.
iCloud via browser has a limit of 1k photos per download.
This is not correct.
You don’t need an iPhone for Photos.app to work.
There are plenty of problems with Photos.app but the issue you describe isn’t by design.
My bad — it appears that Monterey wasn't syncing with iCloud due to "Performance." Photos on Monterey hides the status message at the bottom of the page, and I had to pull the page down twice to finally see it.
Thank you for proving me wrong.
While not free, and not for any other platform than macOS. The program Parachute[1] in the App Store is very nice in downloading both photos from your library as well as files from the various locations.
[1]: https://apps.apple.com/us/app/parachute-backup/id6748614170?...
Another option for iOS at least is PhotoSync. It’s nice, you can pull from photos and push to basically any remote service or local server. I have it backing up to both my nas and b2.
It works well enough, but it's not without flaws either.
The desktop version works reliably, if you can get macOS to keep shares mounted for long enough, and mount them on request. The scheduler is also kinda wonky.
The iOS version has so far never finished an incremental backup overnight of our ~1TB individual libraries. It handles resume/suspend well, but for some reason, while it exports unmodified originals, it doesn't include AAE files, which the desktop version does.
PhotoSync does everything right, with the exception of trying to keep state of what has been exported, which makes little sense as it doesn't support restoring photos.
Anyone know if it works with ADP? I emailed them months ago but no one ever replied.
On a related question, is there a download solution that does work with ADP? I’m looking to mitigate any potential account lockout issues for family members (and, no, they will not switch out of the ecosystem).
It does. It uses PhotoKit to access photos, so it basically uses your Apple Photos app (iOS or Mac) to download the photos.
The only scripted solution I can think of that works with ADP is osxphotos[^1], but that also uses PhotoKit, and requires the user to be signed in.
Personally I use PhotoSync [^2] to backup our photos from phones to a NAS. It works reliably, and supports exporting unmodified originals as well as edited versions, and XMP/AAE metadata alongside it.
^1: https://github.com/RhetTbull/osxphotos
^2: https://www.photosync-app.com/home
Advanced Data Protection
Is there a way to verify this all is safe to use? Like it won’t do something weird privacy wise? Any equivalent for windows?
I have the same paranoia, so I was happy to learn that someone made an open-source downloader for iCloud.
Thanks for the link
I’ve been using usbmuxd+ifuse to copy the photo files straight from the phone. No need to wait for an upload/download to some remote server, just a direct cable from the phone to my computer. I get the original files, and can even move (instead of copy) to clear up the phone.
This works on any iPhone? It mounts the non-privileged DCIM folder or whatever over USB to somewhere on your filesystem? With write access?
Yes, yes, no.
https://wiki.archlinux.org/title/IOS#Transferring_data
I run this periodically from a little shell script; I "should" automate it, but time is scarce.
> icloudpd/icloudpd:latest
Passing your raw iCloud creds into the unverified latest tag is fine until it’s not. Better to pin to a specific tag or hash.
You're not wrong. I know I need to put more work into it. Just haven't had time.
I'm "protected" by the fact Podman doesn't automatically update the latest image even when using the latest tag.
I was more showing how simple icloudpd is to use.
I was just thinking about this today. Apples lack of any 3rd party integration for things like this and iMessage is really annoying sometimes. In addition to a secondary backup, I’d love to automatically sync some photos from a certain album to my parents photo frame. Or if I take a nice nature shot have it sync to a Samsung frame tv. I get the benefits of the walled garden but esp w photos and messaging it seems like opening up a little would allow for some innovation
I use PhotoSync on my phone for this - select which albums to sync where with a rule and away you go.
Thanks for this project. Our family generates about 2TB of media a year, and it’s been like that for a while, so we’re sitting at roughly 12TB total. That’s very much the long tail of personal media.
I’m not ready to pay $60/month, but I do like iCloud’s memories and other photo features. My compromise is simple:
- I use docker-icloudpd to download our iCloud Photos to local storage over time. It’s been the most practical way I’ve found to back up multiple accounts into one place, though it does require occasional re-auth every so often. - I keep only the last ~2 years of media in iCloud and delete older ones after they’re archived locally. - For browsing and searching the older archive, I use Immich, which has been a great self-hosted personal photo cloud experience with a modern app feel.
For storage, I’ve found fast local disk matters a lot once you’re digging up photos from 5+ years ago. Something like an OWC 4M2 with M.2 drives keeps the experience snappy; a typical HDD-based NAS can feel sluggish when you just want to quickly pull up an old memory.
https://github.com/boredazfcuk/docker-icloudpd
Does Photos have features you use that Immich doesn't? I've switched to the latter fully and love it (though I have an Android).
Encryption at rest. If someone breaks into my house and steals my server I'd rather they not be able to get data. I can do LUKS but I really want the data to only ever be decrypted client side.
Music and AI features are still lagging in Immich, and I can understand why. Immich machine learning is not flushed out yet. If Immich has plans for creating marketplace for extensibility like plugins, in the current era of Claude code, I am sure we will end up with many options or features.
Wish I’d seen this 3 days ago. Needed to backup our Shared Library and did the following (about 10K photos/videos at 300gb, had enough space so full downloaded to MacBook, not optimize. 1. Repair iPhoto library, (Wait 24 hrs to re-sync to iCloud) - initially looks like it moved all photos to personal. 2. Select small chunks, by year worked well enough - selecting All gave me the spinning ball. Then export unmodified to external hdd into folders organized by year. 3. Moved entire photo library file to another external hdd. 4. Open iPhoto and select external hdd library as primary library, let it re-sync to iCloud (Wait 24 hrs). iPhoto now running off external HDD library and I’ll backup to separate external HDD monthly. Repair function and wired Ethernet connection were biggest game changers to previous attempts.
Is there any difference in moving Photos Library.photoslibrary to an external HDD and then pointing Photos.app at that?
Not in my experience, previously ran it setup externally with a shared library for at least a year or longer, https://support.apple.com/en-ca/108345. Typically I backup to external drive 2 ways, copy of library itself and unmodified export, only had to go through above after I needed to rebuild the laptop. OP app and or https://photosbackup.app/ , seems like they might enhance my setup, will have to take a closer look.
My concern with backing up iCloud Photos with anything but Apple Photos is that there are some proprietary formats like Live Photos and slow mo video for which exports are lossy. Also, Apple Photos stores all edits non-destructively, so 'flattening' the edits into a single file for export is also a lossy operation.
It seems like an obvious improvement for Time Machine to support full backups while using optimized storage on the primary system.
How about just turning off optimized storage and letting Time Machine do its job?
I have a 1.2T photo library. Carrying that around on my MacBook requires an $800 upgrade to the SSD.
Time Machine's job is to back up my data, it's not strictly to make a 1:1 copy of local storage. It should back up my cloud data too.
My library is large too (roughly a third larger). After years of far more complicated storage/backup solutions, I settled w/ a second Photos library on an external hdd w/ optimize storage disabled. I plug the drive in and open this library every so often to update and then duplicate the drive for an off-site copy. Day to day, I use a Photos library on my primary drive with optimize storage enabled.
I’ve found unreasonable value in being able to search through hundreds of thousands of photos from my phone, so I went all-in on Photos.app. Though one enabling factor is that my photography workflow has drastic simplified in recent years to doing very little post (except for astrophotography, which I try and keep wip out of Photos.app anyway).
I had tried this but found it a little bit weird - switching back and forth on the same device between the 'hard drive w/ full files' library and the 'primary drive optimize storage' didn't really seem easy.
IIRC Photos.app will not even open if the default library you are pointing at is not there (i.e drive was unplugged). Are you able to just open up the library file directly and it will work as expected?
I also recall when changing Photos.app back to the HDD Library it did a ~2h 'rebuild' session before it even started downloading the new photos, but maybe thats acceptable with the 'every so often' approach.
I’ve used this tool for years and it’s great. But it really saves just the raw data. You’d never get it back in to Apple Photos as nice as when you pulled it out. Metadata is missing. Live Photos come out as an image and a similarly named video. But I treat it as the emergency backup. If some Apple DC burns down or they ban my Apple ID for some reason, at least the photos still exist.
Could the first obvious improvement please be its speed? My god. The local Time Machine backup is slower on a 10gb network than Backblaze over the Internet. It isn’t even close.
I reinstalled my system and attempted for weeks to get Time Machine to complete a first backup. Every time I started it, the progress bar would fill up about 60% and then stall, and eventually kernel panic if the system was left idle for hours. Never happened before I reinstalled, though I have had it randomly decide the backup is corrupt and it has to start over. macOS deserves a better first-party backup feature.
> I have had it randomly decide the backup is corrupt and it has to start over.
I have hit this too many times.
I backup ~3-4GB a day with Time Machine to my local NAS and it takes less 10 minutes. Albeit it should take 30 seconds if it was maxing out the network speed.
I’ve done the sudo sysctl debug.lowpri_throttle_enabled=0. But it’s still slow.
10 minutes is great, and my changes wouldn’t seem as extensive as yours. I need to dig deeper.
Asking for anything out of Time Machine is a lost cause. It’s essentially a completed and legacy product.
I migrated to Linux + Pika Backup. For photos I use Ente Photos with their managed cloud storage plus a continuous export to my NAS.
Ente is surprisingly well integrated with iOS, you really don’t need to use Apple’s solution. It automatically backs up photos I take in the background.
The most annoying thing for me is if you set the date for a photo, it gets stored externally rather than modifying the photo metadata. So when you switch platform, every photo which didn't originally have a captured at date ends up reset to the current day every time you move.
For edits, I don't care too much about just baking them in since it's unlikely I'm going back to old photos and want to undo the crop.
In my experience migrating to another provider from iCloud, this hasn’t been a significant issue. Live Photos in particular are not really proprietary in the sense that they’re implemented in an extremely simple way that basically every photo tool understands. ~~Slow motion videos are also not proprietary, they’re just a plain video file.~~ <<< edit: I think I’m wrong about slow motion
> Slow motion videos are also not proprietary, they’re just a plain video file.
I haven't looked into the implementation details, but Photos lets you adjust the section of the video that is played back in slow motion. I thought if you share a slow-mo video, it gets re-encoded to bake this in (i.e., one second at 240fps gets exported as four seconds at 60fps).
Parachute Backup is a tool I use: https://parachuteapps.com/parachute
For those of you on Android / PC-Mac - Syncthing is an excellent tool for P2P syncing between multiple devices (it works over NAT and IPV6 too!).
I'm a big fan of Parachute Backup for this purpose. https://parachuteapps.com/parachute
Do you know if this supports multiple users? Can’t find mention of it in the site.
This great. I want to switch to android but the only thing stopping me is eight years of photos. No more.
You can go to https://privacy.apple.com, log in with your Apple ID, select "Request to transfer a copy of your data" and then select "iCloud photos and videos to Google Photos".
If your end goal is to move the photos to Google photos have a look at https://support.google.com/photos/answer/10502587?sjid=13067...
Does anyone have any idea for why Apple makes it so difficult to keep photos downloaded?
For context, try tapping 'optimize photos' in iPhone storage settings and then figure out how to turn off the feature without using Google. Not only is the toggle nearly impossible to find, but it's also hidden from being searchable
> For context, try tapping 'optimize photos' in iPhone storage settings
Same place it’s always been. In Settings -> App -> Photos, toggle Download and Keep Originals. Same place it is for macOS as well. It’s not that magical. Search for “photos icloud” and you’ll be led to the setting for it.
I used to use the Photos desktop app to move my photos (“select from the app and drop into a folder somewhere” worked best) to a separate folder a lot (and regularly) until I started using ente. Now ente not only backs up to their e2ee cloud but its desktop app keeps those media synced to the OS of my choice on my laptop.
But I can still not escape Apple’s gonorrhoeic naming and organisation.
Pro: FOSS of course; it works, with limitations (that’s mostly Apple) and glitches (that’s entirely ente)
Cons: really subpar non-native apps (desktop app is quite a dumb app as well) :( (and barely and useful additional features that lets a user do some batch/organisational changes or so)
All my vertical videos in iCloud show up cropped horizontal for some reason. If I go to edit I see the whole video. I really do not want to trust any cloud provider to maintain my years of archives of family photos and videos. Glad things like this exist. I just need properly date-foldered files, without no duplciates. Is that so hard?
Sounds like the contact sheet view is just using square preview thumbails?
All my vertical videos in iCloud show up cropped horizontal for some reason.
Turn your phone? /ducks
Related: does anyone know of a way to delete the original videos on trimmed ones? Apparently all edits to videos are non-destructive, so the 10-minute video that I trimmed to 2 seconds still takes up 8gb.
Yeah, choose "Save Video as New Clip" and then delete the original.
I’ve been using this for several years now on a little unraid box to download new photos nightly. There’s a few docker containers that wrap in support for notifying when 2FA is required etc. Always makes me nervous, the access it has, but I’d rather have my photos backed up somewhere I own.
Does anyone know if there’s a way to self host/impersonate iCloud? I’d like to back my iPhone up locally.
You can do manual backups in macOS using Finder, or iTunes on Windows.
There's also a Program called "Apple devices" I believe from Apple for Windows that will let you backup your iPhone/iPad.
If you configure a password for your backup it will backup more (confidential) data than if you don't encrypt your local backup.
https://github.com/libimobiledevice/ifuse might help.
For pictures you can use https://immich.app/
Immich is probably the best option
I'm sure grandparent meant to modify it so they'd just have to click "Backup to cloud" on their iPhone and instead of the iPhone sending their files to Apple's servers, it sends them to a local backup server...
A lot of iPhone apps can backup the iOS photo library these days, including the abovementioned Immich, Google Photos, and Dropbox.
Immich can get pretty close to that experience, with the iPhone app and a local immich instance.
Or ente photos: https://ente.io/
Incredible. I have such a hacked-together system to get my iCloud Photos backed up to an external drive while not filling my main laptop drive. This would be even better.
I use this to sync my wife's photos to Immich and it works great, however the auth process is a bit of a pain (not the fault of icloudpd) and have to reauth every few months.
It is a pain.
I’ve wrapped it in some short scripts which notifies on auth failure and it’s an easy process to run the auth script. But there’s no way to avoid the bi-monthly inconvenience I don’t think.
Why did Apple invent this abomination of a system for storing photos?
We have had file systems for decades. They work well. They sync trivially.
If I'm not mistaken, the Photos app stores originals on disk and then uses a sqlite database to track metadata (and maybe edits to photos as well, given that it has the originals). Seems reasonable to me.
If you are already in the progress of downloading all of your photos, this is a great time to consider completely switching over to Immich.
It's great!
Is there an iCloud Photos uploader?
I have a script to scan files from my camera and add a compressed copy to a folder. This folder was supposed to work with the iCloud for windows (10) program, but one day it just stopped working.
Seems to be looking for a new maintainer. If anyone has the bandwidth, it’s a great piece of software
Wow, I will definitely give this a try. I have tens of thousands of photos in iCloud and I literally can’t export them all at once. Photos app chokes and crashes and manually babysitting smaller batches is a pain. It’s pretty clear they want to make it as hard as possible
iCloud Photos.app seems to not be able to show some photos
But Photometor.app (owned by Apple) can...
So that's a little annoying... I wish I had more visibility on photos not showing up in Photos.app, and what it is that stops it showing them
Ok https://aionlywebsite.pythonanywhere.com/
You know you only need to do a GDPR request to Apple (dedicated page), select images and you get a download link after a few hours
For those in the EU only ?
You can't really file a GDPR request, but the (self-service, web based) download tools works for everybody.
Believe it work for everyone? I am located in Oslo/Norway, so can’t check if it does not apply everywhere.
Only need to go to this page to do the request https://privacy.apple.com/
Fuck, my wife got a notice that she would have to increase her iCloud storage so last week began the process of ordering a backup of all her pictures so I could get them off iCloud and organized on some drives at home. We got 12 zips of the pictures along with csv's and some metadata, and I literally just finished iterating on the script to sort them into year-based folders and convert all the HEIC shit into JPG. It's running literally right now.
Guess I should've searched harder!
I mean can't you go to privacy.apple.com, ask for an archive of your data, and then they'll email you the link to a zip file in a week or two? I'm pretty sure this is what my girlfriend did when she transitioned from an iPhone to a Pixel. I think there's even a specific checkbox for photos/videos
Thanks! I’ve been looking at options for this and didn’t realize Apple had an official method. Just submitted a request, we’ll see how it goes.
Yes... Let me ask my corporate overload and wait 2 weeks for permission to download my family photos.
There is something similar to Google Takeout for Apple Photos. Recently used it to download thousands of photos.
I pay around 10 euros a month to apple just so I can sync my photos from iphone to mac and ipad. That’s the only reason I need the 2 TB for icloud service. With an app like this I could download and keep copies and get rid of iCloud subscription?
This will let you download all of your photos that already exist on iCloud Photos.
Going forward, you’d want to set up some other way to sync photos you take from your phone to your other devices. I can personally recommend Synology Photos for simplicity[1], or Immich[2] for an open-source (and in my opinion, slightly better) alternative you can run on any hardware, if you’d like to set up an always-on NAS. These are “Apple Photos” or “Google Photos” equivalents that you host yourself.
Alternatively, something like Syncthing[3] is a dead-simple way to sync your photos to various other devices as and when they are online, if you’d prefer to manage your photos in an ordinary file manager.
I’d be remiss not to mention that, for any solution where you move off the cloud to a central storage location of your own, you really must make backups to keep your photos safe. The 3-2-1 rule is a standard recommendation.
[1] https://www.synology.com/en-global/dsm/feature/photos
[2] https://immich.app/
[3] https://syncthing.net/
why not use rclone
Because icloudpd supports downloading photos from icloud and rclone does not (yet): https://github.com/rclone/rclone/pull/8734
"why not use rclone (?)"
This is the correct - and obvious - response to something like this.
Unfortunately, I believe that rclone has no support for iCloud photos at this time.
I created this over the holiday break to do mostly the same:
https://github.com/rcarmo/PhotosExport
...when you try to export files using the (restricted) APIs we get, it automatically triggers a download.
Vulnerability hijack incoming with the headline calling for another maintainer
Thanks. I cannot get iCloud sync to work at all. It consumes CPU, asks for logins repeatedly, etc but fails to actually do its job. When I think of its bugs and all the issues with the latest iOS (bugs and performance on recent hardware), I am thinking of exiting the Apple ecosystem entirely.
Became more fascinated with the history of my small hometown (Paris, Texas) TLDR: Much of it was wiped out by a 1916 fire. I spent some time recently vibe-coding this interactive map to provide some kind of historic visualization ( which enabled me to see the impact better ) https://gorch.com/parisfiremap/
> Disable Advanced Data Protection
Nope, bzzzzt, wrong!
That's E2E encryption, right? Wouldn't you just end up with a lot of unreadable files if you didn't turn it off before running this downloader?
You need an extra tool to download your own photos? That's... not a basic feature?
I'm always surprised what kind of antifeatures people in Apple land are willing to accept and still use those things...
As is pointed out elsewhere in the thread, there are at three official ways to download your own photos. This complements those.
You absolutely can download your photos, just not via command line like this project enables.
You do not need an extra tool to download your photos. This one runs from the command line, though.