I have short films, projects that never got made, personal photos, ones of friends, clients, old family ones … i do some writing and store different documents. I want to have a Mari Kundo type hard drive.
Do you guys have any tips? Or some industry standard I’m not aware of?
You must log in or register to comment.
After a few years of trial and error, this is my setup / workflow:
On a higher level:
- One 12Tb HDD on local PC holds the photo’s folder (see below, with 3 subfolders). This HDD gets replaced every 4 yrs.
- This folder gets backupped (one way mirror via Pure Sync) to a Synology NAS where the backupped immediate gets a 2nd copy to a 2n HDD (this means I have 3 copies in my possession: one on the local machine, 2 on the NAS).
- This NAS gets periodically backupped (grandfather-father-son) via Synology Hyper Backup to cheap cold Google Cloud Storage “in case of disaster”. The NAS HDD’s are replaced every 5 yr.
- The albums folders (only 500Gb) gets mirrored periodically to a cloud service (in case One Drive) via Synology Cloud Sync (also a second “in case of really big disaster”)
On a lower level:
- The 12Tb HDD holds:
A. a decharging folder in which all pictures post 2020 reside on a year/month/day basis
(these pictures go into collections via Light room, so no subfolder stuff necessary here)
B. a folder with subfolders per holiday / event / shooting session for everything pre 2020
C. an albums folder with a few hundred subfolders with lighter JPEG exports in it (holds let’s say “the finished product”)
I work in Lightroom where all pictures are accompanied with ther XML file, so in case of disaster (Lightroom settings lost), at least the work I have done on one specific file can be recovered
Definitely like this and the hard drive replacement guidance. I’ve had bad luck when it comes to hard drives. I want to get a NAS setup and maintain my files better. Currently at a breaking point, my two laptops are full, I have two external hard rives that are unusable, one won’t read from a drop lost everything from 2007 to 2009 I keep it around hopping one day in afford to recover it. Then another hard drive that wiped and every time I format it it wipes and acts like a new hard drive. A third external that works but also full.
I have Gopro cloud and a full Google Photos. I just dont have the funds to upgrade, I was hoping Black Friday would be more forgiving but put the kids and wife first so my content creation remains on hold.
Well. In fact there are 2 paths. The first path is to go purely Cloud. It won’t cost that much (Google Drive 5Tb). Problem is working with Lightroom, and having grandfather-father-son type of backups. But this wil work for most people. I sure will.
Second is one system (like mine above, which is just my way of doing things), but be prepared for the cost (3 x HDD + some subscriptions + learning curve (Google Cloud Storage, Hyper Backup, etc…).
So all raw files head over to folder A, then sub folders for the selected photos into folder C? Or do you work out of folder A?. Do you also store the catalog in the subfolders in folder C? Ex. Start shoot raw goes to A, selects goto C into a sub folder called startshoot with the date, then inside that sub folder is all the images and some where a catalog? And exports there as well I assume in another sub folder labeled exports under startshoot?
Only purchase equipment for business needs. They come with extended warranty.
Our work flow depends on shooting on site or offsite. We have to 40TB redundant NAS devices in separate location. One is for prod one is for backup.
I also keep offline backups because NAS devices can get hit with crypto.
Make sure you have a firewall and close all ports no one needs.
Create a V-LAN separate from your home network unless you have a dedicated studio. In that case set up V-Lans and firewalls as well.
Try to not buy all the drives from the same vendor. If you get a bad batch they may all fail. They are covered by warranty but it takes time.
We shoot with A7IV and A7R5. They each have their own purpose and we are a Sony shop so we can swap lenses batteries etc.
For your home storage I would recommend putting a M.2 drive for caching purposes.
However I typically use Samsung T7 or other drives on the go or during post. But for export it goes to the NAS.
LPT if you have a second hand at a shoot just keep an AirPod in so you can talk to each other and make sure everything is in sync. With the new Sony SD cards they promise 900 Mbps write and read speeds.
Which is great because we empty out 2 2TB fast SSDs that support the latest usbc standard. And I can dump 24000 raw files in about 6 minutes.
Last tip, labels and colors. All my drives are labeled. With storage capacity date purchased etc. that way I don’t bring the wrong one 🤣.
Keep inventory of everything, get it insured and make sure you set up the warranty with your OEM for the drives.
Good luck :)
for personal… card dumps are dated. Ideally they are separated by events and the folder named appropriately. I remember places and events more than dates. I ALWAYS include the date in folder names as attributes can change. eg 'Photos - Tasmania Trip 2022.02 (year first for sort order)
for work… similar but Job number, customer name and date - this worked well for a lab
vid, pics in separate sub folders and ‘selected’ in another folder, usually RAW output only.
I recently digitized and archived my entire collection… and my parents. Duped and backed up. It feels so good!
The rule of thumb for backup is min. 3 copies and at least 1 of thoes that is off site.
And a raid does not count as a backup as others have stated, that’s just redundancy for 1 copy to keep you going really… good for a NAS (1 copy), then your working PC (2nd copy) and a off site copy (makes 3). Optimally one is a cold storage ie. not connected unless backing up to it, which can be the off site or 4th copy.
Then you should hopefully be safe!
jpeg folder and raw folder inside month folder inside year folder on two external hdd with CCC daily backup
I have a PC I built as a nas, recently upgraded it to 4x8TB drives in raid 5, this gives me 24tb of storage as you lose one HDD to “parity” data but means a HDD can fail without data being lost.
This “hot copies” to an external 22tb drive overnight and does a full sync end of the week, so I have a local recovery for any data I may accidentally delete within a week. The 2tb smaller is just they didn’t do a 24tb drive at a reasonable price, figure by the time I get to needing the capacity one will be available to swap out to.
Everything is then backed up to a cloud storage service (backblaze, which is ridiculously cheap for unlimited data backups) and can order a drive with all the data on them in absolute worse case I need to recover everything and quickly, rather than waiting for however long it takes to download the 12tb I have with them currently. The service also has a year of file versioning to help against bit rot, if a file is corrupted I have a year with which to notice and grab a replacement from the cloud. They have unlimited versioning but with the amount of data I’m storing it’s a bit cost prohibitive to me.
All my photography is also sent to Amazon Photos (unlimited storage tier on my UK account, limited on my Aus account so your location may vary) - which I view as a free extra precaution given I’m already paying for prime for other reasons.
I’d like to upgrade to a better file system, like btrfs or zfs which will help prevent/ detect bit rot at the system level and be a tad more robust than the ntfs I currently use, but the system is a windows box to make it easy for my partner to also use where needed and going Linux to get these benefits has other knock on complications, badgering them in to windows has an air of unreliability, not something I’m comfortable risking on a storage system. So currently that’s my only risk, which is a low enough probability to not be a massive concern.
The most simple solution is to create folders in hierarchical categories.
I separate my folders into current projects, pending projects, and an archive of old finished projects and multimedia in general.
YYMMDD - ProjectName
That alone should take you pretty far.
If you need to just catalogue and organize your regular photography create a primary folder of photography, and every shoot day is it’s own sub folder. If you own more than one photography device (camera + phone maybe) you can mark each folder by device like:
YYMMDD - DeviceName
You can get way more complex beyond this…but (ISO format) date + description is the fundamental building block.
I’m hoping AI will help me in the future. Between multiple HDs, Laptops, cloud accounts, thumb drives, and boxes of slides and prints dating back to the 70s, this task seems daunting to say the least. Way to many backup of backups. And don’t even get me started on proprietary databases like iPhoto and Lightroom which Need to index and rebuild every time you try to open one.
It doesn’t take long at all with the M series Mac’s. On windows I have no idea. Those are for playing video games and special programs that run on intel. Typically it takes 1200 photo Lightroom library about 3 seconds to load on my end. Maybe it’s not the drives. Maybe you need a M series mac. They are amazing for photography and video
In the market for a MacStudio. I think it’s M series. Will check into it thanks.
If you do absolutely get the higher GPU count. You will enjoy it if your shooting 5k-8k or have a sensor larger than 60 megapixels
6x2TB NVME TrueNas server at home, will upgrade when necessary, with a 4x 4TB HDD server off site as backup
Z:\year\month\date - client\
Super simple and easy to find old stuff, even with literally millions of files. Current year stored locally, past years on NAS, all backed up.
I use this method too and it all gets copied to cloud
I have one pool of storage I use for all sorts of things, that’s 180TB in a single zfs pool in 2 raidz1 vdevs.
For photography I store in the format Photography/YYYY/MM-DD-project-name
I backup my photos (and various other documents) using Restic to Backblaze B2 and Amazon S3 using the glacier deep archive tier.
In general you should follow the 3-2-1 rule. 3 copies of data in 2 storage mediums, at least 1 of which is off-site.
I use a NAS that is then backed up to another NAS in another location. Anything important is also on the line. For paid stuff I don’t format the memory cards used on that shoot until the final work has been delivered.
If you want a Mari Kundo type of fiddly and satisfying non-chronological folder structure let me introduce you to Johnny Decimal: https://johnnydecimal.com/