somewhat OT: External HD

Del Vento

Folks,

I think I've lost some data (have to check if I can recover anything) and that is very sad and annoying. It makes me think life was better when I was kid, and I still can look at the pictures my dad shoot me, and the cassettes he recorded my voice on. Faded and noisy, but still enjoyable. Videos and pictures of my kids? Probably gone, at least the largest one which I could not backup in the cloud or on a second device.

Anyway, I need to purchase an external HD. My only concern is long term reliability, I can be ok with slow(ish) speed and small(ish) size. Ideally, a not-super-expensive device will be best. I simply cannot deal with continuously buying a new device and copying the old data over "just in case". That will kill me.

I can't find any trustworthy information in this regard, every review that I've found deals with super-fast transfer and/or super-large size with no concern whatsoever for long term reliability.

Does any of you have any information in this regard that you are willing to share?

Thanks

HZPiano

Hello,

Del Vento I'm very sorry to hear that.

If you still have some hope of recovering the data, DON'T touch that drive and have someone super knowledgeable see what can be recovered. If it's built into your computer, get it out NOW. Every single simple write action to the drive will greatly diminish any hopes of any file recoveries.

I don't think I can give you much technical advice on a new drive purchase. What I do is, apart from daily (or even more frequent) backups of anything that I worked on to separate media, is to yearly calculate the total storage space needed for all that I have at that time, buy a harddisk of at least that size, and copy everything to that disk. I use Robocopy under Windows (command line tool) which not only copies the files but also verifies (if so switched) each written file against the original. The disk then gets stored as an "archive" never to be touched again for anything else.

Cheers and a lot of good luck with this!

Del Vento

HZPiano If you still have some hope of recovering the data, DON'T touch that drive and have someone super knowledgeable see what can be recovered. If it's built into your computer, get it out NOW. Every single simple write action to the drive will greatly diminish any hopes of any file recoveries.

Thanks for the suggestion. Indeed the drive has been removed a few weeks ago, and it's sitting on the floor in a padded static-proof bag for this purpose.

HZPiano The disk then gets stored as an "archive" never to be touched again for anything else.

I was doing that (with great pain because the "workflow" of taking pictures with different cameras and different computers with different hard drives of different capacity and had to manage what-goes-where and what-is-too-large-to-have-too-many-copies). I somewhat stopped doing that both because of the time consumption, but also because of the inadequacy of the workflow in these conditions (for example if a file is deleted from location 1, should it be deleted from all other locations too? no because it might have been deleted because there was no available space at location 1. Yes because it might have been deleted because it was an out-of-focus picture in a series of 5 and the others were basically identical and in focus).

I will take a look at Robocopy, but (besides that I don't use Windows at the moment, even though that will change soon -- but I will still using Linux on several machine) I doubt it will solve the problem above, which requires tedious and time consuming manual intervention.

HZPiano Cheers and a lot of good luck with this!

Thanks for your suggestions and words of comfort.

HZPiano

Hello,

Del Vento , I am such a(n) (over) thinker too (deep sigh) and use multiple computers as well.

But when it comes to that yearly archiving, I stopped thinking. I just copy everything, never minding doubles, deletions, whatever. The main aim is to have the files there should it ever become necessary to retrieve them. Also, not having to think saves a lot of time which encourages doing it rather than procrastinating (which I do, I admit! because it isn't the most interesting or pleasurable pastime…)

Cheers and again good luck,

MHirsch

I'd rather be using a RAID1 system (mirror). In case one drive fails, you still have the second (mirror).

Something like this:

https://graugear.de/en/portfolio-item/g-25rd2-ac-10g/

And then you can feed it with any 2.5 HDDs/SSDs.

Del Vento

HZPiano But when it comes to that yearly archiving, I stopped thinking. I just copy everything, never minding doubles, deletions, whatever.

Problem with that is twofold:

the data volume quickly became unmanageable, since I was storing pictures and videos in raw format (but I have since stopped doing that)
reorganizing data became another nightmare, because if my first dump was unorganized, then sorting and organizing cause massive duplication (problem 1.) but also difficulty understand what-is-what-and-where-it-is which reduces the usefulness of the backup

MHirsch I'd rather be using a RAID1 system (mirror). In case one drive fails, you still have the second (mirror).

I thought about that in the past, but I never have success with external drive enclosures. Do you have experience and success with the specific one you suggested, or is it just a random one you've found?

Thanks!

DPPianoPhil

Del Vento reorganizing data became another nightmare, because if my first dump was unorganized, then sorting and organizing cause massive duplication (problem 1.) but also difficulty understand what-is-what-and-where-it-is which reduces the usefulness of the backup

That sounds like you need a database management approach to your pictures, where the organization happens without physically moving things around into folders. Have you looked at Lightroom? It's perfect for that. The only overhead is a few to a few hundreds MBs for the database itself. The RAW files don't have to move.

MHirsch

Del Vento Do you have experience and success with the specific one you suggested

I do have a western digital raid1 system, 10 years old. Still works. But it’s a network attached storage, not sure if you need that.

MacMacMac

I use a WD "Elements" USB drive. 8 TB for somewhere around $150 several years ago. Today you can get more even storage for the money.

I use rsync to clone from C: to the external drive. It copies files from C: that are either new or modified, and deletes from the backup any files that are no longer kept on C: (but that latter part can be omitted if you wish).

rsync is a Linux utility, but I run Linux on Windows so I have everything that's Linux … all within Windows. Including rsync and all the other Linux goodies.

Del Vento

MacMacMac I use a WD "Elements" USB drive.

Thanks for actually giving a brand/model recommendation!

MacMacMac I use rsync to clone

Yes, that's what I use(d) too, and it worked well, but it required care for the reason stated before, because it does not distinguish between deletions because the source ran out of space vs deletion because the data was unnecessary (e.g. out of focus picture), vs non-deletion but move to a different location. I've built a system on top of it which (with manual intervention) distinguishes between these cases, but it's tedious and time consuming. I've yet to find a backup system which support this use case, which in my opinion is something that any backup system should have had in place decades ago! But I digress….

n-player As a consumer, what you can do is find reliable, middle-of-the-road HDDs that have been around for a while and are not excoriated in forums (not too fast and not too big).

That was exactly my reason for posting here. In forums and online reviews people rave for huge sizes and fast speeds and the "middle of the road" gets no love and often hate. So I could not find any trustworthy recommendation besides the one @MacMacMac posted above.

n-player That requires enclosure investment.

Which as you say in the previous paragraph are mostly rubbish, so how can a random guy like myself find a right one?

n-player Disclosure - me and family worked for enterprise storage firms.

Thanks, so your opinion is even more valuable to me. Would you suggest going to HDD? As I said, I care only about reliability and not speed/capacity.

Thanks!

n-player

HDDs have high failure rates; all the brands release defective units, batches, and models. Corporations use equipment that mitigate failures through intelligent software and by allowing techs to physically hot-swap failing HDDs, etc.

As a consumer, what you can do is find reliable, middle-of-the-road HDDs that have been around for a while and are not excoriated in forums (not too fast and not too big).

Also, many (most) consumer grade external cases are rubbish. Cheap power supplies and heat are rapid failure points. But there are plenty of others. I had several consumer cases fail in a few days of limited use.

As @MHirsch noted, some RAID system is a good bet for local storage as you can mitigate HDD failure. That requires enclosure investment.

But, whilst considering backup solutions you might also consider the location domain (e.g. periodic off-site backup via the cloud or just physically moving a HDD to a different location for theft, fire, electrical spike, etc. protection).

Disclosure - me and family worked for enterprise storage firms.

MHirsch

n-player me and family worked for enterprise storage firms.

😁

I’m glad you’re not suggesting an EMC or a NetApp 😂

n-player

Del Vento Would you suggest going to HDD? As I said, I care only about reliability and not speed/capacity.

We've been out of the industry too long to make contemporary recommendations in the consumer space other than the very basics you already know. HDDs still are big and cheap. Sorry we can't offer more concrete solutions for you.

In the past couple of years, you will see plenty of news and blogs on HDDs and SSDs downgrading internal hardware-firmware after the reviewers release benchmarks (downgrades materially reducing real-world performance). I can't say if this has been going on long-term, but naturally the drive makers are changing internal components constantly based on pricing and availability so it is difficult for consumers to track.

Cheaper SSDs might be QLC which has lower "endurance" than say SLC, but I don't know how that translates to years of backup use in a consumer environment so may (or may not) be relevant.

Speaking of the longer time horizon, media type would make a difference. I don't know if a SSD or a spinning HDD would be more durable after 10-years in your home use environment, on average. For all the hype around CD & DVD rot, I have never seen or experienced that, not assuming that modern materials are as durable as those used in the 1980s and 1990s.

Following are decent places to start consumer research on storage. They are under the same ownership and rely on advertising so are as "independent" as a typical magazine is:

https://www.anandtech.com/
https://www.tomshardware.com

Frédéric L

For a system backup, Macrium Reflect can be handy : you create an archive of your system disk, and can create a USB flash drive from where you boot and have a recovery instance of Macrium reflect which can create partitions like those of your lost drive.

There is a free version. A paid version adds other functions like incremental backups.