• 1 Post
  • 2.15K Comments
Joined 3 years ago
cake
Cake day: June 18th, 2023

help-circle

  • First and most important:

    In the context of long-term data storage
    ALL DRIVES ARE CONSUMABLES

    I can’t emphasize this enough. If you only skim the rest of my post, re-read the above line and accept it as fundamental truth. “Long-term” means 1+ years, by the way.

    It does not matter what type of drive you buy, how much you spend on it, who manufactured it, etc. The drive will fail at some point, probably when you’re least prepared for it. You need to plan around that. You need to plan for the drive being completely useless and the data on it unrecoverable post-failure. Wasting time and money to acquire the fanciest most bulletproof drives on the market is a pointless resource pit, and has more to do with dick-measuring contests between data-hoarders.

    Knife geeks buy $500+ patterned steel chef’s knives with ebony handles and finely ground edges and bla bla bla. Professional kitchens buy the basic Victorinox with the plastic handle. Why? Because they actually use it, not mount it on a wall to look pretty.

    The knife is a consumable, not an heirloom. So are your storage drives. We call them “spinning rust” for a reason.

    The solution to drive failure is redundancy. Period.

    Unfortunately, this reality runs counter to the desire to maximize available storage. Do not follow the path of desire, that way lies data loss and outer darkness. Fault-tolerant is your watchword. Component failure is unpredictable, no matter how much money you spend. A random manufacturing defect will ruin your day when you least expect it.

    A minimum safe layout is to have 2 live copies of data (one active, one mirror), hot standby for 1 copy (immediate swap-in when the active or mirror fails), and cold standby on the shelf to replace the hot standby when it enters service.

    Note that this does not describe a specific number of disks, but copies of data. The minimum to implement this is 4 disks of identical storage capacity (2 live, 1 hot standby, 1 on the shelf) and a server with slots for 3 disks. If your storage needs expand beyond the capacity of 1 disk, then you need to scale up by the same ratio. A disk is indivisible - having two copies of the same data on a disk does not give you any redundancy value. (I won’t get into striping and mucking about with weird RAID choices in this post because it’s too long already, but basically it’s not worth it - the KISS principle applies, especially in small configurations)

    This means you only get to use 25% of the storage capacity that you buy. Them’s the breaks. Anything less and you’re not taking your data longevity seriously, you might as well just get a consumer-grade external drive and call it a day.

    Buy 4 disks, it doesn’t matter what they are or how much they cost (though if you’re buying used make sure you get a SMART report from the seller and you understand what it means) but keep in mind that your storage capacity is just 1 of the disks. And buy a server that can keep 3 of them online and automatically swap in the standby when one of the disks fails. Spend more money on the server than the disks, it will last longer.

    Remember, long-term is a question of when, not if.



  • NaibofTabr@infosec.pubtoAutism@lemmy.worldAnyone else?
    link
    fedilink
    English
    arrow-up
    7
    ·
    16 hours ago

    The funny thing is, I used to be very risk-averse, just in general, both personally and professionally.

    But after working in risk management for a couple years, I experienced a real shift in my thinking and actions. Today I would describe myself as “risk-conscious”, but not risk -averse in the way I used to be.

    The (in hindsight) very basic concept of graphing out severity vs. probability had a big impact on me, not just in how I thought about risk but how I felt about it.

    It seems paradoxical, but the more I’ve learned about risk, the more willing I’ve become to take risks.

    On the other hand, I’ve become very much more aware of how much risk we all live inside of every day, how many safety nets are built into our society that most people just take for granted, and how suddenly and severely some vulnerabilities can turn into consequences. Sometimes I feel like everyone around me must be going through life with blinders on (although that is a mental trap of the “I am very smart” variety).

    It is a fucking wonder that you can go to a second-hand store, buy a 50-year-old lamp, take it home and plug it into a socket, and not have your house burst into flames or wake up in an emergency room after getting your heart stopped by an electric shock, or have the local transformer set your neighborhood on fire. The amount of work that is being done to manage risks around all of us, to the point that the relevant activities seem mundane, is just crazy.

    The people that spend their time thinking about and writing down and implementing safety standards (that is, managing risk) deserve a lot more appreciation from all of us.










  • Forty-Six

    When the Tao is present in the universe, The horses haul manure.
    When the Tao is absent from the universe,
    War horses are bred outside the city.

    There is no greater sin than desire,
    No greater curse than discontent,
    No greater misfortune than wanting something for oneself.
    Therefore he who knows that enough is enough will always have enough.

    Sixty-One
    […]
    Therefore if a great country gives way to a smaller country,
    It will conquer the smaller country.
    And if a small country submits to a great country,
    It can conquer the great country.
    Therefore those who would conquer must yield,
    And those who conquer do so because they yield.

    A great nation needs more people,
    A small country needs to serve.
    Each gets what it wants.
    It is fitting for a great nation to yield.

    Thirty
    […]
    Force is followed by loss of strength.
    This is not the way of Tao.
    That which goes against the Tao comes to an early end.

    from the Tao The Ching (by Lao Tsu), as translated by Gia-Fu Feng and Jane English

    Book 14 #3

    The Master said: ‘When the way prevails in the state, be enterprising in speech and enterprising in action; but when the Way does not prevail in the state, be enterprising in action but prudent in speech.’

    from The Analects (of Confucius), as translated by Raymond Dawson







  • NaibofTabr@infosec.pubtoLemmy Shitpost@lemmy.worldWhat's "email"?
    link
    fedilink
    English
    arrow-up
    71
    arrow-down
    2
    ·
    4 days ago

    Yes, instead we have gangrene, leptospirosis, trepanning, faith healing, no education unless you’re clergy, forced labor for 95% of the population, a life expectancy of 40 if you’re lucky…

    Oh, and taxes! still got those. Have to pay for Richard’s crusade, you know. Congratulations on being conscripted as a peasant spearman, now get back up and stand in line with the rest of the fodder.