1.91
#...
·
 |
|---|
|
What kind of hard drives do you use to store all your data? Any special brand? Is it better to use 3.5" external drives or 2.5"? If you use a cloud based storage, what do you recommend? |
4.62
#...
·
·
3
likes
|
|---|
I just use an enterprise grade SSD for storing the files and my 7200rpm drive for backup. Ideally you'd have a backup on site and a backup off site and be using a RAID or similar to ensure you have what you need. For me, this is all for fun so I would be annoyed if I lost data but it's not that big of a deal. Not like anyone will be wanting the data for scientific purposes or after I'm gone some day.  |
4.57
#...
·
·
2
likes
|
|---|
|
I would recommend BackBlaze as an offsite backup for everything that needs backup on your PCs. The key advantage of BackBlaze is that, should the worst happen, you can get your multi-terabytes of data back quickly, delivered on hard drive. Most other systems require you to download it all over the web which could take weeks or months for large amounts of data. For local storage: fast SSDs on the PC and external USB drives. |
 #...
·
·
1
like
|
|---|
|
I arrived at my current configuration over about the span of a year. I store imaging data on a 4TB NAS HDD. Pixinsight resides on my 2TB NVME boot drive along with its swap files. I manually backup each image folder when processing is completed to another 4TB NAS HDD. Pixinsight projects are stored on a separate 500GB SSD. I also store images of the boot drive on the 2nd NAS HDD. All in all, a total of 10.5TB of internal storage. Maybe a bit on the high side, but it wasn’t expensive and works for me. Jim |
1.20
#...
·
·
1
like
|
|---|
|
I do the PI processing itself on a iMac Pro with internal SSD hard drive, though all raw data and all archived data are stored on daisy-chained RAID-10 arrays from OtherWord Computing and all are connected to a dedicated UPS with automatic voltage regulation. Final archived data manually backed up and stored off site to another computer (approximately quarterly) and in portable hard drive stored in a fireproof safe (annually) as well as to the cloud (DropBox and GoogleDrive, in an iterative fashion). Though I do not have automated cloud backup, agree that either Backblaze or Crashplan would be the way to go. The nice thing with the RAID arrays is up to 2 discs can fail (more specifically, 1 per array in a RAID 10 config) and not lose data, and you can hot swap drives. Ask me how I know! Though I've not had a drive failure with the OWC units, it will eventually happen. I have two old legacy Pegasus units and over the years, I have replaced virtually all of the drives at one point or another, but thankfully without any data loss. Totally agree with TurtleCat that this hobby is just for fun, so while data loss is not catastrophic, it can be a PITA. I well remember losing a nAperture library that was not well-backed up and the amount of time and effort to rebuilt that library. This probably doesn't help if you are on a Windows system as far as particular brands go, but I think the concept of RAID allows me a little more comfort, although RAID by itself is not an adequate backup strategy. Another dilemma is what to actually save - I think that subject has come up in the forum before. Would be curious to hear brand name recommendations for Windows based users. Rob |
1.51
#...
·
·
5
likes
|
|---|
|
Get a NAS. Synology makes nice ones with great software. Easy to configure. You can go with a 2 bay up to whatever size works for you. I recently purchased a 5 bay Synology Nas1522+. Now I have 24TB with 2 empty bays to increase storage. I should be good for a while. All final images I store in the cloud as well. |
3.34
#...
·
·
1
like
|
|---|
| I store my images on a Western Digital RAID system with spinning hard drives. I have 5 such systems ranging from 8TB to 12TB. My long term storage is on Western Digital USB drives that are offline. I have been using WD drives for decades and find them a very good and reliable company. I do some cloud storage from my MacOS system using Apple storage areas. |
3.01
#...
·
·
1
like
|
|---|
| I installed a 12TB (16 TB before RAID overhead) NAS, and I take a weekly backup of the NAS data on an external drive directly connected to the NAS. I tend to dedicate the SSD of my PC to working space, not for archiving astro images. It should be wise to backup the NAS on a cloud site, for theft and fire protection. |
0.00
#...
·
·
2
likes
|
|---|
| Dedicated Synology NAS Raid AND a portable hard drive backing up the NAS. 15 years of data I would hate to lose. |
0.90
#...
·
·
2
likes
|
|---|
|
If you go the Synology route, a very cool feature is that you can sync multiple NAS together. This is useful if you want to have an off-site copy of your data. You can place the 2nd NAS at a parent's or friend's home, then synchronizing both NAS is just a few clicks away. It's a flexible system, you can configure what kind of synchronization you want, when its running, etc. The backup NAS can be a lower end or older model, since it's only used as a backup. There are loads of them on the second hand market. Read/write speed isn't very fast on these NAS, especially compared to a modern SSD. But that's perfectly fine if you use it as an archive. Your computer's internal SSD is where your working file should be anyway. |
2.10
#...
·
|
|---|
| For processing I use a 1tb m2 drive. Other than that I use a multitude of 16TB drives for my A.P. imaging and backups. Currently using 15 external drives and 3 internal plus the M2. 6 of the externals are 16TB for backing up the other drives. I tend to buy my 16TB drives in pairs. Never really delete anything. |
8.83
#...
·
|
|---|
|
I use a 6 bay Synology NAS with 8TD drives so it gives me around 30TB of storage space with protection against a single drive failure. For me the NAS route is best because the storage space is easily accessible from all machines in my house. I have 2 imaging PC (for 2 mounts) and my office desktop PC where I do my editing. Since the storage is on the network its a trivial matter to have data sync from imaging PC to NAS to the secondary NVME SSD on my editing PC in real time. Once I'm done editing the project files go back to the NAS for storage. One very important thing that needs to be noted here is that RAID arrays ARE NOT backup solutions, they are storage solutions. While a RAID array provides more protection than having data stored on a single drive there are still many scenarios that will result in permanent data loss. So if going the NAS route it would be wise to also implement a backup solution. I personally use Dropbox Business as it provides unlimited storage at a flat rate and the data is easily accessible through the internet in a secure manner. My Dropbox account is connected to the NAS and basically mirrors everything thats on the NAS storage so if the NAS were fail in any way I have a complete backup of everything thats on it that can be downloaded with the click of the mouse. There are other backup solutions that will even mail you a copy of your data upon request (and often a small fee). |
22.75
#...
·
·
1
like
|
|---|
|
48 TB Synology NAS system with an identical off-site back up system. Images from my remote scopes are automatically uploaded as they are taken. In the long term, setting up your own cloud server with NAS is a lot more cost effective than storage using an online cloud service. John |
0.00
#...
·
|
|---|
|
I still have to decide when my big drive fills up, do I go to another internal, archive to an external but have about 6 months to figure it out . When I am done processing a target I put on Blue ray discs Black Blaze for Offsite I have been happy with 12 TB internal (7200) - backs up to Black Blaze. I thought I'd go through 1 drive a year but might be about 1 drive every 2 years https://www.amazon.com/gp/product/B075L6FJH8/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1 |
3.87
#...
·
|
|---|
|
2x1tb NVME working drives, 2x4tb WD blue drives, 1 for storage and the second is a mirror. I will have to add an 8tb soon and use it as the mirror and both 4tb for storage. Offsite external 3.5 inch backup manually updated every month or 2. Unprocessed stack and finished tiff backed up in the cloud. I have tried using software and bios raid in windows and it never has worked right or gracefully recovered from an error. Instead I use SyncBack to run scheduled mirror jobs. This gives me great control over the process and the backup is readable if the primary storage fails. Maybe things have improved but I’m not interested in trying again. If you want raid then a dedicated solution, either internal raid card or external box, is you safest bet. Software raid under Linux has worked flawlessly for more than 15 years though. |
1.51
#...
·
·
2
likes
|
|---|
|
To answer your specific question @Daniel Renner , the only real difference between 3.5" vs 2.5" drives is what sort of mounting options you have. Performance really depends on SSD versus HDD (i.e. spinning disk) as well as specific technology thereof; not size. HDDs give you much more capacity per $ than SSDs, but SSDs far out-perform HDDs in terms of speed, and to some extent, particularly for read-heavy applications, longevity. Industry default is the 3-2-1: 3 copies of your data, two independent on-site copies, and one off-site copy. The way I approach this is: a pair of 4Tb NVME drives in my workstation, which holds my current project files, as well as all stacked/calibrated/reduced images. I then have a big NAS (32 bay HDD, since I'm also a data hoarder) that holds the aforementioned plus all my subs ever. For cloud storage, I use Dropbox along with a local off-site resource I have access to. So for subs, it's more like 3-1-2 (3 copies, one local, two off-site). But for stacked images and project files, it's 4-2-2 (two local, two off-site). Everyone has their own tolerance to data loss though, so from my setup you can probably tell I had a catastrophic data loss experience back in grad school days that has set me on my current course. If you're going the external drives route, just be aware of data degredation: don't expect an external drive stored in a closet for 10 years to have every file intact. So if you have plans to come back to old projects, redundancy and multiple copies are your friend. The other risk with external drives and the like is everything is a manual process, which humans aren't always good at being consistent with. Everything I mentioned re: 3-2-1 above I have automated, so I don't even need to think about "wait, did I remember to move these files there?". |
|
1.91
#...
·
·
1
like
|
|---|
David Koslicki: *thanks for the detailed answer! |
0.00
#...
·
|
|---|
| SSDs are faster and more durable than traditional HDDs but are often more expensive for the same amount of storage. They are great for tasks that require high-speed data access, like running operating systems and frequently accessed applications. However, making an informed decision about the type of storage to use in your business involves more than just speed and durability. It also requires considering factors such as CRM enrichment.Popular cloud storage services include Dropbox, Google Drive, Microsoft OneDrive, and iCloud. The choice of cloud storage depends on your specific needs and preferences. When managing customer relationships, CRM enrichment becomes crucial. You may need a cloud storage provider that integrates seamlessly with CRM systems to enhance customer data and improve your business processes.The choice between 3.5" and 2.5" external drives depends on your usage scenario. If you need higher storage capacity and don't plan to move the drive frequently, a 3.5" external drive may be more suitable. If portability is a priority, a 2.5" drive is more convenient. In both cases, ensuring that your CRM data is backed up securely is essential to maintain efficient customer relationship management.When using cloud-based storage, it's essential to consider factors like data security, privacy, and internet connection speed. Evaluate the terms of service and pricing to determine which cloud storage provider aligns best with your needs. Additionally, check if the provider offers CRM integration or enrichment services to enhance the value of your customer data. |
|
2.10
#...
·
·
1
like
|
|---|
| The hobby (photography) certainly takes a lot of storage and all of that needs to be backed up as well. Losing photographs due to a hardware failure is preventable. I use Retrospect as my backup software. Currently I'm at 76TB of primary storage and 72TB of backup space. The processing drives are 2x2tb m2 and 1x1tb m2. The rest are 12-16TB spinning disks used for storage and backup. Fortunately moores law has continued to apply. Cheers! |
|
3.87
#...
·
|
|---|
frieda ritter: I’m pretty sure that my storage choices have nothing to do with undefined sales catchphrases like CRM integration and enhancement. A great an insightful first post to our forum. I Hope to see many more. |
|
1.51
#...
·
·
1
like
|
|---|
|
For processing I use a 4tb nvme drive. In the same computer is also a 10tb storage drive for stuff that I am currently working on. Backups go to a NAS that I built that is running unraid. It consists of 4 drives for 20tb of storage and also a 1tb ssd for a cache drive. A second ancient desktop that I have installed old hard drives in acts as a second backup for key files such as masters, completed images, and other personal documents. These files are also backed up to google drive and an external hard drive. I have tried to follow the rule of 3-2-1. Backing up in 3 places, 2 different types of media, and 1 off sight. However, it’s not a completely automated process. I still have to periodically copy the important files to the external drive; and of course, nothing syncs with anything until it is copied to the unraid Nas to start the whole process. |
3.81
#...
·
|
|---|
|
Unless you really enjoy managing the storage devices (totally guilty at times) I would just shift to cloud-based storage. I've been storing mine in amazon S3 for awhile now. I use it mainly as an archive, but in theory I could spin up a server and stack a new image directly within AWS. I wouldn't recommend going AWS or cloud service for everyone, but it does have some nice options if you get interested. |
