We've all heard the warnings: back up your data. It's a mantra repeated endlessly, from safeguarding family photos to protecting precious game assets. Yet, the reality is, we often procrastinate, lulled into a false sense of security. I, for one, was a prime example.
My data journey began in 2008, the dawn of the smartphone era. Back then, 250GB hard drives were the pinnacle of storage, and photos and videos were mere kilobytes. I blissfully stored everything on my phone and computer's hard drive, convinced that it was all safe. We all know the regret that follows: formatting an old hard drive, convinced it held nothing of value, only to discover old memories were lost in the process forever.
It wasn't until I hit the terabyte mark that I truly understood the importance of data segmentation. As my computer struggled to handle the weight of operating system files, massive machine learning libraries, and an ever-growing data collection, it became clear that a new approach was needed.
My laptop, limited to NVMe SSD storage, my wallet couldn't hold it all. I was forced to rely on less luxurious external hard drives, creating an unwieldy and unreliable system. The need for a unified, resilient data solution became paramount. The solution, as it turned out, was a multi-layered approach.
To manage my data across multiple devices, I deployed Syncthing, a powerful open-source file synchronization tool, on my computer, phone, and server.
Syncthing works by creating a peer-to-peer network between devices, allowing them to share folders and synchronize files. It's highly customizable, offering granular control over which folders are synced, what changes are tracked, and even a versioning system similar to Google Drive.
To optimize my workflow, I implemented a segmented synchronization strategy:
This approach ensured that I had access to all my data, regardless of which device I was using, while also preventing unintended file modifications.
The challenge was my constant movement between networks I didn't control. While Syncthing offered relay systems for data transfer, I needed a more robust solution. I required a network backbone capable of handling not only connectivity and node discovery but also secure access to internal web services, regardless of network boundaries without a complex hub and spoke infrastructure.
This is where Tailscale came in. A zero-trust networking service that creates a secure, private network over the internet based on Wireguard. It essentially extends my LAN (Local Area Network) with a WAN (Wide Area Network), allowing devices to connect securely, regardless of their physical location without punching hole on firewalls.
By leveraging Tailscale, I established a reliable network infrastructure that ensured consistent connectivity between all my devices, including my computer, phone, camera, and Raspberry Pi. This allowed me to keep all my data in sync using Syncthing, even when I was hopping between Wi-Fi networks or using mobile data.