How to stop ransomware thieves WORMing their way into your data

Sponsored Feature Most of us dislike cyber criminals, but not many of us dislike them quite as much as Anthony Cusimano.

The director of technical marketing at storage company Object First was on the sharp end of an identity theft attack after his details were leaked in the massive 2017 Equifax breach. Thieves armed with these details SIM-jacked his phone, used it to authenticate into his PayPal account, then stole money from Cusimano and his family.

"I became passionate about security for both individuals and businesses," he says.

The attack inspired Cusimano to join the battle against cyber crime and move increasingly into more cybersecurity-focused roles. Today, he spends his working day at Object First helping customers understand the importance of protecting their data from a range of attacks.

Object First specialises in protecting data from encryption by ransomware crooks. Its solution, Ootbi, is designed specifically to work with Veeam backup solutions, providing extra protection in the form of data immutability.

The company was founded by Ramie Timashev and Andrei Baronov, who started Veeam as a backup company for VMware virtual machines in 2006 and then expanded quickly, building it first into a multi-faced backup solution and then into a data management empire.

However, one thing that the two didn't have was a purpose-built object storage system for Veeam. They wanted a hardware appliance that would work seamlessly with their backup software, providing customers with a way to easily store backup data on their own premises, fed directly from Veeam's system. They had specific requirements in mind, the most important of which was to make that backup data tamper-proof.

Timashev and Baronov understood the security risks facing stored data and backups. They had made great progress getting companies to back up their data properly in the first place by creating automated solutions that made it more convenient.

Nice bit of backup data you have there

Then, along came the spectre of ransomware. Beginning as badly-coded malware released ad hoc by individuals or small groups, it exploded into a sophisticated business model with professionally written code.

As more victims hit the headlines, the spread of ransomware hammered home the need to back up your data.

Then, the crooks started coming for the backups.

Data backups were a form of business risk to these new, grown-up ransomware gangs. Like any business, they sought to eliminate the risk. They did it by seeking out backup servers and encrypting or deleting those, too, leaving victims more inclined to pay them.

One answer to this is write once read many (WORM disks, or storage taken offline. WORM disks can't be overwritten, but they are expensive and difficult to manage. Offline hard drives or tape must be connected to the system and then disconnected when the backups are complete, all in the hope that ransomware doesn't target them while they're online.

In search of indelible data

Instead, Object First wanted a system that combined the advantages of both; the immutability of a WORM disk with the convenience of online backup storage that could stay permanently connected to the network. And, naturally, they wanted a solution built specifically for Veeam.

This is what prompted them to begin creating Ootbi (it stands for 'out of the box immutability') three years ago, which eventually led to Object First.

"Ootbi is based on the idea of resiliency domains", explains Cusimano. "You treat every single software stack you have as an individual resiliency domain. If one gets compromised, you still have the others to lean on and recover from."

One component of this is the 3-2-1-1-0 rule: this means, storing three copies of your data, in addition to the original, across two media types, one of which must be off-site. Ootbi satisfies both of these by storing one in the cloud and the other on the customer's premises on its own appliance's NVME flash storage.

That leaves another one and a zero. The zero refers to zero errors, meaning that the storage solution must check that the data is clean going in so that you're not restoring garbage later. The one means that one of the copies must be kept offline, or air-gapped, so that no one can tamper with it.

Ootbi didn't air-gap this data by taking it physically offline. It wanted to handle the offline storage within its own network-connected appliance for maximum efficiency and user convenience.

"How do we make something where the backup lands on a box and there is no digital way that data can be removed from the box once it gets there?" says Cusimano. "That's what we built."

The inner workings of immutability

To build an immutable but connected backup appliance, Object First began by locking down the box as much as possible. Any attacker hoping for privilege escalation on the Linux-based product has a surprise in store: there's no basic or root account that is accessible to users on its hardened version of their customized Linux OS.

Unsurprisingly given its name, Object First also opted for native object storage out of the box with its appliance. Whereas file and block-based storage models tend to store data in hierarchical structures, object storage stores data as uniquely-identifiable units with their own metadata in a single bucket.

Object storage has its historical drawbacks, the main one being its slower speed relative to file and block approaches. However, this is a backup appliance rather than a transactional one, and in any case it uses extremely fast NVME flash for write caching.

Because it's built exclusively for Veeam, the technology also takes advantage of some proprietary work that Veeam did in building its data communications on the Amazon S3 API andVeeam's SOS (Smart Object Storage) API. That enables the backup appliance to eke more performance out of Amazon's cloud-hosted Simple Storage Service than other solutions can, Cusimano says. Ootbi also avoids any compression or de-duplication overhead because Veeam already takes care of those tasks.

Tight integration gives Ootbi support for all Veeam functionality, including simple backup, restore, disaster recovery, Instant Recovery, SureBackup, and hybrid scenarios. The appliance can run failed Instant Recovery workloads directly from backup within minutes, according to Object First.

Object storage also scales quickly and simply thanks to the GUID object labelling. This makes it good at scaling to handle large amounts of static, unstructured data.

"Because the concept was created in the last 20 years, it doesn't have the kind of baggage that that file or block carries," he adds.

The company not only configured its own hardened Linux distribution but also its own customized file system that communicates using the S3 API, which while developed by Amazon is now available as an open protocol.

"We've modified our own file system and we've created our own object storage code base," Cusimano says. "That's proprietary, so we're running our own special sauce on this very normal box."

The S3 API enabled Object First to take advantage of object lock. This introduces write-once-ready-many (WORM) immutability to stop an attacker doing anything even if they did somehow compromise the box. Explicitly built for object storage, it has two modes: governance, and compliance.

Governance mode prevents people overwriting, deleting, or altering the lock settings of a stored object unless they have special permissions. Compliance mode, which is the only mode used in Ootbi's immutable storage, prevents any protected object from being altered or deleted by anyone for the designated retention period (set by the user in Veeam Backup and Recovery).

Software is key

The hardware is effectively a JBOD appliance, with up to ten 16Tb hard drives, another hot spare drive, and a 1.6Tb NVME that acts as a data cache. The hard drives form a RAID 6 array, storing data parity information twice, so that data is recoverable even if two disks fail. This gives customers up to 128Tb of available backup capacity, along with fast data reading thanks to multi-disk striping.

Data arrives from Veeam through two 10Gbit/sec NICs and lands on the NVME cache, which provides a 1Gb per second write speed per node.

The system is designed with expandability in mind. Customers can build a cluster of up to four Ootbi appliances, adding nodes when necessary. This not only increases capacity, but also speed, as each appliance's built-in NIC provides another 1Gb/sec of write speed. It only supports a maximum four-node implementation today, but that's because the company is a small startup focusing on its first sales. The design of its software architecture will allow it to increase that threshold as demand comes in from customers, Cusimano says.

Object First also tailored the system for usability, with an interface that relatively non-technical people can use.

"There's no operating system updates. There's nothing they have to do to make this thing work. You plug it in, you rack and stack the box, you hook it up to your network. You go through two different NIC configurations inside of a text user interface, give it a username and password, and you're configured," Cusimano says. The system automatically optimises its storage, minimising the amount of on-site storage expertise that customers need.

Data backups alone aren't a gold-plated protection against more modern ransomware business models. Double-extortion ransomware gangs will steal your data even if they can't encrypt it, meaning that restoring scrambled files will only solve half of your problems.

With that said, backup protection forms a critical part of a multi-layered defence-in-depth solution that should include employee awareness, anti-phishing scans and malware protection. It will enable you to continue operating after a ransomware attack, making that data immutability worth every penny of your investment.

Sponsored by Object First.