What exactly does the "verify" action do?

#1

The label above the button read: “extract and verify”. How are verified the files ?

Thanks !

#2

We read the individual parts, calculate the checksum on it, compare it to what is stored in our db.
If there is an error in the part, we reconstruct it using the parity parts and write it back.

#3

Thank you for the answer.

Why someone would want to do manually check the files ? C14 isn’t supposed to ensure file integrity without any action from the customer ?

1 Like
#4

Has a customer-initiated verification ever failed or caused data to be recalculated from parity parts? I would hope you have enough monitoring and automatic checks to make sure that never happens. I guess I could imagine wanting it as an end-to-end check (to make sure the c14 data has the same checksum as the client) but in that case I’d want to say “please verify that my data checksum is XYZ”, rather than “please verify that the data in c14 still has the same checksum as just after it was uploaded”. If it’s wrong at all I’d expect it to be a transmission error of some sort.

#5

We run automatic scrubs regularly to ensure that the data does not get corrupt from bitrot or hardware failure, which are not preventable and are the reason why we have parity.
We’ve exposed the scrub action for customers who want to check it more often than we do it.

If a failure were occur, it’s recoverable, we fix it and log it in our system, the user doesn’t see that the data has been repaired. If it’s a non final non-recoverable error (which shouldn’t happen, but some issues like a fire, flood, etc, which are not in the scope of the software the way it is now) we will notify the client.

We use checksums to check data integrity from the temp storage to the final storage, we check part integrity, block integrity and are able to recover from a database meltdown.

The suggestion to provide a way for the client to checksum given hash methods the data he has uploaded on temp storage is something we are missing, I will try to push for it.

1 Like
#6

Nice, thanks :). Do you think you might have a version of C14 that’s automatically replicated in two DC’s someday? Right now the HA SAN product has that, so it may be safer than C14 in some ways.

#7

Having two locations for C14 is on the roadmap, it should be coming in a few months.
Once the hardware is online, the you should be able to save your archives in both location at the same time.

2 Likes
#8

I have around 113gb in C14 now (intensive option) and it’s nice. A few comments:

  • I tried a verify action and it was surprisingly slow, around 14 hours for that fairly small amount of data. That didn’t cause me any problems of course.

  • I chose the AES encryption option and the system emailed me the AES key, which doesn’t seem like a highly secure mode of transport. I’d prefer (at least optionally) to get an email notification saying how to download the key from the web site, but not containing the key in the email text.

  • As mentioned in other threads I wish there were more payment options, like being able to pay further in advance (e.g. 1 year).

  • Other than the above it is a good product and I like it.

#9

Hi @paulr

Thanks for the feedback.

The platform was a bit overloaded for the past few days and caused some delays during archive verification.

I asked our engineering team to enhance the workflow.

Unfortunately, it’s not planned for the moment.

Edouard

#10

So my understanding is, most users should never need to manually trigger a verify (since you guys do it automatically), and the stated 99.999999999% SLA is applicable when no manual verification operations are triggered - is this correct?

#11

Recently the automatic verification took very long time, some hours for little size, you have seen this. In this time the archive is not available.
When we want to make sort of incremental backup with automatic close/open archive it will throw random errors… The support can stop the queue for verification but we have to ask them manually.
It could be find to can stop the verification with the API, isn’t it ?

#12

Sorry for reactivating this thread, but I stumbled upon this:

I am just evaluatung the C14 service. I also just archived a test set.

Please check back with the developers, they are still sending the AES key out by email as of today.