Context
If a record is corrupted on disk we may see the following warning when an attempt is made to read the record:WARNING (migrate): (migrate.c:968) unreadable digest. XXXXXXXXXXXXXXXXX
The solution would be to delete these records by the printed digest.
Method
Prior to deleting these we should capture a debug-record info call output:asinfo -v "debug-record:namespace=<ns-name>;keyd=<hex-digest>[;mode=<mode>]" // where <mode> is one of: pickle or raw or raw-encrypted If above commands fails, a metadata only can be captured: asinfo -v "debug-record-meta:namespace=<ns-name>;keyd=<hex-digest>"We first suggest trying to read the record with standard mode and then followed up by debug-record in raw mode:
asinfo -v 'debug-record:namespace=<ns-name>;keyd=<hex-digest>'
and then with raw mode for us to verify raw-bytes padding:
asinfo -v 'debug-record:namespace=<ns-name>;keyd=<hex-digest>;mode=raw'asbackup tool can be used to send records to dev/null. This would force a full cluster read to check if any other records are imacted.
asbackup --namespace NAMESPACE --output - >/dev/null