Problem Description
Multiple cluster nodes crashed with SIGSEGV on server version 6.2.0.11 when using GeoJson queries and Points-within-Region.Explanation
Crash happened with following stacktrace:Feb 20 2024 07:36:09 GMT: WARNING (as): (signal.c:191) SIGSEGV received, aborting Aerospike Enterprise Edition build 6.2.0.11 os el8 Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:621) stacktrace: registers: rax 0000000000000000 rbx 00007f56f9217000 rcx 000000000000c6ad rdx 0000000000000000 rsi fffffffffffffbff rdi 0000000000000000 rbp 00007f56f9217000 rsp 00007f56fc1f6940 r8 0000000000000063 r9 00000000ffebffff r10 000000007ffef3ff r11 00000000f3bfb7ef r12 00007f56f920ec00 r13 00007f56fc1f6a9c r14 00007f56fc1f6a58 r15 0000000000000010 rip 00007f69c50eca30 Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:633) stacktrace: found 15 frames: 0x6f5909 0x4ec25d 0x7f69c4d7fcf0 0x7f69c50eca30 0x7f69c510e0d5 0x7f69c510e3b3 0x4775d5 0x465947 0x470b38 0x472e15 0x4e94cf 0x6e71be 0x6e6f3a 0x7f69c4d751ca 0x7f69c37ffe73 offset 0x0 Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 0: /usr/bin/asd(cf_log_stack_trace+0xd4) [0x6f5909] Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 1: /usr/bin/asd(as_sig_handle_segv+0x21) [0x4ec25d] Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 2: /lib64/libpthread.so.0(+0x12cf0) [0x7f69c4d7fcf0] Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 3: /lib64/libcrypto.so.1.1(EVP_MD_CTX_copy_ex+0xe0) [0x7f69c50eca30] Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 4: /lib64/libcrypto.so.1.1(HMAC_Final+0x75) [0x7f69c510e0d5] Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 5: /lib64/libcrypto.so.1.1(HMAC+0x93) [0x7f69c510e3b3] Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 6: /usr/bin/asd(as_session_token_is_valid+0x126) [0x4775d5] Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 7: /usr/bin/asd(authenticate+0x4d) [0x465947] Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 8: /usr/bin/asd(cmd_authenticate+0x155) [0x470b38] Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 9: /usr/bin/asd(as_security_transact+0x2dd) [0x472e15] Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 10: /usr/bin/asd() [0x4e94cf] Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 11: /usr/bin/asd() [0x6e71be] Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 12: /usr/bin/asd() [0x6e6f3a] Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 13: /lib64/libpthread.so.0(+0x81ca) [0x7f69c4d751ca] Feb 20 2024 07:36:09 GMT: WARNING (as): (log.c:644) stacktrace: frame 14: /lib64/libc.so.6(clone+0x43) [0x7f69c37ffe73]
Solution
There is a known bug where it can cause double-free when using GeoJson data type and running Points-within-Region query.This fix is in master for version 7.1 and ported to hotfix branches 6.4.0.9, 6.3.0.15, 6.2.0.22, 6.1.0.27 and 7.0.0.2
Notes
There are few other known issues which can cause double-free and lead to server crash.
- Use of both "var" and "call" in an expression may result in a double-free: - >Fix is in master for 7.0 and hotfix branches 6.1.0.22, 6.2.0.17 , 6.3.0.10 and 6.4.0.4
- Deleting a bin via a write operation expression will cause a crash: - > Fix in master for 6.3 and hotfix branches 5.7.0.30, 6.0.0.14, 6.1.0.12, 6.2.0.7