Breaking EvilQuest | Reversing A Custom macOS Ransomware
File Encryption Routine - SentinelLabs
By Jason Reaves
Published: 2020-07-07 · Archived: 2026-04-05 14:18:57 UTC
Executive Summary
A new macOS ransomware threat uses a custom file encryption routine
The routine appears to be partly based on RC2 rather than public key encryption
SentinelLabs has released a public decryptor for use with “EvilQuest” encrypted files
Background
Researchers recently uncovered a new macOS malware threat[1], initially dubbed ‘EvilQuest’ and later
‘ThiefQuest'[2]. The malware exhibits multiple behaviors, including file encryption, data exfiltration and
keylogging[3].
Of particular interest from a research perspective is the custom encryption routine. A cursory inspection of the
malware code suggests that it is not related to public key encryption. At least part of it uses a table normally
associated with RC2. The possible usage of RC2 and time-based seeds for file encryption led me to look deeper at
the code, which allowed me to understand how to break the malware’s encryption routine. As a result, our team
created a decryptor for public use.
Uncarving the Encryption Routine
As mentioned in other reports[4], the function responsible for file encryption is labelled internally as
carve_target.
Before encrypting the file, the function checks whether the file is already encrypted by comparing the last 4 bytes
of the file to a hardcoded DWORD value.
https://labs.sentinelone.com/breaking-evilquest-reversing-a-custom-macos-ransomware-file-encryption-routine/
Page 1 of 10

If the test fails, then file encryption begins by generating a 128 byte key and calling the tpcrypt function, which
basically ends up calling generate_xkey. This function is the key expansion portion followed by tp_encrypt,
which takes the expanded key and uses it to encrypt the data.
https://labs.sentinelone.com/breaking-evilquest-reversing-a-custom-macos-ransomware-file-encryption-routine/
Page 2 of 10

Following this, the key will then be encoded, using time as a seed. A DWORD value will be generated and
utilized.
The encoding routine is simply a ROL-based XOR loop:
At this point, we can see that something interesting happens, and I am unsure if it is intentional by the developer
or not. The key generated is 128 bytes, as we previously mentioned.
https://labs.sentinelone.com/breaking-evilquest-reversing-a-custom-macos-ransomware-file-encryption-routine/
Page 3 of 10

The calculations then used for encoding the key end up performing the loop 4 extra times, producing 132 bytes.
This means that the clear text key used for encoding the file encryption key ends up being appended to the
encoded file encryption key. Taking a look at a completely encrypted file shows that a block of data has been
appended to it.
Reversing the File Encryption
https://labs.sentinelone.com/breaking-evilquest-reversing-a-custom-macos-ransomware-file-encryption-routine/
Page 4 of 10

Fortunately, we don’t have to reverse that much as the actor has left the decryption function, uncarve_target, in
the code. This function takes two parameters: a file location and a seed value that will be used to decode the
onboard file key.
After checking if the file is an encrypted file by examining the last 4 bytes, the function begins reading a structure
of data from the end of the file.
https://labs.sentinelone.com/breaking-evilquest-reversing-a-custom-macos-ransomware-file-encryption-routine/
Page 5 of 10

Following the code execution, we can statically rebuild a version of what this structure might look like:
struct data
{
enc blob[size+12]
long long size
int marker
}
struct enc
{
long long val
int val2                   
long long val3               
char encoded_blob[4 - val % 4 + val]    
}
https://labs.sentinelone.com/breaking-evilquest-reversing-a-custom-macos-ransomware-file-encryption-routine/
Page 6 of 10

The encoded file key will then be decrypted and checked using the two values from the structure and the other
seed value passed to uncarve_target. The file key will be decrypted by eip_decrypt, which is the encrypt-in-place decrypt routine.
The function eip_key will take the two DWORD values and the seed argument to generate the XOR key to
decode the filekey.
Next, the file is set to the beginning and then a temporary file is opened for writing.
The file is then read into an allocated buffer and the key and encoded file data are passed to tpdcrypt.
https://labs.sentinelone.com/breaking-evilquest-reversing-a-custom-macos-ransomware-file-encryption-routine/
Page 7 of 10

As before, we have a key expansion followed this time by a call to tp_decrypt.
A glance inside the key expansion function shows a reference to a hardcoded table which matches RC2 code that
can be found online.
https://labs.sentinelone.com/breaking-evilquest-reversing-a-custom-macos-ransomware-file-encryption-routine/
Page 8 of 10

So now we have enough information to recover the file key:
import struct
import sys
rol = lambda val, r_bits, max_bits=32:
(val << r_bits%max_bits) & (2**max_bits-1) | ((val & (2**max_bits-1)) >> (max_bits-(r_bits%max_bits
data = open(sys.argv[1], 'rb').read()
test = data[-4:]
if test != 'xbexbaxbexdd':
 print("Unknown version")
 sys.exit(-1)
append_length = struct.unpack_from('<I', data[-12:])[0]
append_struct = data[-(append_length+12):]
keySize = struct.unpack_from('<I', append_struct)[0]
if keySize != 0x80:
 print("Weird key?")
 sys.exit(-1)
encoded_data = append_struct[20:20+132]
xorkey = struct.unpack_from('<I', encoded_data[-4:])[0]
https://labs.sentinelone.com/breaking-evilquest-reversing-a-custom-macos-ransomware-file-encryption-routine/
Page 9 of 10

def decode(blob, key):
 out = ""
 for i in range(len(blob)/4):
temp = struct.unpack_from('<I', blob[i*4:])[0]
temp ^= key
key = rol(key, 1)
out += struct.pack('<I', temp)
 return out[:0x80]
temp = decode(encoded_data, xorkey)
print(temp)
Attempting to RC2 decrypt the data, however, only seems to work partially at this time using RC2 routines in both
Python and Golang libraries. Further analysis will be needed to verify what is different.
However, for the purpose of decrypting victim files, we need only take the file key and call the tp_decrypt
function that is located inside the malware itself instead. Dumping the assembly for this function and building it
into a shared object to be executed using the recovered file key appears to work correctly.
Using this method, SentinelLabs created a public decryptor which is available here (this tool is released under the
MIT software license).
Sample
SHA-1: 178b29ba691eea7f366a40771635dd57d8e8f7e8
SHA-256: f409b059205d9a7700d45022dad179f889f18c58c7a284673975271f6af41794
References
1: https://twitter.com/dineshdina04/status/1277668001538433025
2: https://www.bleepingcomputer.com/news/security/thiefquest-ransomware-is-a-file-stealing-mac-wiper-in-disguise/
3: https://blog.malwarebytes.com/mac/2020/06/new-mac-ransomware-spreading-through-piracy/
4: https://objective-see.com/blog/blog_0x59.html
Source: https://labs.sentinelone.com/breaking-evilquest-reversing-a-custom-macos-ransomware-file-encryption-routine/
https://labs.sentinelone.com/breaking-evilquest-reversing-a-custom-macos-ransomware-file-encryption-routine/
Page 10 of 10