Symptoms

https://saltexploit.com/

- very high CPU usage, sometimes only on 1 core, sometimes on all
- Fan spin! (Physical hardware only, of course.)
- Mysterious process named salt-minions is CPU intensive, stemming from /tmp/salt-minions or /usr/bin/
- Additional binary in /var/tmp/ or /usr/bin named salt-store or salt-storer
- Firewalls disabled
- Most applications crashing or taken offline

How to treat for the malware

There are some mutated versions, e.g. v1~5.

1. Stop Salt services. e.g. salt-master, salt-minion, salt-api, etc.

2. Here you are a shell script. (Original code: https://gist.github.com/itskenny0/df20bdb24a2f49b318a91195634ed3c6#file-cleanup-sh)
   Run to your machine.
   

   ## remove for root
   crontab -l | sed "/$i/d" | crontab -
   
   # check other users crontabs
   for f in $(ls /var/spool/cron/crontabs/*); do
     sed -i "/$i/d" $f
   done
   
   # remove ssh public key added by v5
   sed -i '/PbNwmJNcFwSLF12fFBoF\/$/d' /root/.ssh/authorized_keys
   
   for i in $(ls /home/*/.ssh/authorized_keys); do
     sed -i '/PbNwmJNcFwSLF12fFBoF\/$/d' $i
   done
   
   # newer versions seem to set themselves immutable
   chattr -i /tmp/salt-store
   chattr -i /var/tmp/salt-store
   chattr -i /tmp/salt-minions
   chattr -i /usr/bin/salt-store
   
   # remove all the trash
   rm /tmp/salt-minions
   rm /tmp/salt-store
   rm /var/tmp/salt-store
   rm /usr/bin/salt-store
   rm /etc/selinux/config
   rm -rf /tmp/.ICE*
   rm -rf /var/tmp/.ICE*
   rm /root/.wget-hsts
   
   # create apparmor profiles to prevent execution
   #echo 'profile salt-store /var/tmp/salt-store { }' | tee /etc/apparmor.d/salt-store
   #apparmor_parser -r -W /etc/apparmor.d/salt-store
   
   #echo 'profile salt-minions /tmp/salt-minions { }' | tee /etc/apparmor.d/salt-minions
   #apparmor_parser -r -W /etc/apparmor.d/salt-minions
   
   # reenable nmi watchdog
   sysctl kernel.nmi_watchdog=1
   echo '1' >/proc/sys/kernel/nmi_watchdog
   sed -i '/kernel.nmi_watchdog/d' /etc/sysctl.conf
   
   # disable hugepages
   sysctl -w vm.nr_hugepages=0
   
   # enable apparmor
   #systemctl enable apparmor
   #systemctl start apparmor
   
   # kill processes and reenabler
   kill -9 $(ps faux | grep /tmp/.ICE | grep -v grep | awk '{print $2}')
   ps aux | grep ICEd | grep -v grep | cut -c5-15 | xargs -n 1 kill -9
   kill -9 $(pgrep salt-minions)
   kill -9 $(pgrep salt-store)
   
   # fix syslog
   touch /var/log/syslog
   service rsyslog restart

3. Update SaltStack to patch version(2019.2.4)

   1. General case

      $ sudo yum check-update salt
      (omit)
      salt.noarch   2019.2.4-1.el7   salt-py3-2019.2
      
      $ sudo yum list salt
      (omit)
      Installed Packages
      salt.noarch     2019.2.2-1.el7     @salt-py3-2019.2
      Available Packages
      salt.noarch     2019.2.4-1.el7     salt-py3-2019.2
      
      $ sudo yum update salt
      (install all dependencies)
      ====================================================================================================================================================================================================
       Package                                       Arch                                     Version                                             Repository                                         Size
      ====================================================================================================================================================================================================
      Updating:
       salt                                          noarch                                   2019.2.4-1.el7                                      salt-py3-2019.2                                   9.4 M
      Updating for dependencies:
       salt-api                                      noarch                                   2019.2.4-1.el7                                      salt-py3-2019.2                                    19 k
       salt-cloud                                    noarch                                   2019.2.4-1.el7                                      salt-py3-2019.2                                    22 k
       salt-master                                   noarch                                   2019.2.4-1.el7                                      salt-py3-2019.2                                   2.9 M
       salt-minion                                   noarch                                   2019.2.4-1.el7                                      salt-py3-2019.2                                    38 k
       salt-ssh                                      noarch                                   2019.2.4-1.el7                                      salt-py3-2019.2                                    20 k
       salt-syndic                                   noarch                                   2019.2.4-1.el7                                      salt-py3-2019.2                                    19 k
      
      Transaction Summary
      ====================================================================================================================================================================================================
      
      $ sudo yum list salt
      (omit)
      Installed Packages
      salt.noarch   2019.2.4-1.el7     @salt-py3-2019.2

   2. In case of Conda env and pip install

      # Check out which conda env
      $ conda env list
      # conda environments:
      #
      base                  *  /opt/miniconda3
      saltenv                  /opt/miniconda3/envs/saltenv
      
      # Acitivate saltenv
      $ Conda activate saltenv then will prompt like the following
      (saltenv) [root@xxx ~]#
      
      $ pip install salt==2019.2.4
      ...
      (done)

4. [Salt-master server-side only] ACL for 4505-4506 ports by Firewall(Recommendation)
   <Sample>
   

   $ firewall-cmd --permanent --zone=public --add-rich-rule='rule family="ipv4" source address="xxx.xxx.xxx.0/24" port port="4505-4506" protocol="tcp" accept'
   $ firewall-cmd --reload

References

https://saltexploit.com/

https://github.com/saltstack/salt/issues/57057

https://docs.saltstack.com/en/latest/topics/releases/2019.2.4.html

https://gist.github.com/itskenny0/df20bdb24a2f49b318a91195634ed3c6#file-cleanup-sh