Required Hardware Spec.

OS

• CentOS 7 x86_64 기준

CloudHub에서 사용되는 Port

개념도

참고 아래에서 화살표를 받는 쪽이 Port Listening 상태로 표기함.

Open

External Ports: Internet 구간

• 443: CloudHub Portal에서 사용하는 웹 포트.

Internal Ports (Backend Server 간): Trusted Network 구간

• 8000 : Salt-API Server - CH Server와 데이터 송수신.

• 9094 : Kapacitor - InfluxDB와 데이터 송수신.

Intermediate Ports (Agent ↔︎ Backend Server 간): Semi-Trusted Network 구간

• 4505-4506 : Salt-Master - Salt-Minion과 데이터 송수신.

• 8086 : InfluxDB - telegraf로부터 데이터 수신.

아래는 firewalld 설정 파일 예시이며,
주의 물리적인 보안 장비를 따로 사용하여 설정할 경우는 위 설명한 용도와 내용에 맞게 ACL을 설정하여 사용하여야 합니다.

Install Pre-required Packages

Setting InfluxDB and Kapacitor containers via Docker Compose

1. Docker 설치 확인

   • 버전 : Docker version 18.09.4 이상

     $ docker --version Docker version 18.09.4, build d14af54266

   • 버전이 낮으면 이전 버전을 삭제 후 설치

     • 이전 버전 삭제

       $ yum remove docker \ docker-client \ docker-client-latest \ docker-common \ docker-latest \ docker-latest-logrotate \ docker-logrotate \ docker-engine

     • 저장소 설정 및 추가

       $ yum install -y yum-utils \ device-mapper-persistent-data \ lvm2 $ yum-config-manager \ --add-repo \ https://download.docker.com/linux/centos/docker-ce.repo

     • Install Docker Engine - Community

       $ yum install docker-ce docker-ce-cli containerd.io

2. Docker log rotation 설정

   • logrotate을 사용하여 Docker Log를 관리 합니다.

   • Log 파일 경로 : /var/lib/docker/containers/*/*.log

   • Log 설정 파일 : $ vim /etc/logrotate.d/docker-container

     /var/lib/docker/containers/*/*.log { daily rotate 7 missingok dateext compress notifempty }

     daily : 매일 log를 rotate 합니다.
     rotate 7 : 최대 log.1, log.2 등 7개의 파일을 보관합니다.
     missingok : log가 없는 경우 에러 없이 다음 rotate로 넘어갑니다.
     dateext : log 파일에 YYYYMMDD 형식의 확장자를 추가합니다.
     compress : 이전 log를 압축합니다.
     notifempty :  log 내용이 없으면 rotation 하지 않습니다.

3. Docker 실행

   $ systemctl enable docker $ systemctl start docker $ systemctl status docker ● docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled) Active: active (running) since Mon 2020-03-09 05:30:36 EDT; 3s ago Docs: https://docs.docker.com Main PID: 2036 (dockerd) Tasks: 8 Memory: 42.1M CGroup: /system.slice/docker.service └─2036 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ... ... Hint: Some lines were ellipsized, use -l to show in full.

4. Docker Compose 설치 확인

   • 버전 : Docker Compose version 1.24 이상

     $ docker-compose --version docker-compose version 1.24.0, build 0aa59064

   • 버전이 낮거나 미 설치 시 Docker Compose 설치

     • 파일 다운로드

       $ curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

     • 다운로드 파일에 권한 적용

       $ sudo chmod +x /usr/local/bin/docker-compose

     • 버전 확인

       $ docker-compose --version docker-compose version 1.25.4, build 8d51620a

5. Git 설치

   $ yum install -y git

6. Snetsystems/Sandbox Download

   $ cd {Sandbox 다운로드 할 디렉토리 이동} $ git clone https://github.com/snetsystems/sandbox.git

7. Sandbox 실행

   $ cd sandbox $ ./sandbox up Using latest, stable releases Spinning up Docker Images... If this is your first time starting sandbox this might take a minute... Creating network "sandbox_default" with the default driver Building influxdb Step 1/2 : ARG INFLUXDB_TAG Step 2/2 : FROM influxdb:$INFLUXDB_TAG latest: Pulling from library/influxdb c0c53f743a40: Downloading [=================================> ] 30.81MB/45.38MB c0c53f743a40: Pull complete 66997431d390: Pull complete ... ... Successfully built a59cd84d6569 Successfully tagged kapacitor:latest Creating sandbox_influxdb_1 ... done Creating sandbox_kapacitor_1 ... done Opening tabs in browser...

Setting SaltStack for master

Yum install

1. SaltStack 저장소 등록

   $ yum install -y epel-release $ rpm --import https://repo.saltproject.io/py3/redhat/7/x86_64/archive/3001.4/SALTSTACK-GPG-KEY.pub $ curl -fsSL https://repo.saltproject.io/py3/redhat/7/x86_64/archive/3001.4.repo | sudo tee /etc/yum.repos.d/salt.repo $ yum clean expire-cache

2. Salt-Master, Salt-API, Salt-Minion 설치

   $ yum install -y salt-master salt-api salt-minion

3. Config 설정

   1. Salt-Master의 Config 와 Salt-API의 Config 파일을 설정해야 합니다.

   2. Salt-Master : $ vim /etc/salt/master.d/master.conf

   3. 주의 Host에 여러 Interface가 있을 경우 Salt-Minion과 연결 가능한 네트워크 Interface IP로 설정해야 합니다.
      ex) interface: 61.254.65.58
      아래 예와 같이 any(0.0.0.0)로 설정해도 master ↔︎ minion 간 통신은 문제가 없으나,
      CloudHub UI에서 원하지 않는 IP로 표시될 수 있습니다.

      ## log_level ## # One of 'garbage', 'trace', 'debug', info', 'warning'(default), 'error', 'critical'. log_level: info ##### Primary configuration settings ##### ########################################## # The address of the interface to bind to: interface: 0.0.0.0 # The tcp port used by the publisher: publish_port: 4505 # Allow minions to push files to the master. This is disabled by default, for # security purposes. # file_recv: True ##### State System settings ##### ########################################## # The state system uses a "top" file to tell the minions what environment to # use and what modules to use. The state_top file is defined relative to the # root of the base environment as defined in "File Server settings" below. state_top: top.sls ##### File Server settings ##### ########################################## # Salt runs a lightweight file server written in zeromq to deliver files to # minions. This file server is built into the master daemon and does not # require a dedicated port. # The file server works on environments passed to the master, each environment # can have multiple root directories, the subdirectories in the multiple file # roots cannot match, otherwise the downloaded files will not be able to be # reliably ensured. A base environment is required to house the top file. file_roots: base: - /srv/salt/prod qa: - /srv/salt/qa - /srv/salt/prod dev: - /srv/salt/dev - /srv/salt/qa - /srv/salt/prod # File Server Backend # # Salt supports a modular fileserver backend system, this system allows # the salt master to link directly to third party systems to gather and # manage the files available to minions. Multiple backends can be # configured and will be searched for the requested file in the order in which # they are defined here. The default setting only enables the standard backend # "roots" which uses the "file_roots" option. fileserver_backend: - roots ##### Security settings ##### ########################################## # The external auth system uses the Salt auth modules to authenticate and # validate users to access areas of the Salt system. external_auth: pam: saltdev: - .* - '@runner' - '@wheel' - '@jobs' # Allow eauth users to specify the expiry time of the tokens they generate. # A boolean applies to all users or a dictionary of whitelisted eauth backends # and usernames may be given. token_expire_user_override: pam: - saltdev ##### API Server settings ##### ########################################## rest_cherrypy: port: 8000 disable_ssl: True

   4. Salt-Minion : $ vim /etc/salt/minion.d/minion.conf

      ## log_level ## # One of 'garbage', 'trace', 'debug', info', 'warning'(default), 'error', 'critical'. log_level: info master: <master ip address> id: <유일한 minion id 설정해야 하며, 생략 시, hostname = minion id>

4. 계정 생성 및 패스워드 설정
   아래 추가된 password는 추후 salt-api authentication을 위한 pam_token를 발급 받는데 쓰이므로, 잘 기억해두어야 합니다.

   $ useradd saltdev $ passwd <password> Changing password for user saltdev. New password:

5. 서비스 시작

   $ systemctl enable salt-master $ systemctl start salt-master $ systemctl enable salt-api $ systemctl start salt-api $ systemctl enable salt-minion $ systemctl start salt-minion

For off internet

• 상단 설치 과정은 인터넷이 허용되는 환경에서의 Pre-required Packages 설치 방법입니다.

• 인터넷이 안되는 폐쇄망 환경에서 Package Deployment on Private Network(Server)설치 방법 입니다.

Download telegraf into salt file_roots path on salt-master

아래 링크에서 설치하고자 하는 버전을 다운로드 할 수 있습니다.

Download URL: https://github.com/snetsystems/telegraf/releases

Install CloudHub Portal

Download

아래 링크에서 설치하고자 하는 버전을 다운로드 할 수 있습니다.

Download URL: https://github.com/snetsystems/cloudhub/releases

Install CloudHub admin portal package

• 설치명령어

  • $ sudo yum localinstall cloudhub-0.9.1~17fe47a.x86_64.rpm

• Log 파일

  • service로 데몬 실행 시 log file : /var/log/cloudhub

  • systemd로 데몬 실행 시 log file: /var/log/messages

• Environment or Argument file path

  • 실행(help) 옵션

    • $ cloudhub -h

  • 파일 : /etc/default/cloudhub

  • Argument 설정: 예> 61.250.122.xx

    $ cat /etc/default/cloudhub CLOUDHUB_OPTS="-l=debug \ --auth-duration=0 \ -t=74c1e9e2450886060b5bf736b935cd0bf960837f \ -i=<client-id> \ -s=<client-secret> \ -u=salt:http://61.250.122.xx:8000/run \ -k=salt:<salt-token>"

• 서비스 시작/정지/재시작

  • $ sudo systemctl {start|stop|restart} cloudhub

• 서비스 확인

  $ systemctl status cloudhub.service ● cloudhub.service Loaded: loaded (/usr/lib/systemd/system/cloudhub.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2020-02-20 03:55:04 EST; 1s ago Main PID: 23498 (cloudhub) Tasks: 5 Memory: 26.9M CGroup: /system.slice/cloudhub.service └─23498 /usr/bin/cloudhub Feb 20 03:55:04 localhost.localdomain systemd[1]: Started cloudhub.service. Feb 20 03:55:04 localhost.localdomain cloudhub[23498]: time="2020-02-20T03:55:04-05:00" level=info msg="Serving CloudHub at http://[::]:8888" component=server

Enabling HTTPS on CloudHub Portal

• http가 기본으로 설정되어 있으며 https로 변경 하려면 cloudhub.service 중단 후 설정파일을 수정하고 'systemctl daemon-reload'를 하여 서비스를 재시작 합니다.

  $ cat /etc/systemd/system/multi-user.target.wants/cloudhub.service [Unit] After=network-online.target [Service] User=root Group=root Environment="HOST=0.0.0.0" #Environment="PORT=8888" Environment="PORT=443" Environment="TLS_CERTIFICATE=/usr/lib/cloudhub/key/cloudhub_self_signed.pem" Environment="BOLT_PATH=/var/lib/cloudhub/cloudhub-v1.db" Environment="CANNED_PATH=/usr/share/cloudhub/cloudhub-canned" Environment="PROTOBOARDS_PATH=/usr/share/cloudhub/cloudhub-protoboards" EnvironmentFile=-/etc/default/cloudhub ExecStart=/usr/bin/cloudhub $CLOUDHUB_OPTS KillMode=control-group Restart=on-failure [Install] WantedBy=multi-user.target