사전 점검

Required Hardware Spec.

OS

• CentOS 7 x86_64 기준

CloudHub에서 사용되는 Port 및 방화벽 설정

개념도

...

External Ports: Internet 구간

• 443: CloudHub Portal에서 사용하는 웹 포트.

Internal Ports (Backend Server 간): Trusted Network 구간

• 8000 : Salt-API Server - CH Server와 데이터 송수신.

• 9094 : Kapacitor - InfluxDB와 데이터 송수신.

Intermediate Ports (Agent ↔︎ Backend Server 간): Semi-Trusted Network 구간

• 4505-4506 : Salt-Master - Salt-Minion과 데이터 송수신.

• 8086 : InfluxDB - telegraf로부터 데이터 수신.

아래는 firewalld 설정 파일 예시이며,

<!-- /etc/firewalld/zones/public.xml -->
<?xml version="1.0" encoding="utf-8"?>
<zone>
  <short>Public</short>
  <description>For use in public areas.</description>
  <service name="ssh"/>
  <service name="dhcpv6-client"/>
  <port protocol="tcp" port="443"/>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.0/24"/>
    <port protocol="tcp" port="4505-4506"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="8000"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="8086"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="9094"/>
    <accept/>
  </rule>
</zone>

기 설치된 Docker 버전 확인 및 이전 버전 삭제

• 버전 : Docker version 18.09.4 이상

  Code Block
  language bash
  $ docker --version Docker version 18.09.4, build d14af54266

• 버전이 낮으면 이전 버전을 삭제 후 설치

  • 이전 버전 삭제

    Code Block
    language bash
    $ yum remove docker \ docker-client \ docker-client-latest \ docker-common \ docker-latest \ docker-latest-logrotate \ docker-logrotate \ docker-engine

의존성 설치

Docker 설치

• 저장소 설정 및 추가

  Code Block
  language bash
  $ yum install -y yum-utils \ device-mapper-persistent-data \ lvm2 $ yum-config-manager \ --add-repo \ https://download.docker.com/linux/centos/docker-ce.repo

• Install Docker Engine - Community

  Code Block
  language bash
  $ yum install docker-ce docker-ce-cli containerd.io

Docker Compose 설치

...

사전 점검

Required Hardware Spec.

OS

• CentOS 7 x86_64 기준

CloudHub에서 사용되는 Port 및 방화벽 설정

개념도

...

External Ports: Internet 구간

• 443: CloudHub Portal에서 사용하는 웹 포트.

Internal Ports (Backend Server 간): Trusted Network 구간

• 8000 : Salt-API Server - CH Server와 데이터 송수신.

• 9094 : Kapacitor - InfluxDB와 데이터 송수신.

Intermediate Ports (Agent ↔︎ Backend Server 간): Semi-Trusted Network 구간

• 4505-4506 : Salt-Master - Salt-Minion과 데이터 송수신.

• 8086 : InfluxDB - telegraf로부터 데이터 수신.

아래는 firewalld 설정 파일 예시이며,

<!-- /etc/firewalld/zones/public.xml -->
<?xml version="1.0" encoding="utf-8"?>
<zone>
  <short>Public</short>
  <description>For use in public areas.</description>
  <service name="ssh"/>
  <service name="dhcpv6-client"/>
  <port protocol="tcp" port="443"/>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.0/24"/>
    <port protocol="tcp" port="4505-4506"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="8000"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="8086"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="9094"/>
    <accept/>
  </rule>
</zone>

기 설치된 Docker 버전 확인 및 이전 버전 삭제

• 버전 : Docker version 18.09.4 이상

  Code Block
  language bash
  $ docker

...

• --version

...

• Docker version

...

• 18.

...

• 09.

...

• 4, build

...

• d14af54266

• 버전이 낮거나 미 설치 시 Docker Compose [재]설치

  파일 다운로드

  낮으면 이전 버전을 삭제 후 설치

  • 이전 버전 삭제

    Code Block
    language bash
    $ rmyum -f /usr/local/bin/docker-compose $ curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

  • 실행 권한 부여

    Code Block
    language bash
    $ sudo chmod +x /usr/local/bin/docker-compose

  • 버전 확인

    Code Block
    language bash
    $ docker-compose --version docker-compose version 1.29.2, build 5becea4c

Docker log rotation 설정

• Log 설정 파일 : $ vim /etc/logrotate.d/docker-container

  Code Block
  language yaml
  /var/lib/docker/containers/*/*.log { daily rotate 7 missingok dateext compress notifempty }

Docker 서비스 실행

$ systemctl enable docker
$ systemctl start docker
$ systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2020-03-09 05:30:36 EDT; 3s ago
     Docs: https://docs.docker.com
 Main PID: 2036 (dockerd)
    Tasks: 8
   Memory: 42.1M
   CGroup: /system.slice/docker.service
           └─2036 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

...
...
Hint: Some lines were ellipsized, use -l to show in full.

Git 설치

$ yum install -y git

CloudHub Components 설치

Snetsystems/Sandbox 구동

...

• • remove docker \ docker-client \ docker-client-latest \ docker-common \ docker-latest \ docker-latest-logrotate \ docker-logrotate \ docker-engine

의존성 설치

Docker 설치

• 저장소 설정 및 추가

  Code Block
  language bash
  $ yum install -y yum-utils \ device-mapper-persistent-data \ lvm2 $ yum-config-manager \ --add-repo \ https://download.docker.com/linux/centos/docker-ce.repo

• Install Docker Engine - Community

  Code Block
  language bash
  $ yum install docker-ce docker-ce-cli containerd.io

Docker Compose 설치

• 버전 : Docker Compose version 1.24 이상

$ docker-compose --version
docker-compose version 1.24.0, build 0aa59064

• 버전이 낮거나 미 설치 시 Docker Compose [재]설치

  • 파일 다운로드

    Code Block
    language bash
    $

...

• • rm

...

• • -f /usr/local/bin/docker-compose $

...

• • curl

...

• • -L "https://github.com

...

• • /docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

  • 실행 권한 부여

    Code Block
    language bash
    $ sudo chmod +x /usr/local/bin/docker-compose

  • 버전 확인

    Code Block
    language bash
    $ docker-compose --version docker-compose version 1.29.2, build 5becea4c

Docker log rotation 설정

• Log 설정 파일 : $ vim /etc/logrotate.d/docker-container

  Code Block
  language yaml
  /var/lib/docker/containers/*/*.log { daily rotate 7 missingok dateext compress copytruncate notifempty }

Docker 서비스 실행

$ systemctl cdenable sandboxdocker
$ ./sandboxsystemctl upstart docker
$ systemctl status docker
● docker-compose ps
.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/python2.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.24.3) or chardet (2.2.1) doesn't match a supported version!
  RequestsDependencyWarning)
       Name                    Commandsystemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2020-03-09 05:30:36 EDT; 3s ago
     Docs: https://docs.docker.com
 Main PID: 2036 (dockerd)
    Tasks: 8
   Memory: 42.1M
   CGroup: /system.slice/docker.service
           └─2036 /usr/bin/dockerd -H  State                                                  Ports
------------------------------------------------------------------fd:// --containerd=/run/containerd/containerd.sock

...
...
Hint: Some lines were ellipsized, use -l to show in full.

Git 설치

$ yum install -y git

CloudHub Components 설치

Snetsystems/Sandbox 구동

Clone Snetsystems/Sandbox

$ cd {Sandbox 다운로드 할 디렉토리 이동}
$ git clone https://github.com/snetsystems/sandbox.git

# Server Single node mode로 구성하기 위해, 아래에서 etcd: 이하 configuration 부분 주석 또는 삭제.
$ vim sandbox/docker-compose.yml
...(생략)
#etcd:
#    # Full tag list: quay.io/coreos/etcd:v3.3.11
#    #image: gcr.io/etcd-development/etcd:${ETCD_TAG}
#    image: quay.io/coreos/etcd:${ETCD_TAG}
#    entrypoint: /usr/local/bin/etcd
...(생략)

Sandbox 실행

$ cd sandbox
$ ./sandbox up

$ docker-compose ps
/usr/lib/python2.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.24.3) or chardet (2.2.1) doesn't match a supported version!
  RequestsDependencyWarning)
       Name                    Command            State                                                  Ports
------------------------------------------------------------------------------------------------------
sandbox_influxdb_1    /entrypoint.sh influxd      Up      0.0.0.0:8082->8082/tcp,:::8082->8082/tcp, 0.0.0.0:8086->8086/tcp,:::8086->8086/tcp,
                                  ----------------------------------------------------------------------------------------------------------------
sandbox_influxdb_1    /entrypoint.sh influxd      Up      0.0.0.0:8082->8082/tcp,:::8082->8082/tcp, 0.0.0.0:8086->8086/tcp,:::8086->8086/tcp,
                                                          0.0.0.0:8089->8089/udp,:::8089->8089/udp
sandbox_kapacitor_1   /entrypoint.sh kapacitord   Up      0.0.0.0:9094->9092/tcp,:::9094->9092/tcp

Sandbox 서비스 파일 생성 및 활성화

$ cd <Sandbox 다운로드 받은 디렉토리>
$ ./sandbox install <Working Directory>

# 서비스 파일 확인
$ cat /etc/systemd/system/snet-sandbox.service

# 서비스 Enabled 상태 확인
$ systemctl status snet-sandbox

SaltStack 설치([On/Off]line 설치 겸용)

Predefinitions

설명의 편의를 위해, 아래와 같이 사전 정의된 경로를 사용하는 것을 전제로 합니다.

Conda 설치

위 첨부 파일에서 Miniconda3 배포 파일 다운로드 후,

아래 명령으로 적절히 용량이 큰 디스크 경로에 설치 권장.

$ wget https://seversky.atlassian.net/wiki/download/attachments/1883635713/Miniconda3-latest-Linux-x86_64.sh
$ chmod 755 Miniconda3-latest-Linux-x86_64.sh # 쉘 실행 권한 부여
$ ./Miniconda3-latest-Linux-x86_64.sh -b -u -p /opt/miniconda3 # /opt/miniconda3에 설치

conda 환경 변수 등록

conda 환경 변수 설정을 위해 conda init 명령어를 입력해야 합니다.

$ export PATH=/opt/miniconda3/bin:$PATH
$ conda init

conda 기본 환경을 비활성화

$ conda config --set auto_activate_base false

Download from Github release page

https://github.com/snetsystems/salt/releases 에서 원하는 버전의 인스톨 파일들을 다운로드 합니다.
여기서는 “3001.4-15-g1d946bf”버전으로 인스톨 할 것이며, 관련 파일들을 적당한 경로에 다운로드 합니다.

1. Conda 가상환경 압축 파일: 예> saltenv-3001.4-15-g1d946bf-linux.tar.gz

2. install_salt_svc_for_conda_systemd.sh

saltenv unpack

/opt/miniconda3/envs/에 saltenv-3001.4-15-g1d946bf-linux.tar.gz를 압축 해제합니다.

그리고 나서, conda env list로 확인하면 정상 등록되어 있을 것 입니다.

$ tar xzf saltenv-3001.4-15-g1d946bf-linux.tar.gz -C /opt/miniconda3/envs/
$ pwd
/opt/miniconda3/envs
$ ll
total 219044
drwxr-xr-x. 13 root root       195 May 16 11:08 saltenv

$ conda env list
# conda environments:
#
base                  *  /opt/miniconda3
saltenv                  /opt/miniconda3/envs/saltenv

Config 설정

Salt-Master와 Salt-API의 Config 설정

Config path :

$ mkdir -p /opt/miniconda3/envs/saltenv/master/etc/salt && vim /opt/miniconda3/envs/saltenv/master/etc/salt/master

## log_level ##
# One of 'garbage', 'trace', 'debug', info', 'warning'(default), 'error', 'critical'.
log_level: info
root_dir: /opt/miniconda3/envs/saltenv/master

##### Primary configuration settings #####
##########################################
# The address of the interface to bind to:
interface: x.x.x.x
 
# The tcp port used by the publisher:
publish_port: 4505
 
# Allow minions to push files to the master. This is disabled by default, for
# security purposes.
# file_recv: True
 
#####      State System settings     #####
##########################################
# The state system uses a "top" file to tell the minions what environment to
# use and what modules to use. The state_top file is defined relative to the
# root of the base environment as defined in "File Server settings" below.
state_top: top.sls
 
#####      File Server settings      #####
##########################################
# Salt runs a lightweight file server written in zeromq to deliver files to
# minions. This file server is built into the master daemon and does not
# require a dedicated port.
 
# The file server works on environments passed to the master, each environment
# can have multiple root directories, the subdirectories in the multiple file
# roots cannot match, otherwise the downloaded files will not be able to be
# reliably ensured. A base environment is required to house the top file.
file_roots:
  base:
    - /srv/salt/prod
 
# File Server Backend
#
# Salt supports a modular fileserver backend system, this system allows
# the salt master to link directly to third party systems to gather and
# manage the files available to minions. Multiple backends can be
# configured and will be searched for the requested file in the order in which
# they are defined here. The default setting only enables the standard backend
# "roots" which uses the "file_roots" option.
fileserver_backend:
  - roots
 
#####        Security settings       #####
##########################################
# The external auth system uses the Salt auth modules to authenticate and
# validate users to access areas of the Salt system.
external_auth:
  pam:
    saltenv:
      - .*
      - '@runner'
      - '@wheel'
      - '@jobs'
 
# Allow eauth users to specify the expiry time of the tokens they generate.
# A boolean applies to all users or a dictionary of whitelisted eauth backends
# and usernames may be given.
token_expire_user_override:
  pam:
    - saltenv

#####        API Server settings     #####
##########################################
rest_cherrypy:
  port: 8887
  0.0.0.0:8089->8089/udp,:::8089->8089/udp
sandbox_kapacitor_1   /entrypoint.sh kapacitord   Up      0.0.0.0:9094->9092/tcp,:::9094->9092/tcp

SaltStack 설치([On/Off]line 설치 겸용)

Predefinitions

설명의 편의를 위해, 아래와 같이 사전 정의된 경로를 사용하는 것을 전제로 합니다.

...

Conda Installed Path

...

/opt/miniconda3

Conda 설치

위 첨부 파일에서 Miniconda3 배포 파일 다운로드 후,

아래 명령으로 적절히 용량이 큰 디스크 경로에 설치 권장.

chmod 755 Miniconda3-latest-Linux-x86_64.sh # 쉘 실행 권한 부여
./Miniconda3-latest-Linux-x86_64.sh -b -u -p /opt/miniconda3 # /opt/miniconda3에 설치

Download from Github release page

https://github.com/snetsystems/salt/releases 에서 원하는 버전의 인스톨 파일들을 다운로드 합니다.
여기서는 “3001.4-6-ge56e905”버전으로 인스톨 할 것이며, 관련 파일들을 적당한 경로에 다운로드 합니다.

1. Conda 가상환경 압축 파일: 예> saltenv-3001.4-6-ge56e905-linux.tar.gz

2. install_salt_svc_for_conda_centos7.sh

saltenv unpack

/opt/miniconda3/envs/에 saltenv-3001.4-6-ge56e905-linux.tar.gz를 압축 해제합니다.

그리고 나서, conda env list로 확인하면 정상 등록되어 있을 것 입니다.

$ tar czf saltenv-3001.4-6-ge56e905-linux.tar.gz -C /opt/miniconda3/envs/
$ pwd
/opt/miniconda3/envs
$ ll
total 219044
drwxr-xr-x. 13 root root       195 May 16 11:08 saltenv

$ conda env list
# conda environments:
#
base                  *  /opt/miniconda3
saltenv                  /opt/miniconda3/envs/saltenv

Config 설정

Salt-Master와 Salt-API의 Config 설정.

Config path : $ vim /opt/miniconda3/etc/salt/master

## log_level ##
# One of 'garbage', 'trace', 'debug', info', 'warning'(default), 'error', 'critical'.
log_level: info
root_dir: /opt/miniconda3/envs/saltenv

##### Primary configuration settings #####
##########################################
# The address of the interface to bind to:
interface: x.x.x.x
 
# The tcp port used by the publisher:
publish_port: 4505
 
# Allow minions to push files to the master. This is disabled by default, for
# security purposes.
# file_recv: True
 
#####      State System settings     #####
##########################################
# The state system uses a "top" file to tell the minions what environment to
# use and what modules to use. The state_top file is defined relative to the
# root of the base environment as defined in "File Server settings" below.
state_top: top.sls
 
#####      File Server settings      #####
##########################################
# Salt runs a lightweight file server written in zeromq to deliver files to
# minions. This file server is built into the master daemon and does not
# require a dedicated port.
 
# The file server works on environments passed to the master, each environment
# can have multiple root directories, the subdirectories in the multiple file
# roots cannot match, otherwise the downloaded files will not be able to be
# reliably ensured. A base environment is required to house the top file.
file_roots:
  base:
    - /srv/salt/prod
 
# File Server Backend
#
# Salt supports a modular fileserver backend system, this system allows
# the salt master to link directly to third party systems to gather and
# manage the files available to minions. Multiple backends can be
# configured and will be searched for the requested file in the order in which
# they are defined here. The default setting only enables the standard backend
# "roots" which uses the "file_roots" option.
fileserver_backend:
  - roots
 
#####        Security settings       #####
##########################################
# The external auth system uses the Salt auth modules to authenticate and
# validate users to access areas of the Salt system.
external_auth:
  pam:
    saltdev:
      - .*
      - '@runner'
      - '@wheel'
      - '@jobs'
 
# Allow eauth users to specify the expiry time of the tokens they generate.
# A boolean applies to all users or a dictionary of whitelisted eauth backends
# and usernames may be given.
token_expire_user_override:
  pam:
    - saltdev

#####        API Server settings     #####
##########################################
rest_cherrypy:
  port: 8000
  disable_ssl: True

Salt-Minion 설정

Config path: $ vim /opt/miniconda3/etc/salt/minion

## log_level ##
# One of 'garbage', 'trace', 'debug', info', 'warning'(default), 'error', 'critical'.
log_level: info
root_dir: /opt/miniconda3/envs/saltenv
master: <master ip address>
id: <유일한 minion id 설정해야 하며, 생략 시, hostname = minion id>

계정 생성 및 패스워드 설정

$ useradd saltdev
$ passwd <password>
Changing password for user saltdev.
New password:

Register & Start Salt Services

$ ./install_salt_svc_for_conda_centos7.sh를 실행하면 아래 예제와 같이 서비스가 설치되며, 자동 실행됩니다.

# install_salt_svc_for_conda_centos7.sh 사용법
$ ./install_salt_svc_for_conda_centos7.sh -h

usage: ./install_salt_svc_for_conda_centos7.sh [options]

Install SNET salt services

-p          Set installed miniconda prefix path (default: /opt/miniconda3/envs/saltenv

Install Master + Minion + API Service

$ ./install_salt_svc_for_conda_centos7.sh
Do you want to install salt-minion only? [y/n]n
Do you want to start the 'snet-salt-minion' service? [y/n]y

• 서비스 명: snet-salt-master.service, snet-salt-minion.service, snet-salt-api.service

• Logrotate 설정.

  Code Block
  # ll /etc/logrotate.d/snet-salt -rw-r--r--. 1 root root 592 May 16 12:39 /etc/logrotate.d/snet-salt

Install Minion Service Only

$ ./install_salt_svc_for_conda_centos7.sh
Do you want to install salt-minion only? [y/n]y
Do you want to start the 'snet-salt-minion' service? [y/n]y

• 서비스 명: snet-salt-minion.service

• Logrotate 설정.

  Code Block
  # ll /etc/logrotate.d/snet-salt -rw-r--r--. 1 root root 592 May 16 12:39 /etc/logrotate.d/snet-salt

Download telegraf into salt file_roots path on salt-master

아래 링크에서 설치하고자 하는 버전을 다운로드 할 수 있습니다.
Download URL: https://github.com/snetsystems/telegraf/releases

아래 예제와 같이 지정된 경로에 원하는 버전의 rpm 파일을 다운로드 합니다.

$ mkdir -p /opt/miniconda3/srv/salt/prod/telegraf          # Telegraf package path
$ cd /opt/miniconda3/srv/salt/prod/telegraf
$ wget https://github.com/snetsystems/telegraf/releases/download/v1.19.3-snet/telegraf-1.19.3-snet-1_x86_64.rpm

Install CloudHub Portal

Download

아래 링크에서 설치하고자 하는 버전을 다운로드 할 수 있습니다.

Download URL: https://github.com/snetsystems/cloudhub/releases

Install

$ yum localinstall cloudhub-0.9.1~17fe47a.x86_64.rpm

Configure

1. $ vim /etc/systemd/system/multi-user.target.wants/cloudhub.service

   Code Block
   language yaml
   [Unit] After=network-online.target [Service] User=root Group=root Environment="HOST=0.0.0.0" Environment="PORT=443" # default: 8888 Environment="TLS_CERTIFICATE=/usr/lib/cloudhub/key/cloudhub_self_signed.pem" #Environment="TLS_PRIVATE_KEY=my.key" Environment="BOLT_PATH=/var/lib/cloudhub/cloudhub-v1.db" #Environment="ETCD_ENDPOINTS={ETCD_CLIENT_IP}:2379" Environment="CANNED_PATH=/usr/share/cloudhub/cloudhub-canned" Environment="PROTOBOARDS_PATH=/usr/share/cloudhub/cloudhub-protoboards" EnvironmentFile=-/etc/default/cloudhub ExecStart=/usr/bin/cloudhub $CLOUDHUB_OPTS KillMode=control-group Restart=on-failure [Install] WantedBy=multi-user.target

2. localhost 인증서(cloudhub_self_signed.pem) 발급

   Code Block
   $ cd /usr/lib/cloudhub/key/ $ openssl req -x509 -newkey rsa:4096 -sha256 -nodes -keyout cloudhub_self_signed.pem -out cloudhub_self_signed.pem -subj "/CN=localhost" -days 365

3. <salt_token> 발급
   /wiki/spaces/CM/pages/254050538
   

   Status
   colour Purple title 노트

   위 페이지 접근 권한이 없는 경우에는 Snetsystems CloudHub 팀에 문의/발급 받은 후,
   아래 4번 -k=salt:<salt_token>에 기입하세요.

4. $ vim /etc/default/cloudhub

   Code Block
   language yaml
   CLOUDHUB_OPTS="-l=debug \ --auth-duration=0 \ -t=74c1e9e2450886060b5bf736b935cd0bf960837f \ -i=any \ -s=any \ -u=salt:http://{salt-master-ip}:8000/run \ -k=salt:<salt_token> \ -u=vsphere:on \ -u=aws:on \ -u=k8s:on \ --login-auth-type=basic \ --password-policy='(?=.*[0-9]{1,50})(?=.*[~`!@#$%\\^&*()-+=]{1,50})(?=.*[a-zA-Z]{2,50}).{8,50}$' \ --password-policy-message='Enter at least 8 digits using at least 1 number and at least 1 special sentence, and at least 2 English characters.' \ --mail-subject='[Notifiy] $user_id Password is reset' \ --mail-body-message='Reset OTP for $user_id is '$user_pw'.<br>Do not forget this!<br>This OTP will use just once when login into cloudhub.<br>Then you must set up the new password' \ --kapacitor-url=http://{Kapacitor_IP}:9094 \ --influxdb-url=http://{InfluxDB_IP}:8086 \ --retry-policy=count:3 \ --retry-policy=delaytime:5 \ --retry-policy=type:delay"

Run

...

ssl_crt: /usr/lib/cloudhub/key/cloudhub_self_signed.crt
  ssl_key: /usr/lib/cloudhub/key/cloudhub_self_signed.key

rest_cherrypy_ssl: True

Salt-Minion 설정

Config path:

$ mkdir -p /opt/miniconda3/envs/saltenv/minion/etc/salt && vim /opt/miniconda3/envs/saltenv/minion/etc/salt/minion

## log_level ##
# One of 'garbage', 'trace', 'debug', info', 'warning'(default), 'error', 'critical'.
log_level: info
root_dir: /opt/miniconda3/envs/saltenv/minion
master: <master ip address>
id: <유일한 minion id 설정해야 하며, 생략 시, hostname = minion id>

계정 생성 및 패스워드 설정

$ useradd saltenv
$ passwd saltenv
Changing password for user saltenv.
New password:

Register & Start Salt Services

$ ./install_salt_svc_for_conda_systemd.sh를 실행하면 아래 예제와 같이 서비스가 설치되며, 자동 실행됩니다.

# install_salt_svc_for_conda_systemd.sh 사용법
$ ./install_salt_svc_for_conda_systemd.sh -h

usage: ./install_salt_svc_for_conda_systemd.sh [options]

Install SNET salt services

-p          Set installed miniconda prefix path (default: /opt/miniconda3/envs/saltenv

Install Master + Minion + API Service

$ ./install_salt_svc_for_conda_systemd.sh
Do you want to install salt-minion only? [y/n]n
Do you want to start whole 'snet-salt' services? [y/n]y

• 서비스 명: snet-salt-master.service, snet-salt-minion.service, snet-salt-api.service

• Logrotate 설정.

  Code Block
  # ll /etc/logrotate.d/snet-salt -rw-r--r--. 1 root root 592 May 16 12:39 /etc/logrotate.d/snet-salt

Install Minion Service Only

$ ./install_salt_svc_for_conda_systemd.sh
Do you want to install salt-minion only? [y/n]y
Do you want to start the 'snet-salt-minion' service? [y/n]y

• 서비스 명: snet-salt-minion.service

• Logrotate 설정.

  Code Block
  # ll /etc/logrotate.d/snet-salt -rw-r--r--. 1 root root 592 May 16 12:39 /etc/logrotate.d/snet-salt

Download telegraf into salt file_roots path on salt-master

아래 링크에서 설치하고자 하는 버전을 다운로드 할 수 있습니다.
Download URL: https://github.com/snetsystems/telegraf/releases

아래 예제와 같이 지정된 경로에 원하는 버전의 rpm 파일을 다운로드 합니다.

$ mkdir -p /srv/salt/prod/telegraf          # Telegraf package path
$ cd /srv/salt/prod/telegraf
$ wget https://github.com/snetsystems/telegraf/releases/download/v1.23.4-snet-1/telegraf-1.23.4-snet-1-1.x86_64.rpm

Install CloudHub Portal

Download

아래 링크에서 설치하고자 하는 버전을 다운로드 할 수 있습니다.

Download URL: https://github.com/snetsystems/cloudhub/releases

Install

$ yum localinstall CloudHub-1.4.2.520d7bd.x86_64.rpm

Configure

1. $ vim /etc/systemd/system/multi-user.target.wants/cloudhub.service

   Code Block
   language yaml
   [Unit] After=network-online.target [Service] User=root Group=root Environment="HOST=0.0.0.0" Environment="PORT=8888" Environment="TLS_CERTIFICATE=/usr/lib/cloudhub/key/cloudhub_self_signed.pem" #Environment="TLS_PRIVATE_KEY=my.key" Environment="BOLT_PATH=/var/lib/cloudhub/cloudhub-v1.db" #Environment="ETCD_ENDPOINTS={ETCD_CLIENT_IP}:2379" Environment="CANNED_PATH=/usr/share/cloudhub/cloudhub-canned" Environment="PROTOBOARDS_PATH=/usr/share/cloudhub/cloudhub-protoboards" EnvironmentFile=-/etc/default/cloudhub ExecStart=/usr/bin/cloudhub $CLOUDHUB_OPTS KillMode=control-group Restart=on-failure [Install] WantedBy=multi-user.target

2. localhost 인증서(cloudhub_self_signed.pem) 발급

   Code Block
   $ cd /usr/lib/cloudhub/key/ $ openssl req -x509 -newkey rsa:4096 -sha256 -nodes -keyout cloudhub_self_signed.pem -out cloudhub_self_signed.pem -subj "/CN=localhost" -days 365 $ openssl rsa -in cloudhub_self_signed.pem -text > cloudhub_self_signed.key $ openssl x509 -inform PEM -in cloudhub_self_signed.pem -out cloudhub_self_signed.crt

3. <salt_token> 발급
   /wiki/spaces/CM/pages/254050538
   

   Status
   colour Purple title 노트

   위 페이지 접근 권한이 없는 경우에는 Snetsystems CloudHub 팀에 문의/발급 받은 후,
   아래 4번 -k=salt:<salt_token>에 기입하세요.

4. $ vim /etc/default/cloudhub

   Code Block
   language yaml
   CLOUDHUB_OPTS="-l=debug \ --auth-duration=0 \ -t=74c1e9e2450886060b5bf736b935cd0bf960837f \ -i=any \ -s=any \ -u=salt:https://{salt-master-ip}:8000/run \ -k=salt:<salt_token> \ -u=vsphere:on \ -u=aws:on \ -u=k8s:on \ -u=salt-env-path:/opt/miniconda3/envs/saltenv/master \ --login-auth-type=basic \ --password-policy='(?=.*[0-9]{1,50})(?=.*[~`!@#$%\\^&*()-+=]{1,50})(?=.*[a-zA-Z]{2,50}).{8,50}$' \ --password-policy-message='Enter at least 8 digits using at least 1 number and at least 1 special sentence, and at least 2 English characters.' \ --mail-subject='[Notifiy] $user_id Password is reset' \ --mail-body-message='Reset OTP for $user_id is '$user_pw'.<br>Do not forget this!<br>This OTP will use just once when login into cloudhub.<br>Then you must set up the new password' \ --kapacitor-url=http://{Kapacitor_IP}:9094 \ --influxdb-url=http://{InfluxDB_IP}:8086 \ --retry-policy=count:3 \ --retry-policy=delaytime:5 \ --retry-policy=type:delay"

Run

$ sudo systemctl {start|stop|restart} cloudhub

Nginx 설치

yum 외부 저장소 추가

$ cd /etc/yum.repos.d/
$ ls
CentOS-Base.repo  CentOS-CR.repo  CentOS-Debuginfo.repo  CentOS-Media.repo  CentOS-Sources.repo  CentOS-Vault.repo  CentOS-fasttrack.repo  microsoft-prod.repo

/etc/yum.repos.d/ 경로에 nginx.repo 파일 추가

$ vim /etc/yum.repos.d/nginx.repo

nginx.repo 파일

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
enabled=1

yum install 명령어를 이용해서 설치합니다.

$ yum install -y nginx

$ semanage port -a -t http_port_t -p tcp 443
$ semanage port -a -t http_port_t -p tcp 8000
$ semanage port -a -t http_port_t -p tcp 8888
$ semanage port -a -t http_port_t -p tcp 8887

$ semanage port -m -t http_port_t -p tcp xxxx

Nginx 서버 설정

$ vim /etc/nginx/conf.d/default.conf

upstream app {
    server {Cloudhub-Portal-ip}:8888;
}

upstream salt {
    server {salt-master-ip}:8887;
}

server {
    listen 8000 ssl http2;
    server_name _;
    ssl_certificate /usr/lib/cloudhub/key/cloudhub_self_signed.crt;
    ssl_certificate_key /usr/lib/cloudhub/key/cloudhub_self_signed.key;

    location / {
       proxy_pass https://salt;
       proxy_set_header Host $host;
       proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       proxy_set_header X-Forwarded-Proto $scheme;
     }
}

server {
        listen  443 ssl http2 ;
        server_name _;
        #access_log /var/log/nginx/proxy/access.log;
        #error_log /var/log/nginx/proxy/error.log;
        ssl_certificate /usr/lib/cloudhub/key/cloudhub_self_signed.crt;
        ssl_certificate_key /usr/lib/cloudhub/key/cloudhub_self_signed.key;

        location / {
               # First attempt to serve request as file, then
               # as directory, then fall back to displaying a 404.
               proxy_pass https://app;
               proxy_set_header Host $host;
               proxy_set_header X-Real-IP $remote_addr;
               proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
               proxy_set_header X-Forwarded-Proto $scheme;
        }

        location /cloudhub/v1/WebTerminalHandler {
               proxy_http_version 1.1;
               proxy_pass https://app;
               proxy_set_header       Upgrade $http_upgrade;
               proxy_set_header       Connection "upgrade";
               proxy_set_header Host $host;
               proxy_read_timeout 28800s;
               proxy_send_timeout 28800s;
       }
}

Nginx start

$ sudo systemctl start nginx
$ sudo systemctl enable nginx

CloudHub Portal UI 초기 설정

Steps

1. 로그인 창에서 Sign up(가입) 합니다.

2. 로그인 후, 아래와 같이 초기 설정에서 단계 별로 적절한 값으로 설정합니다.

   Image Modified

   초기 설정으로는 InfluxDB가 설치된 host IP 정도만 맞게 설정하면 됩니다.
   Connection Name는 Database(=Group) Name과 동일하면 편리합니다.

   Image Modified

   Add Connection → Next를 반복 클릭하여 마무리 합니다.

3. 아래 메뉴 화면에서 원하는 Group(Organization)을 추가합니다.

   Image Modified

4. “Agent Configuration > Minions” 메뉴 화면에서 새로 등록된 minion을 Operation 버튼을 통해 “Accept”합니다.

   Image Modified

5. “Agent Configuration > Collector Control”으로 이동 후, 설치하고자 하는 minion host를 선택하고 콤보박스에서 telegraf를 선택 후, INSTALL 합니다.

   Image Modified

6. Group을 지정한 후, [[outputs.influxdb]]을 설정한 후, Test 후, Apply 합니다.

   Image Modified

   그 외 필요한 설정은

...

1. Telegraf Configuration 항목을 참고 합니다.
   또한, Plugins 목록의 “?” 버튼을 클릭하면 해당 Plugin의 기본 Configuration 방법에 대한 help 창이 출력 됩니다. Copy & Paste를 통해 편리하게 Configuration을 편집할 수 있습니다.

주의 사항

위 설정까지 완료하고 나면, InfluxDB에 동적으로 Database가 생성됩니다.