사전 점검

Required Hardware Spec.

OS

• CentOS 7 x86_64 기준

CloudHub에서 사용되는 Port 및 방화벽 설정

개념도

...

External Ports: Internet 구간

• 443: CloudHub Portal에서 사용하는 웹 포트.

Internal Ports (Backend Server 간): Trusted Network 구간

• 8000 : Salt-API Server - CH Server와 데이터 송수신.

• 9094 : Kapacitor - InfluxDB와 데이터 송수신.

Intermediate Ports (Agent ↔︎ Backend Server 간): Semi-Trusted Network 구간

• 4505-4506 : Salt-Master - Salt-Minion과 데이터 송수신.

• 8086 : InfluxDB - telegraf로부터 데이터 수신.

아래는 firewalld 설정 파일 예시이며,

<!-- /etc/firewalld/zones/public.xml -->
<?xml version="1.0" encoding="utf-8"?>
<zone>
  <short>Public</short>
  <description>For use in public areas.</description>
  <service name="ssh"/>
  <service name="dhcpv6-client"/>
  <port protocol="tcp" port="443"/>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.0/24"/>
    <port protocol="tcp" port="4505-4506"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="8000"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="8086"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="9094"/>
    <accept/>
  </rule>
</zone>

기 설치된 Docker 버전 확인 및 이전 버전 삭제

• 버전 : Docker version 18.09.4 이상

  Code Block
  language bash
  $ docker --version Docker version 18.09.4, build d14af54266

• 버전이 낮으면 이전 버전을 삭제 후 설치

  • 이전 버전 삭제

    Code Block
    language bash
    $ yum remove docker \ docker-client \ docker-client-latest \ docker-common \ docker-latest \ docker-latest-logrotate \ docker-logrotate \ docker-engine

의존성 설치

Docker 설치

...

사전 점검

Required Hardware Spec.

OS

• CentOS 7 x86_64 기준

CloudHub에서 사용되는 Port 및 방화벽 설정

개념도

...

External Ports: Internet 구간

• 443: CloudHub Portal에서 사용하는 웹 포트.

Internal Ports (Backend Server 간): Trusted Network 구간

• 8000 : Salt-API Server - CH Server와 데이터 송수신.

• 9094 : Kapacitor - InfluxDB와 데이터 송수신.

Intermediate Ports (Agent ↔︎ Backend Server 간): Semi-Trusted Network 구간

• 4505-4506 : Salt-Master - Salt-Minion과 데이터 송수신.

• 8086 : InfluxDB - telegraf로부터 데이터 수신.

아래는 firewalld 설정 파일 예시이며,

<!-- /etc/firewalld/zones/public.xml -->
<?xml version="1.0" encoding="utf-8"?>
<zone>
  <short>Public</short>
  <description>For use in public areas.</description>
  <service name="ssh"/>
  <service name="dhcpv6-client"/>
  <port protocol="tcp" port="443"/>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.0/24"/>
    <port protocol="tcp" port="4505-4506"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="8000"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="8086"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="9094"/>
    <accept/>
  </rule>
</zone>

기 설치된 Docker 버전 확인 및 이전 버전 삭제

• 버전 : Docker version 18.09.4 이상

  Code Block
  language bash
  $ docker --version Docker version 18.09.4, build d14af54266

• 버전이 낮으면 이전 버전을 삭제 후 설치

  • 이전 버전 삭제

    Code Block
    language bash
    $ yum remove

  install

  • docker

  -y

  • \

  yum-utils
  \
  device-mapper-persistent-data \

  • docker-client \

  lvm2
  $
  yum

  • docker-

  config

  • client-

  manager

  • latest \

  --add-repo
  \
  https://download.docker.com/linux/centos/docker-ce.repo

  Install Docker Engine - Community

  Code Block

  language	bash

  $
  yum
  install

  • docker-

  ce docker-ce-cli containerd.io

Docker Compose 설치

• 버전 : Docker Compose version 1.24 이상

$ docker-compose --version
docker-compose version 1.24.0, build 0aa59064

• 버전이 낮거나 미 설치 시 Docker Compose [재]설치

  • 파일 다운로드

    Code Block
    language bash
    $ rm -f /usr/local/bin/docker-compose $ curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

  • 실행 권한 부여

    Code Block
    language bash
    $ sudo chmod +x /usr/local/bin/docker-compose

  • 버전 확인

    common \ docker-latest \ docker-latest-logrotate \ docker-logrotate \ docker-engine

의존성 설치

Docker 설치

• 저장소 설정 및 추가

  Code Block
  language bash
  $

  docker-compose --version docker-compose version 1.29.2, build 5becea4c

Docker log rotation 설정

• Log 설정 파일 : $ vim /etc/logrotate.d/docker-container

  Code Block
  language yaml
  /var/lib/docker/containers/*/*.log { daily rotate 7 missingok dateext compress notifempty }

...

• yum install -y yum-utils \ device-mapper-persistent-data \ lvm2 $ yum-config-manager \ --add-repo \ https://download.docker.com/linux/centos/docker-ce.repo

• Install Docker Engine - Community

  Code Block
  language bash
  $

...

• yum

...

• install docker

...

• -ce docker

...

Git 설치

$ yum install -y git

CloudHub Components 설치

Snetsystems/Sandbox 구동

...

• -ce-cli containerd.io

Docker Compose 설치

• 버전 : Docker Compose version 1.24 이상

$ docker-compose --version
docker-compose version 1.24.0, build 0aa59064

• 버전이 낮거나 미 설치 시 Docker Compose [재]설치

  • 파일 다운로드

    Code Block
    language bash
    $ rm -f /usr/local/bin/docker-compose $ curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

  • 실행 권한 부여

    Code Block
    language bash
    $ sudo chmod +x /usr/local/bin/docker-compose

  • 버전 확인

    Code Block
    language bash
    $ docker-compose --version docker-compose version 1.29.2, build 5becea4c

Docker log rotation 설정

• Log 설정 파일 : $ vim /etc/logrotate.d/docker-container

  Code Block
  language yaml
  /var/lib/docker/containers/*/*.log { daily rotate 7 missingok dateext compress copytruncate notifempty }

Docker 서비스 실행

$ systemctl cdenable {Sandboxdocker
다운로드$ 할systemctl 디렉토리start 이동}docker
$ systemctl status gitdocker
clone https://github.com/snetsystems/sandbox.git

# Server Single node mode로 구성하기 위해, 아래에서 etcd: 이하 configuration 부분 주석 또는 삭제.
$ vim sandbox/docker-compose.yml
...(생략)
#etcd:
#    # Full tag list: quay.io/coreos/etcd:v3.3.11
#    #image: gcr.io/etcd-development/etcd:${ETCD_TAG}
#    image: quay.io/coreos/etcd:${ETCD_TAG}
#    entrypoint: /usr/local/bin/etcd
...(생략)

Sandbox 실행

$ cd sandbox
$ ./sandbox up

$ docker-compose ps
/usr/lib/python2.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.24.3) or chardet (2.2.1) doesn't match a supported version!
  RequestsDependencyWarning)
       Name                    Command            State                                                  Ports
-------------------------------------------------------------------------------------------------● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2020-03-09 05:30:36 EDT; 3s ago
     Docs: https://docs.docker.com
 Main PID: 2036 (dockerd)
    Tasks: 8
   Memory: 42.1M
   CGroup: /system.slice/docker.service
           └─2036 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

...
...
Hint: Some lines were ellipsized, use -l to show in full.

Git 설치

$ yum install -y git

CloudHub Components 설치

Snetsystems/Sandbox 구동

Clone Snetsystems/Sandbox

$ cd {Sandbox 다운로드 할 디렉토리 이동}
$ git clone https://github.com/snetsystems/sandbox.git

# Server Single node mode로 구성하기 위해, 아래에서 etcd: 이하 configuration 부분 주석 또는 삭제.
$ vim sandbox/docker-compose.yml
...(생략)
#etcd:
#    # Full tag list: quay.io/coreos/etcd:v3.3.11
#    #image: gcr.io/etcd-development/etcd:${ETCD_TAG}
#    image: quay.io/coreos/etcd:${ETCD_TAG}
#    entrypoint: /usr/local/bin/etcd
...(생략)

Sandbox 실행

$ cd sandbox
$ ./sandbox up

$ docker-compose ps
/usr/lib/python2.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.24.3) or chardet (2.2.1) doesn't match a supported version!
  RequestsDependencyWarning)
       Name                    Command            State                                                  Ports
----------------------------------------------------------------------------------------------------
sandbox_influxdb_1    /entrypoint.sh influxd      Up      0.0.0.0:8082->8082/tcp,:::8082->8082/tcp, 0.0.0.0:8086->8086/tcp,:::8086->8086/tcp,
                                                          0.0.0.0:8089->8089/udp,:::8089->8089/udp
sandbox_kapacitor_1   /entrypoint.sh kapacitord   Up      0.0.0.0:9094->9092/tcp,:::9094->9092/tcp

SaltStack 설치([On/Off]line 설치 겸용)

Predefinitions

설명의 편의를 위해, 아래와 같이 사전 정의된 경로를 사용하는 것을 전제로 합니다.

...

Conda Installed Path

...

/opt/miniconda3

Conda 설치

위 첨부 파일에서 Miniconda3 배포 파일 다운로드 후,

아래 명령으로 적절히 용량이 큰 디스크 경로에 설치 권장.

chmod 755 Miniconda3-latest-Linux-x86_64.sh # 쉘 실행 권한 부여
./Miniconda3-latest-Linux-x86_64.sh -u -p /opt/miniconda3 # /opt/miniconda3에 설치

conda 환경 변수 등록

conda 환경 변수 설정을 위해 conda init 명령어를 입력해야 합니다.

$ /opt/miniconda3/bin/conda init

conda 기본 환경을 비활성화

$ conda config --set auto_activate_base false

Download from Github release page

https://github.com/snetsystems/salt/releases 에서 원하는 버전의 인스톨 파일들을 다운로드 합니다.
여기서는 “3001.4-6-ge56e905”버전으로 인스톨 할 것이며, 관련 파일들을 적당한 경로에 다운로드 합니다.

1. Conda 가상환경 압축 파일: 예> saltenv-3001.4-6-ge56e905-linux.tar.gz

2. install_salt_svc_for_conda_centos7.sh

saltenv unpack

/opt/miniconda3/envs/에 saltenv-3001.4-6-ge56e905-linux.tar.gz를 압축 해제합니다.

그리고 나서, conda env list로 확인하면 정상 등록되어 있을 것 입니다.

$ tar xzf saltenv-3001.4-6-ge56e905-linux.tar.gz -C /opt/miniconda3/envs/
$ pwd
/opt/miniconda3/envs
$ ll
total 219044
drwxr-xr-x. 13 root root       195 May 16 11:08 saltenv

$ conda env list
# conda environments:
#
base                  *  /opt/miniconda3
saltenv                  /opt/miniconda3/envs/saltenv

Config 설정

Salt-Master와 Salt-API의 Config 설정.

Config path : $ vim /opt/miniconda3/envs/saltenv/etc/salt/master

-------------------------------------------------------------
sandbox_influxdb_1    /entrypoint.sh influxd      Up      0.0.0.0:8082->8082/tcp,:::8082->8082/tcp, 0.0.0.0:8086->8086/tcp,:::8086->8086/tcp,
                                                          0.0.0.0:8089->8089/udp,:::8089->8089/udp
sandbox_kapacitor_1   /entrypoint.sh kapacitord   Up      0.0.0.0:9094->9092/tcp,:::9094->9092/tcp

Sandbox 서비스 파일 생성 및 활성화

$ cd <Sandbox 다운로드 받은 디렉토리>
$ ./sandbox install <Working Directory>

# 서비스 파일 확인
$ cat /etc/systemd/system/snet-sandbox.service

# 서비스 Enabled 상태 확인
$ systemctl status snet-sandbox

SaltStack 설치([On/Off]line 설치 겸용)

Predefinitions

설명의 편의를 위해, 아래와 같이 사전 정의된 경로를 사용하는 것을 전제로 합니다.

Conda 설치

위 첨부 파일에서 Miniconda3 배포 파일 다운로드 후,

아래 명령으로 적절히 용량이 큰 디스크 경로에 설치 권장.

$ wget https://seversky.atlassian.net/wiki/download/attachments/1883635713/Miniconda3-latest-Linux-x86_64.sh
$ chmod 755 Miniconda3-latest-Linux-x86_64.sh # 쉘 실행 권한 부여
$ ./Miniconda3-latest-Linux-x86_64.sh -b -u -p /opt/miniconda3 # /opt/miniconda3에 설치

conda 환경 변수 등록

conda 환경 변수 설정을 위해 conda init 명령어를 입력해야 합니다.

$ export PATH=/opt/miniconda3/bin:$PATH
$ conda init

conda 기본 환경을 비활성화

$ conda config --set auto_activate_base false

Download from Github release page

https://github.com/snetsystems/salt/releases 에서 원하는 버전의 인스톨 파일들을 다운로드 합니다.
여기서는 “3001.4-15-g1d946bf”버전으로 인스톨 할 것이며, 관련 파일들을 적당한 경로에 다운로드 합니다.

1. Conda 가상환경 압축 파일: 예> saltenv-3001.4-15-g1d946bf-linux.tar.gz

2. install_salt_svc_for_conda_systemd.sh

saltenv unpack

/opt/miniconda3/envs/에 saltenv-3001.4-15-g1d946bf-linux.tar.gz를 압축 해제합니다.

그리고 나서, conda env list로 확인하면 정상 등록되어 있을 것 입니다.

$ tar xzf saltenv-3001.4-15-g1d946bf-linux.tar.gz -C /opt/miniconda3/envs/
$ pwd
/opt/miniconda3/envs
$ ll
total 219044
drwxr-xr-x. 13 root root       195 May 16 11:08 saltenv

$ conda env list
# conda environments:
#
base                  *  /opt/miniconda3
saltenv                  /opt/miniconda3/envs/saltenv

Config 설정

Salt-Master와 Salt-API의 Config 설정

Config path :

$ mkdir -p /opt/miniconda3/envs/saltenv/master/etc/salt && vim /opt/miniconda3/envs/saltenv/master/etc/salt/master

## log_level ##
# One of 'garbage', 'trace', 'debug', info', 'warning'(default), 'error', 'critical'.
log_level: info
root_dir: /opt/miniconda3/envs/saltenv/master

##### Primary configuration settings #####
##########################################
# The address of the interface to bind to:
interface: x.x.x.x
 
# The tcp port used by the publisher:
publish_port: 4505
 
# Allow minions to push files to the master. This is disabled by default, for
# security purposes.
# file_recv: True
 
#####      State System settings     #####
##########################################
# The state system uses a "top" file to tell the minions what environment to
# use and what modules to use. The state_top file is defined relative to the
# root of the base environment as defined in "File Server settings" below.
state_top: top.sls
 
#####      File Server settings      #####
##########################################
# Salt runs a lightweight file server written in zeromq to deliver files to
# minions. This file server is built into the master daemon and does not
# require a dedicated port.
 
# The file server works on environments passed to the master, each environment
# can have multiple root directories, the subdirectories in the multiple file
# roots cannot match, otherwise the downloaded files will not be able to be
# reliably ensured. A base environment is required to house the top file.
file_roots:
  base:
    - /srv/salt/prod
 
# File Server Backend
#
# Salt supports a modular fileserver backend system, this system allows
# the salt master to link directly to third party systems to gather and
# manage the files available to minions. Multiple backends can be
# configured and will be searched for the requested file in the order in which
# they are defined here. The default setting only enables the standard backend
# "roots" which uses the "file_roots" option.
fileserver_backend:
  - roots
 
#####        Security settings       #####
##########################################
# The external auth system uses the Salt auth modules to authenticate and
# validate users to access areas of the Salt system.
external_auth:
  pam:
    saltenv:
      - .*
      - '@runner'
      - '@wheel'
      - '@jobs'
 
# Allow eauth users to specify the expiry time of the tokens they generate.
# A boolean applies to all users or a dictionary of whitelisted eauth backends
# and usernames may be given.
token_expire_user_override:
  pam:
    - saltenv

#####        API Server settings     #####
##########################################
rest_cherrypy:
  port: 8887
  ssl_crt: /usr/lib/cloudhub/key/cloudhub_self_signed.crt
  ssl_key: /usr/lib/cloudhub/key/cloudhub_self_signed.key

rest_cherrypy_ssl: True

Salt-Minion 설정

Config path:

$ mkdir -p /opt/miniconda3/envs/saltenv/minion/etc/salt && vim /opt/miniconda3/envs/saltenv/minion/etc/salt/minion

## log_level ##
# One of 'garbage', 'trace', 'debug', info', 'warning'(default), 'error', 'critical'.
log_level: info
root_dir: /opt/miniconda3/envs/saltenv

##### Primary configuration settings #####
##########################################
# The address of the interface to bind to:
interface: x.x.x.x
 
# The tcp port used by the publisher:
publish_port: 4505
 
# Allow minions to push files to the master. This is disabled by default, for
# security purposes.
# file_recv: True
 
#####      State System settings     #####
##########################################
# The state system uses a "top" file to tell the minions what environment to
# use and what modules to use. The state_top file is defined relative to the
# root of the base environment as defined in "File Server settings" below.
state_top: top.sls
 
#####      File Server settings      #####
##########################################
# Salt runs a lightweight file server written in zeromq to deliver files to
# minions. This file server is built into the master daemon and does not
# require a dedicated port.
 
# The file server works on environments passed to the master, each environment
# can have multiple root directories, the subdirectories in the multiple file
# roots cannot match, otherwise the downloaded files will not be able to be
# reliably ensured. A base environment is required to house the top file.
file_roots:
  base:
    - /srv/salt/prod
 
# File Server Backend
#
# Salt supports a modular fileserver backend system, this system allows
# the salt master to link directly to third party systems to gather and
# manage the files available to minions. Multiple backends can be
# configured and will be searched for the requested file in the order in which
# they are defined here. The default setting only enables the standard backend
# "roots" which uses the "file_roots" option.
fileserver_backend:
  - roots
 
#####        Security settings       #####
##########################################
# The external auth system uses the Salt auth modules to authenticate and
# validate users to access areas of the Salt system.
external_auth:
  pam:
    saltdev:
      - .*
      - '@runner'
      - '@wheel'
      - '@jobs'
 
# Allow eauth users to specify the expiry time of the tokens they generate.
# A boolean applies to all users or a dictionary of whitelisted eauth backends
# and usernames may be given.
token_expire_user_override:
  pam:
    - saltdev

#####        API Server settings     #####
##########################################
rest_cherrypy:
  port: 8000
  disable_ssl: True

Salt-Minion 설정

Config path: $ vim /opt/miniconda3/envs/saltenv/etc/salt/minion

## log_level ##
# One of 'garbage', 'trace', 'debug', info', 'warning'(default), 'error', 'critical'.
log_level: info
root_dir: /opt/miniconda3/envs/saltenv
master: <master ip address>
id: <유일한 minion id 설정해야 하며, 생략 시, hostname = minion id>

계정 생성 및 패스워드 설정

$ useradd saltdev
$ passwd <password>
Changing password for user saltdev.
New password:

Register & Start Salt Services

$ ./install_salt_svc_for_conda_centos7.sh를 실행하면 아래 예제와 같이 서비스가 설치되며, 자동 실행됩니다.

# install_salt_svc_for_conda_centos7.sh 사용법
$ ./install_salt_svc_for_conda_centos7.sh -h

usage: ./install_salt_svc_for_conda_centos7.sh [options]

Install SNET salt services

-p          Set installed miniconda prefix path (default: /opt/miniconda3/envs/saltenv

Install Master + Minion + API Service

$ ./install_salt_svc_for_conda_centos7.sh
Do you want to install salt-minion only? [y/n]n
Do you want to start whole 'snet-salt' services? [y/n]y

• 서비스 명: snet-salt-master.service, snet-salt-minion.service, snet-salt-api.service

• Logrotate 설정.

  Code Block
  # ll /etc/logrotate.d/snet-salt -rw-r--r--. 1 root root 592 May 16 12:39 /etc/logrotate.d/snet-salt

Install Minion Service Only

$ ./install_salt_svc_for_conda_centos7.sh
Do you want to install salt-minion only? [y/n]y
Do you want to start the 'snet-salt-minion' service? [y/n]y

• 서비스 명: snet-salt-minion.service

• Logrotate 설정.

  Code Block
  # ll /etc/logrotate.d/snet-salt -rw-r--r--. 1 root root 592 May 16 12:39 /etc/logrotate.d/snet-salt

Download telegraf into salt file_roots path on salt-master

아래 링크에서 설치하고자 하는 버전을 다운로드 할 수 있습니다.
Download URL: https://github.com/snetsystems/telegraf/releases

아래 예제와 같이 지정된 경로에 원하는 버전의 rpm 파일을 다운로드 합니다.

$ mkdir -p /srv/salt/prod/telegraf          # Telegraf package path
$ cd /srv/salt/prod/telegraf
$ wget https://github.com/snetsystems/telegraf/releases/download/v1.19.3-snet/telegraf-1.19.3-snet-1_x86_64.rpm

Install CloudHub Portal

Download

아래 링크에서 설치하고자 하는 버전을 다운로드 할 수 있습니다.

Download URL: https://github.com/snetsystems/cloudhub/releases

Install

$ yum localinstall cloudhub-0.9.1~17fe47a.x86_64.rpm

Configure

1. $ vim /etc/systemd/system/multi-user.target.wants/cloudhub.service

   Code Block
   language yaml
   [Unit] After=network-online.target [Service] User=root Group=root Environment="HOST=0.0.0.0" Environment="PORT=443" # default: 8888 Environment="TLS_CERTIFICATE=/usr/lib/cloudhub/key/cloudhub_self_signed.pem" #Environment="TLS_PRIVATE_KEY=my.key" Environment="BOLT_PATH=/var/lib/cloudhub/cloudhub-v1.db" #Environment="ETCD_ENDPOINTS={ETCD_CLIENT_IP}:2379" Environment="CANNED_PATH=/usr/share/cloudhub/cloudhub-canned" Environment="PROTOBOARDS_PATH=/usr/share/cloudhub/cloudhub-protoboards" EnvironmentFile=-/etc/default/cloudhub ExecStart=/usr/bin/cloudhub $CLOUDHUB_OPTS KillMode=control-group Restart=on-failure [Install] WantedBy=multi-user.target

2. localhost 인증서(cloudhub_self_signed.pem) 발급

   Code Block
   $ cd /usr/lib/cloudhub/key/ $ openssl req -x509 -newkey rsa:4096 -sha256 -nodes -keyout cloudhub_self_signed.pem -out cloudhub_self_signed.pem -subj "/CN=localhost" -days 365

3. <salt_token> 발급
   /wiki/spaces/CM/pages/254050538
   

   Status
   colour Purple title 노트

   위 페이지 접근 권한이 없는 경우에는 Snetsystems CloudHub 팀에 문의/발급 받은 후,
   아래 4번 -k=salt:<salt_token>에 기입하세요.

4. $ vim /etc/default/cloudhub

   Code Block
   language yaml
   CLOUDHUB_OPTS="-l=debug \ --auth-duration=0 \ -t=74c1e9e2450886060b5bf736b935cd0bf960837f \ -i=any \ -s=any \ -u=salt:http://{salt-master-ip}:8000/run \ -k=salt:<salt_token> \ -u=vsphere:on \ -u=aws:on \ -u=k8s:on \ --login-auth-type=basic \ --password-policy='(?=.*[0-9]{1,50})(?=.*[~`!@#$%\\^&*()-+=]{1,50})(?=.*[a-zA-Z]{2,50}).{8,50}$' \ --password-policy-message='Enter at least 8 digits using at least 1 number and at least 1 special sentence, and at least 2 English characters.' \ --mail-subject='[Notifiy] $user_id Password is reset' \ --mail-body-message='Reset OTP for $user_id is '$user_pw'.<br>Do not forget this!<br>This OTP will use just once when login into cloudhub.<br>Then you must set up the new password' \ --kapacitor-url=http://{Kapacitor_IP}:9094 \ --influxdb-url=http://{InfluxDB_IP}:8086 \ --retry-policy=count:3 \ --retry-policy=delaytime:5 \ --retry-policy=type:delay"

Run

...

/minion
master: <master ip address>
id: <유일한 minion id 설정해야 하며, 생략 시, hostname = minion id>

계정 생성 및 패스워드 설정

$ useradd saltenv
$ passwd saltenv
Changing password for user saltenv.
New password:

Register & Start Salt Services

$ ./install_salt_svc_for_conda_systemd.sh를 실행하면 아래 예제와 같이 서비스가 설치되며, 자동 실행됩니다.

# install_salt_svc_for_conda_systemd.sh 사용법
$ ./install_salt_svc_for_conda_systemd.sh -h

usage: ./install_salt_svc_for_conda_systemd.sh [options]

Install SNET salt services

-p          Set installed miniconda prefix path (default: /opt/miniconda3/envs/saltenv

Install Master + Minion + API Service

$ ./install_salt_svc_for_conda_systemd.sh
Do you want to install salt-minion only? [y/n]n
Do you want to start whole 'snet-salt' services? [y/n]y

• 서비스 명: snet-salt-master.service, snet-salt-minion.service, snet-salt-api.service

• Logrotate 설정.

  Code Block
  # ll /etc/logrotate.d/snet-salt -rw-r--r--. 1 root root 592 May 16 12:39 /etc/logrotate.d/snet-salt

Install Minion Service Only

$ ./install_salt_svc_for_conda_systemd.sh
Do you want to install salt-minion only? [y/n]y
Do you want to start the 'snet-salt-minion' service? [y/n]y

• 서비스 명: snet-salt-minion.service

• Logrotate 설정.

  Code Block
  # ll /etc/logrotate.d/snet-salt -rw-r--r--. 1 root root 592 May 16 12:39 /etc/logrotate.d/snet-salt

Download telegraf into salt file_roots path on salt-master

아래 링크에서 설치하고자 하는 버전을 다운로드 할 수 있습니다.
Download URL: https://github.com/snetsystems/telegraf/releases

아래 예제와 같이 지정된 경로에 원하는 버전의 rpm 파일을 다운로드 합니다.

$ mkdir -p /srv/salt/prod/telegraf          # Telegraf package path
$ cd /srv/salt/prod/telegraf
$ wget https://github.com/snetsystems/telegraf/releases/download/v1.23.4-snet-1/telegraf-1.23.4-snet-1-1.x86_64.rpm

Install CloudHub Portal

Download

아래 링크에서 설치하고자 하는 버전을 다운로드 할 수 있습니다.

Download URL: https://github.com/snetsystems/cloudhub/releases

Install

$ yum localinstall CloudHub-1.4.2.520d7bd.x86_64.rpm

Configure

1. $ vim /etc/systemd/system/multi-user.target.wants/cloudhub.service

   Code Block
   language yaml
   [Unit] After=network-online.target [Service] User=root Group=root Environment="HOST=0.0.0.0" Environment="PORT=8888" Environment="TLS_CERTIFICATE=/usr/lib/cloudhub/key/cloudhub_self_signed.pem" #Environment="TLS_PRIVATE_KEY=my.key" Environment="BOLT_PATH=/var/lib/cloudhub/cloudhub-v1.db" #Environment="ETCD_ENDPOINTS={ETCD_CLIENT_IP}:2379" Environment="CANNED_PATH=/usr/share/cloudhub/cloudhub-canned" Environment="PROTOBOARDS_PATH=/usr/share/cloudhub/cloudhub-protoboards" EnvironmentFile=-/etc/default/cloudhub ExecStart=/usr/bin/cloudhub $CLOUDHUB_OPTS KillMode=control-group Restart=on-failure [Install] WantedBy=multi-user.target

2. localhost 인증서(cloudhub_self_signed.pem) 발급

   Code Block
   $ cd /usr/lib/cloudhub/key/ $ openssl req -x509 -newkey rsa:4096 -sha256 -nodes -keyout cloudhub_self_signed.pem -out cloudhub_self_signed.pem -subj "/CN=localhost" -days 365 $ openssl rsa -in cloudhub_self_signed.pem -text > cloudhub_self_signed.key $ openssl x509 -inform PEM -in cloudhub_self_signed.pem -out cloudhub_self_signed.crt

3. <salt_token> 발급
   /wiki/spaces/CM/pages/254050538
   

   Status
   colour Purple title 노트

   위 페이지 접근 권한이 없는 경우에는 Snetsystems CloudHub 팀에 문의/발급 받은 후,
   아래 4번 -k=salt:<salt_token>에 기입하세요.

4. $ vim /etc/default/cloudhub

   Code Block
   language yaml
   CLOUDHUB_OPTS="-l=debug \ --auth-duration=0 \ -t=74c1e9e2450886060b5bf736b935cd0bf960837f \ -i=any \ -s=any \ -u=salt:https://{salt-master-ip}:8000/run \ -k=salt:<salt_token> \ -u=vsphere:on \ -u=aws:on \ -u=k8s:on \ -u=salt-env-path:/opt/miniconda3/envs/saltenv/master \ --login-auth-type=basic \ --password-policy='(?=.*[0-9]{1,50})(?=.*[~`!@#$%\\^&*()-+=]{1,50})(?=.*[a-zA-Z]{2,50}).{8,50}$' \ --password-policy-message='Enter at least 8 digits using at least 1 number and at least 1 special sentence, and at least 2 English characters.' \ --mail-subject='[Notifiy] $user_id Password is reset' \ --mail-body-message='Reset OTP for $user_id is '$user_pw'.<br>Do not forget this!<br>This OTP will use just once when login into cloudhub.<br>Then you must set up the new password' \ --kapacitor-url=http://{Kapacitor_IP}:9094 \ --influxdb-url=http://{InfluxDB_IP}:8086 \ --retry-policy=count:3 \ --retry-policy=delaytime:5 \ --retry-policy=type:delay"

Run

$ sudo systemctl {start|stop|restart} cloudhub

Nginx 설치

yum 외부 저장소 추가

$ cd /etc/yum.repos.d/
$ ls
CentOS-Base.repo  CentOS-CR.repo  CentOS-Debuginfo.repo  CentOS-Media.repo  CentOS-Sources.repo  CentOS-Vault.repo  CentOS-fasttrack.repo  microsoft-prod.repo

/etc/yum.repos.d/ 경로에 nginx.repo 파일 추가

$ vim /etc/yum.repos.d/nginx.repo

nginx.repo 파일

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
enabled=1

yum install 명령어를 이용해서 설치합니다.

$ yum install -y nginx

$ semanage port -a -t http_port_t -p tcp 443
$ semanage port -a -t http_port_t -p tcp 8000
$ semanage port -a -t http_port_t -p tcp 8888
$ semanage port -a -t http_port_t -p tcp 8887

$ semanage port -m -t http_port_t -p tcp xxxx

Nginx 서버 설정

$ vim /etc/nginx/conf.d/default.conf

upstream app {
    server {Cloudhub-Portal-ip}:8888;
}

upstream salt {
    server {salt-master-ip}:8887;
}

server {
    listen 8000 ssl http2;
    server_name _;
    ssl_certificate /usr/lib/cloudhub/key/cloudhub_self_signed.crt;
    ssl_certificate_key /usr/lib/cloudhub/key/cloudhub_self_signed.key;

    location / {
       proxy_pass https://salt;
       proxy_set_header Host $host;
       proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       proxy_set_header X-Forwarded-Proto $scheme;
     }
}

server {
        listen  443 ssl http2 ;
        server_name _;
        #access_log /var/log/nginx/proxy/access.log;
        #error_log /var/log/nginx/proxy/error.log;
        ssl_certificate /usr/lib/cloudhub/key/cloudhub_self_signed.crt;
        ssl_certificate_key /usr/lib/cloudhub/key/cloudhub_self_signed.key;

        location / {
               # First attempt to serve request as file, then
               # as directory, then fall back to displaying a 404.
               proxy_pass https://app;
               proxy_set_header Host $host;
               proxy_set_header X-Real-IP $remote_addr;
               proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
               proxy_set_header X-Forwarded-Proto $scheme;
        }

        location /cloudhub/v1/WebTerminalHandler {
               proxy_http_version 1.1;
               proxy_pass https://app;
               proxy_set_header       Upgrade $http_upgrade;
               proxy_set_header       Connection "upgrade";
               proxy_set_header Host $host;
               proxy_read_timeout 28800s;
               proxy_send_timeout 28800s;
       }
}

Nginx start

$ sudo systemctl start nginx
$ sudo systemctl enable nginx

CloudHub Portal UI 초기 설정

Steps

...