사전 점검

Required Hardware Spec.

OS

• CentOS 7 x86_64 기준

CloudHub에서 사용되는 Port 및 방화벽 설정

개념도

...

External Ports: Internet 구간

• 443: CloudHub Portal에서 사용하는 웹 포트.

Internal Ports (Backend Server 간): Trusted Network 구간

• 8000 : Salt-API Server - CH Server와 데이터 송수신.

• 9094 : Kapacitor - InfluxDB와 데이터 송수신.

Intermediate Ports (Agent ↔︎ Backend Server 간): Semi-Trusted Network 구간

• 4505-4506 : Salt-Master - Salt-Minion과 데이터 송수신.

• 8086 : InfluxDB - telegraf로부터 데이터 수신.

아래는 firewalld 설정 파일 예시이며,

...

...

사전 점검

Required Hardware Spec.

OS

• CentOS 7 x86_64 기준

CloudHub에서 사용되는 Port 및 방화벽 설정

개념도

...

External Ports: Internet 구간

• 443: CloudHub Portal에서 사용하는 웹 포트.

Internal Ports (Backend Server 간): Trusted Network 구간

• 8000 : Salt-API Server - CH Server와 데이터 송수신.

• 9094 : Kapacitor - InfluxDB와 데이터 송수신.

Intermediate Ports (Agent ↔︎ Backend Server 간): Semi-Trusted Network 구간

• 4505-4506 : Salt-Master - Salt-Minion과 데이터 송수신.

• 8086 : InfluxDB - telegraf로부터 데이터 수신.

아래는 firewalld 설정 파일 예시이며,

<!-- /etc/firewalld/zones/public.xml -->
<?xml version="1.0" encoding="utf-8"?>
<zone>
  <short>Public</short>
  <description>For use in public areas.</description>
  <service name="ssh"/>
  <service name="dhcpv6-client"/>
  <port protocol="tcp" port="443"/>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.0/24"/>
    <port protocol="tcp" port="4505-4506"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="8000"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="8086"/>
    <accept/>
  </rule>
  <rule family="ipv4">
    <source address="xxx.xxx.xxx.xxx"/>
    <port protocol="tcp" port="9094"/>
    <accept/>
  </rule>
</zone>

...

• Log 설정 파일 : $ vim /etc/logrotate.d/docker-container

  Code Block
  language yaml
  /var/lib/docker/containers/*/*.log { daily rotate 7 missingok dateext compress copytruncate notifempty }

Docker 서비스 실행

...

$ cd sandbox
$ ./sandbox up

$ docker-compose ps
/usr/lib/python2.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.24.3) or chardet (2.2.1) doesn't match a supported version!
  RequestsDependencyWarning)
       Name                    Command            State                                                  Ports
--------------------------------------------------------------------------------------------------------------------------------------------------------------
sandbox_influxdb_1    /entrypoint.sh influxd      Up      0.0.0.0:8082->8082/tcp,:::8082->8082/tcp, 0.0.0.0:8086->8086/tcp,:::8086->8086/tcp,
                                                          0.0.0.0:8089->8089/udp,:::8089->8089/udp
sandbox_kapacitor_1   /entrypoint.sh kapacitord       Up      0.0.0.0:80899094->8089>9092/udptcp,:::8089->8089/udp
sandbox_kapacitor_1   /entrypoint.sh kapacitord   Up      0.0.0.0:9094->9092/tcp,:::9094->9092/tcp:::9094->9092/tcp

Sandbox 서비스 파일 생성 및 활성화

$ cd <Sandbox 다운로드 받은 디렉토리>
$ ./sandbox install <Working Directory>

# 서비스 파일 확인
$ cat /etc/systemd/system/snet-sandbox.service

# 서비스 Enabled 상태 확인
$ systemctl status snet-sandbox

SaltStack 설치([On/Off]line 설치 겸용)

...

설명의 편의를 위해, 아래와 같이 사전 정의된 경로를 사용하는 것을 전제로 합니다.

Conda 설치

...

아래 명령으로 적절히 용량이 큰 디스크 경로에 설치 권장.

$ wget https://seversky.atlassian.net/wiki/download/attachments/1883635713/Miniconda3-latest-Linux-x86_64.sh
$ chmod 755 Miniconda3-latest-Linux-x86_64.sh # 쉘 실행 권한 부여
$ ./Miniconda3-latest-Linux-x86_64.sh -b -u -p /opt/miniconda3 # /opt/miniconda3에 설치

...

Config 설정

Salt-Master와 Salt-API의 Config 설정

...

Config path :

Code Block
$ mkdir -p /opt/miniconda3/envs/saltenv/master/etc/salt && vim /opt/miniconda3/envs/saltenv/master/etc/salt/master

Note
Host에 여러 Interface가 있을 경우 Salt-Minion과 연결 가능한 네트워크 Interface IP로 설정해야 합니다. ex) interface: 61.254.65.58 아래 예와 같이 any(0.0.0.0)로 설정해도 master ↔︎ minion 간 통신은 문제가 없으나, CloudHub UI에서 원하지 않는 IP로 표시될 수 있습니다. 자동 생성될 부산물 파일들을, 가상 환경 내로 격리하기 위해 root_dir을 명시해야 합니다. 아래 설정 파일에는 주석 포함하여 영문 외 2 Byte 문자(예, 한글, 일본어, 중국어 등)가 포함되지 않도록 주의합니다.

Code Block
language yaml
## log_level ## # One of 'garbage', 'trace', 'debug', info', 'warning'(default), 'error', 'critical'. log_level: info root_dir: /opt/miniconda3/envs/saltenv/master ##### Primary configuration settings ##### ########################################## # The address of the interface to bind to: interface: x.x.x.x # The tcp port used by the publisher: publish_port: 4505 # Allow minions to push files to the master. This is disabled by default, for # security purposes. # file_recv: True ##### State System settings ##### ########################################## # The state system uses a "top" file to tell the minions what environment to # use and what modules to use. The state_top file is defined relative to the # root of the base environment as defined in "File Server settings" below. state_top: top.sls ##### File Server settings ##### ########################################## # Salt runs a lightweight file server written in zeromq to deliver files to # minions. This file server is built into the master daemon and does not # require a dedicated port. # The file server works on environments passed to the master, each environment # can have multiple root directories, the subdirectories in the multiple file # roots cannot match, otherwise the downloaded files will not be able to be # reliably ensured. A base environment is required to house the top file. file_roots: base: - /srv/salt/prod # File Server Backend # # Salt supports a modular fileserver backend system, this system allows # the salt master to link directly to third party systems to gather and # manage the files available to minions. Multiple backends can be # configured and will be searched for the requested file in the order in which # they are defined here. The default setting only enables the standard backend # "roots" which uses the "file_roots" option. fileserver_backend: - roots ##### Security settings ##### ########################################## # The external auth system uses the Salt auth modules to authenticate and # validate users to access areas of the Salt system. external_auth: pam: saltenv: - .* - '@runner' - '@wheel' - '@jobs' # Allow eauth users to specify the expiry time of the tokens they generate. # A boolean applies to saltdev:all users or a dictionary of whitelisted -eauth .*backends # and usernames may be - '@runner' given. token_expire_user_override: pam: - '@wheel'saltenv ##### - '@jobs' API #Server Allowsettings eauth users to specify the##### expiry time of the tokens they generate. # A boolean applies to all users or a dictionary of whitelisted eauth backends # and usernames may be given. token_expire_user_override: pam: - saltdev ##### API Server settings ##### ########################################## rest_cherrypy: port: 8887 ssl_crt: /usr/lib/cloudhub/key/cloudhub_self_signed.crt ssl_key: /usr/lib/cloudhub/key/cloudhub_self_signed.key rest_cherrypy_ssl: True

...

########################################## rest_cherrypy: port: 8887 ssl_crt: /usr/lib/cloudhub/key/cloudhub_self_signed.crt ssl_key: /usr/lib/cloudhub/key/cloudhub_self_signed.key rest_cherrypy_ssl: True

Salt-Minion 설정

Config path:

$ mkdir -p /opt/miniconda3/envs/saltenv/minion/etc/salt && vim /opt/miniconda3/envs/saltenv/minion/etc/salt/minion

## log_level ##
# One of 'garbage', 'trace', 'debug', info', 'warning'(default), 'error', 'critical'.
log_level: info
root_dir: /opt/miniconda3/envs/saltenv/minion
master: <master ip address>
id: <유일한 minion id 설정해야 하며, 생략 시, hostname = minion id>

계정 생성 및 패스워드 설정

$ useradd saltdevsaltenv
$ passwd <password>saltenv
Changing password for user saltdevsaltenv.
New password:

Register & Start Salt Services

...

1. $ vim /etc/systemd/system/multi-user.target.wants/cloudhub.service

   Code Block
   language yaml
   [Unit] After=network-online.target [Service] User=root Group=root Environment="HOST=0.0.0.0" Environment="PORT=8888" Environment="TLS_CERTIFICATE=/usr/lib/cloudhub/key/cloudhub_self_signed.pem" #Environment="TLS_PRIVATE_KEY=my.key" Environment="BOLT_PATH=/var/lib/cloudhub/cloudhub-v1.db" #Environment="ETCD_ENDPOINTS={ETCD_CLIENT_IP}:2379" Environment="CANNED_PATH=/usr/share/cloudhub/cloudhub-canned" Environment="PROTOBOARDS_PATH=/usr/share/cloudhub/cloudhub-protoboards" EnvironmentFile=-/etc/default/cloudhub ExecStart=/usr/bin/cloudhub $CLOUDHUB_OPTS KillMode=control-group Restart=on-failure [Install] WantedBy=multi-user.target

2. localhost 인증서(cloudhub_self_signed.pem) 발급

   Code Block
   $ cd /usr/lib/cloudhub/key/ $ openssl req -x509 -newkey rsa:4096 -sha256 -nodes -keyout cloudhub_self_signed.pem -out cloudhub_self_signed.pem -subj "/CN=localhost" -days 365 $ openssl rsa -in cloudhub_self_signed.pem -text > cloudhub_self_signed.key $ openssl x509 -inform PEM -in cloudhub_self_signed.pem -out cloudhub_self_signed.crt

3. <salt_token> 발급
   /wiki/spaces/CM/pages/254050538
   

   Status
   colour Purple title 노트

   위 페이지 접근 권한이 없는 경우에는 Snetsystems CloudHub 팀에 문의/발급 받은 후,
   아래 4번 -k=salt:<salt_token>에 기입하세요.

4. $ vim /etc/default/cloudhub

   Code Block
   language yaml
   CLOUDHUB_OPTS="-l=debug \ --auth-duration=0 \ -t=74c1e9e2450886060b5bf736b935cd0bf960837f \ -i=any \ -s=any \ -u=salt:httphttps://{salt-master-ip}:8000/run \ -k=salt:<salt_token> \ -u=vsphere:on \ -u=aws:on \ -u=k8s:on \ -u=salt-env-path:/opt/miniconda3/envs/saltenv/master \ --login-auth-type=basic \ --password-policy='(?=.*[0-9]{1,50})(?=.*[~`!@#$%\\^&*()-+=]{1,50})(?=.*[a-zA-Z]{2,50}).{8,50}$' \ --password-policy-message='Enter at least 8 digits using at least 1 number and at least 1 special sentence, and at least 2 English characters.' \ --mail-subject='[Notifiy] $user_id Password is reset' \ --mail-body-message='Reset OTP for $user_id is '$user_pw'.<br>Do not forget this!<br>This OTP will use just once when login into cloudhub.<br>Then you must set up the new password' \ --kapacitor-url=http://{Kapacitor_IP}:9094 \ --influxdb-url=http://{InfluxDB_IP}:8086 \ --retry-policy=count:3 \ --retry-policy=delaytime:5 \ --retry-policy=type:delay"

...