Wednesday, 25 May 2016

SALT-SSH: AttributeError: 'module' object has no attribute 'fromstringlist'

This error triggers when the salt-master is CentOS6 and the target is CentOS7. 
The salt-master on CentOS6 would copy ElementTree.py to the minion on CentOS7 but 
python2.6's lib cannot run directly on python2.7.

To overcome this issue,

On the salt-master:

1). Find the thin.py

# find / -name thin.py
/usr/lib/python2.6/site-packages/salt/runners/thin.py
/usr/lib/python2.6/site-packages/salt/utils/thin.py

In /usr/lib/python2.6/site-packages/salt/utils/thin.py

try:
    import xml
    HAS_XML = True
except ImportError:
    HAS_XML = False

Modify as:

try:
    import xml
    HAS_XML = False
except ImportError:
    HAS_XML = False


2). remove the thin.tgz from cache

# rm /var/cache/salt/master/thin/thin.tgz

3). Perform the salt-ssh command again with -W
( -W for Select a random temp dir to deploy on the remote system. The dir will be cleaned
after the execution.)

# salt-ssh -W rnd test.ping

# salt-ssh -W rnd disk.usage

Wednesday, 24 February 2016

State system (High State & Low State) in SALT STACK:

For this example we are using top.sls and httpd_require.sls wihch contain,

# cat top.sls
base:
 '*':
  - httpd_require
root@ranjith1:/srv/salt# cat httpd_require.sls
install_httpd:
 pkg.installed:
   - name: httpd
 service.running:
  - name: httpd
  - enable: True
  - require:
    - file: install_httpd
 file.managed:
  - name: /var/www/html/index.html
  - source: salt://index1.html
  - user: root
  - group: root
  - mode: 644
  - require:
    - pkg: install_httpd

High state:

We can see all the aspects of high state system while working with state files( .sls), There are three specific components.

- High data:
- SLS file:
- High State

  - Each individual State represents a piece of high data.
   - for example:
       file.managed:
    - name: /var/www/html/index.html
    - source: salt://index1.html
    - user: root
    - group: root
    - mode: 644
    - require:
      - pkg: install_httpd
  - Salt will compile all relevant SLS inside the top.sls, When these files are tied together using includes, and further glued together for use inside an environment using a top.sls file, they form a High State.
we can see this using 'state.show_highstate',




First, an order is declared, All States that are set to be first will have their order adjusted accordingly. Salt will then add 10000 to the last defined number (which is 0 by default), and add any States that are not explicitly ordered.

Salt will also add some variables that it uses internally, to know which environment (__env__) to execute the State in, and which SLS file (__sls__) the State declaration came from, Remember that the order is still no more than a starting point; the actual High State will be executed based first on requisites, and then on order.

In other words, "High" data refers generally to data as it is seen by the user.

Low States:

"Low" data refers generally to data as it is ingested and used by Salt.

Once the final High State has been generated, it will be sent to the State compiler. This will reformat the State data into a format that Salt uses internally to evaluate
each declaration, and feed data into each State module (which will in turn call the execution modules, as necessary). As with high data, low data can be broken into individual components:
- Low State
- Low chunks
- State module
- Execution module(s)



Together, all this comprises a Low State. Each individual item is a Low Chunk. The first Low Chunk on this list looks like this:

- __env__: base
  __id__: install_httpd
  __sls__: httpd_require
  fun: installed
  name: http
  order: 10000
  state: pkg

Each low chunk maps to a State module (in this case, pkg) and a function inside that State module (in this case, installed). An ID is also provided at this level
(__id__). Salt will map relationships (that is, requisites) between States using a combination of State and __id__. If a name has not been declared by the user, then
Salt will automatically use the __id__ as the name.Once a function inside a State module has been called, it will usually map to one or more execution modules which actually do the work.

Saturday, 2 January 2016

Integrate SaltStack with other applications through SALT-API(rest_cherrypy):

 I was looking for the config management / remote execution tools and found SALT-STACK which fulfills the exact needs. 

 But, when it comes to integrate with other / our custom applications we had rest_tornado, rest_wsgi and rest_cherrypy options, i must thanks to Benjamin Cane and The Reluctant Tecchie who gave me the simple and best understanding about the SALT-API(rest_cherrypy) and configuring it to allow third party services to initiate SaltStack executions.

 Well, I'm using Ubuntu server as my salt-master(single salt-master and one Minion: 'my-api-minion' connected.) so i took the apt-get method to install the package,


# apt-get install salt-api

Generating the key to create SSL Certificate.


# openssl genrsa -out /etc/ssl/private/key.pem 4096

Sign the key and generate a certificate:

# openssl req -new -x509 -key /etc/ssl/private/key.pem -out /etc/ssl/private/cert.pem -days 1826

rest_cherrypy configuration:

added below values in /etc/salt/master:

rest_cherrypy:
  host: <your host IP> or
  port: 8000
  ssl_crt: /etc/ssl/private/cert.pem
  ssl_key: /etc/ssl/private/key.pem


External Auth configuration:

external_auth:
 pam:
  <saltuser>:
   - .*


The above values were also updated in /etc/salt/master after the 'rest_cherrypy:' config, kindly make sure the '<saltuser>' available or create the user.

# service salt-master restart
# service salt-api start


We're done configuring! Let's test this with curl,

# curl -ksi https://<your host IP>:8000/
HTTP/1.1 200 OK
Content-Length: 165
Access-Control-Expose-Headers: GET, POST
Vary: Accept-Encoding
Server: CherryPy/4.0.0
Allow: GET, HEAD, POST
Access-Control-Allow-Credentials: true
Date: Sat, 02 Jan 2016 08:12:45 GMT
Access-Control-Allow-Origin: *
Content-Type: application/json
Set-Cookie: session_id=1b1034009baa00ec797e94869215d77fd3d1b1e1; expires=Sat, 02 Jan 2016 18:12:45 GMT; Path=/

{"clients": ["_is_master_running", "local", "local_async", "local_batch", "runner", "runner_async", "ssh", "ssh_async", "wheel", "wheel_async"], "return": "Welcome"}

I got the value: "return": "Welcome",

Let try few more things,


# curl -ksi https://<your host IP>:8000 -H "Accept: application/x-yaml" -d client='local' -d tgt='my-api-minion' -d fun='test.ping'

Now you can see <title>401 Unauthorized</title>, we should create 'X-Auth-Token' using the salt-username and password to make things work !!

# curl -ksi https://<your host IP>:8000/login "Accept: application/json" -d username='saltuser' -d password='password' -d eauth='pam'

HTTP/1.1 200 OK
Content-Length: 177
Access-Control-Expose-Headers: GET, POST
Vary: Accept-Encoding
Server: CherryPy/4.0.0
Allow: GET, HEAD, POST
Access-Control-Allow-Credentials: true
Date: Sat, 02 Jan 2016 08:18:35 GMT
Access-Control-Allow-Origin: *
X-Auth-Token: 21d20f77641c15d64a8d5f3fd49ed2aad68076fa
Content-Type: application/json
Set-Cookie: session_id=21d20f77641c15d64a8d5f3fd49ed2aad68076fa; expires=Sat, 02 Jan 2016 18:18:35 GMT; Path=/

{"return": [{"perms": [".*"], "start": 1451722715.398141, "token": "21d20f77641c15d64a8d5f3fd49ed2aad68076fa", "expire": 1451765915.398142, "user": "saltuser", "eauth": "pam"}]}

fine, we got the token now.

# curl -ksi https://192.168.1 -H "Accept: application/x-yaml" -H "X-Auth-Token: 21d20f77641c15d64a8d5f3fd49ed2aad68076fa" -d client='local' -d tgt='my-api-minion' -d fun='test.ping'
HTTP/1.1 200 OK
Content-Length: 30
Access-Control-Expose-Headers: GET, POST
Access-Control-Allow-Credentials: true
Vary: Accept-Encoding
Server: CherryPy/4.0.0
Allow: GET, HEAD, POST
Cache-Control: private
Date: Sat, 02 Jan 2016 08:18:47 GMT
Access-Control-Allow-Origin: *
Content-Type: application/x-yaml
Set-Cookie: session_id=21d20f77641c15d64a8d5f3fd49ed2aad68076fa; expires=Sat, 02 Jan 2016 18:18:47 GMT; Path=/

return:
- my-api-minion: true

The importent values are,

 target(tgt), function(fun) and argument(arg).


In the above command am using my minion name instead of '*' which make sure that particular minion's availability.

Using state file:

Already i have touch.sls which contain,


# cat /srv/salt/touch.sls
create_file:
 file.touch:
  - name: /salt-api-check.txt


Lemme check:

# curl -ksi https://<your host IP>:8000 -H "Allow-Control-Allow-Headers: X-Auth-Token" -H "Accept: application/x-yaml" -H "X-Auth-Token: 21d20f77641c15d64a8d5f3fd49ed2aad68076fa" -d client='local' -d tgt='my-api-minion' -d fun='state.sls' -d arg='touch'
HTTP/1.1 200 OK
Content-Length: 311
Access-Control-Expose-Headers: GET, POST
Access-Control-Allow-Credentials: true
Vary: Accept-Encoding
Server: CherryPy/4.0.0
Allow: GET, HEAD, POST
Cache-Control: private
Date: Sat, 02 Jan 2016 08:26:35 GMT
Access-Control-Allow-Origin: *
Content-Type: application/x-yaml
Set-Cookie: session_id=21d20f77641c15d64a8d5f3fd49ed2aad68076fa; expires=Sat, 02 Jan 2016 18:26:35 GMT; Path=/

return:
- my-api-minion:
    file_|-create_file_|-/salt-api-check.txt_|-touch:
      __run_num__: 0
      changes:
        new: /salt-api-check.txt
      comment: Created empty file /salt-api-check.txt
      duration: 5.636
      name: /salt-api-check.txt
      result: true
      start_time: '13:57:56.274299'

its working.      **Don't forget to generate the new token if you restart the services.**