ChatOps Bot: Query Node Info Without Manual Login

Try Your Best To Avoid Any SSH Operations. Yes, I deeply believe in this principle. But my reality is the automation is never so perfect. I still need to login and check system status sometimes. Though the chance is rare.

It may happen at nights or even when I’m on vacation. So what I can do? Just carry my laptop with me wherever I go? This is certainly bad, isn’t it?

We’re living in the world of ChatOps. And mobile phones dominate our daily life (Sadly!) So why don’t we implement a ChatOps bot for this? Here comes a slack command: /chatqueryhost. (Note: The solution is not limited to Slack)

ChatOps Bot: Query Node Info Without Manual Login


Original Article: https://dennyzhang.com/chatqueryhost

Connect with Denny In LinkedIn Or MailList


Check More Discussion In LinkedIn!

ChatOps Bot: Query Node Info Without Manual Login


1. Install node_usage.py For Each Nodes.

It will get status for each node as a json output.

  • OS resource utilization: RAM, CPU, and disk.
  • [Optional] Get service status, or tail log files.

(See why I choose Python over Shell: GoodBye Shell, Hello Python!)

ChatOps Bot: Query Node Info Without Manual Login

2. Start A Web Server, Serving Slack Requests.

It will start a webserver with Python flask + uwsgi.

  • Get requests from Slack input.
  • Run remote ssh command, which is literally node_usage.py.

Code is GitHub. [1] (Warning. I admit: It’s not well documented. Just F.Y.I).

You can design and implement your own ChatOps solution for this.

And here are some suggestions I’ve for you, my friends:

  1. Slack command must return within 3 seconds.[2]
  2. Instead of running ssh commands, it’s safer to wrap py script as agents. Then the agent serves as a tcp or http server.
  3. Support find node by fuzzy match. Your env may have tens of nodes, if not hundereds of. So support people to identify node by giving part of hostname.

2 Responses to ChatOps Bot: Query Node Info Without Manual Login

  1. Not saying the idea doesn’t have merit. It does, and it’s useful. Just that I’m concerned about how easy this would be for a rogue employee to abuse. Admittedly I’ll need to dig into the workings more to dissect how it works. But this is the area that most concerns me.

    • James, I agree we shouldn’t forget security.

      1. For small teams, I think we’re fine.
      2. For big groups, we can enforce whitelist check based on slack users. Thus only selective members can run this in slack.

      Do you see any risks or issues about this approach?

Leave a Reply

Your email address will not be published. Required fields are marked *